[KYUUBI #6765] [CI] speed up the script for freeing disk space by batching removals

# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- batching removals to avoid repeatedly reading and updating the apt-get databases
- 50% speed up the execution time for freeing disk space from 2 minutes or more , to about 1 minutues.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6765 from bowenliang123/speedup-freedisk.

Closes #6765

dd475c243 [Bowen Liang] update (+1 squashed commit) Squashed commits: [688793f47] update

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Kent Yao <yao@apache.org>
This commit is contained in:
Bowen Liang 2024-11-14 18:31:39 +08:00 committed by Kent Yao
parent 2453efe145
commit e03207baa2
No known key found for this signature in database
GPG Key ID: F7051850A0AF904D

View File

@ -19,6 +19,8 @@
# This script is inspired by Apache Spark
set -x
echo "=================================="
echo "Free up disk space on CI system"
echo "=================================="
@ -28,28 +30,34 @@ dpkg-query -Wf '${Installed-Size}\t${Package}\n' | sort -n | tail -n 100
df -h
echo "Removing large packages"
sudo rm -rf /usr/share/dotnet/
sudo rm -rf /usr/share/php/
sudo rm -rf /usr/local/graalvm/
sudo rm -rf /usr/local/.ghcup/
sudo rm -rf /usr/local/share/powershell
sudo rm -rf /usr/local/share/chromium
sudo rm -rf /usr/local/lib/android
sudo rm -rf /usr/local/lib/node_modules
sudo rm -rf /opt/az
sudo rm -rf /opt/hostedtoolcache/CodeQL
sudo rm -rf /opt/hostedtoolcache/go
sudo rm -rf /opt/hostedtoolcache/node
sudo apt-get remove --purge -y '^aspnet.*'
sudo apt-get remove --purge -y '^dotnet-.*'
sudo apt-get remove --purge -y '^llvm-.*'
sudo apt-get remove --purge -y 'php.*'
sudo apt-get remove --purge -y '^temurin-\d{n,}.*'
sudo apt-get remove --purge -y snapd google-chrome-stable microsoft-edge-stable firefox
sudo apt-get remove --purge -y azure-cli google-cloud-sdk mono-devel msbuild powershell libgl1-mesa-dri
sudo apt-get autoremove --purge -y
sudo apt-get clean
sudo rm -rf /usr/share/dotnet/ \
/usr/share/php/ \
/usr/local/graalvm/ \
/usr/local/.ghcup/ \
/usr/local/share/powershell \
/usr/local/share/chromium \
/usr/local/lib/android \
/usr/local/lib/node_modules \
/opt/az \
/opt/hostedtoolcache/CodeQL \
/opt/hostedtoolcache/go \
/opt/hostedtoolcache/node
df -h
sudo apt-get remove --purge -y \
'^aspnet.*' \
'^dotnet-.*' \
'^llvm.*' \
'php.*' \
'^temurin-\d{n,}.*' \
snapd google-chrome-stable microsoft-edge-stable firefox \
azure-cli google-cloud-sdk mono-devel msbuild powershell libgl1-mesa-dri
df -h
sudo apt-get autoremove --purge -y \
&& sudo apt-get clean \
&& sudo apt-get autoclean
df -h