[KYUUBI #6047] [CI] Free up disk space

# 🔍 Description
## Issue References 🔗

I saw [the K8s IT](https://github.com/apache/kyuubi/actions/runs/7776268590/job/21203214430?pr=6045) failed with

```
Error: No space left on device : '/home/runner/runners/2.312.0/_diag/pages/131e7057-723d-4281-b296-b876237f4318_21774d93-c106-57cd-810b-f0e6bf96d865_1.log'
```

## Describe Your Solution 🔧

Borrow the solution from Apache Spark, just uninstall the un-used large pre-installed packages.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

Wait CI report

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6047 from pan3793/ci-space.

Closes #6047

272df2790 [Cheng Pan] nit
8c78115a4 [Cheng Pan] nit
a9c8b035c [Cheng Pan] nit
ab4c3e461 [Cheng Pan] keep jdk8
9d1ec4e44 [Cheng Pan] extend
47f90b95e [Cheng Pan] nit
30b82cdf0 [Cheng Pan] fix
352689f89 [Cheng Pan] [CI] Free up disk space for K8s IT

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
This commit is contained in:
Cheng Pan 2024-02-05 12:12:15 +08:00
parent f67140e650
commit 5f99ec8d6a
No known key found for this signature in database
GPG Key ID: 8001952629BCC75D
5 changed files with 85 additions and 8 deletions

55
.github/scripts/free_disk_space.sh vendored Executable file
View File

@ -0,0 +1,55 @@
#!/usr/bin/env bash
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# This script is inspired by Apache Spark
echo "=================================="
echo "Free up disk space on CI system"
echo "=================================="
echo "Listing 100 largest packages"
dpkg-query -Wf '${Installed-Size}\t${Package}\n' | sort -n | tail -n 100
df -h
echo "Removing large packages"
sudo rm -rf /usr/share/dotnet/
sudo rm -rf /usr/share/php/
sudo rm -rf /usr/local/graalvm/
sudo rm -rf /usr/local/.ghcup/
sudo rm -rf /usr/local/share/powershell
sudo rm -rf /usr/local/share/chromium
sudo rm -rf /usr/local/lib/android
sudo rm -rf /usr/local/lib/node_modules
sudo rm -rf /opt/az
sudo rm -rf /opt/hostedtoolcache/CodeQL
sudo rm -rf /opt/hostedtoolcache/go
sudo rm -rf /opt/hostedtoolcache/node
sudo apt-get remove --purge -y '^aspnet.*'
sudo apt-get remove --purge -y '^dotnet-.*'
sudo apt-get remove --purge -y '^llvm-.*'
sudo apt-get remove --purge -y 'php.*'
sudo apt-get remove --purge -y '^temurin-\d{n,}.*'
sudo apt-get remove --purge -y snapd google-chrome-stable microsoft-edge-stable firefox
sudo apt-get remove --purge -y azure-cli google-cloud-sdk mono-devel msbuild powershell libgl1-mesa-dri
sudo apt-get autoremove --purge -y
sudo apt-get clean
df -h

View File

@ -30,6 +30,8 @@ jobs:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- name: Free up disk space
run: ./.github/scripts/free_disk_space.sh
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
- name: Update and Upgrade
@ -79,6 +81,8 @@ jobs:
spark: [ '3.4', '3.3' ]
steps:
- uses: actions/checkout@v4
- name: Free up disk space
run: ./.github/scripts/free_disk_space.sh
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
- name: Update and Upgrade

View File

@ -87,6 +87,8 @@ jobs:
SPARK_LOCAL_IP: localhost
steps:
- uses: actions/checkout@v4
- name: Free up disk space
run: ./.github/scripts/free_disk_space.sh
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
- name: Setup JDK ${{ matrix.java }}
@ -144,6 +146,8 @@ jobs:
- '3.4'
steps:
- uses: actions/checkout@v4
- name: Free up disk space
run: ./.github/scripts/free_disk_space.sh
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
- name: Setup JDK ${{ matrix.java }}
@ -200,6 +204,8 @@ jobs:
comment: 'verify-on-flink-1.18-binary'
steps:
- uses: actions/checkout@v4
- name: Free up disk space
run: ./.github/scripts/free_disk_space.sh
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
- name: Setup JDK ${{ matrix.java }}
@ -256,6 +262,8 @@ jobs:
comment: 'verify-on-hive-2.1-cdh6-binary'
steps:
- uses: actions/checkout@v4
- name: Free up disk space
run: ./.github/scripts/free_disk_space.sh
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
- name: Setup JDK ${{ matrix.java }}
@ -302,6 +310,8 @@ jobs:
comment: [ "normal" ]
steps:
- uses: actions/checkout@v4
- name: Free up disk space
run: ./.github/scripts/free_disk_space.sh
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
- name: Setup JDK ${{ matrix.java }}
@ -341,6 +351,8 @@ jobs:
comment: [ "normal" ]
steps:
- uses: actions/checkout@v4
- name: Free up disk space
run: ./.github/scripts/free_disk_space.sh
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
- name: Setup JDK ${{ matrix.java }}
@ -375,6 +387,8 @@ jobs:
SPARK_LOCAL_IP: localhost
steps:
- uses: actions/checkout@v4
- name: Free up disk space
run: ./.github/scripts/free_disk_space.sh
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
- name: Setup JDK 8
@ -400,8 +414,9 @@ jobs:
name: Kyuubi Server On Kubernetes Integration Test
runs-on: ubuntu-22.04
steps:
- name: Checkout
uses: actions/checkout@v4
- uses: actions/checkout@v4
- name: Free up disk space
run: ./.github/scripts/free_disk_space.sh
# https://github.com/docker/build-push-action
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
@ -472,8 +487,9 @@ jobs:
name: Spark Engine On Kubernetes Integration Test
runs-on: ubuntu-22.04
steps:
- name: Checkout
uses: actions/checkout@v4
- uses: actions/checkout@v4
- name: Free up disk space
run: ./.github/scripts/free_disk_space.sh
- name: Cache Engine Archives
uses: ./.github/actions/cache-engine-archives
- name: Setup Minikube

View File

@ -27,8 +27,9 @@ jobs:
if: ${{ startsWith(github.repository, 'apache/') }}
runs-on: ubuntu-22.04
steps:
- name: Checkout
uses: actions/checkout@v4
- uses: actions/checkout@v4
- name: Free up disk space
run: ./.github/scripts/free_disk_space.sh
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx

View File

@ -42,10 +42,11 @@ jobs:
- branch: branch-1.8
profiles: -Pflink-provided,spark-provided,hive-provided,spark-3.4
steps:
- name: Checkout repository
uses: actions/checkout@v4
- uses: actions/checkout@v4
with:
ref: ${{ matrix.branch }}
- name: Free up disk space
run: ./.github/scripts/free_disk_space.sh
- name: Setup JDK 8
uses: actions/setup-java@v4
with: