[KYUUBI #6047] [CI] Free up disk space
# 🔍 Description ## Issue References 🔗 I saw [the K8s IT](https://github.com/apache/kyuubi/actions/runs/7776268590/job/21203214430?pr=6045) failed with ``` Error: No space left on device : '/home/runner/runners/2.312.0/_diag/pages/131e7057-723d-4281-b296-b876237f4318_21774d93-c106-57cd-810b-f0e6bf96d865_1.log' ``` ## Describe Your Solution 🔧 Borrow the solution from Apache Spark, just uninstall the un-used large pre-installed packages. ## Types of changes 🔖 - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Test Plan 🧪 Wait CI report --- # Checklist 📝 - [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html) **Be nice. Be informative.** Closes #6047 from pan3793/ci-space. Closes #6047 272df2790 [Cheng Pan] nit 8c78115a4 [Cheng Pan] nit a9c8b035c [Cheng Pan] nit ab4c3e461 [Cheng Pan] keep jdk8 9d1ec4e44 [Cheng Pan] extend 47f90b95e [Cheng Pan] nit 30b82cdf0 [Cheng Pan] fix 352689f89 [Cheng Pan] [CI] Free up disk space for K8s IT Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Cheng Pan <chengpan@apache.org>
This commit is contained in:
parent
f67140e650
commit
5f99ec8d6a
55
.github/scripts/free_disk_space.sh
vendored
Executable file
55
.github/scripts/free_disk_space.sh
vendored
Executable file
@ -0,0 +1,55 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
#
|
||||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
#
|
||||
|
||||
# This script is inspired by Apache Spark
|
||||
|
||||
echo "=================================="
|
||||
echo "Free up disk space on CI system"
|
||||
echo "=================================="
|
||||
|
||||
echo "Listing 100 largest packages"
|
||||
dpkg-query -Wf '${Installed-Size}\t${Package}\n' | sort -n | tail -n 100
|
||||
df -h
|
||||
|
||||
echo "Removing large packages"
|
||||
sudo rm -rf /usr/share/dotnet/
|
||||
sudo rm -rf /usr/share/php/
|
||||
sudo rm -rf /usr/local/graalvm/
|
||||
sudo rm -rf /usr/local/.ghcup/
|
||||
sudo rm -rf /usr/local/share/powershell
|
||||
sudo rm -rf /usr/local/share/chromium
|
||||
sudo rm -rf /usr/local/lib/android
|
||||
sudo rm -rf /usr/local/lib/node_modules
|
||||
|
||||
sudo rm -rf /opt/az
|
||||
sudo rm -rf /opt/hostedtoolcache/CodeQL
|
||||
sudo rm -rf /opt/hostedtoolcache/go
|
||||
sudo rm -rf /opt/hostedtoolcache/node
|
||||
|
||||
sudo apt-get remove --purge -y '^aspnet.*'
|
||||
sudo apt-get remove --purge -y '^dotnet-.*'
|
||||
sudo apt-get remove --purge -y '^llvm-.*'
|
||||
sudo apt-get remove --purge -y 'php.*'
|
||||
sudo apt-get remove --purge -y '^temurin-\d{n,}.*'
|
||||
sudo apt-get remove --purge -y snapd google-chrome-stable microsoft-edge-stable firefox
|
||||
sudo apt-get remove --purge -y azure-cli google-cloud-sdk mono-devel msbuild powershell libgl1-mesa-dri
|
||||
sudo apt-get autoremove --purge -y
|
||||
sudo apt-get clean
|
||||
|
||||
df -h
|
||||
4
.github/workflows/gluten.yml
vendored
4
.github/workflows/gluten.yml
vendored
@ -30,6 +30,8 @@ jobs:
|
||||
runs-on: ubuntu-22.04
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Free up disk space
|
||||
run: ./.github/scripts/free_disk_space.sh
|
||||
- name: Tune Runner VM
|
||||
uses: ./.github/actions/tune-runner-vm
|
||||
- name: Update and Upgrade
|
||||
@ -79,6 +81,8 @@ jobs:
|
||||
spark: [ '3.4', '3.3' ]
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Free up disk space
|
||||
run: ./.github/scripts/free_disk_space.sh
|
||||
- name: Tune Runner VM
|
||||
uses: ./.github/actions/tune-runner-vm
|
||||
- name: Update and Upgrade
|
||||
|
||||
24
.github/workflows/master.yml
vendored
24
.github/workflows/master.yml
vendored
@ -87,6 +87,8 @@ jobs:
|
||||
SPARK_LOCAL_IP: localhost
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Free up disk space
|
||||
run: ./.github/scripts/free_disk_space.sh
|
||||
- name: Tune Runner VM
|
||||
uses: ./.github/actions/tune-runner-vm
|
||||
- name: Setup JDK ${{ matrix.java }}
|
||||
@ -144,6 +146,8 @@ jobs:
|
||||
- '3.4'
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Free up disk space
|
||||
run: ./.github/scripts/free_disk_space.sh
|
||||
- name: Tune Runner VM
|
||||
uses: ./.github/actions/tune-runner-vm
|
||||
- name: Setup JDK ${{ matrix.java }}
|
||||
@ -200,6 +204,8 @@ jobs:
|
||||
comment: 'verify-on-flink-1.18-binary'
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Free up disk space
|
||||
run: ./.github/scripts/free_disk_space.sh
|
||||
- name: Tune Runner VM
|
||||
uses: ./.github/actions/tune-runner-vm
|
||||
- name: Setup JDK ${{ matrix.java }}
|
||||
@ -256,6 +262,8 @@ jobs:
|
||||
comment: 'verify-on-hive-2.1-cdh6-binary'
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Free up disk space
|
||||
run: ./.github/scripts/free_disk_space.sh
|
||||
- name: Tune Runner VM
|
||||
uses: ./.github/actions/tune-runner-vm
|
||||
- name: Setup JDK ${{ matrix.java }}
|
||||
@ -302,6 +310,8 @@ jobs:
|
||||
comment: [ "normal" ]
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Free up disk space
|
||||
run: ./.github/scripts/free_disk_space.sh
|
||||
- name: Tune Runner VM
|
||||
uses: ./.github/actions/tune-runner-vm
|
||||
- name: Setup JDK ${{ matrix.java }}
|
||||
@ -341,6 +351,8 @@ jobs:
|
||||
comment: [ "normal" ]
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Free up disk space
|
||||
run: ./.github/scripts/free_disk_space.sh
|
||||
- name: Tune Runner VM
|
||||
uses: ./.github/actions/tune-runner-vm
|
||||
- name: Setup JDK ${{ matrix.java }}
|
||||
@ -375,6 +387,8 @@ jobs:
|
||||
SPARK_LOCAL_IP: localhost
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Free up disk space
|
||||
run: ./.github/scripts/free_disk_space.sh
|
||||
- name: Tune Runner VM
|
||||
uses: ./.github/actions/tune-runner-vm
|
||||
- name: Setup JDK 8
|
||||
@ -400,8 +414,9 @@ jobs:
|
||||
name: Kyuubi Server On Kubernetes Integration Test
|
||||
runs-on: ubuntu-22.04
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v4
|
||||
- name: Free up disk space
|
||||
run: ./.github/scripts/free_disk_space.sh
|
||||
# https://github.com/docker/build-push-action
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v3
|
||||
@ -472,8 +487,9 @@ jobs:
|
||||
name: Spark Engine On Kubernetes Integration Test
|
||||
runs-on: ubuntu-22.04
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v4
|
||||
- name: Free up disk space
|
||||
run: ./.github/scripts/free_disk_space.sh
|
||||
- name: Cache Engine Archives
|
||||
uses: ./.github/actions/cache-engine-archives
|
||||
- name: Setup Minikube
|
||||
|
||||
@ -27,8 +27,9 @@ jobs:
|
||||
if: ${{ startsWith(github.repository, 'apache/') }}
|
||||
runs-on: ubuntu-22.04
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v4
|
||||
- name: Free up disk space
|
||||
run: ./.github/scripts/free_disk_space.sh
|
||||
- name: Set up QEMU
|
||||
uses: docker/setup-qemu-action@v3
|
||||
- name: Set up Docker Buildx
|
||||
|
||||
5
.github/workflows/publish-snapshot-nexus.yml
vendored
5
.github/workflows/publish-snapshot-nexus.yml
vendored
@ -42,10 +42,11 @@ jobs:
|
||||
- branch: branch-1.8
|
||||
profiles: -Pflink-provided,spark-provided,hive-provided,spark-3.4
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ matrix.branch }}
|
||||
- name: Free up disk space
|
||||
run: ./.github/scripts/free_disk_space.sh
|
||||
- name: Setup JDK 8
|
||||
uses: actions/setup-java@v4
|
||||
with:
|
||||
|
||||
Loading…
Reference in New Issue
Block a user