weiguoqiang/cert-manager
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Gracefully handle 404s when fetching old CRDs

Browse Source 
If we're in the middle of a cert-manager release we'll have a git tag
created for that release, but won't have a GitHub release created yet.
That means that an attempt to download old CRDs for our versionchecker
test will fail for that version, with a 404 error.

An alternative approach would be to use the GitHub API to query for
existing non-draft releases - but that introduces a new point of failure
whereby we can easily hit a rate limit, or else introduces the need for
a GitHub API token.

The GitHub API also has the issue that it doesn't present every release
in one API call, which complicates fetching releases using curl and uses
even more rate-limit capacity.

The approach here is simple; we ignore tags for which the release 404s,
download manifests for which the release gives a 200, and bubble up any
other errors.

Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
This commit is contained in:
Ashley Davis 2022-03-30 15:16:17 +01:00
parent f4f40f689e
commit 7ea4496dcb
No known key found for this signature in database
GPG Key ID: DD14CC017E32BEB1
2 changed files with 60 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

57
hack/fetch-old-crd.sh Executable file
View File

@ -0,0 +1,57 @@
#!/usr/bin/env bash
# Copyright 2022 The cert-manager Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -eu -o pipefail
# This script fetches old CRDs from GitHub releases but gracefully exits without an error
# if it encounters a 404. This handles the case where a git tag exists but no release
# exists, which would otherwise cause fetching the CRDs to fail.
function print_help() {
echo "usage: $0 <url-to-fetch> <file-to-write>" > /dev/stderr
}
if [[ -z "${1:-}" ]]; then
print_help
exit 1
fi
if [[ -z "${2:-}" ]]; then
print_help
exit 1
fi
url=$1
destfile=$2
# make curl write to a temp file, since we don't want to write to destfile if
# we get a 404 from GitHub
outfile=$(mktemp)
trap 'rm -f -- "$outfile"' EXIT
STATUSCODE=$(curl --retry 3 --compressed --silent --location --output $outfile --write-out "%{http_code}" $url)
if test $STATUSCODE -eq 404; then
# If a tag exists without a release, then we'll get a 404 here. This could happen during a release, for example.
# In this case, we don't error and don't write anything to destfile
exit 0
elif test $STATUSCODE -ne 200; then
echo "Got status code $STATUSCODE for '$url' - possibly broken or in-progress release / GitHub down / rate limit" > /dev/stderr
exit 1
fi
cp $outfile $destfile

6
make/test.mk
View File

@ -85,9 +85,9 @@ test/integration/versionchecker/testdata/test_manifests.tar: bin/scratch/oldcrds
bin/scratch/oldcrds.tar: bin/scratch/git/upstream-tags.txt | bin/scratch/oldcrds
@# First, download the CRDs for all releases listed in upstream-tags.txt
<bin/scratch/git/upstream-tags.txt xargs -I% -P5 \
curl --compressed -sSfL \
-o bin/scratch/oldcrds/%.yaml \
"https://github.com/cert-manager/cert-manager/releases/download/%/cert-manager.yaml"
./hack/fetch-old-crd.sh \
"https://github.com/cert-manager/cert-manager/releases/download/%/cert-manager.yaml" \
bin/scratch/oldcrds/%.yaml
@# Next, tar up the old CRDs together
tar cf $@ -C bin/scratch/oldcrds .