This uses cmctl instead of kubectl_cert-manager, uses make instead of
bazel and fixes an incorrect container name in
test/fixtures/upgrade/overlay/cainjector-ops.yaml
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
This is needed because go and other tools will ignore directories
starting with "_" or "." but would treat a dir called "bin" as a regular
directory.
This in turn meant that when we vendored Go in bin, these tools would by
default scan the whole stdlib included with the bundled vendored go.
See https://pkg.go.dev/cmd/go#hdr-Package_lists_and_patterns for details
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
- "make help" should be good enough for most people in most situations
to build and test cert-manager
- "make clean-all" can be a one-stop-shop to start fresh.
- "make which-go" makes it clearer whether go vendoring is being used
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
If we're in the middle of a cert-manager release we'll have a git tag
created for that release, but won't have a GitHub release created yet.
That means that an attempt to download old CRDs for our versionchecker
test will fail for that version, with a 404 error.
An alternative approach would be to use the GitHub API to query for
existing non-draft releases - but that introduces a new point of failure
whereby we can easily hit a rate limit, or else introduces the need for
a GitHub API token.
The GitHub API also has the issue that it doesn't present every release
in one API call, which complicates fetching releases using curl and uses
even more rate-limit capacity.
The approach here is simple; we ignore tags for which the release 404s,
download manifests for which the release gives a 200, and bubble up any
other errors.
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
Previously, the XML was saved to $ARTIFACTS, which was not picked up by
the Prow UI. The XML files must be of the form: junit_*.xml.
This format is required by our Spyglass configuration (Spyglass is the
name the Prow UI) [1].
[1]: https://github.com/jetstack/testing/blob/d04c4641f/config/config.yaml#L80-L81
Signed-off-by: Maël Valais <mael@vls.dev>
It seems like Ginkgo isn't running from the root of the repository, that's why
it can't find binaries, e.g., bin/tools/etcd.
Signed-off-by: Maël Valais <mael@vls.dev>
The commands can be run concurrently, with the exception of e2e that
has to be run after e2e-setup is done. The e2e target does not check
whether cert-manager and the addons are installed.
The two only scripts that were kept are:
- make/e2e.sh (previously called ./devel/run-e2e.sh)
- make/cluster.sh (previsouly called ./devel/cluster/create.sh)
The reason for the removal of the other scripts is that they didn't
have that much logic and could easily ported to Make, improving greatly
the execution speed thanks to make's concurrency.
make/e2e.sh now behaves "as expected" when using -ginkgo.focus or
GINKGO_FOCUS; previously, the logs would not be shown before the end
of the test.
make/cluster.sh has lost the ability to create an OpenShift 3.11 cluster.
for running the end-to-end tests. The two reasons are that OpenShift 4
wasn't supported by the script devel/cluster/create.sh, and OpenShift
3.11 is not supported by cert-manager anymore.
The Makefile targets that were used in the Prow jobs (verify, verify_deps,
verify_chart, verify_upgrade, and cluster) have been kept around. They
now show a warning to encourage people to use the new Make-based targets.
When running one of the deprecated targets the Makefile won't check the
presence of the system tools such as Go and jq, since Bazel takes care of
these dependencies.
On version change, downloaded tools and images are re-downloaded. The
command 'make clean' now keeps the downloaded images and tools.
Note that a lot of attention has been put into having a Make system that works
flawlessly both on Linux and on BSDs (such as macOS).
You will note that some recursive calls to make are made, and $(MAKE)
instead of plain "make" is used in that case. If we didn't use $(MAKE),
we would have concurrency issues, and warnings such as:
make[1]: warning: jobserver unavailable: using -j1. Add `+' to parent make rule.
Signed-off-by: Maël Valais <mael@vls.dev>
These lean heavily on `go test` for everything possible.
Also adds setup for versionchecker test in make, and a script for
extracting CRDs from templated rendered YAML files
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
