These enable scanning of each of our container images on linux/amd64
to check for vulnerabilities. These targets can then be used in CI as
an indicator that we might need to take a look at upgrading dependencies
or base images.
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
I also took the opportunity to document the three most important "build"
variables in "make help".
Manually rebased to adopt $(BINDIR) changes
Signed-off-by: Maël Valais <mael@vls.dev>
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
This adds multiple retries on every attempt we make to use curl, which
should help to reduce flakes. Uses a $(CURL) variable where possible so
that we have the same invocation everywhere.
Also switches to using the more verbose curl arguments, in an attempt to
make it easier to reason about how curl is configured.
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
previously we'd relied on rosetta because these tools hadn't been built
for darwin-arm64, but now they've started to be built and we can use
arm64 versions directly
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
we don't _need_ to remove these and we can keep them around for
longer, but we don't need them to be in files we actually use and edit.
putting the targets in a separate file feels cleaner!
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
We only use python in one place and probably won't start using it more
without some kind of policy change. We don't need to require that everyone
has it installed, and can instead only require it for people who're running
the boilerplate check
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
- includes a run of make update-crds which causes some trivial changes
- updates version of YQ to latest
- makes hack/update-crds.sh just call make
- makes hack/verify-crds.sh just call make
- moves functionality of hack/verify-crds.sh to hack/check-crds.sh,
using the makefile for generating alternative CRDs for comparison
- removes the bazel test associated with CRDs
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
This is needed because go and other tools will ignore directories
starting with "_" or "." but would treat a dir called "bin" as a regular
directory.
This in turn meant that when we vendored Go in bin, these tools would by
default scan the whole stdlib included with the bundled vendored go.
See https://pkg.go.dev/cmd/go#hdr-Package_lists_and_patterns for details
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
this removes the update functionality from make/cluster.sh in favour of
the simpler logic in hack/latest-kind-images.sh
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
- "make help" should be good enough for most people in most situations
to build and test cert-manager
- "make clean-all" can be a one-stop-shop to start fresh.
- "make which-go" makes it clearer whether go vendoring is being used
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
I think having a separate file for storing the various kind image
digest isn't necessary. From now on, make/cluster.sh is the "source of
truth" for everything related to kind. If you would like to see the
image that is going to be used for creating the kind cluster, you can
run:
make/cluster.sh --show-image
To self-update the digests with the latest available digests, run:
make/cluster.sh --update-images
Signed-off-by: Maël Valais <mael@vls.dev>
This is to work around the fact that binaries in hostPath-mounted
directories cannot be executed even if the permissions are correct.
Signed-off-by: Maël Valais <mael@vls.dev>
I had written instructions that would show up when you don't have one of
the mandatory system tools such as go or jq. After some feedback, I
decided to remove these instructions since we are concerned that these
instructions may become out of date.
Signed-off-by: Maël Valais <mael@vls.dev>
The commands can be run concurrently, with the exception of e2e that
has to be run after e2e-setup is done. The e2e target does not check
whether cert-manager and the addons are installed.
The two only scripts that were kept are:
- make/e2e.sh (previously called ./devel/run-e2e.sh)
- make/cluster.sh (previsouly called ./devel/cluster/create.sh)
The reason for the removal of the other scripts is that they didn't
have that much logic and could easily ported to Make, improving greatly
the execution speed thanks to make's concurrency.
make/e2e.sh now behaves "as expected" when using -ginkgo.focus or
GINKGO_FOCUS; previously, the logs would not be shown before the end
of the test.
make/cluster.sh has lost the ability to create an OpenShift 3.11 cluster.
for running the end-to-end tests. The two reasons are that OpenShift 4
wasn't supported by the script devel/cluster/create.sh, and OpenShift
3.11 is not supported by cert-manager anymore.
The Makefile targets that were used in the Prow jobs (verify, verify_deps,
verify_chart, verify_upgrade, and cluster) have been kept around. They
now show a warning to encourage people to use the new Make-based targets.
When running one of the deprecated targets the Makefile won't check the
presence of the system tools such as Go and jq, since Bazel takes care of
these dependencies.
On version change, downloaded tools and images are re-downloaded. The
command 'make clean' now keeps the downloaded images and tools.
Note that a lot of attention has been put into having a Make system that works
flawlessly both on Linux and on BSDs (such as macOS).
You will note that some recursive calls to make are made, and $(MAKE)
instead of plain "make" is used in that case. If we didn't use $(MAKE),
we would have concurrency issues, and warnings such as:
make[1]: warning: jobserver unavailable: using -j1. Add `+' to parent make rule.
Signed-off-by: Maël Valais <mael@vls.dev>
These lean heavily on `go test` for everything possible.
Also adds setup for versionchecker test in make, and a script for
extracting CRDs from templated rendered YAML files
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
Note that darwin/arm64 has to be special cased currently because of a
lack of official support for that platform in upstream etcd and
kube-apiserver. We instead install the amd64 versions, which is what
we did for bazel.
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
when on temp target, the object being verified will still be written and
so a second invocation of `make tools` will succeed and install an
unverified version
For more information see attached issue
Fixes#4678
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
Includes targets for:
- all "server" binaries, for all arches
- all containers for all server binaries for all arches
- all client binaries (kubectl plugin / cmctl) for all arches
- the cert-manager helm chart + signature
- the cert-manager static manifests + CRDs
- tools which bazel would download, with checksum verification
- (commented out) a signed SHA256SUM file for client binaries
Upgrades from the bazel flow include that:
- we use OS-specific base images rather than just using amd64 everywhere
- we easily add support for signing artifacts at build time
- we add ".exe" to the end of windows executables
- we add a zip file for windows executables, for easier consumption
- we concatenate YAML files more robustly
- staging a full release should be much faster
- hopefully, it's easier to change things!
- licenses are trimmed down to reduce bloat in images (the license
bundle was 1.4MB in size alone)
Changes from the bazel flow include:
- containers no longer have a symlink to the binary at an unusual
path, but instead just have the binary at a more predictable path
(e.g. /app/cmd/webhook/webhook instead of
/app/cmd/webhook/webhook.runfiles/com_github_jetstack_cert_manager/cmd/webhook/webhook_/webhook)
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
