Commit Graph

68 Commits

Author SHA1 Message Date
Maël Valais
f70baa8d3c make: test-ci: hide fuzz_xxxx test results in the JUnit XML
The Prow UI can't handle that many test cases (over 40,000 test cases,
53,000 XML lines).

Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-29 10:45:08 +02:00
Maël Valais
f8187d17f2 make: e2e-setup-certmanager: fix the acmesolver image
The default value for --acme-http01-solver-image was off.

 quay.io/jetstack/cert-manager-acmesolver:v1.7.0-beta.0-260-gd0abb71ef0a78d
 <-------------------------------------->
           incorrect

 cert-manager-acmesolver-amd64:v1.7.0-beta.0-260-gd0abb71ef0a78d
 <--------------------------->
           correct

Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-29 10:45:08 +02:00
Maël Valais
fbb372a1a4 make/cluster.sh: don't display the networking info, it isn't helpful
Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-29 10:45:08 +02:00
Maël Valais
37628bf0ed make: e2e-setup-sampleexternalissuer: bump from v0.0.0-30 to v0.1.1
Not sure how the end-to-end tests were ever able to run previously. My
guess: the sample-external-issuer end-to-end suite is disabled.

Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-29 10:45:08 +02:00
Maël Valais
b33f5280a8 make/e2e.sh: the "trace" func was showing quotes in the wrong place
Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-29 10:45:08 +02:00
Maël Valais
66aef65267 make: test-ci now writes JUnit XML to $ARTIFACTS/junit_make-test-ci.xml
Previously, the XML was saved to $ARTIFACTS, which was not picked up by
the Prow UI. The XML files must be of the form: junit_*.xml.

This format is required by our Spyglass configuration (Spyglass is the
name the Prow UI) [1].

[1]: https://github.com/jetstack/testing/blob/d04c4641f/config/config.yaml#L80-L81

Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-29 10:45:08 +02:00
jetstack-bot
7091eaabd8
Merge pull request #4985 from SgtCoDFish/bumpbase
Bump base images to latest versions
2022-03-25 18:50:21 +00:00
Maël Valais
904a00fefb make: tools version is now properly switched when switching branches
Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-25 18:03:35 +01:00
Maël Valais
c6f1f9bc24 make: in CI, copy binaries from "bin/downlaoded" to "bin/tools"
This is to work around the fact that binaries in hostPath-mounted
directories cannot be executed even if the permissions are correct.

Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-25 18:01:54 +01:00
Ashley Davis
d9d960e5f7
bump base images to latest versions
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
2022-03-25 12:04:34 +00:00
Ashley Davis
5d17ec5d2d
add a build source indicator to metadata
This makes it easier to tell when a build was made by the makefile
workflow and therefore to adjust `cmrel publish` to adapt to changes

Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
2022-03-21 21:37:03 +00:00
Ashley Davis
a2a5f3f564
clear up the release target a little, ensure all things are built
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
2022-03-18 16:24:20 +00:00
Ashley Davis
e3c3939113
some small QoL tweaks in makefiles
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
2022-03-18 16:14:46 +00:00
Ashley Davis
968aeed6d7
fix phony names for some targets
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
2022-03-18 14:45:50 +00:00
Maël Valais
d30f47293d make: warn people about e2e-setup-traefik and e2e-setup-haproxyingress
These two targets are not used by our end-to-end tests meaning that they
may either be broken or removed in the future.

Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-16 17:51:44 +01:00
Joakim Ahrlin
ff2d82f381 make: e2e-setup-haproxyingress: add haproxyingress arm64 image
Signed-off-by: Joakim Ahrlin <joakim.ahrlin@gmail.com>
2022-03-16 17:51:27 +01:00
Maël Valais
56888639fd make: e2e-setup-traefik: fix dditionalArguments -> additionalArguments
Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-16 14:06:55 +01:00
Maël Valais
9c8cf7564b docs: suggest -j8 instead of -j to avoid fork bombs
Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-16 14:05:07 +01:00
Maël Valais
3ed6ba3621 docs: mention direnv with export PATH=bin/tools
Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-16 14:03:38 +01:00
Maël Valais
3f8f12a7af make: remove comments I re-added when moving release_containers.mk to release.mk
Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-16 13:53:42 +01:00
Maël Valais
82190e7656 make: e2e-setup: pebble can now be built on M1 macs
Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-16 13:30:21 +01:00
Maël Valais
0df0a2cf21 make: e2e-ci: export ARTIFACTS so that it goes to make/e2e.sh
Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-15 17:04:25 +01:00
Maël Valais
029db6258f make: test-ci now tests $(WHAT), like "make test"
Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-15 16:14:44 +01:00
Maël Valais
48a31521b3 make: vendor-go wasn't run before e2e-setup
Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-15 15:55:19 +01:00
Maël Valais
9e56f63b0f make: e2e-setup: use eafxx/bind instead of sameersbn/bind
eafxx's container images are multi-arch which means M1 macs can run
bind on Colima.

Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-14 21:25:19 +01:00
Maël Valais
5b3b8a83ef make: e2e: add CGO_ENABLED=0 to make/e2e.sh
This is because we are running these tests in a non-libc container
(alpine) which means we can't use CGO.

Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-14 17:18:51 +01:00
Maël Valais
daf412c09d make: test-ci: xargs --max-procs doesn't exist in the busybox
Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-14 16:47:43 +01:00
Maël Valais
bbf5ab3c77 make: e2e-setup now skips bind and sampleexternalissuer on arm64
Otherwise, the target 'e2e-setup' fails. At least, that gives people a
chance to have some of the end-to-end tests pass. I added a warning to
let people know that they can still force using CRI_ARCH=amd64, which is
possible on Docker Desktop thanks to the QEMU emulation that kicks in
when a container with the arch amd64 tries to execute.

Note that this work around does not work on Colima at the moment.

Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-14 15:34:48 +01:00
Maël Valais
06d5f86e52 make: e2e-setup: projectcontour actually has an arm64 image
Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-14 15:33:19 +01:00
Joakim Ahrlin
63405375b8 fix kyverno_arm64 image format
Signed-off-by: Joakim Ahrlin <joakim.ahrlin@gmail.com>
2022-03-14 11:00:52 +01:00
Maël Valais
9e76ccd981 verify-errexit: add set -e to pass CI
Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-13 12:32:08 +01:00
Maël Valais
8fd19044f2 make: integration-test: KUBEBUILDER_ASSETS needs to be absolute
It seems like Ginkgo isn't running from the root of the repository, that's why
it can't find binaries, e.g., bin/tools/etcd.

Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-13 12:32:08 +01:00
Maël Valais
7003a26baa make: remove 'how to install system tools' instructions such as go or jq
I had written instructions that would show up when you don't have one of
the mandatory system tools such as go or jq. After some feedback, I
decided to remove these instructions since we are concerned that these
instructions may become out of date.

Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-13 12:32:08 +01:00
Maël Valais
3405edf821 make: add the targets 'e2e-setup-kind', 'e2e-setup-kind', and 'e2e'
The commands can be run concurrently, with the exception of e2e that
has to be run after e2e-setup is done. The e2e target does not check
whether cert-manager and the addons are installed.

The two only scripts that were kept are:

- make/e2e.sh      (previously called ./devel/run-e2e.sh)
- make/cluster.sh  (previsouly called ./devel/cluster/create.sh)

The reason for the removal of the other scripts is that they didn't
have that much logic and could easily ported to Make, improving greatly
the execution speed thanks to make's concurrency.

make/e2e.sh now behaves "as expected" when using -ginkgo.focus or
GINKGO_FOCUS; previously, the logs would not be shown before the end
of the test.

make/cluster.sh has lost the ability to create an OpenShift 3.11 cluster.
for running the end-to-end tests. The two reasons are that OpenShift 4
wasn't supported by the script devel/cluster/create.sh, and OpenShift
3.11 is not supported by cert-manager anymore.

The Makefile targets that were used in the Prow jobs (verify, verify_deps,
verify_chart, verify_upgrade, and cluster) have been kept around. They
now show a warning to encourage people to use the new Make-based targets.
When running one of the deprecated targets the Makefile won't check the
presence of the system tools such as Go and jq, since Bazel takes care of
these dependencies.

On version change, downloaded tools and images are re-downloaded. The
command 'make clean' now keeps the downloaded images and tools.

Note that a lot of attention has been put into having a Make system that works
flawlessly both on Linux and on BSDs (such as macOS).

You will note that some recursive calls to make are made, and $(MAKE)
instead of plain "make" is used in that case. If we didn't use $(MAKE),
we would have concurrency issues, and warnings such as:

  make[1]: warning: jobserver unavailable: using -j1. Add `+' to parent make rule.

Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-13 12:32:08 +01:00
Maël Valais
7aaff9fbb7 make: fix manifests.mk data race due to bin/helm/cert-manager/templates/%.yaml
It looked like this:

cp: cannot create regular file 'bin/helm/cert-manager/templates/startupapicheck-psp-clusterrole.yaml': File exists
cp: cannot create regular file 'bin/helm/cert-manager/templates/service.yaml': File exists
cp: cannot create regular file 'bin/helm/cert-manager/templates/webhook-psp.yaml': File exists
cp: cannot create regular file 'bin/helm/cert-manager/templates/rbac.yaml': File exists

Signed-off-by: Maël Valais <mael@vls.dev>
2022-03-13 12:17:24 +01:00
Ashley Davis
216f95f563
add install for controller-gen and other "gen" tools
k8s codegen tools are a bit of a special case; this is explained in a
comment.

Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
2022-03-05 17:18:50 +00:00
Ashley Davis
ccd03ccdd9
Remove outdated, duplicated comment in Makefile
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
2022-03-05 17:05:31 +00:00
Ashley Davis
bc070dfff0
add checks for required binaries in environment
this helps with better error messages for end users

Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
2022-03-05 17:05:22 +00:00
Ashley Davis
e22f30420b
add target for boilerplate verification in CI presubmits
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
2022-03-05 16:37:26 +00:00
Ashley Davis
89d7742c56
add verify-errexit target to ci presubmits
Also modifies the script to run without bazel, and tweaks it so that
it'll detect errors in itself too.

Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
2022-03-05 16:37:26 +00:00
Maël Valais
b150828a5b make: bin/release-version needs to be rebuilt every time
We use FORCE instead of .PHONY because this is a real file that can be
used as a prerequisite. If we were to use .PHONY, then the file's
timestamp would not be used to check whether targets should be rebuilt,
and they would get constantly rebuilt.

Signed-off-by: Maël Valais <mael@vls.dev>
2022-02-25 17:30:48 +01:00
Maël Valais
a85b1abd37 make: work around the multiple copies of each binaries with hard links
Signed-off-by: Maël Valais <mael@vls.dev>
2022-02-25 17:07:44 +01:00
Maël Valais
6e6667acac make: make sure bin/scratch/build-context/* are intermediate
That should make so 'make' removes these intermediate files.

Signed-off-by: Maël Valais <mael@vls.dev>
2022-02-25 17:06:55 +01:00
Maël Valais
2b252b30ad make: explain why we have the "build-context" dir
Signed-off-by: Maël Valais <mael@vls.dev>
2022-02-25 16:38:53 +01:00
Maël Valais
b33c50342c make: fix "release-version" description
Signed-off-by: Maël Valais <mael@vls.dev>
2022-02-25 16:38:53 +01:00
Maël Valais
6e5bf073c2 make: explain why the target "bin/release-version" exists
Signed-off-by: Maël Valais <mael@vls.dev>
2022-02-25 16:38:53 +01:00
Maël Valais
ac16285f2e make: asthetic, fix the spacing between targets
Signed-off-by: Maël Valais <mael@vls.dev>
2022-02-25 16:38:53 +01:00
Maël Valais
c09dbb07e4 make: speed up 'docker build' with separate dir contexts
Previously, we had one .dockerignore that would do its best to only have
the binaries and licenses copied into the Docker (or nerdctl, or
buildah). Unfortunately, that meant it had to copy all of bin/server and
bin/cmctl, which could become quite large (I measured 1.6 GB).

Instead of relying on a single .dockerignore file, we copy the licenses
and binaries into a "scratch context" directory. The downside is that
all the binaries are in two different places (bin/server and
bin/scratch/containers). Note that we can't use symlinks because Docker
won't dereference them.

Signed-off-by: Maël Valais <mael@vls.dev>
2022-02-25 16:38:53 +01:00
Maël Valais
3f7bac6ad3 make: rebuild images when a different commit is checked out
When switching branches, the Go files may not change. But since the
images contain the commit hash, e.g.:

  cert-manager-controller-amd64:v1.7.0-beta.0-142-gfc0819af6

It is surprising when trying to deploy to Kind: the git commit that is
checked out does not match the commit hash of the image.

To avoid confusion, I added bin/release-version that gets updated only
when the currently checked out commit changes.

Signed-off-by: Maël Valais <mael@vls.dev>
2022-02-25 16:38:53 +01:00
Ashley Davis
dadb2aa8a2
log latest known tag when prepping integration tests
Signed-off-by: Ashley Davis <ashley.davis@jetstack.io>
2022-02-18 16:43:37 +00:00