On Mon, Jun 29, 2020 at 08:51:51PM +0200, Andrea Bolognani wrote:
Instead of having static job definitions for FreeBSD and macOS, use a generic template for both and fill in the details that are actually different, such as the list of packages to install, in the GitLab CI job, right before calling cirrus-run.
The target-specific information are provided by lcitool, so that keeping them up to date is just a matter of running the refresh script when necessary.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- .gitlab-ci.yml | 36 ++++++++++++++- ci/cirrus/build.yml | 26 +++++++++++ ci/cirrus/freebsd-12.yml.j2 | 73 ------------------------------- ci/cirrus/libvirt-freebsd-12.vars | 7 +++ ci/cirrus/libvirt-macos-1015.vars | 7 +++ ci/cirrus/macos-1015.yml.j2 | 38 ---------------- ci/cirrus/refresh | 22 ++++++++++ 7 files changed, 97 insertions(+), 112 deletions(-) create mode 100644 ci/cirrus/build.yml delete mode 100644 ci/cirrus/freebsd-12.yml.j2 create mode 100644 ci/cirrus/libvirt-freebsd-12.vars create mode 100644 ci/cirrus/libvirt-macos-1015.vars delete mode 100644 ci/cirrus/macos-1015.yml.j2 create mode 100755 ci/cirrus/refresh
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index e6eb2f9905..5565750b7e 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -58,11 +58,35 @@ stages: # Jobs that we delegate to Cirrus CI because they require an operating # system other than Linux. These jobs will only run if the required # setup has been performed on the GitLab account (see ci/README.rst). +# +# The Cirrus CI configuration is generated by replacing target-specific +# variables in a generic template: some of these variables are provided +# when the GitLab CI job is defined, others are taken from a shell +# snippet generated using lcitool. +# +# Note that the $PATH environment variable has to be treated with +# special care, because we can't just override it at the GitLab CI job +# definition level or we risk breaking it completely.
Informative, thanks :). I just wish I didn't have to google what semantics the various types of parameter expansion in Bash had (obviously not your fault).
.cirrus_build_job_template: &cirrus_build_job_definition stage: builds image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master script: - - cirrus-run ci/cirrus/$NAME.yml.j2 + - source ci/cirrus/libvirt-$NAME.vars + - sed -e "s|[@]CI_REPOSITORY_URL@|$CI_REPOSITORY_URL|g" + -e "s|[@]CI_COMMIT_REF_NAME@|$CI_COMMIT_REF_NAME|g" + -e "s|[@]CI_COMMIT_SHA@|$CI_COMMIT_SHA|g" + -e "s|[@]CIRRUS_VM_INSTANCE_TYPE@|$CIRRUS_VM_INSTANCE_TYPE|g" + -e "s|[@]CIRRUS_VM_IMAGE_SELECTOR@|$CIRRUS_VM_IMAGE_SELECTOR|g" + -e "s|[@]CIRRUS_VM_IMAGE_NAME@|$CIRRUS_VM_IMAGE_NAME|g" + -e "s|[@]INSTALL_COMMAND@|$INSTALL_COMMAND|g" + -e "s|[@]PATH@|$PATH_EXTRA${PATH_EXTRA:+:}\$PATH|g" + -e "s|[@]PKG_CONFIG_PATH@|$PKG_CONFIG_PATH|g" + -e "s|[@]PKGS@|$PKGS|g" + -e "s|[@]MAKE@|$MAKE|g" + -e "s|[@]PYTHON@|$PYTHON|g" + <ci/cirrus/build.yml >ci/cirrus/$NAME.yml + - cat ci/cirrus/$NAME.yml
was ^this 'cat' just for debugging purposes or was that intended to be part of the job output in production? ...
diff --git a/ci/cirrus/refresh b/ci/cirrus/refresh new file mode 100755 index 0000000000..b84910a645 --- /dev/null +++ b/ci/cirrus/refresh @@ -0,0 +1,22 @@ +#!/bin/sh + +if test -z "$1" +then + echo "syntax: $0 PATH-TO-LCITOOL" + exit 1 +fi + +LCITOOL=$1 + +if ! test -x "$LCITOOL" +then + echo "$LCITOOL is not executable" + exit 1 +fi + +HOSTS=$($LCITOOL hosts | grep -E 'freebsd-12|macos') + +for host in $HOSTS +do + $LCITOOL dockerfile "$host" libvirt --variables >"$host.vars" +done
Overall, I prefer this to v1 because of the further level of abstraction this introduces. However, the final format we're producing here is a YAML template rather than a Dockerfile, so introducing just an option for the "dockerfile" subcommand rather than a dedicated "cirrus-ci-file" (or similar) subcommand doesn't feel completely right, especially from the long run perspective. At some point we'd like to generate the whole gitlab-ci.yml file, so that one would very likely get a dedicated subcommand as well - it should, as it's just another "plugin" generator. If we expand beyond libvirt, say QEMU, we may need to add a different CI provider plugin as well. Regards, Erik