Re: [libvirt PATCH 2/3] cirrus: Add templates and refresh script
On Mon, Jun 29, 2020 at 03:32:22PM +0100, Daniel P. Berrangé wrote:
On Mon, Jun 29, 2020 at 03:58:43PM +0200, Andrea Bolognani wrote:
> This is similar to what we already use for Dockerfiles, with one
> key difference: while we still rely on lcitool taking care of the
> complicated work for us, in this case we're only provided with a
> bunch of variables and we have to do the last bit of work (that
> is replacing them inside an existing template) ourselves.
>
> Signed-off-by: Andrea Bolognani <abologna(a)redhat.com>
> ---
> ci/cirrus/refresh | 32 ++++++++++++++++++++++++++++++
> ci/cirrus/templates/freebsd-12.yml | 25 +++++++++++++++++++++++
> ci/cirrus/templates/macos-1015.yml | 27 +++++++++++++++++++++++++
These files are largely the identical content to the same name
files in the dir above which feels like it is easy to optimize
away.
> 3 files changed, 84 insertions(+)
> create mode 100755 ci/cirrus/refresh
> create mode 100644 ci/cirrus/templates/freebsd-12.yml
> create mode 100644 ci/cirrus/templates/macos-1015.yml
>
> diff --git a/ci/cirrus/refresh b/ci/cirrus/refresh
> new file mode 100755
> index 0000000000..51deca94ac
> --- /dev/null
> +++ b/ci/cirrus/refresh
> @@ -0,0 +1,32 @@
> +#!/bin/sh
> +
> +if test -z "$1"
> +then
> + echo "syntax: $0 PATH-TO-LCITOOL"
> + exit 1
> +fi
> +
> +LCITOOL=$1
> +
> +if ! test -x "$LCITOOL"
> +then
> + echo "$LCITOOL is not executable"
> + exit 1
> +fi
> +
> +for infile in templates/*
> +do
> + outfile="${infile##*/}.j2"
> + host="${outfile%%.*}"
> +
> + eval $("$LCITOOL" dockerfile "libvirt-$host" libvirt
--variables)
> +
> + sed -e "s|[@]PKGS@|$PKGS|g" \
> + -e "s|[@]CROSS_PKGS@|$CROSS_PKGS|g" \
> + -e "s|[@]PYPI_PKGS@|$PYPI_PKGS|g" \
> + -e "s|[@]CPAN_PKGS@|$CPAN_PKGS|g" \
> + -e "s|[@]MAKE@|$MAKE|g" \
> + -e "s|[@]NINJA@|$NINJA|g" \
> + -e "s|[@]PYTHON@|$PYTHON|g" \
> + <"$infile" >"$outfile"
> +done
I feel like this should really be reduced to just:
$LCITOOL cirrusci libvirt-$host libvirt > $outfile
where the 'cirrusci' command generates this content....
I agree with the idea of having a new command generating the content below, I
tend to look at this as different generator plugins/backends that we support,
so having a dedicated command makes much more sense to me.
> diff --git a/ci/cirrus/templates/freebsd-12.yml
b/ci/cirrus/templates/freebsd-12.yml
> new file mode 100644
> index 0000000000..228ea67827
> --- /dev/null
> +++ b/ci/cirrus/templates/freebsd-12.yml
> @@ -0,0 +1,25 @@
> +freebsd_instance:
> + image_family: freebsd-12-1
> +
> +env:
> + CI_REPOSITORY_URL: {{ CI_REPOSITORY_URL }}
> + CI_COMMIT_REF_NAME: {{ CI_COMMIT_REF_NAME }}
> + CI_COMMIT_SHA: {{ CI_COMMIT_SHA }}
> + PKGS: @PKGS@
> + MAKE: @MAKE@
> + PYTHON: @PYTHON@
> +
> +freebsd_12_task:
> + install_script:
> + - pkg install -y $PKGS
> + clone_script:
> + - git clone --depth 100 "$CI_REPOSITORY_URL" .
> + - git fetch origin "$CI_COMMIT_REF_NAME"
> + - git reset --hard "$CI_COMMIT_SHA"
...down to this point.
> + build_script:
> + - mkdir build
> + - cd build
> + - ../autogen.sh --prefix=$(pwd)/install-root
> + - $MAKE -j3
> + - $MAKE -j3 install
> + - $MAKE -j3 dist
This part can then be stored in ci/cirrus/build.yml since it is
common to freebsd & macos.
So now in gitlab-ci.yml we can just concatenate the two into a
temp file:
cat ci/cirrus/$NAME.yml ci/cirrus/build.yml > ci/cirrus/$NAME.yml.j2
cirrus-run ci/cirrus/$NAME.yml.j2
How about instead of storing the same build.yml in every secondary project's ci
directory we store the build.yml in libvirt-ci repo and generate the whole
thing with lcitool? I think the way the build YAML would look like for every
single subproject, we could achieve the level of similarity we have for lcitool
native builds - IOW the build job abstraction we have across projects under
guests/playbooks/build/projects could also be achieved here with cirrus, just
an idea.
Erik