10:19 a.m.
On Mon, Feb 06, 2023 at 03:45:12PM +0000, Daniel P. Berrangé wrote:
On Mon, Feb 06, 2023 at 02:52:57PM +0100, Erik Skultety wrote:
> This is a follow up to:
> https://listman.redhat.com/archives/libvir-list/2023-January/237201.html
>
> The effort here is to unify the way builds/tests are executed in GitLab CI vs
> local container executions and make another step forward in terms of
> reproducibility of (specifically) GitLab environments.
>
> Even though code to run all but one (coverity) jobs from GitLab via the
> build.sh script is added with this series, local behavior remains the same as
> before this series. The reason for that is that that will require more patches
> ridding of the Makefile which is currently used and instead integrate usage of
> lcitool with the ci/helper Python script which is currently the entry point for
> local container executions.
snip
> .gitlab-ci.yml | 56 ++++++++++-------------
> ci/Makefile | 16 ++++---
> ci/build.sh | 121 +++++++++++++++++++++++++++++++++++++++++++------
> ci/helper | 21 ++++++---
> 4 files changed, 155 insertions(+), 59 deletions(-)
> mode change 100644 => 100755 ci/build.sh
I'm really super unenthusiastic about this change, and also the similar
change to add an ci/integration.sh. Shell scripting is something we
worked hard to eliminate from libvirt. It is an awful language
to do anything non-trivial with, error handling, lack of data
structures, variable handling, portability are all bug generators.
I know the .gitlab-ci.yml 'script' commands are technically shell
script, but they are pretty trivial bits and don't have to worry
about portability for dash vs bash etc, or complex control logic.
The majority of it is just a simple list of commands to invoke,
with the occasional conditional.
The build.sh script is by contrast significantly more complex. By
the very nature of taking "N" separate gitlab jobs and multiplexing
them onto the one shell script, we're now having todo command line
arg parsing in shell and de-multiplexing back to commands. The CI
logic is now split up across even more sources - the gitlab config,
the build.sh script and the meson.build files, which I think is
worse for maintaining this too.
True, except that even despite that, and I know what I'm talking about since I
wrote the integration jobs templates, GitLab is super picky; you can't debug
the syntax problems properly; certain common syntactic sugars don't work and
have to be replaced by less common counterparts; using YAML for bash formatting
leads to many table-flipping moments; you have to wait for Nx10 minutes before
it fails with some ridiculous error you would have caught early locally but you
couldn't because of the stage and artifact dependencies. So, once you have it
finally in place it's unarguably nice, but the road towards that isn't really
rainbow and unicorns either, it's still shell after all. Since I'm mainly
working on this part of libvirt, I myself would appreciate if I could easily
just run a single script instead of copy pasting commands one-by-one to a local
VM to catch stupid errors quickly rather than wait for GitLab to fail.
I appreciate the goal of being able to run CI commands locally, but
I'm not feeling this approach is a net win. I'd much rather just
having developers invoke the meson command directly, which is not
that hard.
Well, the big picture here is about making running integration tests locally
less painful than it is now...plus increase the amount of automation involved.
That's the major driver here - libvirt needs to be built locally in a safe
environment before integration tests could be run against it without touching
the host. So, with this script, while I agree with everything you said about
Bash, was just one step towards getting the automation I mentioned in place. I
also considered Python (yeah..why), but that would be super akward.
TL;DR: if we don't do anything about how we currently maintain the build
recipes (note we're maintaining 2 already), everybody will continue ignoring the
integration test suite, unless we enable merge requests where the status quo
would be somewhat (but not completely) eliminated. With integration tests you
can't ignore the aspect of getting feedback early compared to waiting for
GitLab CI.
If we do really want to provide something that 100% mirrors the
gitlab CI job commands though, I'm even more convinced now that
we should be using the .gitlab-ci.yml as the canonical source.
Since I last mentioned that idea, I found out that this should
be something we can achieve via the gitlab runner 'exec' command.
Haven't heard of ^this one, but I wonder how we can get something meaningful
out of it for the purposes I mentioned above.
Erik
I've not quite figured out the right incantations though. I can
get it to work for a simple repo, but not for the lbivirt.git
yet as it seems to not find the included yml files.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|