Re: [libvirt PATCH 0/5] ci: Use GitLab container registry
On Mon, Jun 01, 2020 at 04:51:19PM +0200, Pavel Hrdina wrote:
On Fri, May 29, 2020 at 03:00:39PM +0200, Andrea Bolognani wrote:
> Branch: https://gitlab.com/abologna/libvirt/-/tree/ci-full-gitlab-registry
> Pipeline: https://gitlab.com/abologna/libvirt/pipelines/150891361
>
> This is what we're already doing with the subprojects we've migrated
> to GitLab CI and, as of earlier today, all projects under the
> libosinfo umbrella.
>
> Once this is merged, we can stop publishing container images on Quay
> and archive the libvirt-dockerfiles repository.
>
> Patch 3/5 has been trimmed in order to comply with the size limits
> of the mailing list. You can grab the unabridged version with
>
> $ git fetch https://gitlab.com/abologna/libvirt ci-full-gitlab-registry
This is a lot of files and lines of text/code. I was wondering about
building the dockerfiles as part of the container_job_definition.
To me it seems like a lot of duplication and a lot of noise in the
future if we decide to change the dockerfiles generation. The main
difference that I can think of is that with files in repository we
need to regenerate all the dockerfiles to apply changes made in
libvirt-ci but with the automatic generation we would have that for
free.
The key reason for keeping the dockerfiles in the libvirt.git repo and
NOT auto-generating them on the fly is to ensure the CI process is
self-contained, with no dependancy on external moving parts in other
git repos.
If you automatically generated dockerfiles from libvirt-ci.git, then
you end up with unstable CI when changes in libvirt.git need to be
made in lock-step with changes in libvirt-ci.git. If you change
libvirt.git first, CI will break if it runs before libvirt-ci.git
is updated. If you change libvirt-ci.git first, then CI will break
if it runs before libvirt.git is updated. This is a no-win situation.
This is especially painful when you consider that a user's fork of
libvirt.git may not updated to current master. Or consider a user
who needs to make changes to libvirt.git that require updated
dockerfiles and needs to be able to test them before any change
in libvirt-ci.git is present.
We've seen these problems many times with our current Jenkins setup
where CI breaks for a period when we have to do matching updates
between both libvirt-ci.git & libvirt.git.
With dockerfiles kept in libvirt.git we know that the containers
we're building will always contain exactly what we need. This also
makes it easy for users to experiment with changes, as they can
modify the dockerfiles directly to add/remove pieces. Such changes
can be propagated back to libvirt-ci.git out of band.
Both approaches have some benefits and drawbacks and I guess we
could
have some variable to prevent automatic generation of dockerfiles to
make sure that unwanted changes in libvirt-ci doesn't affect CI for
all libvirt repositories, on the other hand it would automatically
check that changes to libvirt-ci doesn't break anything.
Changes to libvirt-ci that affect the dockerfiles, should come with
a URL pointer to a merge request against the affected project(s),
showing the succesful CI run with updated dockerfiles.
I personally don't like the need to introduce 5000+ lines just
for
compilation testing.
That's a tiny proportion of the code we have in libvirt.git, so IMHO
it is not worth worrying about. The benefits of having the CI self
contained far outweigh the downside.
Essentially we are prioritizing the main libvirt.git as that is the
primary content from the POV of most contributors.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|