On Wed, Jun 13, 2018 at 02:58:55PM +0200, Andrea Bolognani wrote:
On Wed, 2018-06-13 at 11:32 +0100, Daniel P. Berrangé wrote:
On Tue, Jun 12, 2018 at 12:12:12PM +0200, Andrea Bolognani wrote:
The pre-built images have been hand-crafted using the build dependencies recorded in the libvirt-jenkins-ci repository: of course that's not something that we want to keep doing manually going forward, so figuring out a sensible way to generate Dockerfiles and potentially even Docker images automatically is pretty high on the priority list.
When first testing I produced a custom Ubuntu docker image with not much effort. I was just creating a file in "libvirt-jenkins-ci" repo called "images/ubuntu-18.04.docker" that contains
FROM ubuntu:18.04 RUN apt-get update ENV PACKAGES \ ::PACKAGE-LIST:: \ RUN apt-get -y install $PACKAGES
Pretty much exactly how I've created the images you can find on Docker Hub, except for
RUN mkdir /build WORKDIR /build
this bit, which AFAICT is entirely unnecessary.
WORKDIR /build, avoids the need for the '-w /build' arg in the .travis.yml docker run command. I think there's a slight plus to having the workdir set automatically, avoiding the need for a -w arg.
::PACKAGE-LIST:: can be built by reading the guests/vars/projects/libvirt.yml file, and then expanding it based on guest/vars/mappings.yml
I hadn't written code for that bit, but it just needs a short python script to read the two yaml files and map the data sets.
Yeah, same here: I just extracted the package list from the output of a 'lcitool update' run this time around, but I was planning on writing a tool to do that for me just like you mention.
The one thing I haven't quite figured out yet is where to store the resulting Dockerfiles. If we committed them to some repository we could take advantage of Docker Hub's autobuild support, which would be pretty neat; on the other hand, being generated content, they have no business being committed, plus it would be tricky to ensure the generated files are always in sync with the source mappings without introducing a bunch of scaffoling to the libvirt-jenkins-ci repository.
I think we should just have the dockerfile templates (ie with the ::PACKAGE:: placeholder) in the libvirt-jenkins-ci repo. We don't need to store the expanded dockerfile. Then we can have a CI job somewhere that automatically rebuilds the & uploads new docker images whenever a change is pushed to libvirt-jenkins-ci.
I was only going to do packages forthe libvirt.yml, but we can expand to cover the other modules too quite easily, as its just taking the union of all the project files.
I don't think we can/want to do that.
The way build dependencies for projects are set up right now, we expect to build eg. libvirt-glib against our own copy of libvirt rather than the distro-provided one, so libvirt is *not* included among the packages required by the libvirt-glib project.
So if we included build dependencies for all projects in the Docker images, that would make them way bigger and you would still be unable to build libvirt-glib or whatever on top of them. Perhaps we can find some way out of this later, but I'd rather move one step at the time instead of trying to solve all the things in one fell swoop :)
Yeah, lets only focus on core libvirt right now until we have an immediate need for more.
ie rather than
Name: libvirt/build Tag: ubuntu-18.04
we should have
Name: libvirt/ubuntu Tag: 18.04
I don't mind having several images and using the tag only for the version number, if that's something that will make the result look less alien to Docker users; however, I think we should keep the names consistent with what we use on our CentOS CI, so it would be ubuntu:18 instead of ubuntu:18.04.
NB that is ambiguous as Ubuntu does two releases a year, 18.04 and 18.10
Though perhaps make clear it is for CI, so
Name: libvirt/ci-ubuntu Tag: 18.04
Just like for the guests you can create and manage with lcitool, while these images will be primarily used for Travis CI they should be usable for developers as well, which is why I picked the name "build" instead of "ci" or "travis-ci" in the first place.
Sure, 'libvirt/build-' is fine
At the same time, I didn't use just the distribution name because I didn't want to give the impression that by pulling them you would get an OS with libvirt already installed.
Agreed. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|