[libvirt-ci PATCH] guests: introduce libvirt-dist and libvirt-minimal projects
The language bindings have traditionally built against a full libvirt, however, this is serious overkill because all they should really need most of the time is access to the main API, and the test driver if they want to run functional tests. The full libvirt build with virt drivers is only needed for integration testing which most bindings won't do. Thus this introduces a new project "libvirt-minimal" which lists the bare minimum dependencies required to build libvirt. Taking libvirt-perl as an example, if creating a container using the current "libvirt,libvirt-perl" project set, the result with 1.4 GB in size for Fedora 31. With the "libvirt-minimal,libvirt-perl" set, the size is just 777 MB. Some projects also wish to have the ability to build against the distro provided libvirt instead of the latest git master, and for this purpose a "libvirt-dist" project is defined which pulls in the package needed for building against the distro libvirt. In the lcitool dockerfile command, there is a check that the requested project is configured against the requested host. These new projects are not listed against any host, because we don't want them installed by default in the VMs. Thus the dockerfile check is no longer valid and is removed. Whomever is invoking lcitool knows what combination they want and whether it will work. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- guests/lcitool | 7 ------- guests/vars/mappings.yml | 5 +++++ guests/vars/projects/libvirt-dist.yml | 3 +++ guests/vars/projects/libvirt-minimal.yml | 13 +++++++++++++ 4 files changed, 21 insertions(+), 7 deletions(-) create mode 100644 guests/vars/projects/libvirt-dist.yml create mode 100644 guests/vars/projects/libvirt-minimal.yml diff --git a/guests/lcitool b/guests/lcitool index 0c89e13..abc87d2 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -1007,13 +1007,6 @@ class Application: for project in projects: if project.rfind("+mingw") >= 0: raise Exception("Obsolete syntax, please use --cross-arch") - if project not in facts["projects"]: - raise Exception( - "Host {} doesn't support project {}".format( - host, - project, - ) - ) varmap = self._dockerfile_build_varmap(facts, mappings, pip_mappings, projects, cross_arch) self._dockerfile_format(facts, cross_arch, varmap) diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index 753f0fe..a9ab3d5 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -450,6 +450,11 @@ mappings: rpm: libuuid-devel cross-policy-deb: foreign + libvirt-devel: + rpm: libvirt-devel + deb: libvirt-dev + pkg: libvirt + libxml2: deb: libxml2-dev pkg: libxml2 diff --git a/guests/vars/projects/libvirt-dist.yml b/guests/vars/projects/libvirt-dist.yml new file mode 100644 index 0000000..aa95f23 --- /dev/null +++ b/guests/vars/projects/libvirt-dist.yml @@ -0,0 +1,3 @@ +--- +packages: + - libvirt-devel diff --git a/guests/vars/projects/libvirt-minimal.yml b/guests/vars/projects/libvirt-minimal.yml new file mode 100644 index 0000000..9402c4f --- /dev/null +++ b/guests/vars/projects/libvirt-minimal.yml @@ -0,0 +1,13 @@ +--- +packages: + - glib2 + - gnutls + - libnl3 + - libnlroute3 + - libpcap + - libtirpc + - libxml2 + - python3-docutils + - rpcgen + - xmllint + - xsltproc -- 2.25.4
On Wed, 2020-04-29 at 11:38 +0100, Daniel P. Berrangé wrote:
Thus this introduces a new project "libvirt-minimal" which lists the bare minimum dependencies required to build libvirt.
I don't like the name libvirt-minimal because it appears to follows the convention used for actual libvirt-based projects such as libvirt-glib. What about libvirt+minimal, following the project+variant convention used for all MinGW builds[1] and in the past for libvirt+website?
Some projects also wish to have the ability to build against the distro provided libvirt instead of the latest git master, and for this purpose a "libvirt-dist" project is defined which pulls in the package needed for building against the distro libvirt.
I don't like this name either :) One reason is the same detailed above, but there's another one which is tied to semantics: in lcitool, "$project" has always been used to mean "the list of packages necessary to build $project from source". This is the case for all the projects that exist today[2] and even for libvirt+minimal, but this one is different: in this case, libvirt-dist means "the list of packages necessary to build a project that *depends* on libvirt from source". That's quite different. I think a better approach would be to once again use variants: for example, if you wanted to build libvirt-python against the version of libvirt included in the OS, instead of having something like # libvirt-python.yml --- packages: - python3 ... # libvirt-dist.yml --- packages: - libvirt which is then used like $ lcitool dockerfile $OS libvirt-dist,libvirt-python you'd have # libvirt-python+dist.yml --- packages: - libvirt - python-3 ... which is used like $ lcitool dockerfile $OS libvirt-python+dist This would achieve the same result with less typing and without subverting the existing semantics. Another nice property of this approach is that it naturally extends to cover projects that depend on more than just libvirt: for example, we could have # libvirt-dbus+dist.yml --- packages: - libvirt - libvirt-glib ... for use in libvirt-dbus CI. What do you think? [...]
+++ b/guests/vars/mappings.yml + libvirt-devel: + rpm: libvirt-devel + deb: libvirt-dev + pkg: libvirt
The mapping should be called just "libvirt". [1] Applying this convention to MinGW builds is increasingly inaccurate now that we have proper multi-arch support, and in fact at least at the Dockerfile generator level we've gotten away with this convention; eradicating it completely is something that I hope I'll be able to spend time on in the near future. [2] Except for base, blacklist and jenkins, but those are internal projects which are never exposed to the user so they don't need to follow the rules as strictly :) -- Andrea Bolognani / Red Hat / Virtualization
On Wed, Apr 29, 2020 at 08:52:06PM +0200, Andrea Bolognani wrote:
On Wed, 2020-04-29 at 11:38 +0100, Daniel P. Berrangé wrote:
Thus this introduces a new project "libvirt-minimal" which lists the bare minimum dependencies required to build libvirt.
I don't like the name libvirt-minimal because it appears to follows the convention used for actual libvirt-based projects such as libvirt-glib.
What about libvirt+minimal, following the project+variant convention used for all MinGW builds[1] and in the past for libvirt+website?
Some projects also wish to have the ability to build against the distro provided libvirt instead of the latest git master, and for this purpose a "libvirt-dist" project is defined which pulls in the package needed for building against the distro libvirt.
I don't like this name either :)
One reason is the same detailed above, but there's another one which is tied to semantics: in lcitool, "$project" has always been used to mean "the list of packages necessary to build $project from source".
This is the case for all the projects that exist today[2] and even for libvirt+minimal, but this one is different: in this case, libvirt-dist means "the list of packages necessary to build a project that *depends* on libvirt from source". That's quite different.
I think a better approach would be to once again use variants: for example, if you wanted to build libvirt-python against the version of libvirt included in the OS, instead of having something like
# libvirt-python.yml --- packages: - python3 ...
# libvirt-dist.yml --- packages: - libvirt
which is then used like
$ lcitool dockerfile $OS libvirt-dist,libvirt-python
you'd have
# libvirt-python+dist.yml --- packages: - libvirt - python-3 ...
which is used like
$ lcitool dockerfile $OS libvirt-python+dist
This would achieve the same result with less typing and without subverting the existing semantics.
This results in defining the combinatorial expansion of project sets which just looks like unecessary duplication & work to me. It also gives different syntax for configuring a container to build from git vs dist. There is only ever one project here "libvirt-project" and nothing about it is is changing, except for which "libvirt" it is being built against. It supports any libvirt, whether a full git build or a minimal git build, or a distro build or some other build: $ lcitool dockerfile $OS libvirt,libvirt-python $ lcitool dockerfile $OS libvirt-dist,libvirt-python $ lcitool dockerfile $OS libvirt-minimal,libvirt-python We could call it "libvirt+dist" instead "libvirt-dist" if we want
Another nice property of this approach is that it naturally extends to cover projects that depend on more than just libvirt: for example, we could have
# libvirt-dbus+dist.yml --- packages: - libvirt - libvirt-glib ...
for use in libvirt-dbus CI.
I think that's a downside not a plus, because it makes the combinatorial expansion even bigger. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Thu, 2020-04-30 at 12:17 +0100, Daniel P. Berrangé wrote:
On Wed, Apr 29, 2020 at 08:52:06PM +0200, Andrea Bolognani wrote:
# libvirt-python+dist.yml --- packages: - libvirt - python-3 ...
which is used like
$ lcitool dockerfile $OS libvirt-python+dist
This would achieve the same result with less typing and without subverting the existing semantics.
This results in defining the combinatorial expansion of project sets which just looks like unecessary duplication & work to me. It also gives different syntax for configuring a container to build from git vs dist.
There is only ever one project here "libvirt-project" and nothing about it is is changing, except for which "libvirt" it is being built against. It supports any libvirt, whether a full git build or a minimal git build, or a distro build or some other build:
$ lcitool dockerfile $OS libvirt,libvirt-python $ lcitool dockerfile $OS libvirt-dist,libvirt-python $ lcitool dockerfile $OS libvirt-minimal,libvirt-python
Okay, this is a pretty solid argument. It also opens up the possibility of some external project, that uses libvirt but is not known to lcitool, using $ lcitool dockerfile $OS libvirt+dist to generate a base container for their CI use, which coud be useful. The semantics of $project and $project+dist are still different enough that we should document them properly in lcitool's README.
We could call it "libvirt+dist" instead "libvirt-dist" if we want
Yes please. -- Andrea Bolognani / Red Hat / Virtualization
participants (2)
-
Andrea Bolognani -
Daniel P. Berrangé