[libvirt] [jenkins-ci PATCH] Switch libvirt-dbus builds to Meson
libvirt-dbus has switched its build system to meson and requires 0.49.0 or newer. It means, let's *not* build libvirt-dbus on: - Debian 9 (or older); - Ubuntu 18 (or older); - CentOS 7 Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- .../playbooks/build/projects/libvirt-dbus.yml | 37 +++---------------- guests/vars/projects/libvirt-dbus.yml | 1 + jenkins/projects/libvirt-dbus.yaml | 30 +++------------ 3 files changed, 12 insertions(+), 56 deletions(-) diff --git a/guests/playbooks/build/projects/libvirt-dbus.yml b/guests/playbooks/build/projects/libvirt-dbus.yml index a6ff025..a031ba7 100644 --- a/guests/playbooks/build/projects/libvirt-dbus.yml +++ b/guests/playbooks/build/projects/libvirt-dbus.yml @@ -1,9 +1,9 @@ --- - set_fact: name: libvirt-dbus + # libvirt-dbus depends on meson 0.49.0 which is not available on + # CentOS 7, Debian 9, Ubuntu 18; machines: - - libvirt-centos-7 - - libvirt-debian-9 - libvirt-debian-10 - libvirt-debian-sid - libvirt-fedora-29 @@ -12,42 +12,15 @@ - libvirt-freebsd-11 - libvirt-freebsd-12 - libvirt-freebsd-current - - libvirt-ubuntu-16 - - libvirt-ubuntu-18 archive_format: xz git_url: '{{ git_urls["libvirt-dbus"][git_remote] }}' - include: '{{ playbook_base }}/jobs/prepare.yml' -- include: '{{ playbook_base }}/jobs/autotools-build-job.yml' -- include: '{{ playbook_base }}/jobs/autotools-syntax-check-job.yml' +- include: '{{ playbook_base }}/jobs/meson-build-job.yml' +- include: '{{ playbook_base }}/jobs/meson-check-job.yml' +- include: '{{ playbook_base }}/jobs/meson-rpm-job.yml' vars: - # CentOS 7 doesn't include Python 3, while the versions of flake8 - # and pyflakes currently available on FreeBSD (3.5.0 and 2.0.0 - # respectively) are not compatible machines: - - libvirt-debian-9 - - libvirt-debian-10 - - libvirt-debian-sid - libvirt-fedora-29 - libvirt-fedora-30 - libvirt-fedora-rawhide - - libvirt-ubuntu-16 - - libvirt-ubuntu-18 -- include: '{{ playbook_base }}/jobs/autotools-check-job.yml' - vars: - # CentOS 7 doesn't include Python 3 and the version in Ubuntu - # 16.04 is too old - machines: - - libvirt-debian-9 - - libvirt-debian-10 - - libvirt-debian-sid - - libvirt-fedora-29 - - libvirt-fedora-30 - - libvirt-fedora-rawhide - - libvirt-freebsd-11 - - libvirt-freebsd-12 - - libvirt-freebsd-current - - libvirt-ubuntu-18 -- include: '{{ playbook_base }}/jobs/autotools-rpm-job.yml' - vars: - machines: '{{ rpm_machines }}' diff --git a/guests/vars/projects/libvirt-dbus.yml b/guests/vars/projects/libvirt-dbus.yml index ce902cb..406ad86 100644 --- a/guests/vars/projects/libvirt-dbus.yml +++ b/guests/vars/projects/libvirt-dbus.yml @@ -3,6 +3,7 @@ packages: - dbus-daemon - flake8 - glib2 + - meson - python3 - python3-dbus - python3-gi diff --git a/jenkins/projects/libvirt-dbus.yaml b/jenkins/projects/libvirt-dbus.yaml index 2b72fb4..82eb892 100644 --- a/jenkins/projects/libvirt-dbus.yaml +++ b/jenkins/projects/libvirt-dbus.yaml @@ -1,9 +1,9 @@ --- - project: name: libvirt-dbus + # libvirt-dbus requires meson 0.49.0, which is not available on + # CentOS 7 and Debian 9; machines: - - libvirt-centos-7 - - libvirt-debian-9 - libvirt-debian-10 - libvirt-fedora-29 - libvirt-fedora-30 @@ -14,31 +14,13 @@ archive_format: xz git_url: '{git_urls[libvirt-dbus][default]}' jobs: - - autotools-build-job: + - meson-build-job: parent_jobs: 'libvirt-glib-build' - - autotools-syntax-check-job: + - meson-check-job: parent_jobs: 'libvirt-dbus-build' - # CentOS 7 doesn't include Python 3, while the versions of flake8 - # and pyflakes currently available on FreeBSD (3.5.0 and 2.0.0 - # respectively) are not compatible - machines: - - libvirt-debian-9 - - libvirt-debian-10 - - libvirt-fedora-29 - - libvirt-fedora-30 - - libvirt-fedora-rawhide - - autotools-check-job: - parent_jobs: 'libvirt-dbus-syntax-check' - # CentOS 7 doesn't include Python 3 and the version in Ubuntu - # 16.04 is too old + - meson-rpm-job: + parent_jobs: 'libvirt-dbus-check' machines: - - libvirt-debian-9 - - libvirt-debian-10 - libvirt-fedora-29 - libvirt-fedora-30 - libvirt-fedora-rawhide - - libvirt-freebsd-11 - - libvirt-freebsd-12 - - autotools-rpm-job: - parent_jobs: 'libvirt-dbus-check' - machines: '{rpm_machines}' -- 2.23.0
On Mon, 2019-09-23 at 13:23 +0200, Fabiano Fidêncio wrote:
name: libvirt-dbus + # libvirt-dbus depends on meson 0.49.0 which is not available on + # CentOS 7, Debian 9, Ubuntu 18; machines: - - libvirt-centos-7 - - libvirt-debian-9 - libvirt-debian-10 - libvirt-debian-sid - libvirt-fedora-29 @@ -12,42 +12,15 @@ - libvirt-freebsd-11 - libvirt-freebsd-12 - libvirt-freebsd-current - - libvirt-ubuntu-16 - - libvirt-ubuntu-18
Based on the discussion we had on libvir-list, our approach for CI should not be to drop target platforms on the basis that they doen't include a recent enough Meson, but rather to make a suitable Meson version available through other means such as pip. I would even go as far as arguing that implementing such a solution in libvirt-jenkins-ci is a pre-requisite, if not for merging the libvirt-dbus patches making the switch to Meson, certainly for cutting a release that includes them. The general understanding is that we only test what we claim to support and only claim to support what we test. There are some cases where we're not following this mantra to the letter, and efforts are already underway to address the most obvious ones, but that's no excuse to make the situation worse, especially considering that in the coming months we expect to move more and more projects to Meson. As a positive side-effect of figuring this out, we should be able to start building osinfo-db-tools again on platforms such as Ubuntu 18.04, dropping which I already though was problematic at the time. tl;dr NACK until we figure out how to do this without dropping several target platforms in the process -- Andrea Bolognani / Red Hat / Virtualization
On Mon, Sep 23, 2019 at 02:33:43PM +0200, Andrea Bolognani wrote:
On Mon, 2019-09-23 at 13:23 +0200, Fabiano Fidêncio wrote:
name: libvirt-dbus + # libvirt-dbus depends on meson 0.49.0 which is not available on + # CentOS 7, Debian 9, Ubuntu 18; machines: - - libvirt-centos-7 - - libvirt-debian-9 - libvirt-debian-10 - libvirt-debian-sid - libvirt-fedora-29 @@ -12,42 +12,15 @@ - libvirt-freebsd-11 - libvirt-freebsd-12 - libvirt-freebsd-current - - libvirt-ubuntu-16 - - libvirt-ubuntu-18
Based on the discussion we had on libvir-list, our approach for CI should not be to drop target platforms on the basis that they doen't include a recent enough Meson, but rather to make a suitable Meson version available through other means such as pip.
I would even go as far as arguing that implementing such a solution in libvirt-jenkins-ci is a pre-requisite, if not for merging the libvirt-dbus patches making the switch to Meson, certainly for cutting a release that includes them.
The general understanding is that we only test what we claim to support and only claim to support what we test. There are some cases where we're not following this mantra to the letter, and efforts are already underway to address the most obvious ones, but that's no excuse to make the situation worse, especially considering that in the coming months we expect to move more and more projects to Meson.
As a positive side-effect of figuring this out, we should be able to start building osinfo-db-tools again on platforms such as Ubuntu 18.04, dropping which I already though was problematic at the time.
tl;dr NACK until we figure out how to do this without dropping several target platforms in the process
In theory you can 'pip install meson'. The issue is likely to then be whether ninja is available on the platform in question, as that's a native app, not python. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Mon, 2019-09-23 at 13:51 +0100, Daniel P. Berrangé wrote:
On Mon, Sep 23, 2019 at 02:33:43PM +0200, Andrea Bolognani wrote:
tl;dr NACK until we figure out how to do this without dropping several target platforms in the process
In theory you can 'pip install meson'. The issue is likely to then be whether ninja is available on the platform in question, as that's a native app, not python.
Yeah, I forgot about the Ninja bit and mentioned it in a separate message before reading your reply :) Anyway, while I agree that in theory this is extremely simple to solve, I'd like to see that theory put into practice before we cut any release that requires Meson. -- Andrea Bolognani / Red Hat / Virtualization
On Mon, Sep 23, 2019 at 03:03:47PM +0200, Andrea Bolognani wrote:
On Mon, 2019-09-23 at 13:51 +0100, Daniel P. Berrangé wrote:
On Mon, Sep 23, 2019 at 02:33:43PM +0200, Andrea Bolognani wrote:
tl;dr NACK until we figure out how to do this without dropping several target platforms in the process
In theory you can 'pip install meson'. The issue is likely to then be whether ninja is available on the platform in question, as that's a native app, not python.
Yeah, I forgot about the Ninja bit and mentioned it in a separate message before reading your reply :)
Anyway, while I agree that in theory this is extremely simple to solve, I'd like to see that theory put into practice before we cut any release that requires Meson.
Meson 0.49.0 depends on Ninja 1.5 or newer and that is available on all supported distros. On some of them the package is called `ninja` and the rest uses `ninja-build`. On CentOS/RHEL 7 the package is available only from EPEL. On RHEL 8 the package is available from CodeReady Linux Builder repository. On CentOS 8 it should be available as well. I'm not sure if we have EPEL repository in our CI VMs, but it should be fairly easy to enable and in fact if there is someone using upstream libvirt on CentOS/RHEL 7 they most likely have EPEL enabled as well. Installing Meson using pip should be simple to implement. Pavel
On Mon, 2019-09-23 at 15:43 +0200, Pavel Hrdina wrote:
On Mon, Sep 23, 2019 at 03:03:47PM +0200, Andrea Bolognani wrote:
Anyway, while I agree that in theory this is extremely simple to solve, I'd like to see that theory put into practice before we cut any release that requires Meson.
Meson 0.49.0 depends on Ninja 1.5 or newer and that is available on all supported distros. On some of them the package is called `ninja` and the rest uses `ninja-build`.
On CentOS/RHEL 7 the package is available only from EPEL.
On RHEL 8 the package is available from CodeReady Linux Builder repository.
On CentOS 8 it should be available as well.
Thanks for taking the time to investigate this! It looks like we're indeed covered as far as Ninja is concerned.
I'm not sure if we have EPEL repository in our CI VMs, but it should be fairly easy to enable and in fact if there is someone using upstream libvirt on CentOS/RHEL 7 they most likely have EPEL enabled as well.
We don't have it enabled, but a while ago we agreed that it's acceptable to add it to the CentOS guests, so installing Ninja from EPEL shouldn't be a problem.
Installing Meson using pip should be simple to implement.
I agree that it's almost certainly just a matter of sitting down and, you know, actually implementing it :) But let me reiterate that I don't think we should consider releasing a version of libvirt-dbus that uses Meson for builds until we have proper Meson support in our CI. Perhaps we're all on the same page already and this wouldn't even need mentioning, but I'm being extremely clear about my position just in case. -- Andrea Bolognani / Red Hat / Virtualization
On Mon, Sep 23, 2019 at 2:51 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
On Mon, Sep 23, 2019 at 02:33:43PM +0200, Andrea Bolognani wrote:
On Mon, 2019-09-23 at 13:23 +0200, Fabiano Fidêncio wrote:
name: libvirt-dbus + # libvirt-dbus depends on meson 0.49.0 which is not available on + # CentOS 7, Debian 9, Ubuntu 18; machines: - - libvirt-centos-7 - - libvirt-debian-9 - libvirt-debian-10 - libvirt-debian-sid - libvirt-fedora-29 @@ -12,42 +12,15 @@ - libvirt-freebsd-11 - libvirt-freebsd-12 - libvirt-freebsd-current - - libvirt-ubuntu-16 - - libvirt-ubuntu-18
Based on the discussion we had on libvir-list, our approach for CI should not be to drop target platforms on the basis that they doen't include a recent enough Meson, but rather to make a suitable Meson version available through other means such as pip.
I would even go as far as arguing that implementing such a solution in libvirt-jenkins-ci is a pre-requisite, if not for merging the libvirt-dbus patches making the switch to Meson, certainly for cutting a release that includes them.
The general understanding is that we only test what we claim to support and only claim to support what we test. There are some cases where we're not following this mantra to the letter, and efforts are already underway to address the most obvious ones, but that's no excuse to make the situation worse, especially considering that in the coming months we expect to move more and more projects to Meson.
As a positive side-effect of figuring this out, we should be able to start building osinfo-db-tools again on platforms such as Ubuntu 18.04, dropping which I already though was problematic at the time.
tl;dr NACK until we figure out how to do this without dropping several target platforms in the process
In theory you can 'pip install meson'. The issue is likely to then be whether ninja is available on the platform in question, as that's a native app, not python.
Just would like to point that using 'pip install meson' may be problematic as it'd always install the latest meson present in pip (which, in Debian 9's case would be 0.51.2 ... which *may* require a newer version of the Ninja than the one present in the system). So, maybe we should agree building it ourselves and, at least, have control of what's the meson (and, consequently) the ninja version being used. Best Regards, -- Fabiano Fidêncio
On Mon, Sep 23, 2019 at 03:37:15PM +0200, Fabiano Fidêncio wrote:
On Mon, Sep 23, 2019 at 2:51 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
On Mon, Sep 23, 2019 at 02:33:43PM +0200, Andrea Bolognani wrote:
On Mon, 2019-09-23 at 13:23 +0200, Fabiano Fidêncio wrote:
name: libvirt-dbus + # libvirt-dbus depends on meson 0.49.0 which is not available on + # CentOS 7, Debian 9, Ubuntu 18; machines: - - libvirt-centos-7 - - libvirt-debian-9 - libvirt-debian-10 - libvirt-debian-sid - libvirt-fedora-29 @@ -12,42 +12,15 @@ - libvirt-freebsd-11 - libvirt-freebsd-12 - libvirt-freebsd-current - - libvirt-ubuntu-16 - - libvirt-ubuntu-18
Based on the discussion we had on libvir-list, our approach for CI should not be to drop target platforms on the basis that they doen't include a recent enough Meson, but rather to make a suitable Meson version available through other means such as pip.
I would even go as far as arguing that implementing such a solution in libvirt-jenkins-ci is a pre-requisite, if not for merging the libvirt-dbus patches making the switch to Meson, certainly for cutting a release that includes them.
The general understanding is that we only test what we claim to support and only claim to support what we test. There are some cases where we're not following this mantra to the letter, and efforts are already underway to address the most obvious ones, but that's no excuse to make the situation worse, especially considering that in the coming months we expect to move more and more projects to Meson.
As a positive side-effect of figuring this out, we should be able to start building osinfo-db-tools again on platforms such as Ubuntu 18.04, dropping which I already though was problematic at the time.
tl;dr NACK until we figure out how to do this without dropping several target platforms in the process
In theory you can 'pip install meson'. The issue is likely to then be whether ninja is available on the platform in question, as that's a native app, not python.
Just would like to point that using 'pip install meson' may be problematic as it'd always install the latest meson present in pip (which, in Debian 9's case would be 0.51.2 ... which *may* require a newer version of the Ninja than the one present in the system).
I can't remember the syntax, but I'm pretty sure you can request a specific version with pip. OpenStack uses pip and intentionally fixes on non-latest versions of certain stuff. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Mon, Sep 23, 2019 at 3:40 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
On Mon, Sep 23, 2019 at 03:37:15PM +0200, Fabiano Fidêncio wrote:
On Mon, Sep 23, 2019 at 2:51 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
On Mon, Sep 23, 2019 at 02:33:43PM +0200, Andrea Bolognani wrote:
On Mon, 2019-09-23 at 13:23 +0200, Fabiano Fidêncio wrote:
name: libvirt-dbus + # libvirt-dbus depends on meson 0.49.0 which is not available on + # CentOS 7, Debian 9, Ubuntu 18; machines: - - libvirt-centos-7 - - libvirt-debian-9 - libvirt-debian-10 - libvirt-debian-sid - libvirt-fedora-29 @@ -12,42 +12,15 @@ - libvirt-freebsd-11 - libvirt-freebsd-12 - libvirt-freebsd-current - - libvirt-ubuntu-16 - - libvirt-ubuntu-18
Based on the discussion we had on libvir-list, our approach for CI should not be to drop target platforms on the basis that they doen't include a recent enough Meson, but rather to make a suitable Meson version available through other means such as pip.
I would even go as far as arguing that implementing such a solution in libvirt-jenkins-ci is a pre-requisite, if not for merging the libvirt-dbus patches making the switch to Meson, certainly for cutting a release that includes them.
The general understanding is that we only test what we claim to support and only claim to support what we test. There are some cases where we're not following this mantra to the letter, and efforts are already underway to address the most obvious ones, but that's no excuse to make the situation worse, especially considering that in the coming months we expect to move more and more projects to Meson.
As a positive side-effect of figuring this out, we should be able to start building osinfo-db-tools again on platforms such as Ubuntu 18.04, dropping which I already though was problematic at the time.
tl;dr NACK until we figure out how to do this without dropping several target platforms in the process
In theory you can 'pip install meson'. The issue is likely to then be whether ninja is available on the platform in question, as that's a native app, not python.
Just would like to point that using 'pip install meson' may be problematic as it'd always install the latest meson present in pip (which, in Debian 9's case would be 0.51.2 ... which *may* require a newer version of the Ninja than the one present in the system).
I can't remember the syntax, but I'm pretty sure you can request a specific version with pip. OpenStack uses pip and intentionally fixes on non-latest versions of certain stuff.
True: `pip install meson==0.49.0` Best Regards, -- Fabiano Fidêncio
On Mon, 2019-09-23 at 14:33 +0200, Andrea Bolognani wrote:
Based on the discussion we had on libvir-list, our approach for CI should not be to drop target platforms on the basis that they doen't include a recent enough Meson, but rather to make a suitable Meson version available through other means such as pip.
We should also figure out which version of Ninja Meson 0.49 depends on because, unlike Meson, Ninja is a native tool and it might be more annoying to side-load it. All the more reason to get this sorted out before progressing much further on our path to converting everything to Meson. -- Andrea Bolognani / Red Hat / Virtualization
participants (4)
-
Andrea Bolognani -
Daniel P. Berrangé -
Fabiano Fidêncio -
Pavel Hrdina