[libvirt] [jenkins-ci PATCH 00/22] Add CentOS 8 to libvirt-jenkins-ci
In order to add CentOS 8 to jenkins-ci, we had to: - Adjust lcitool to generate unattended scripts based on templates instead of using the templates themselves, as a new parameter is expected on kickstart files for RHEL / CentOS 8 (or newer); - Enable the EPEL and PowerTools repos; Then, the following projects have been added: - libvirt - Some adjustments had to be done as some dependencies got removed from CentOS 8; - libvirt-glib - libvirt-dbus - Some adjustments had to be done as flake8 doesn't pull all the needed dependencies when installed on CentOS 8; - libvirt-go - libvirt-go-xml - libvirt-ocaml - libvirt-perl - libvirt-python - virt-viewer - Some adjustments had to be done as libgovirt-devel and gtk-vnc2-devel are not present anymore on CentOS 8; - osinfo-db-tools - oisnfo-db - libosinfo - virt-manager The projects which were not added due to missing dependencies are: - libvirt-cim - libvirt-sandbox - libvirt-tck This series is written atop of: https://www.redhat.com/archives/libvir-list/2019-December/msg00461.html Fabiano Fidêncio (22): lcitool: Generate the unattended script file guests,lcitool: Use install_url in the unattended install files guests,jenkins: Add CentOS 8 guests: Install EPEL repo on all CentOS guests guests: Enable PowerTools repo on CentOS 8 guests guests,jenkins: Build libvirt on CentOS 8 guests guests,jenkins: Build libvirt-glib on CentOS guests guests,libvirt-dbus: Add python3-entrypoints dependency guests,libvirt-dbus: Add python3-mccabe dependency guests,libvirt-dbus: Add python3-pycodestile dependency guests,libvirt-dbus: Add python3-pyflakes dependency guests,jenkins: Build libvirt-dbus on CentOS 8 guests guests,jenkins: Build libvirt-go on CentOS 8 guests guests,jenkins: Build libvirt-go-xml on CentOS 8 guests guests,jenkins: Build libvirt-ocaml on CentOS 8 guests guests,jenkins: Build libvirt-perl on CentOS 8 guests guests,jenkins: Build libvirt-python on CentOS 8 guests guests,jenkins: Build virt-viewer on CentOS 8 guests guests,jenkins: Build osinfo-db-tools on CentOS 8 guests guests,jenkins: Build osinfo-db on CentOS 8 guests guests,jenkins: Build libosinfo on CentOS 8 guests guests,jenkins: Build virt-manager on CentOS 8 guests guests/configs/kickstart.cfg | 4 ++++ guests/host_vars/libvirt-centos-8/docker.yml | 2 ++ guests/host_vars/libvirt-centos-8/install.yml | 2 ++ guests/host_vars/libvirt-centos-8/main.yml | 22 ++++++++++++++++++ guests/inventory | 1 + guests/lcitool | 23 ++++++++++++++++++- guests/playbooks/build/jobs/defaults.yml | 2 ++ guests/playbooks/build/projects/libosinfo.yml | 1 + .../playbooks/build/projects/libvirt-dbus.yml | 2 ++ .../build/projects/libvirt-go-xml.yml | 1 + .../playbooks/build/projects/libvirt-go.yml | 1 + .../build/projects/osinfo-db-tools.yml | 1 + .../playbooks/build/projects/virt-manager.yml | 3 +++ guests/playbooks/update/tasks/base.yml | 9 +++++++- guests/vars/mappings.yml | 16 +++++++++++++ guests/vars/projects/libvirt-dbus.yml | 4 ++++ jenkins/jobs/defaults.yaml | 2 ++ jenkins/projects/libosinfo.yaml | 1 + jenkins/projects/libvirt-dbus.yaml | 2 ++ jenkins/projects/libvirt-go-xml.yaml | 1 + jenkins/projects/libvirt-go.yaml | 1 + jenkins/projects/libvirt.yaml | 1 + jenkins/projects/osinfo-db-tools.yaml | 1 + jenkins/projects/virt-manager.yaml | 3 +++ 24 files changed, 104 insertions(+), 2 deletions(-) create mode 100644 guests/host_vars/libvirt-centos-8/docker.yml create mode 100644 guests/host_vars/libvirt-centos-8/install.yml create mode 100644 guests/host_vars/libvirt-centos-8/main.yml -- 2.23.0
Let's always generate the unattended script file as, by doing this, we can trea the files we have as templates, being able to change them accordingly to whatever is needed by specific distros. It's important to note that we *must* be careful and keep generating those files using their "expected" filename, preferrably using the same name of the template, as some of the distros require that. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/lcitool | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/guests/lcitool b/guests/lcitool index 8436ce7..3985e90 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -23,9 +23,11 @@ import json import os import platform import random +import shutil import string import subprocess import sys +import tempfile import textwrap import yaml @@ -537,7 +539,14 @@ class Application: raise Exception( "Host {} doesn't support installation".format(host) ) - initrd_inject = os.path.join(base, "configs", install_config) + initrd_template = os.path.join(base, "configs", install_config) + with open(initrd_template) as template: + content = template.read() + + tempdir = tempfile.mkdtemp() + initrd_inject = os.path.join(tempdir, install_config) + + open(initrd_inject, "w").write(content) # preseed files must use a well-known name to be picked up by # d-i; for kickstart files, we can use whatever name we please @@ -587,6 +596,8 @@ class Application: except Exception as ex: raise Exception("Failed to install '{}': {}".format(host, ex)) + shutil.rmtree(tempdir, ignore_errors=True) + def _action_update(self, args): self._execute_playbook("update", args.hosts, args.projects, args.git_revision) -- 2.23.0
On Mon, 2019-12-09 at 16:20 +0100, Fabiano Fidêncio wrote:
Let's always generate the unattended script file as, by doing this, we can trea the files we have as templates, being able to change them
s/trea/treat/
accordingly to whatever is needed by specific distros.
It's important to note that we *must* be careful and keep generating those files using their "expected" filename, preferrably using the same
s/preferrably/preferably/ [...]
@@ -537,7 +539,14 @@ class Application: raise Exception( "Host {} doesn't support installation".format(host) )
It would be nice if you added a short comment here, explaining the fact that we're generating the file on the fly starting from a template.
- initrd_inject = os.path.join(base, "configs", install_config) + initrd_template = os.path.join(base, "configs", install_config) + with open(initrd_template) as template:
Missing "r" here. It's probably the default, but we're passing it explicitly to open() everywhere else so let's be consistent.
+ content = template.read() + + tempdir = tempfile.mkdtemp() + initrd_inject = os.path.join(tempdir, install_config) + + open(initrd_inject, "w").write(content)
Shouldn't this also use a 'with' clause to ensure the file is closed correctly after writing to it?
@@ -587,6 +596,8 @@ class Application: except Exception as ex: raise Exception("Failed to install '{}': {}".format(host, ex))
+ shutil.rmtree(tempdir, ignore_errors=True)
So if we error out at any point, the temporary directory will be left behind... Eh, I can live with that :) -- Andrea Bolognani / Red Hat / Virtualization
Passing `url --url={{ install_url }}` to a kickstart file is, according to Red Hat official documentation[0], mandatory for RHEL 8 and onwards (which includes CentOS 8). Using it for RHEL 7, CentOS 7, and Fedora does not cause any issue. [0]: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/htm... Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/configs/kickstart.cfg | 4 ++++ guests/lcitool | 10 ++++++++++ 2 files changed, 14 insertions(+) diff --git a/guests/configs/kickstart.cfg b/guests/configs/kickstart.cfg index 713557a..a176e25 100644 --- a/guests/configs/kickstart.cfg +++ b/guests/configs/kickstart.cfg @@ -44,6 +44,10 @@ part swap --fstype=swap --size=256 bootloader --location=mbr --timeout=1 +# Set the installation url +url --url={{ install_url }} + + # Configure networking # # The only network interface available to the guest will come up diff --git a/guests/lcitool b/guests/lcitool index 3985e90..9ef5795 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -539,9 +539,19 @@ class Application: raise Exception( "Host {} doesn't support installation".format(host) ) + + unattended_options = [ + "install_url", + ] + initrd_template = os.path.join(base, "configs", install_config) with open(initrd_template) as template: content = template.read() + for option in unattended_options: + content = content.replace( + "{{ " + option + " }}", + facts[option] + ) tempdir = tempfile.mkdtemp() initrd_inject = os.path.join(tempdir, install_config) -- 2.23.0
On Mon, 2019-12-09 at 16:20 +0100, Fabiano Fidêncio wrote:
guests,lcitool: Use install_url in the unattended install files
s/^guests,//
+++ b/guests/configs/kickstart.cfg +# Set the installation url +url --url={{ install_url }}
I know it doesn't make any difference in practice, but setting this up right in between configuring the bootloader and the network is a... rather peculiar choice. Please move this to the very top of the file, and include a proper comment for it like # Installation source # # The operating system will be installed from the following URL Everything else looks good, so with these nits fixed Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
This commit allows calling `lcitool install libvirt-centos-8`. `lcitool update libvirt-centos-8 $project` does *not* work as: - EPEL repository is only enabled for CentOS 7; - The reason EPEL is needed because of some base packages (as screen); - There's no task to enable PowerTools repository; - PowerTools is needed because the development packages are distributed via this repo; - There's *no* project supported; Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/host_vars/libvirt-centos-8/docker.yml | 2 ++ guests/host_vars/libvirt-centos-8/install.yml | 2 ++ guests/host_vars/libvirt-centos-8/main.yml | 7 +++++++ guests/inventory | 1 + guests/playbooks/build/jobs/defaults.yml | 2 ++ jenkins/jobs/defaults.yaml | 2 ++ 6 files changed, 16 insertions(+) create mode 100644 guests/host_vars/libvirt-centos-8/docker.yml create mode 100644 guests/host_vars/libvirt-centos-8/install.yml create mode 100644 guests/host_vars/libvirt-centos-8/main.yml diff --git a/guests/host_vars/libvirt-centos-8/docker.yml b/guests/host_vars/libvirt-centos-8/docker.yml new file mode 100644 index 0000000..10c2a50 --- /dev/null +++ b/guests/host_vars/libvirt-centos-8/docker.yml @@ -0,0 +1,2 @@ +--- +docker_base: centos:centos8 diff --git a/guests/host_vars/libvirt-centos-8/install.yml b/guests/host_vars/libvirt-centos-8/install.yml new file mode 100644 index 0000000..46facc1 --- /dev/null +++ b/guests/host_vars/libvirt-centos-8/install.yml @@ -0,0 +1,2 @@ +--- +install_url: http://mirror.centos.org/centos-8/8/BaseOS/x86_64/os/ diff --git a/guests/host_vars/libvirt-centos-8/main.yml b/guests/host_vars/libvirt-centos-8/main.yml new file mode 100644 index 0000000..a147183 --- /dev/null +++ b/guests/host_vars/libvirt-centos-8/main.yml @@ -0,0 +1,7 @@ +--- +package_format: 'rpm' +package_manager: 'dnf' +os_name: 'CentOS' +os_version: '8' + +ansible_python_interpreter: /usr/bin/python3 diff --git a/guests/inventory b/guests/inventory index ecdcc34..f062310 100644 --- a/guests/inventory +++ b/guests/inventory @@ -1,4 +1,5 @@ libvirt-centos-7 +libvirt-centos-8 libvirt-debian-9 libvirt-debian-10 libvirt-debian-sid diff --git a/guests/playbooks/build/jobs/defaults.yml b/guests/playbooks/build/jobs/defaults.yml index 0175a05..5e4ec03 100644 --- a/guests/playbooks/build/jobs/defaults.yml +++ b/guests/playbooks/build/jobs/defaults.yml @@ -1,6 +1,7 @@ --- all_machines: - libvirt-centos-7 + - libvirt-centos-8 - libvirt-debian-9 - libvirt-debian-10 - libvirt-debian-sid @@ -15,6 +16,7 @@ all_machines: - libvirt-ubuntu-1804 rpm_machines: - libvirt-centos-7 + - libvirt-centos-8 - libvirt-fedora-30 - libvirt-fedora-31 - libvirt-fedora-rawhide diff --git a/jenkins/jobs/defaults.yaml b/jenkins/jobs/defaults.yaml index 676ecbf..9232d42 100644 --- a/jenkins/jobs/defaults.yaml +++ b/jenkins/jobs/defaults.yaml @@ -4,6 +4,7 @@ node: libvirt all_machines: - libvirt-centos-7 + - libvirt-centos-8 - libvirt-debian-9 - libvirt-debian-10 - libvirt-fedora-30 @@ -13,6 +14,7 @@ - libvirt-freebsd-12 rpm_machines: - libvirt-centos-7 + - libvirt-centos-8 - libvirt-fedora-30 - libvirt-fedora-31 - libvirt-fedora-rawhide -- 2.23.0
On Mon, 2019-12-09 at 16:20 +0100, Fabiano Fidêncio wrote:
guests,jenkins: Add CentOS 8
s/,jenkins// [...]
diff --git a/guests/playbooks/build/jobs/defaults.yml b/guests/playbooks/build/jobs/defaults.yml index 0175a05..5e4ec03 100644 --- a/guests/playbooks/build/jobs/defaults.yml +++ b/guests/playbooks/build/jobs/defaults.yml @@ -1,6 +1,7 @@ --- all_machines: - libvirt-centos-7 + - libvirt-centos-8 - libvirt-debian-9 - libvirt-debian-10 - libvirt-debian-sid @@ -15,6 +16,7 @@ all_machines: - libvirt-ubuntu-1804 rpm_machines: - libvirt-centos-7 + - libvirt-centos-8 - libvirt-fedora-30 - libvirt-fedora-31 - libvirt-fedora-rawhide diff --git a/jenkins/jobs/defaults.yaml b/jenkins/jobs/defaults.yaml index 676ecbf..9232d42 100644 --- a/jenkins/jobs/defaults.yaml +++ b/jenkins/jobs/defaults.yaml @@ -4,6 +4,7 @@ node: libvirt all_machines: - libvirt-centos-7 + - libvirt-centos-8 - libvirt-debian-9 - libvirt-debian-10 - libvirt-fedora-30 @@ -13,6 +14,7 @@ - libvirt-freebsd-12 rpm_machines: - libvirt-centos-7 + - libvirt-centos-8 - libvirt-fedora-30 - libvirt-fedora-31 - libvirt-fedora-rawhide
These changes are not wrong per se, but I would prefer it if the series had a different structure: * set the stage + this is your first two patches * add support for installing a CentOS 8 guest + all the other hunks of this patch plus the next two + './lcitool install libvirt-centos-8' now works + happens entirely inside the guests/ directory * add projects to CentOS 8 one by one + one patch per project + just tweak the mappings as needed and add the corresponding project to host_vars/libvirt-centos-8/main.yml when done + './lcitool update libvirt-centos-8 $project' should succeed with the patch applied + also requires touching guests/ only * start building projects on CentOS 8 + single patch that enables everything + touches guests/playbooks/build/ and jenkins/ only This structure makes more sense to me because adding libvirt-centos-8 to all_machines feels really wrong when you know that, at that point in time, you wouldn't actually be able to build a single project on the resulting guest. So, in conclusion: Reviewed-by: Andrea Bolognani <abologna@redhat.com> to all other hunks, but save these last few for later. -- Andrea Bolognani / Red Hat / Virtualization
On Mon, 2019-12-09 at 18:29 +0100, Andrea Bolognani wrote:
These changes are not wrong per se, but I would prefer it if the series had a different structure:
* set the stage + this is your first two patches
* add support for installing a CentOS 8 guest + all the other hunks of this patch plus the next two + './lcitool install libvirt-centos-8' now works + happens entirely inside the guests/ directory
* add projects to CentOS 8 one by one + one patch per project + just tweak the mappings as needed and add the corresponding project to host_vars/libvirt-centos-8/main.yml when done + './lcitool update libvirt-centos-8 $project' should succeed with the patch applied + also requires touching guests/ only
Taking half a step back after giving a second look to the rest of the series, once you split the changes to playbooks/build/ and jenkins/ off to a separate patch there's very little point in having one patch per project, as the changes to mappings.yml and host_vars/ in patches 6,7 and 12-22 are small and straighforward enough that it would make more sense to have them all together. So let's do that instead. I don't see the need to add further comments to this incarnation of the series. The only remaining question IMHO is whether we can get away without patches 8-11 - I really don't like those :) -- Andrea Bolognani / Red Hat / Virtualization
EPEL repository should be enabled on all supported CentOS guests as it brings in packages which are part of the "base packages" for libvirt-jenkins-ci. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/playbooks/update/tasks/base.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/guests/playbooks/update/tasks/base.yml b/guests/playbooks/update/tasks/base.yml index 5153917..965e442 100644 --- a/guests/playbooks/update/tasks/base.yml +++ b/guests/playbooks/update/tasks/base.yml @@ -15,7 +15,6 @@ state: latest when: - os_name == 'CentOS' - - os_version == '7' - name: Update installed packages package: -- 2.23.0
On Mon, 2019-12-09 at 16:20 +0100, Fabiano Fidêncio wrote:
EPEL repository should be enabled on all supported CentOS guests as it brings in packages which are part of the "base packages" for libvirt-jenkins-ci.
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/playbooks/update/tasks/base.yml | 1 - 1 file changed, 1 deletion(-)
Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
This is needed as all the development packages will come from the PowerTools repository, which is not enabled by default. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/playbooks/update/tasks/base.yml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/guests/playbooks/update/tasks/base.yml b/guests/playbooks/update/tasks/base.yml index 965e442..b3ab738 100644 --- a/guests/playbooks/update/tasks/base.yml +++ b/guests/playbooks/update/tasks/base.yml @@ -9,6 +9,14 @@ - os_name == 'Fedora' - os_version == 'Rawhide' +- name: Enable PowerTools repository + command: '{{ package_manager }} config-manager --set-enabled PowerTools -y' + args: + warn: no + when: + - os_name == 'CentOS' + - os_version == '8' + - name: Enable EPEL repository package: name: epel-release -- 2.23.0
On Mon, 2019-12-09 at 16:20 +0100, Fabiano Fidêncio wrote:
This is needed as all the development packages will come from the PowerTools repository, which is not enabled by default.
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/playbooks/update/tasks/base.yml | 8 ++++++++ 1 file changed, 8 insertions(+)
Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
In order to perform a libvirt `update` and `build` on a CentOS 8 guest, a few changes in the mappings have been done. The changes are: - libssh2 is not longer present: - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/htm... - librbd1-devel has been replaced by librdb-devel: - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/htm... Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/host_vars/libvirt-centos-8/main.yml | 3 +++ guests/vars/mappings.yml | 2 ++ jenkins/projects/libvirt.yaml | 1 + 3 files changed, 6 insertions(+) diff --git a/guests/host_vars/libvirt-centos-8/main.yml b/guests/host_vars/libvirt-centos-8/main.yml index a147183..b924f1f 100644 --- a/guests/host_vars/libvirt-centos-8/main.yml +++ b/guests/host_vars/libvirt-centos-8/main.yml @@ -1,4 +1,7 @@ --- +projects: + - libvirt + package_format: 'rpm' package_manager: 'dnf' os_name: 'CentOS' diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index 56310f0..f0614a3 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -379,6 +379,7 @@ mappings: deb: librbd-dev Fedora: librbd-devel CentOS7: librbd1-devel + CentOS8: librbd-devel OpenSUSE: librbd-devel cross-policy-deb: foreign @@ -404,6 +405,7 @@ mappings: deb: libssh2-1-dev pkg: libssh2 rpm: libssh2-devel + CentOS8: cross-policy-deb: foreign libtirpc: diff --git a/jenkins/projects/libvirt.yaml b/jenkins/projects/libvirt.yaml index fdc24bc..fa8fb89 100644 --- a/jenkins/projects/libvirt.yaml +++ b/jenkins/projects/libvirt.yaml @@ -14,6 +14,7 @@ # commands with more arguments than FreeBSD supports machines: - libvirt-centos-7 + - libvirt-centos-8 - libvirt-debian-9 - libvirt-debian-10 - libvirt-fedora-30 -- 2.23.0
On Mon, 2019-12-09 at 16:20 +0100, Fabiano Fidêncio wrote:
guests,jenkins: Build libvirt on CentOS 8 guests
s/,jenkins// [...]
+++ b/jenkins/projects/libvirt.yaml @@ -14,6 +14,7 @@ # commands with more arguments than FreeBSD supports machines: - libvirt-centos-7 + - libvirt-centos-8 - libvirt-debian-9 - libvirt-debian-10 - libvirt-fedora-30
You need to update guests/playbooks/build/projects/libvirt.yml at the same time; in any case, as I said earlier, I expect all these changes to be in a single patch that goes in at the very end of the series. For the hunks that change mappings and host_vars, Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/host_vars/libvirt-centos-8/main.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/guests/host_vars/libvirt-centos-8/main.yml b/guests/host_vars/libvirt-centos-8/main.yml index b924f1f..06a7844 100644 --- a/guests/host_vars/libvirt-centos-8/main.yml +++ b/guests/host_vars/libvirt-centos-8/main.yml @@ -1,6 +1,7 @@ --- projects: - libvirt + - libvirt-glib package_format: 'rpm' package_manager: 'dnf' -- 2.23.0
On Mon, 2019-12-09 at 16:20 +0100, Fabiano Fidêncio wrote:
guests,jenkins: Build libvirt-glib on CentOS guests
I should have spotted this while reviewing the previous patch, but it applies to many subsequent ones as well so I guess now will work too :) Since with the changes in series structure I requested we're not really "building" anything, the patch subject should be something like guests: Add $project to CentOS 8 This applies to all subsequent patches, so I'm not going to point it out again. With the subject fixed, Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
Seems that for all OSes where run libvirt-dbus tests python3-entrypoints is pulled together when installing flake8. However, this is not the case for CentOS 8 and that's the reason for explicitly adding the dependency here. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/vars/mappings.yml | 3 +++ guests/vars/projects/libvirt-dbus.yml | 1 + 2 files changed, 4 insertions(+) diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index f0614a3..79e7acd 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -830,6 +830,9 @@ mappings: CentOS7: python36-devel cross-policy-deb: foreign + python3-entrypoints: + CentOS8: python3-entrypoints + python3-gi: deb: python3-gi pkg: py36-gobject3 diff --git a/guests/vars/projects/libvirt-dbus.yml b/guests/vars/projects/libvirt-dbus.yml index 481693d..248637b 100644 --- a/guests/vars/projects/libvirt-dbus.yml +++ b/guests/vars/projects/libvirt-dbus.yml @@ -4,5 +4,6 @@ packages: - flake8 - glib2 - python3-dbus + - python3-entrypoints - python3-gi - python3-pytest -- 2.23.0
On Mon, 2019-12-09 at 16:20 +0100, Fabiano Fidêncio wrote:
Seems that for all OSes where run libvirt-dbus tests python3-entrypoints is pulled together when installing flake8. However, this is not the case for CentOS 8 and that's the reason for explicitly adding the dependency here.
There's a bug report tracking this: https://bugzilla.redhat.com/show_bug.cgi?id=1757463 The packaging issue was since fixed in Fedora, but unfortunately it has not been backported to EPEL yet. Do you think there's a chance we could convince the EPEL maintainers to pick up the very simple patch[1] that makes this unnecessary and get a working build within a short time? I'm really not fond of this work around :( [1] https://src.fedoraproject.org/rpms/python-flake8/c/4b0a28a01de1d24a7d1509165... -- Andrea Bolognani / Red Hat / Virtualization
On Mon, Dec 09, 2019 at 06:45:10PM +0100, Andrea Bolognani wrote:
On Mon, 2019-12-09 at 16:20 +0100, Fabiano Fidêncio wrote:
Seems that for all OSes where run libvirt-dbus tests python3-entrypoints is pulled together when installing flake8. However, this is not the case for CentOS 8 and that's the reason for explicitly adding the dependency here.
There's a bug report tracking this:
https://bugzilla.redhat.com/show_bug.cgi?id=1757463
The packaging issue was since fixed in Fedora, but unfortunately it has not been backported to EPEL yet.
Do you think there's a chance we could convince the EPEL maintainers to pick up the very simple patch[1] that makes this unnecessary and get a working build within a short time?
It looks like the blocking problem in EPEL build system got fixed in stable 2 days ago. So I've commented on the flake8 bz to request a new build Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Mon, 2019-12-09 at 17:51 +0000, Daniel P. Berrangé wrote:
On Mon, Dec 09, 2019 at 06:45:10PM +0100, Andrea Bolognani wrote:
On Mon, 2019-12-09 at 16:20 +0100, Fabiano Fidêncio wrote:
Seems that for all OSes where run libvirt-dbus tests python3-entrypoints is pulled together when installing flake8. However, this is not the case for CentOS 8 and that's the reason for explicitly adding the dependency here.
There's a bug report tracking this:
https://bugzilla.redhat.com/show_bug.cgi?id=1757463
The packaging issue was since fixed in Fedora, but unfortunately it has not been backported to EPEL yet.
Do you think there's a chance we could convince the EPEL maintainers to pick up the very simple patch[1] that makes this unnecessary and get a working build within a short time?
It looks like the blocking problem in EPEL build system got fixed in stable 2 days ago. So I've commented on the flake8 bz to request a new build
That's great news! Thanks :) -- Andrea Bolognani / Red Hat / Virtualization
Seems that for all OSes where we run libvirt-dbus tests python3-mccabe is pulled together when installing flake8. However, this is not the case for CentOS 8 and that's the reason for explicitly adding the dependency here. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/vars/mappings.yml | 3 +++ guests/vars/projects/libvirt-dbus.yml | 1 + 2 files changed, 4 insertions(+) diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index 79e7acd..7b16a59 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -850,6 +850,9 @@ mappings: FreeBSD: py36-lxml CentOS7: python36-lxml + python3-mccabe: + CentOS8: python3-mccabe + python3-nose: default: python3-nose FreeBSD: py36-nose diff --git a/guests/vars/projects/libvirt-dbus.yml b/guests/vars/projects/libvirt-dbus.yml index 248637b..528fcc6 100644 --- a/guests/vars/projects/libvirt-dbus.yml +++ b/guests/vars/projects/libvirt-dbus.yml @@ -6,4 +6,5 @@ packages: - python3-dbus - python3-entrypoints - python3-gi + - python3-mccabe - python3-pytest -- 2.23.0
Seems that for all OSes where run libvirt-dbus tests python3-pycodestile is pulled together when installing flake8. However, this is not the case for CentOS 8 and that's the reason for explicitly adding the dependency here. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/vars/mappings.yml | 3 +++ guests/vars/projects/libvirt-dbus.yml | 1 + 2 files changed, 4 insertions(+) diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index 7b16a59..722887d 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -864,6 +864,9 @@ mappings: Ubuntu1604: python3-pip Ubuntu1804: python3-pip + python3-pycodestyle: + CentOS8: python3-pycodestyle + python3-pytest: default: python3-pytest FreeBSD: py36-pytest diff --git a/guests/vars/projects/libvirt-dbus.yml b/guests/vars/projects/libvirt-dbus.yml index 528fcc6..9a87e11 100644 --- a/guests/vars/projects/libvirt-dbus.yml +++ b/guests/vars/projects/libvirt-dbus.yml @@ -7,4 +7,5 @@ packages: - python3-entrypoints - python3-gi - python3-mccabe + - python3-pycodestyle - python3-pytest -- 2.23.0
Seems that for all OSes where run libvirt-dbus tests python3-pyflakes is pulled together when installing flake8. However, this is not the case for CentOS 8 and that's the reason for explicitly adding the dependency here. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/vars/mappings.yml | 3 +++ guests/vars/projects/libvirt-dbus.yml | 1 + 2 files changed, 4 insertions(+) diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index 722887d..2ee3848 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -867,6 +867,9 @@ mappings: python3-pycodestyle: CentOS8: python3-pycodestyle + python3-pyflakes: + CentOS8: python3-pyflakes + python3-pytest: default: python3-pytest FreeBSD: py36-pytest diff --git a/guests/vars/projects/libvirt-dbus.yml b/guests/vars/projects/libvirt-dbus.yml index 9a87e11..b5d3d32 100644 --- a/guests/vars/projects/libvirt-dbus.yml +++ b/guests/vars/projects/libvirt-dbus.yml @@ -8,4 +8,5 @@ packages: - python3-gi - python3-mccabe - python3-pycodestyle + - python3-pyflakes - python3-pytest -- 2.23.0
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/host_vars/libvirt-centos-8/main.yml | 1 + guests/playbooks/build/projects/libvirt-dbus.yml | 2 ++ jenkins/projects/libvirt-dbus.yaml | 2 ++ 3 files changed, 5 insertions(+) diff --git a/guests/host_vars/libvirt-centos-8/main.yml b/guests/host_vars/libvirt-centos-8/main.yml index 06a7844..9b81b6f 100644 --- a/guests/host_vars/libvirt-centos-8/main.yml +++ b/guests/host_vars/libvirt-centos-8/main.yml @@ -1,6 +1,7 @@ --- projects: - libvirt + - libvirt-dbus - libvirt-glib package_format: 'rpm' diff --git a/guests/playbooks/build/projects/libvirt-dbus.yml b/guests/playbooks/build/projects/libvirt-dbus.yml index 8c946af..66bc1fa 100644 --- a/guests/playbooks/build/projects/libvirt-dbus.yml +++ b/guests/playbooks/build/projects/libvirt-dbus.yml @@ -14,6 +14,7 @@ # Python3 version in Ubuntu 16.04 and python3-pytest version # in CentOS 7 are too old. machines: + - libvirt-centos-8 - libvirt-debian-9 - libvirt-debian-10 - libvirt-debian-sid @@ -25,6 +26,7 @@ # RPM build is still not possible on CentOS7 as it does not # have the needed RPM macros for meson. machines: + - libvirt-centos-8 - libvirt-fedora-30 - libvirt-fedora-31 - libvirt-fedora-rawhide diff --git a/jenkins/projects/libvirt-dbus.yaml b/jenkins/projects/libvirt-dbus.yaml index dfc159c..e71e2f9 100644 --- a/jenkins/projects/libvirt-dbus.yaml +++ b/jenkins/projects/libvirt-dbus.yaml @@ -15,6 +15,7 @@ # Python3 version in Ubuntu 16.04 and python3-pytest version # in CentOS 7 are too old. machines: + - libvirt-centos-8 - libvirt-debian-9 - libvirt-debian-10 - libvirt-fedora-30 @@ -25,6 +26,7 @@ # RPM build is still not possible on CentOS7 as it does not # have the needed RPM macros for meson. machines: + - libvirt-centos-8 - libvirt-fedora-30 - libvirt-fedora-31 - libvirt-fedora-rawhide -- 2.23.0
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/host_vars/libvirt-centos-8/main.yml | 1 + guests/playbooks/build/projects/libvirt-go.yml | 1 + jenkins/projects/libvirt-go.yaml | 1 + 3 files changed, 3 insertions(+) diff --git a/guests/host_vars/libvirt-centos-8/main.yml b/guests/host_vars/libvirt-centos-8/main.yml index 9b81b6f..a273d3d 100644 --- a/guests/host_vars/libvirt-centos-8/main.yml +++ b/guests/host_vars/libvirt-centos-8/main.yml @@ -3,6 +3,7 @@ projects: - libvirt - libvirt-dbus - libvirt-glib + - libvirt-go package_format: 'rpm' package_manager: 'dnf' diff --git a/guests/playbooks/build/projects/libvirt-go.yml b/guests/playbooks/build/projects/libvirt-go.yml index 36322b1..eb6a2f2 100644 --- a/guests/playbooks/build/projects/libvirt-go.yml +++ b/guests/playbooks/build/projects/libvirt-go.yml @@ -2,6 +2,7 @@ - set_fact: name: libvirt-go machines: + - libvirt-centos-8 - libvirt-debian-9 - libvirt-debian-10 - libvirt-debian-sid diff --git a/jenkins/projects/libvirt-go.yaml b/jenkins/projects/libvirt-go.yaml index 63d039a..75e484b 100644 --- a/jenkins/projects/libvirt-go.yaml +++ b/jenkins/projects/libvirt-go.yaml @@ -2,6 +2,7 @@ - project: name: libvirt-go machines: + - libvirt-centos-8 - libvirt-debian-9 - libvirt-debian-10 - libvirt-fedora-30 -- 2.23.0
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/host_vars/libvirt-centos-8/main.yml | 1 + guests/playbooks/build/projects/libvirt-go-xml.yml | 1 + jenkins/projects/libvirt-go-xml.yaml | 1 + 3 files changed, 3 insertions(+) diff --git a/guests/host_vars/libvirt-centos-8/main.yml b/guests/host_vars/libvirt-centos-8/main.yml index a273d3d..7a55c4b 100644 --- a/guests/host_vars/libvirt-centos-8/main.yml +++ b/guests/host_vars/libvirt-centos-8/main.yml @@ -4,6 +4,7 @@ projects: - libvirt-dbus - libvirt-glib - libvirt-go + - libvirt-go-xml package_format: 'rpm' package_manager: 'dnf' diff --git a/guests/playbooks/build/projects/libvirt-go-xml.yml b/guests/playbooks/build/projects/libvirt-go-xml.yml index 5cacb7c..665aab5 100644 --- a/guests/playbooks/build/projects/libvirt-go-xml.yml +++ b/guests/playbooks/build/projects/libvirt-go-xml.yml @@ -2,6 +2,7 @@ - set_fact: name: libvirt-go-xml machines: + - libvirt-centos-8 - libvirt-debian-9 - libvirt-debian-10 - libvirt-debian-sid diff --git a/jenkins/projects/libvirt-go-xml.yaml b/jenkins/projects/libvirt-go-xml.yaml index 8b4e2c5..9338768 100644 --- a/jenkins/projects/libvirt-go-xml.yaml +++ b/jenkins/projects/libvirt-go-xml.yaml @@ -2,6 +2,7 @@ - project: name: libvirt-go-xml machines: + - libvirt-centos-8 - libvirt-debian-9 - libvirt-debian-10 - libvirt-fedora-30 -- 2.23.0
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/host_vars/libvirt-centos-8/main.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/guests/host_vars/libvirt-centos-8/main.yml b/guests/host_vars/libvirt-centos-8/main.yml index 7a55c4b..93e66e3 100644 --- a/guests/host_vars/libvirt-centos-8/main.yml +++ b/guests/host_vars/libvirt-centos-8/main.yml @@ -5,6 +5,7 @@ projects: - libvirt-glib - libvirt-go - libvirt-go-xml + - libvirt-ocaml package_format: 'rpm' package_manager: 'dnf' -- 2.23.0
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/host_vars/libvirt-centos-8/main.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/guests/host_vars/libvirt-centos-8/main.yml b/guests/host_vars/libvirt-centos-8/main.yml index 93e66e3..793ad3e 100644 --- a/guests/host_vars/libvirt-centos-8/main.yml +++ b/guests/host_vars/libvirt-centos-8/main.yml @@ -6,6 +6,7 @@ projects: - libvirt-go - libvirt-go-xml - libvirt-ocaml + - libvirt-perl package_format: 'rpm' package_manager: 'dnf' -- 2.23.0
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/host_vars/libvirt-centos-8/main.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/guests/host_vars/libvirt-centos-8/main.yml b/guests/host_vars/libvirt-centos-8/main.yml index 793ad3e..63ac54b 100644 --- a/guests/host_vars/libvirt-centos-8/main.yml +++ b/guests/host_vars/libvirt-centos-8/main.yml @@ -7,6 +7,7 @@ projects: - libvirt-go-xml - libvirt-ocaml - libvirt-perl + - libvirt-python package_format: 'rpm' package_manager: 'dnf' -- 2.23.0
In order to perform a virt-viewer `update` and `build` on a CentOS 8 guest, a few changes in the mappings have been done. The changes are: - libgovirt-devel and gtk-vnc2-devel are not present anymore - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/htm... Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/host_vars/libvirt-centos-8/main.yml | 1 + guests/vars/mappings.yml | 2 ++ 2 files changed, 3 insertions(+) diff --git a/guests/host_vars/libvirt-centos-8/main.yml b/guests/host_vars/libvirt-centos-8/main.yml index 63ac54b..ce8cde5 100644 --- a/guests/host_vars/libvirt-centos-8/main.yml +++ b/guests/host_vars/libvirt-centos-8/main.yml @@ -8,6 +8,7 @@ projects: - libvirt-ocaml - libvirt-perl - libvirt-python + - virt-viewer package_format: 'rpm' package_manager: 'dnf' diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index 2ee3848..acd2cec 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -238,6 +238,7 @@ mappings: deb: libgtk-vnc-2.0-dev pkg: gtk-vnc rpm: gtk-vnc2-devel + CentOS8: cross-policy-deb: foreign hal: @@ -332,6 +333,7 @@ mappings: libgovirt: rpm: libgovirt-devel + CentOS8: Debian: libgovirt-dev cross-policy-deb: foreign -- 2.23.0
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/host_vars/libvirt-centos-8/main.yml | 1 + guests/playbooks/build/projects/osinfo-db-tools.yml | 1 + jenkins/projects/osinfo-db-tools.yaml | 1 + 3 files changed, 3 insertions(+) diff --git a/guests/host_vars/libvirt-centos-8/main.yml b/guests/host_vars/libvirt-centos-8/main.yml index ce8cde5..61ef995 100644 --- a/guests/host_vars/libvirt-centos-8/main.yml +++ b/guests/host_vars/libvirt-centos-8/main.yml @@ -8,6 +8,7 @@ projects: - libvirt-ocaml - libvirt-perl - libvirt-python + - osinfo-db-tools - virt-viewer package_format: 'rpm' diff --git a/guests/playbooks/build/projects/osinfo-db-tools.yml b/guests/playbooks/build/projects/osinfo-db-tools.yml index 18f0f0f..b5024ae 100644 --- a/guests/playbooks/build/projects/osinfo-db-tools.yml +++ b/guests/playbooks/build/projects/osinfo-db-tools.yml @@ -13,6 +13,7 @@ # RPM build is still not possible on CentOS7 as it does not # have the needed RPM macros for meson. machines: + - libvirt-centos-8 - libvirt-fedora-30 - libvirt-fedora-31 - libvirt-fedora-rawhide diff --git a/jenkins/projects/osinfo-db-tools.yaml b/jenkins/projects/osinfo-db-tools.yaml index f28e72f..7831df4 100644 --- a/jenkins/projects/osinfo-db-tools.yaml +++ b/jenkins/projects/osinfo-db-tools.yaml @@ -15,6 +15,7 @@ # RPM build is still not possible on CentOS7 as it does not # have the needed RPM macros for meson. machines: + - libvirt-centos-8 - libvirt-fedora-30 - libvirt-fedora-31 - libvirt-fedora-rawhide -- 2.23.0
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/host_vars/libvirt-centos-8/main.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/guests/host_vars/libvirt-centos-8/main.yml b/guests/host_vars/libvirt-centos-8/main.yml index 61ef995..e764964 100644 --- a/guests/host_vars/libvirt-centos-8/main.yml +++ b/guests/host_vars/libvirt-centos-8/main.yml @@ -8,6 +8,7 @@ projects: - libvirt-ocaml - libvirt-perl - libvirt-python + - osinfo-db - osinfo-db-tools - virt-viewer -- 2.23.0
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/host_vars/libvirt-centos-8/main.yml | 1 + guests/playbooks/build/projects/libosinfo.yml | 1 + jenkins/projects/libosinfo.yaml | 1 + 3 files changed, 3 insertions(+) diff --git a/guests/host_vars/libvirt-centos-8/main.yml b/guests/host_vars/libvirt-centos-8/main.yml index e764964..87234d6 100644 --- a/guests/host_vars/libvirt-centos-8/main.yml +++ b/guests/host_vars/libvirt-centos-8/main.yml @@ -1,5 +1,6 @@ --- projects: + - libosinfo - libvirt - libvirt-dbus - libvirt-glib diff --git a/guests/playbooks/build/projects/libosinfo.yml b/guests/playbooks/build/projects/libosinfo.yml index a759204..6391323 100644 --- a/guests/playbooks/build/projects/libosinfo.yml +++ b/guests/playbooks/build/projects/libosinfo.yml @@ -13,6 +13,7 @@ # RPM build is still not possible on CentOS7 as it does not # have the needed RPM macros for meson. machines: + - libvirt-centos-8 - libvirt-fedora-30 - libvirt-fedora-31 - libvirt-fedora-rawhide diff --git a/jenkins/projects/libosinfo.yaml b/jenkins/projects/libosinfo.yaml index cde10ec..a4a9797 100644 --- a/jenkins/projects/libosinfo.yaml +++ b/jenkins/projects/libosinfo.yaml @@ -17,6 +17,7 @@ # RPM build is still not possible on CentOS7 as it does not # have the needed RPM macros for meson. machines: + - libvirt-centos-8 - libvirt-fedora-30 - libvirt-fedora-31 - libvirt-fedora-rawhide -- 2.23.0
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/host_vars/libvirt-centos-8/main.yml | 1 + guests/playbooks/build/projects/virt-manager.yml | 3 +++ jenkins/projects/virt-manager.yaml | 3 +++ 3 files changed, 7 insertions(+) diff --git a/guests/host_vars/libvirt-centos-8/main.yml b/guests/host_vars/libvirt-centos-8/main.yml index 87234d6..aae2313 100644 --- a/guests/host_vars/libvirt-centos-8/main.yml +++ b/guests/host_vars/libvirt-centos-8/main.yml @@ -11,6 +11,7 @@ projects: - libvirt-python - osinfo-db - osinfo-db-tools + - virt-manager - virt-viewer package_format: 'rpm' diff --git a/guests/playbooks/build/projects/virt-manager.yml b/guests/playbooks/build/projects/virt-manager.yml index c0d4294..886cb73 100644 --- a/guests/playbooks/build/projects/virt-manager.yml +++ b/guests/playbooks/build/projects/virt-manager.yml @@ -5,6 +5,7 @@ # Ubuntu 16.04 has Python 3 but not the libxml2 bindings, so it can't # build the project either machines: + - libvirt-centos-8 - libvirt-debian-9 - libvirt-debian-10 - libvirt-debian-sid @@ -29,6 +30,7 @@ # so skip the test suite there for the time being. See # https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=224902 machines: + - libvirt-centos-8 - libvirt-debian-9 - libvirt-debian-10 - libvirt-debian-sid @@ -39,6 +41,7 @@ - include: '{{ playbook_base }}/jobs/python-distutils-rpm-job.yml' vars: machines: + - libvirt-centos-8 - libvirt-fedora-30 - libvirt-fedora-31 - libvirt-fedora-rawhide diff --git a/jenkins/projects/virt-manager.yaml b/jenkins/projects/virt-manager.yaml index 3dc8e2e..7106471 100644 --- a/jenkins/projects/virt-manager.yaml +++ b/jenkins/projects/virt-manager.yaml @@ -5,6 +5,7 @@ # Ubuntu 16.04 has Python 3 but not the libxml2 bindings, so it can't # build the project either machines: + - libvirt-centos-8 - libvirt-debian-9 - libvirt-debian-10 - libvirt-fedora-30 @@ -28,6 +29,7 @@ # so skip the test suite there for the time being. See # https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=224902 machines: + - libvirt-centos-8 - libvirt-debian-9 - libvirt-debian-10 - libvirt-fedora-30 @@ -36,6 +38,7 @@ - python-distutils-rpm-job: parent_jobs: 'virt-manager-check' machines: + - libvirt-centos-8 - libvirt-fedora-30 - libvirt-fedora-31 - libvirt-fedora-rawhide -- 2.23.0
participants (3)
-
Andrea Bolognani -
Daniel P. Berrangé -
Fabiano Fidêncio