[libvirt] [jenkins-ci PATCH v2 0/4] Fix virt-manager build failure

Changes from [v1]: * expand coverage instead of reducing it, which makes for a far more acceptable solution. [v1] https://www.redhat.com/archives/libvir-list/2018-February/msg00354.html Andrea Bolognani (4): jobs: Make Python version configurable projects: Build libvirt-python for Python 3 too projects: Build virt-manager using Python 3 guests: Fix Python installation guests/host_vars/libvirt-centos-7/main.yml | 1 - guests/host_vars/libvirt-debian-8/main.yml | 1 - guests/host_vars/libvirt-ubuntu-14/main.yml | 1 - guests/host_vars/libvirt-ubuntu-16/main.yml | 1 - guests/vars/mappings.yml | 50 +++++++++++++++++------------ guests/vars/projects/libvirt-python.yml | 2 ++ guests/vars/projects/virt-manager.yml | 8 ++--- jobs/python-distutils.yaml | 22 ++++++------- projects/libvirt-python.yaml | 29 +++++++++++++++-- projects/virt-manager.yaml | 14 ++++---- 10 files changed, 80 insertions(+), 49 deletions(-) -- 2.14.3

Some projects, namely libvirt-python, support Python 2 and Python 3, so we'd like to have CI coverage of both; other projects, namely virt-manager, no longer support Python 2 so we need to start building them using Python 3 instead. After this change, the names of jobs as well as workspaces include the Python version so that they can happily live side by side. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- jobs/python-distutils.yaml | 22 +++++++++++----------- projects/libvirt-python.yaml | 7 +++++-- projects/virt-manager.yaml | 11 +++++++---- 3 files changed, 23 insertions(+), 17 deletions(-) diff --git a/jobs/python-distutils.yaml b/jobs/python-distutils.yaml index 47b25f1..122c759 100644 --- a/jobs/python-distutils.yaml +++ b/jobs/python-distutils.yaml @@ -1,11 +1,11 @@ - job-template: id: python-distutils-build-job - name: '{name}-{branch}-build' + name: '{name}-{branch}-{python}-build' project-type: matrix - description: '{title} Build' + description: '{title} Build ({python})' command_pre_build: '' - workspace: '{name}-{branch}' + workspace: '{name}-{branch}-{python}' child-workspace: '.' block-downstream: true block-upstream: true @@ -43,8 +43,8 @@ {global_env} {local_env} {command_pre_build} - python2 setup.py build - python2 setup.py install --prefix=$VIRT_PREFIX + {python} ./setup.py build + {python} ./setup.py install --prefix=$VIRT_PREFIX publishers: - email: recipients: '{obj:spam}' @@ -54,9 +54,9 @@ - job-template: id: python-distutils-check-job - name: '{name}-{branch}-check' + name: '{name}-{branch}-{python}-check' project-type: matrix - description: '{title} Check' + description: '{title} Check ({python})' workspace: '{name}-{branch}' child-workspace: '.' block-downstream: true @@ -83,7 +83,7 @@ - shell: | {global_env} {local_env} - python2 setup.py test + {python} ./setup.py test publishers: - email: recipients: '{obj:spam}' @@ -92,9 +92,9 @@ - job-template: id: python-distutils-rpm-job - name: '{name}-{branch}-rpm' + name: '{name}-{branch}-{python}-rpm' project-type: matrix - description: '{title} RPM' + description: '{title} RPM ({python})' workspace: '{name}-{branch}' child-workspace: '.' block-downstream: true @@ -122,7 +122,7 @@ {global_env} {local_env} sed -i -e 's/BuildRequires: libvirt.*devel.*//' *.spec.in - python2 setup.py rpm + {python} ./setup.py rpm publishers: - email: recipients: '{obj:spam}' diff --git a/projects/libvirt-python.yaml b/projects/libvirt-python.yaml index dd8ac13..540238d 100644 --- a/projects/libvirt-python.yaml +++ b/projects/libvirt-python.yaml @@ -14,11 +14,14 @@ title: Libvirt Python jobs: - python-distutils-build-job: + python: python2 parent_jobs: 'libvirt-master-build' - python-distutils-check-job: - parent_jobs: 'libvirt-python-master-build' + python: python2 + parent_jobs: 'libvirt-python-master-{python}-build' - python-distutils-rpm-job: - parent_jobs: 'libvirt-python-master-check' + python: python2 + parent_jobs: 'libvirt-python-master-{python}-check' machines: - libvirt-centos-6 - libvirt-centos-7 diff --git a/projects/virt-manager.yaml b/projects/virt-manager.yaml index c1d198b..2d13c15 100644 --- a/projects/virt-manager.yaml +++ b/projects/virt-manager.yaml @@ -13,15 +13,18 @@ title: Virtual Machine Manager jobs: - python-distutils-build-job: + python: python2 parent_jobs: - - 'libvirt-python-master-build' + - 'libvirt-python-master-{python}-build' - 'libosinfo-master-build' command_pre_build: | - python2 setup.py configure --prefix=$VIRT_PREFIX + {python} ./setup.py configure --prefix=$VIRT_PREFIX - python-distutils-check-job: - parent_jobs: 'virt-manager-master-build' + python: python2 + parent_jobs: 'virt-manager-master-{python}-build' - python-distutils-rpm-job: - parent_jobs: 'virt-manager-master-check' + python: python2 + parent_jobs: 'virt-manager-master-{python}-check' machines: - libvirt-centos-7 - libvirt-fedora-26 -- 2.14.3

We don't need a second RPM job, because the existing one already builds both Python 2 and Python 3 RPMs if the OS supports it. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- projects/libvirt-python.yaml | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/projects/libvirt-python.yaml b/projects/libvirt-python.yaml index 540238d..e065c5d 100644 --- a/projects/libvirt-python.yaml +++ b/projects/libvirt-python.yaml @@ -16,9 +16,31 @@ - python-distutils-build-job: python: python2 parent_jobs: 'libvirt-master-build' + - python-distutils-build-job: + python: python3 + parent_jobs: 'libvirt-master-build' + machines: + - libvirt-debian-8 + - libvirt-debian-9 + - libvirt-fedora-26 + - libvirt-fedora-27 + - libvirt-fedora-rawhide + - libvirt-freebsd-10 + - libvirt-freebsd-11 - python-distutils-check-job: python: python2 parent_jobs: 'libvirt-python-master-{python}-build' + - python-distutils-check-job: + python: python3 + parent_jobs: 'libvirt-python-master-{python}-build' + machines: + - libvirt-debian-8 + - libvirt-debian-9 + - libvirt-fedora-26 + - libvirt-fedora-27 + - libvirt-fedora-rawhide + - libvirt-freebsd-10 + - libvirt-freebsd-11 - python-distutils-rpm-job: python: python2 parent_jobs: 'libvirt-python-master-{python}-check' -- 2.14.3

virt-manager is Python 3 only now, so wa can no longer build it using Python 2. That means installing the Python 3 version of all its dependencies and no longer building on OSs that don't ship Python 3, such as CentOS. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- guests/host_vars/libvirt-centos-7/main.yml | 1 - guests/host_vars/libvirt-debian-8/main.yml | 1 - guests/host_vars/libvirt-ubuntu-14/main.yml | 1 - guests/host_vars/libvirt-ubuntu-16/main.yml | 1 - guests/vars/mappings.yml | 41 ++++++++++++++++------------- guests/vars/projects/virt-manager.yml | 8 +++--- projects/virt-manager.yaml | 9 +++---- 7 files changed, 29 insertions(+), 33 deletions(-) diff --git a/guests/host_vars/libvirt-centos-7/main.yml b/guests/host_vars/libvirt-centos-7/main.yml index 30c826a..8338f99 100644 --- a/guests/host_vars/libvirt-centos-7/main.yml +++ b/guests/host_vars/libvirt-centos-7/main.yml @@ -13,5 +13,4 @@ projects: - libvirt-sandbox - osinfo-db - osinfo-db-tools - - virt-manager - virt-viewer diff --git a/guests/host_vars/libvirt-debian-8/main.yml b/guests/host_vars/libvirt-debian-8/main.yml index f097792..ecf9cd3 100644 --- a/guests/host_vars/libvirt-debian-8/main.yml +++ b/guests/host_vars/libvirt-debian-8/main.yml @@ -11,5 +11,4 @@ projects: - libvirt-python - libvirt-tck - osinfo-db-tools - - virt-manager - virt-viewer diff --git a/guests/host_vars/libvirt-ubuntu-14/main.yml b/guests/host_vars/libvirt-ubuntu-14/main.yml index 18b013d..c11dd5b 100644 --- a/guests/host_vars/libvirt-ubuntu-14/main.yml +++ b/guests/host_vars/libvirt-ubuntu-14/main.yml @@ -7,5 +7,4 @@ projects: - libvirt-python - libvirt-tck - osinfo-db-tools - - virt-manager - virt-viewer diff --git a/guests/host_vars/libvirt-ubuntu-16/main.yml b/guests/host_vars/libvirt-ubuntu-16/main.yml index 7baecf7..190f174 100644 --- a/guests/host_vars/libvirt-ubuntu-16/main.yml +++ b/guests/host_vars/libvirt-ubuntu-16/main.yml @@ -11,5 +11,4 @@ projects: - libvirt-sandbox - libvirt-tck - osinfo-db-tools - - virt-manager - virt-viewer diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index 440123c..579deb4 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -555,21 +555,6 @@ mappings: pkg: python27 rpm: python2-devel - python2-gi: - deb: python-gi - pkg: py27-gobject3 - rpm: python-gobject - CentOS6: - - python2-ipaddr: - default: python-ipaddr - FreeBSD: py27-ipaddr - - python2-libxml2: - default: python-libxml2 - pkg: py27-libxml2 - CentOS: libxml2-python - python2-lxml: default: python-lxml Fedora: python2-lxml @@ -580,16 +565,28 @@ mappings: Fedora: python2-nose FreeBSD: py27-nose - python2-requests: - default: python-requests - Fedora: python2-requests - FreeBSD: py27-requests + python3: + default: python3 + CentOS: python3-devel: deb: python3-dev pkg: python36 Fedora: python3-devel + python3-gi: + deb: python3-gi + pkg: py36-gobject3 + rpm: python3-gobject + CentOS: + + python3-libxml2: + default: python3-libxml2 + FreeBSD: py36-libxml2 + CentOS: + Ubuntu: + Debian8: + python3-lxml: default: python3-lxml FreeBSD: py36-lxml @@ -600,6 +597,12 @@ mappings: FreeBSD: py36-nose CentOS: + python3-requests: + default: python3-requests + FreeBSD: py36-requests + CentOS: + Ubuntu12: + qemu-img: default: qemu-utils rpm: qemu-img diff --git a/guests/vars/projects/virt-manager.yml b/guests/vars/projects/virt-manager.yml index 474017f..e9e5324 100644 --- a/guests/vars/projects/virt-manager.yml +++ b/guests/vars/projects/virt-manager.yml @@ -1,8 +1,8 @@ --- packages: - intltool - - python2-gi - - python2-ipaddr - - python2-libxml2 - - python2-requests + - python3 + - python3-gi + - python3-libxml2 + - python3-requests - unzip diff --git a/projects/virt-manager.yaml b/projects/virt-manager.yaml index 2d13c15..b5b0df5 100644 --- a/projects/virt-manager.yaml +++ b/projects/virt-manager.yaml @@ -2,8 +2,6 @@ - project: name: virt-manager machines: - - libvirt-centos-7 - - libvirt-debian-8 - libvirt-debian-9 - libvirt-fedora-26 - libvirt-fedora-27 @@ -13,20 +11,19 @@ title: Virtual Machine Manager jobs: - python-distutils-build-job: - python: python2 + python: python3 parent_jobs: - 'libvirt-python-master-{python}-build' - 'libosinfo-master-build' command_pre_build: | {python} ./setup.py configure --prefix=$VIRT_PREFIX - python-distutils-check-job: - python: python2 + python: python3 parent_jobs: 'virt-manager-master-{python}-build' - python-distutils-rpm-job: - python: python2 + python: python3 parent_jobs: 'virt-manager-master-{python}-check' machines: - - libvirt-centos-7 - libvirt-fedora-26 - libvirt-fedora-27 - libvirt-fedora-rawhide -- 2.14.3

Clean up the mappings to make sure we install the python2 and python3 packages on FreeBSD instead of python27 and python36, because the former are the ones that contain the respective command - which build jobs will invoke. Make the dependency on the above explicit for the libvirt-python jobs. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- guests/vars/mappings.yml | 9 +++++++-- guests/vars/projects/libvirt-python.yml | 2 ++ 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index 579deb4..e3781b9 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -550,9 +550,14 @@ mappings: pkg: polkit rpm: polkit-devel + python2: + default: python + Fedora: python2 + FreeBSD: python2 + python2-devel: deb: python-dev - pkg: python27 + pkg: python2 rpm: python2-devel python2-lxml: @@ -571,7 +576,7 @@ mappings: python3-devel: deb: python3-dev - pkg: python36 + pkg: python3 Fedora: python3-devel python3-gi: diff --git a/guests/vars/projects/libvirt-python.yml b/guests/vars/projects/libvirt-python.yml index 72dcb4a..774da26 100644 --- a/guests/vars/projects/libvirt-python.yml +++ b/guests/vars/projects/libvirt-python.yml @@ -1,8 +1,10 @@ --- packages: + - python2 - python2-devel - python2-lxml - python2-nose + - python3 - python3-devel - python3-lxml - python3-nose -- 2.14.3

On Thu, Feb 08, 2018 at 02:07:22PM +0100, Andrea Bolognani wrote:
Changes from [v1]:
* expand coverage instead of reducing it, which makes for a far more acceptable solution.
[v1] https://www.redhat.com/archives/libvir-list/2018-February/msg00354.html
Andrea Bolognani (4): jobs: Make Python version configurable projects: Build libvirt-python for Python 3 too projects: Build virt-manager using Python 3 guests: Fix Python installation
For all patches Reviewed-by: Daniel P. Berrange <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
participants (2)
-
Andrea Bolognani
-
Daniel P. Berrangé