[libvirt] [jenkins-ci PATCH v2 00/12] Meson, more meson!
The first 5 patches in the series are adding the machinery for installing meson via pip, which will be needed for systems which do not have the minimum required version, as CentOS 7, Debian 9, Ubuntu 16, and Ubuntu 18. Next 3 patches build and test osinfo-db-tools, osinfo-db, and libosinfo on all supported machines as those now have the minimum requirements needed. Following these patches, there's a patch re-enabling virt-manager builds on the systems which are, again, building libosinfo. Next 2 are basically re-enabling syntax-check job for libvirt-dbus on CentOS 7 and removing some misleading comment. Finally, last patch in the series does the libvirt-dbus builds switch to using meson instead of autotools. Fabiano Fidêncio (12): mappings: Add python2-setuptools guests: Add python2-setuptools to the base project mappings: Add python3-pip guests: Add python3-pip to the base project guests: Install meson via pip osinfo-db-tools: Build again on all machines osinfo-db: Build again on all machines libosinfo: Build again on all machines virt-manager: Build again on Debian9 and Ubuntu18 libvirt-dbus: Take advantage of Python 3 on CentOS 7 libvirt-dbus: Remove misleading comment Switch libvirt-dbus to Meson guests/host_vars/libvirt-centos-7/main.yml | 4 +++ guests/host_vars/libvirt-debian-9/main.yml | 4 +++ guests/host_vars/libvirt-ubuntu-16/main.yml | 4 +++ guests/host_vars/libvirt-ubuntu-18/main.yml | 4 +++ guests/lcitool | 17 +++++++++ guests/playbooks/build/projects/libosinfo.yml | 14 ++------ .../playbooks/build/projects/libvirt-dbus.yml | 28 +++++---------- .../build/projects/osinfo-db-tools.yml | 14 ++------ guests/playbooks/build/projects/osinfo-db.yml | 6 ++-- .../playbooks/build/projects/virt-manager.yml | 4 +++ guests/playbooks/update/tasks/packages.yml | 35 +++++++++++++++++++ guests/vars/mappings.yml | 15 ++++++++ guests/vars/projects/base.yml | 2 ++ jenkins/projects/libosinfo.yaml | 10 ++---- jenkins/projects/libvirt-dbus.yaml | 25 +++++-------- jenkins/projects/osinfo-db-tools.yaml | 12 ++----- jenkins/projects/osinfo-db.yaml | 13 ++----- jenkins/projects/virt-manager.yaml | 2 ++ 18 files changed, 125 insertions(+), 88 deletions(-) -- 2.23.0
python-setuptools, which will only be used on CentOS7, is needed in order to properly use the pip module for installing meson. Without this package, we would see failures as `ImportError: No module named pkg_resources` when trying to use the pip module, even when using its python3 version. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/vars/mappings.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index 5c32c94..8640d15 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -787,6 +787,9 @@ mappings: rpm: perl-generators CentOS7: + python2-setuptools: + CentOS7: python2-setuptools + pkg-config: default: pkgconf rpm: pkgconfig -- 2.23.0
On Tue, 2019-10-08 at 09:58 +0200, Fabiano Fidêncio wrote:
python-setuptools, which will only be used on CentOS7, is needed in order to properly use the pip module for installing meson.
Without this package, we would see failures as `ImportError: No module named pkg_resources` when trying to use the pip module, even when using its python3 version.
It's a bummer that we still can't leave Python 2 behind :(
+++ b/guests/vars/mappings.yml @@ -787,6 +787,9 @@ mappings: rpm: perl-generators CentOS7:
+ python2-setuptools: + CentOS7: python2-setuptools
This belongs further down, after python2-nose. With that fixed, Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/vars/projects/base.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/guests/vars/projects/base.yml b/guests/vars/projects/base.yml index f008de3..a8c83ac 100644 --- a/guests/vars/projects/base.yml +++ b/guests/vars/projects/base.yml @@ -25,6 +25,7 @@ packages: - patch - perl - pkg-config + - python2-setuptools - rpmbuild - screen - strace -- 2.23.0
On Tue, 2019-10-08 at 09:58 +0200, Fabiano Fidêncio wrote:
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/vars/projects/base.yml | 1 + 1 file changed, 1 insertion(+)
Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
pip will be used to install meson on OSes which do not have support to the minimum required version of the project. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> --- guests/vars/mappings.yml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index 8640d15..ec9e771 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -790,6 +790,12 @@ mappings: python2-setuptools: CentOS7: python2-setuptools + python3-pip: + CentOS7: python3-pip + Debian9: python3-pip + Ubuntu16: python3-pip + Ubuntu18: python3-pip + pkg-config: default: pkgconf rpm: pkgconfig -- 2.23.0
On Tue, 2019-10-08 at 09:58 +0200, Fabiano Fidêncio wrote:
+++ b/guests/vars/mappings.yml @@ -790,6 +790,12 @@ mappings: python2-setuptools: CentOS7: python2-setuptools
+ python3-pip: + CentOS7: python3-pip + Debian9: python3-pip + Ubuntu16: python3-pip + Ubuntu18: python3-pip
This also belong further down. With that fixed, Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> --- guests/vars/projects/base.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/guests/vars/projects/base.yml b/guests/vars/projects/base.yml index a8c83ac..ec04fdf 100644 --- a/guests/vars/projects/base.yml +++ b/guests/vars/projects/base.yml @@ -26,6 +26,7 @@ packages: - perl - pkg-config - python2-setuptools + - python3-pip - rpmbuild - screen - strace -- 2.23.0
Let's install meson via pip whenever meson is not available at all or does not match the minimum required version. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/lcitool | 17 +++++++++++ guests/playbooks/update/tasks/packages.yml | 35 ++++++++++++++++++++++ guests/vars/mappings.yml | 6 ++++ 3 files changed, 58 insertions(+) diff --git a/guests/lcitool b/guests/lcitool index 5b5b622..6a15c0c 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -303,6 +303,7 @@ class Projects: with open(mappings_path, "r") as infile: mappings = yaml.safe_load(infile) self._mappings = mappings["mappings"] + self._pip_mappings = mappings["pip_mappings"] except Exception as ex: raise Exception("Can't load mappings: {}".format(ex)) @@ -340,6 +341,9 @@ class Projects: def get_mappings(self): return self._mappings + def get_pip_mappings(self): + return self._pip_mappings + def get_packages(self, project): return self._packages[project] @@ -583,6 +587,7 @@ class Application: def _action_dockerfile(self, args): mappings = self._projects.get_mappings() + pip_mappings = self._projects.get_pip_mappings() hosts = self._inventory.expand_pattern(args.hosts) if len(hosts) > 1: @@ -617,6 +622,7 @@ class Application: pkgs = {} cross_pkgs = {} + pip_pkgs = {} base_keys = ["default", package_format, os_name, os_full] cross_keys = [] if args.cross_arch: @@ -641,21 +647,27 @@ class Application: for key in keys: if key in mappings[package]: pkgs[package] = mappings[package][key] + if package in pip_mappings and key in pip_mappings[package]: + pip_pkgs[package] = pip_mappings[package][key] if package not in pkgs: continue + if package in pip_pkgs and pkgs[package] is not None: + del pip_pkgs[package] if cross_policy == "foreign" and pkgs[package] is not None: cross_pkgs[package] = pkgs[package] if pkgs[package] is None or cross_policy in ["skip", "foreign"]: del pkgs[package] pkg_align = " \\\n" + (" " * len("RUN " + package_manager + " ")) + pip_pkg_align = " \\\n" + (" " * len("RUN pip3 install ")) print("FROM {}".format(facts["docker_base"])) varmap = {} varmap["package_manager"] = package_manager varmap["pkgs"] = pkg_align[1:] + pkg_align.join(sorted(set(pkgs.values()))) + varmap["pip_pkgs"] = pip_pkg_align[1:] + pip_pkg_align.join(sorted(set(pip_pkgs.values()))) if package_format == "deb": if args.cross_arch: deb_arch = Util.native_arch_to_deb_arch(args.cross_arch) @@ -714,6 +726,11 @@ class Application: {package_manager} clean all -y """).format(**varmap)) + if pip_pkgs: + sys.stdout.write(textwrap.dedent(""" + RUN pip3 install {pip_pkgs} + """).format(**varmap)) + def run(self): args = self._parser.parse_args() if args.debug: diff --git a/guests/playbooks/update/tasks/packages.yml b/guests/playbooks/update/tasks/packages.yml index ec8a4c4..5724c28 100644 --- a/guests/playbooks/update/tasks/packages.yml +++ b/guests/playbooks/update/tasks/packages.yml @@ -100,3 +100,38 @@ package: name: '{{ flattened|sort }}' state: '{{ state }}' + +- set_fact: + pip_temp: {} + +- name: '{{ project }}: Verify pip mappings' + fail: + msg: 'No mappings defined for {{ item }}' + with_items: + '{{ packages }}' + when: + - pip_mappings[item] is undefined and mappings[item] is undefined + +- name: '{{ project }}: Look up pip mappings (default)' + set_fact: + pip_temp: '{{ pip_temp|combine({ item: pip_mappings[item]["default"] }) }}' + with_items: + '{{ packages }}' + when: + - pip_mappings[item]["default"] is defined + +- set_fact: + pip_executable: pip3 + +- name: '{{ project }}: Install packages from pip' + pip: + name: '{{ pip_temp[item] }}' + executable: '{{ pip_executable }}' + state: '{{ state }}' + with_items: + '{{ packages }}' + when: + - temp[item] is defined + - temp[item] == None + - pip_temp[item] is defined + - pip_temp[item] != None diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index ec9e771..3907db4 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -1007,3 +1007,9 @@ mappings: deb: zlib1g-dev rpm: zlib-static cross-policy-deb: foreign + + +pip_mappings: + + meson: + default: meson==0.49.0 -- 2.23.0
On Tue, 2019-10-08 at 09:58 +0200, Fabiano Fidêncio wrote:
+++ b/guests/lcitool pkg_align = " \\\n" + (" " * len("RUN " + package_manager + " ")) + pip_pkg_align = " \\\n" + (" " * len("RUN pip3 install "))
s/pip3 install /pip3 /
+ if pip_pkgs: + sys.stdout.write(textwrap.dedent(""" + RUN pip3 install {pip_pkgs} + """).format(**varmap)) +
For cross Dockerfiles, this results in ENV directives being generated in between RUN directives, which is kinda messy. Please shuffle stuff around so that you avoid that.
+++ b/guests/playbooks/update/tasks/packages.yml +- name: '{{ project }}: Verify pip mappings' + fail: + msg: 'No mappings defined for {{ item }}' + with_items: + '{{ packages }}' + when: + - pip_mappings[item] is undefined and mappings[item] is undefined
This can be just when: - pip_mappings[item] is undefined since we checked the other part earlier.
+- set_fact: + pip_executable: pip3
You use this only once... No need to define a fact for it.
+- name: '{{ project }}: Install packages from pip' + pip: + name: '{{ pip_temp[item] }}' + executable: '{{ pip_executable }}' + state: '{{ state }}' + with_items: + '{{ packages }}' + when: + - temp[item] is defined + - temp[item] == None + - pip_temp[item] is defined + - pip_temp[item] != None
I would flatten and sort the list before acting on it, same as we do for native packages.
+++ b/guests/vars/mappings.yml +pip_mappings: + + meson: + default: meson==0.49.0
Please add a comment explaining how these mappings differ from the native ones. -- Andrea Bolognani / Red Hat / Virtualization
As now we've the needed packages for building and testing osinfo-db-tools on all systems supported by libvirt-jenkins-ci, let's build osinfo-db-tools everywhere. RPM build is still not possible in CentOS7 as it doesn't have the needed RPM macros for Meson. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> --- guests/host_vars/libvirt-centos-7/main.yml | 1 + guests/host_vars/libvirt-debian-9/main.yml | 1 + guests/host_vars/libvirt-ubuntu-16/main.yml | 1 + guests/host_vars/libvirt-ubuntu-18/main.yml | 1 + .../playbooks/build/projects/osinfo-db-tools.yml | 14 +++----------- jenkins/projects/osinfo-db-tools.yaml | 12 +++--------- 6 files changed, 10 insertions(+), 20 deletions(-) diff --git a/guests/host_vars/libvirt-centos-7/main.yml b/guests/host_vars/libvirt-centos-7/main.yml index 4e3ce85..908aeef 100644 --- a/guests/host_vars/libvirt-centos-7/main.yml +++ b/guests/host_vars/libvirt-centos-7/main.yml @@ -7,6 +7,7 @@ projects: - libvirt-ocaml - libvirt-perl - libvirt-python + - osinfo-db-tools - virt-viewer package_format: 'rpm' diff --git a/guests/host_vars/libvirt-debian-9/main.yml b/guests/host_vars/libvirt-debian-9/main.yml index c388ee8..054f4e0 100644 --- a/guests/host_vars/libvirt-debian-9/main.yml +++ b/guests/host_vars/libvirt-debian-9/main.yml @@ -10,6 +10,7 @@ projects: - libvirt-python - libvirt-sandbox - libvirt-tck + - osinfo-db-tools - virt-viewer package_format: 'deb' diff --git a/guests/host_vars/libvirt-ubuntu-16/main.yml b/guests/host_vars/libvirt-ubuntu-16/main.yml index bed11d4..89d4c96 100644 --- a/guests/host_vars/libvirt-ubuntu-16/main.yml +++ b/guests/host_vars/libvirt-ubuntu-16/main.yml @@ -10,6 +10,7 @@ projects: - libvirt-python - libvirt-sandbox - libvirt-tck + - osinfo-db-tools - virt-viewer package_format: 'deb' diff --git a/guests/host_vars/libvirt-ubuntu-18/main.yml b/guests/host_vars/libvirt-ubuntu-18/main.yml index 199b2bb..7ce6711 100644 --- a/guests/host_vars/libvirt-ubuntu-18/main.yml +++ b/guests/host_vars/libvirt-ubuntu-18/main.yml @@ -10,6 +10,7 @@ projects: - libvirt-python - libvirt-sandbox - libvirt-tck + - osinfo-db-tools - virt-viewer package_format: 'deb' diff --git a/guests/playbooks/build/projects/osinfo-db-tools.yml b/guests/playbooks/build/projects/osinfo-db-tools.yml index 67e5f00..68eb620 100644 --- a/guests/playbooks/build/projects/osinfo-db-tools.yml +++ b/guests/playbooks/build/projects/osinfo-db-tools.yml @@ -1,17 +1,7 @@ --- - set_fact: name: osinfo-db-tools - # osinfo-db-tools depends on meson 0.49.0, which is not available on - # CentOS 7, Debian 9, Ubuntu 18; - machines: - - libvirt-debian-10 - - libvirt-debian-sid - - libvirt-fedora-29 - - libvirt-fedora-30 - - libvirt-fedora-rawhide - - libvirt-freebsd-11 - - libvirt-freebsd-12 - - libvirt-freebsd-current + machines: '{{ all_machines }}' archive_format: xz git_url: '{{ git_urls["osinfo-db-tools"][git_remote] }}' @@ -20,6 +10,8 @@ - include: '{{ playbook_base }}/jobs/meson-check-job.yml' - include: '{{ playbook_base }}/jobs/meson-rpm-job.yml' vars: + # RPM build is still not possible on CentOS7 as it does not + # have the needed RPM macros for meson. machines: - libvirt-fedora-29 - libvirt-fedora-30 diff --git a/jenkins/projects/osinfo-db-tools.yaml b/jenkins/projects/osinfo-db-tools.yaml index 8609b46..121da37 100644 --- a/jenkins/projects/osinfo-db-tools.yaml +++ b/jenkins/projects/osinfo-db-tools.yaml @@ -1,15 +1,7 @@ --- - project: name: osinfo-db-tools - # osinfo-db-tools requires meson 0.49.0, which is not available on - # CentOS 7 and Debian 9; - machines: - - libvirt-debian-10 - - libvirt-fedora-29 - - libvirt-fedora-30 - - libvirt-fedora-rawhide - - libvirt-freebsd-11 - - libvirt-freebsd-12 + machines: '{all_machines}' title: osinfo database tools archive_format: xz git_url: '{git_urls[osinfo-db-tools][default]}' @@ -20,6 +12,8 @@ parent_jobs: 'osinfo-db-tools-build' - meson-rpm-job: parent_jobs: 'osinfo-db-tools-check' + # RPM build is still not possible on CentOS7 as it does not + # have the needed RPM macros for meson. machines: - libvirt-fedora-29 - libvirt-fedora-30 -- 2.23.0
As now we've the needed packages for building and testing osinfo-db on all systems supported by libvirt-jenkins-ci, let's build osinfo-db everywhere. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> --- guests/host_vars/libvirt-centos-7/main.yml | 1 + guests/host_vars/libvirt-debian-9/main.yml | 1 + guests/host_vars/libvirt-ubuntu-16/main.yml | 1 + guests/host_vars/libvirt-ubuntu-18/main.yml | 1 + guests/playbooks/build/projects/osinfo-db.yml | 6 ++---- jenkins/projects/osinfo-db.yaml | 13 ++----------- 6 files changed, 8 insertions(+), 15 deletions(-) diff --git a/guests/host_vars/libvirt-centos-7/main.yml b/guests/host_vars/libvirt-centos-7/main.yml index 908aeef..17f3123 100644 --- a/guests/host_vars/libvirt-centos-7/main.yml +++ b/guests/host_vars/libvirt-centos-7/main.yml @@ -7,6 +7,7 @@ projects: - libvirt-ocaml - libvirt-perl - libvirt-python + - osinfo-db - osinfo-db-tools - virt-viewer diff --git a/guests/host_vars/libvirt-debian-9/main.yml b/guests/host_vars/libvirt-debian-9/main.yml index 054f4e0..c6d774c 100644 --- a/guests/host_vars/libvirt-debian-9/main.yml +++ b/guests/host_vars/libvirt-debian-9/main.yml @@ -10,6 +10,7 @@ projects: - libvirt-python - libvirt-sandbox - libvirt-tck + - osinfo-db - osinfo-db-tools - virt-viewer diff --git a/guests/host_vars/libvirt-ubuntu-16/main.yml b/guests/host_vars/libvirt-ubuntu-16/main.yml index 89d4c96..7fc7411 100644 --- a/guests/host_vars/libvirt-ubuntu-16/main.yml +++ b/guests/host_vars/libvirt-ubuntu-16/main.yml @@ -10,6 +10,7 @@ projects: - libvirt-python - libvirt-sandbox - libvirt-tck + - osinfo-db - osinfo-db-tools - virt-viewer diff --git a/guests/host_vars/libvirt-ubuntu-18/main.yml b/guests/host_vars/libvirt-ubuntu-18/main.yml index 7ce6711..7197037 100644 --- a/guests/host_vars/libvirt-ubuntu-18/main.yml +++ b/guests/host_vars/libvirt-ubuntu-18/main.yml @@ -10,6 +10,7 @@ projects: - libvirt-python - libvirt-sandbox - libvirt-tck + - osinfo-db - osinfo-db-tools - virt-viewer diff --git a/guests/playbooks/build/projects/osinfo-db.yml b/guests/playbooks/build/projects/osinfo-db.yml index 1952827..be3fe39 100644 --- a/guests/playbooks/build/projects/osinfo-db.yml +++ b/guests/playbooks/build/projects/osinfo-db.yml @@ -21,14 +21,12 @@ $MAKE install OSINFO_DB_TARGET="--system" - include: '{{ playbook_base }}/jobs/generic-check-job.yml' vars: + machines: '{{ all_machines }}' command: | $MAKE check - include: '{{ playbook_base }}/jobs/generic-rpm-job.yml' vars: - machines: - - libvirt-fedora-29 - - libvirt-fedora-30 - - libvirt-fedora-rawhide + machines: '{{ rpm_machines }}' command: | {{ strip_buildrequires }} rpmbuild --clean --define "_topdir `pwd`/rpmbuild" --define "_sourcedir `pwd`" -ba osinfo-db.spec diff --git a/jenkins/projects/osinfo-db.yaml b/jenkins/projects/osinfo-db.yaml index 7074fd1..87b6748 100644 --- a/jenkins/projects/osinfo-db.yaml +++ b/jenkins/projects/osinfo-db.yaml @@ -1,13 +1,7 @@ --- - project: name: osinfo-db - machines: - - libvirt-debian-10 - - libvirt-fedora-29 - - libvirt-fedora-30 - - libvirt-fedora-rawhide - - libvirt-freebsd-11 - - libvirt-freebsd-12 + machines: '{all_machines}' title: osinfo database archive_format: xz git_url: '{git_urls[osinfo-db][default]}' @@ -23,10 +17,7 @@ $MAKE check - generic-rpm-job: parent_jobs: 'osinfo-db-check' - machines: - - libvirt-fedora-29 - - libvirt-fedora-30 - - libvirt-fedora-rawhide + machines: '{rpm_machines}' command: | {strip_buildrequires} rpmbuild --clean --define "_topdir `pwd`/rpmbuild" --define "_sourcedir `pwd`" -ba osinfo-db.spec -- 2.23.0
On Tue, 2019-10-08 at 09:58 +0200, Fabiano Fidêncio wrote:
+++ b/guests/playbooks/build/projects/osinfo-db.yml @@ -21,14 +21,12 @@ $MAKE install OSINFO_DB_TARGET="--system" - include: '{{ playbook_base }}/jobs/generic-check-job.yml' vars: + machines: '{{ all_machines }}' command: | $MAKE check
These changes are not correct: you got it right...
+++ b/jenkins/projects/osinfo-db.yaml @@ -1,13 +1,7 @@ --- - project: name: osinfo-db - machines: - - libvirt-debian-10 - - libvirt-fedora-29 - - libvirt-fedora-30 - - libvirt-fedora-rawhide - - libvirt-freebsd-11 - - libvirt-freebsd-12 + machines: '{all_machines}'
... in the Jenkins part, so just do the same thing in the Ansible part too. With that fixed, Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
As now we've the needed packages for building and testing libosinfo on all systems supported by libvirt-jenkins-ci, let's build libosinfo everywhere. RPM build is still not possible in CentOS7 as it doesn't have the needed RPM macros for Meson. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/host_vars/libvirt-centos-7/main.yml | 1 + guests/host_vars/libvirt-debian-9/main.yml | 1 + guests/host_vars/libvirt-ubuntu-16/main.yml | 1 + guests/host_vars/libvirt-ubuntu-18/main.yml | 1 + guests/playbooks/build/projects/libosinfo.yml | 14 +++----------- jenkins/projects/libosinfo.yaml | 10 +++------- 6 files changed, 10 insertions(+), 18 deletions(-) diff --git a/guests/host_vars/libvirt-centos-7/main.yml b/guests/host_vars/libvirt-centos-7/main.yml index 17f3123..94e29af 100644 --- a/guests/host_vars/libvirt-centos-7/main.yml +++ b/guests/host_vars/libvirt-centos-7/main.yml @@ -1,5 +1,6 @@ --- projects: + - libosinfo - libvirt - libvirt-cim - libvirt-dbus diff --git a/guests/host_vars/libvirt-debian-9/main.yml b/guests/host_vars/libvirt-debian-9/main.yml index c6d774c..6ac3808 100644 --- a/guests/host_vars/libvirt-debian-9/main.yml +++ b/guests/host_vars/libvirt-debian-9/main.yml @@ -1,5 +1,6 @@ --- projects: + - libosinfo - libvirt - libvirt-dbus - libvirt-glib diff --git a/guests/host_vars/libvirt-ubuntu-16/main.yml b/guests/host_vars/libvirt-ubuntu-16/main.yml index 7fc7411..179dd03 100644 --- a/guests/host_vars/libvirt-ubuntu-16/main.yml +++ b/guests/host_vars/libvirt-ubuntu-16/main.yml @@ -1,5 +1,6 @@ --- projects: + - libosinfo - libvirt - libvirt-dbus - libvirt-glib diff --git a/guests/host_vars/libvirt-ubuntu-18/main.yml b/guests/host_vars/libvirt-ubuntu-18/main.yml index 7197037..e9a5a7b 100644 --- a/guests/host_vars/libvirt-ubuntu-18/main.yml +++ b/guests/host_vars/libvirt-ubuntu-18/main.yml @@ -1,5 +1,6 @@ --- projects: + - libosinfo - libvirt - libvirt-dbus - libvirt-glib diff --git a/guests/playbooks/build/projects/libosinfo.yml b/guests/playbooks/build/projects/libosinfo.yml index f06dbb3..bb583c7 100644 --- a/guests/playbooks/build/projects/libosinfo.yml +++ b/guests/playbooks/build/projects/libosinfo.yml @@ -1,17 +1,7 @@ --- - set_fact: name: libosinfo - # libosinfo depends on meson 0.49.0, which is not available on - # CentOS 7, Debian 9, Ubuntu 18; - machines: - - libvirt-debian-10 - - libvirt-debian-sid - - libvirt-fedora-29 - - libvirt-fedora-30 - - libvirt-fedora-rawhide - - libvirt-freebsd-11 - - libvirt-freebsd-12 - - libvirt-freebsd-current + machines: '{{ all_machines }}' archive_format: xz git_url: '{{ git_urls["libosinfo"][git_remote] }}' @@ -20,6 +10,8 @@ - include: '{{ playbook_base }}/jobs/meson-check-job.yml' - include: '{{ playbook_base }}/jobs/meson-rpm-job.yml' vars: + # RPM build is still not possible on CentOS7 as it does not + # have the needed RPM macros for meson. machines: - libvirt-fedora-29 - libvirt-fedora-30 diff --git a/jenkins/projects/libosinfo.yaml b/jenkins/projects/libosinfo.yaml index b38524e..90fc465 100644 --- a/jenkins/projects/libosinfo.yaml +++ b/jenkins/projects/libosinfo.yaml @@ -3,13 +3,7 @@ name: libosinfo # libosinfo depends on meson 0.49.0, which is not available on # CentOS 7, Debian 9, Ubuntu 18; - machines: - - libvirt-debian-10 - - libvirt-fedora-29 - - libvirt-fedora-30 - - libvirt-fedora-rawhide - - libvirt-freebsd-11 - - libvirt-freebsd-12 + machines: '{all_machines}' title: libosinfo archive_format: xz git_url: '{git_urls[libosinfo][default]}' @@ -20,6 +14,8 @@ parent_jobs: 'libosinfo-build' - meson-rpm-job: parent_jobs: 'libosinfo-check' + # RPM build is still not possible on CentOS7 as it does not + # have the needed RPM macros for meson. machines: - libvirt-fedora-29 - libvirt-fedora-30 -- 2.23.0
On Tue, 2019-10-08 at 09:58 +0200, Fabiano Fidêncio wrote:
As now we've the needed packages for building and testing libosinfo on all systems supported by libvirt-jenkins-ci, let's build libosinfo everywhere.
RPM build is still not possible in CentOS7 as it doesn't have the needed RPM macros for Meson.
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/host_vars/libvirt-centos-7/main.yml | 1 + guests/host_vars/libvirt-debian-9/main.yml | 1 + guests/host_vars/libvirt-ubuntu-16/main.yml | 1 + guests/host_vars/libvirt-ubuntu-18/main.yml | 1 + guests/playbooks/build/projects/libosinfo.yml | 14 +++----------- jenkins/projects/libosinfo.yaml | 10 +++------- 6 files changed, 10 insertions(+), 18 deletions(-)
Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
As osinfo-db-tools, osindo-db, and libosinfo can now be built again on Debian9 and Ubuntu18, let's enable virt-manager builds on those OSes as well. This commit partilly reverts 570143fc41fa270. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/host_vars/libvirt-centos-7/main.yml | 1 + guests/host_vars/libvirt-debian-9/main.yml | 1 + guests/host_vars/libvirt-ubuntu-16/main.yml | 1 + guests/host_vars/libvirt-ubuntu-18/main.yml | 1 + guests/playbooks/build/projects/virt-manager.yml | 4 ++++ jenkins/projects/virt-manager.yaml | 2 ++ 6 files changed, 10 insertions(+) diff --git a/guests/host_vars/libvirt-centos-7/main.yml b/guests/host_vars/libvirt-centos-7/main.yml index 94e29af..49b0ed7 100644 --- a/guests/host_vars/libvirt-centos-7/main.yml +++ b/guests/host_vars/libvirt-centos-7/main.yml @@ -10,6 +10,7 @@ projects: - libvirt-python - osinfo-db - osinfo-db-tools + - virt-manager - virt-viewer package_format: 'rpm' diff --git a/guests/host_vars/libvirt-debian-9/main.yml b/guests/host_vars/libvirt-debian-9/main.yml index 6ac3808..6b685a4 100644 --- a/guests/host_vars/libvirt-debian-9/main.yml +++ b/guests/host_vars/libvirt-debian-9/main.yml @@ -13,6 +13,7 @@ projects: - libvirt-tck - osinfo-db - osinfo-db-tools + - virt-manager - virt-viewer package_format: 'deb' diff --git a/guests/host_vars/libvirt-ubuntu-16/main.yml b/guests/host_vars/libvirt-ubuntu-16/main.yml index 179dd03..6ac9505 100644 --- a/guests/host_vars/libvirt-ubuntu-16/main.yml +++ b/guests/host_vars/libvirt-ubuntu-16/main.yml @@ -13,6 +13,7 @@ projects: - libvirt-tck - osinfo-db - osinfo-db-tools + - virt-manager - virt-viewer package_format: 'deb' diff --git a/guests/host_vars/libvirt-ubuntu-18/main.yml b/guests/host_vars/libvirt-ubuntu-18/main.yml index e9a5a7b..5b5bf0c 100644 --- a/guests/host_vars/libvirt-ubuntu-18/main.yml +++ b/guests/host_vars/libvirt-ubuntu-18/main.yml @@ -13,6 +13,7 @@ projects: - libvirt-tck - osinfo-db - osinfo-db-tools + - virt-manager - virt-viewer package_format: 'deb' diff --git a/guests/playbooks/build/projects/virt-manager.yml b/guests/playbooks/build/projects/virt-manager.yml index 0078fbe..f955f4c 100644 --- a/guests/playbooks/build/projects/virt-manager.yml +++ b/guests/playbooks/build/projects/virt-manager.yml @@ -5,6 +5,7 @@ # Ubuntu 16.04 has Python 3 but not the libxml2 bindings, so it can't # build the project either machines: + - libvirt-debian-9 - libvirt-debian-10 - libvirt-debian-sid - libvirt-fedora-29 @@ -13,6 +14,7 @@ - libvirt-freebsd-11 - libvirt-freebsd-12 - libvirt-freebsd-current + - libvirt-ubuntu-18 archive_format: gz git_url: '{{ git_urls["virt-manager"][git_remote] }}' @@ -27,11 +29,13 @@ # so skip the test suite there for the time being. See # https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=224902 machines: + - libvirt-debian-9 - libvirt-debian-10 - libvirt-debian-sid - libvirt-fedora-29 - libvirt-fedora-30 - libvirt-fedora-rawhide + - libvirt-ubuntu-18 - include: '{{ playbook_base }}/jobs/python-distutils-rpm-job.yml' vars: machines: diff --git a/jenkins/projects/virt-manager.yaml b/jenkins/projects/virt-manager.yaml index fe6b903..2577ea9 100644 --- a/jenkins/projects/virt-manager.yaml +++ b/jenkins/projects/virt-manager.yaml @@ -5,6 +5,7 @@ # Ubuntu 16.04 has Python 3 but not the libxml2 bindings, so it can't # build the project either machines: + - libvirt-debian-9 - libvirt-debian-10 - libvirt-fedora-29 - libvirt-fedora-30 @@ -27,6 +28,7 @@ # so skip the test suite there for the time being. See # https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=224902 machines: + - libvirt-debian-9 - libvirt-debian-10 - libvirt-fedora-29 - libvirt-fedora-30 -- 2.23.0
On Tue, 2019-10-08 at 09:58 +0200, Fabiano Fidêncio wrote:
+++ b/guests/host_vars/libvirt-centos-7/main.yml @@ -10,6 +10,7 @@ projects: - libvirt-python - osinfo-db - osinfo-db-tools + - virt-manager - virt-viewer
This hunk...
+++ b/guests/host_vars/libvirt-ubuntu-16/main.yml @@ -13,6 +13,7 @@ projects: - libvirt-tck - osinfo-db - osinfo-db-tools + - virt-manager - virt-viewer
and this one are incorrect: we still can't build virt-manager on CentOS 7 and Ubuntu 16.04. Please drop them.
+++ b/guests/playbooks/build/projects/virt-manager.yml @@ -5,6 +5,7 @@ # virt-manager is Python 3 only, so it can't be built on CentOS 7; # Ubuntu 16.04 has Python 3 but not the libxml2 bindings, so it can't # build the project either machines: + - libvirt-debian-9 - libvirt-debian-10 - libvirt-debian-sid - libvirt-fedora-29
The comment should be updated, as the motivation for not building virt-manager is the same for both CentOS 7 and Ubuntu 16.04 now. With these issues addressed, Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
As latest CentOS 7 has added python3, CentOS 7 has support to python3, let's run syntax-check job the project. Also, let's update the comment explaining that we don't run `make check` on CentOS 7 because python3-pytest version is too old. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- guests/playbooks/build/projects/libvirt-dbus.yml | 8 ++++---- jenkins/projects/libvirt-dbus.yaml | 10 +++++----- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/guests/playbooks/build/projects/libvirt-dbus.yml b/guests/playbooks/build/projects/libvirt-dbus.yml index 218bcc1..7474d5f 100644 --- a/guests/playbooks/build/projects/libvirt-dbus.yml +++ b/guests/playbooks/build/projects/libvirt-dbus.yml @@ -9,10 +9,10 @@ - include: '{{ playbook_base }}/jobs/autotools-build-job.yml' - include: '{{ playbook_base }}/jobs/autotools-syntax-check-job.yml' vars: - # CentOS 7 doesn't include Python 3, while the versions of flake8 - # and pyflakes currently available on FreeBSD (3.5.0 and 2.0.0 + # flake8 and pyflakes versions available on FreeBSD (3.5.0 and 2.0.0 # respectively) are not compatible machines: + - libvirt-centos-7 - libvirt-debian-9 - libvirt-debian-10 - libvirt-debian-sid @@ -23,8 +23,8 @@ - libvirt-ubuntu-18 - include: '{{ playbook_base }}/jobs/autotools-check-job.yml' vars: - # CentOS 7 doesn't include Python 3 and the version in Ubuntu - # 16.04 is too old + # Python3 version in Ubuntu 16.04 and python3-pytest version + # in CentOS 7 are too old machines: - libvirt-debian-9 - libvirt-debian-10 diff --git a/jenkins/projects/libvirt-dbus.yaml b/jenkins/projects/libvirt-dbus.yaml index f7a52ef..1693d24 100644 --- a/jenkins/projects/libvirt-dbus.yaml +++ b/jenkins/projects/libvirt-dbus.yaml @@ -10,10 +10,10 @@ parent_jobs: 'libvirt-glib-build' - autotools-syntax-check-job: parent_jobs: 'libvirt-dbus-build' - # CentOS 7 doesn't include Python 3, while the versions of flake8 - # and pyflakes currently available on FreeBSD (3.5.0 and 2.0.0 - # respectively) are not compatible + # flake8 and pyflakes versions currently available on FreeBSD + # (3.5.0 and 2.0.0 respectively) are not compatible. machines: + - libvirt-centos-7 - libvirt-debian-9 - libvirt-debian-10 - libvirt-fedora-29 @@ -21,8 +21,8 @@ - libvirt-fedora-rawhide - autotools-check-job: parent_jobs: 'libvirt-dbus-syntax-check' - # CentOS 7 doesn't include Python 3 and the version in Ubuntu - # 16.04 is too old + # Python 3 version in Ubuntu 16.04 and python3-pytest version + # in CentOS 7 are too old machines: - libvirt-debian-9 - libvirt-debian-10 -- 2.23.0
On Tue, 2019-10-08 at 09:58 +0200, Fabiano Fidêncio wrote:
+++ b/guests/playbooks/build/projects/libvirt-dbus.yml @@ -9,10 +9,10 @@ - include: '{{ playbook_base }}/jobs/autotools-build-job.yml' - include: '{{ playbook_base }}/jobs/autotools-syntax-check-job.yml' vars: - # CentOS 7 doesn't include Python 3, while the versions of flake8 - # and pyflakes currently available on FreeBSD (3.5.0 and 2.0.0 + # flake8 and pyflakes versions available on FreeBSD (3.5.0 and 2.0.0 # respectively) are not compatible
This comment...
+++ b/jenkins/projects/libvirt-dbus.yaml @@ -10,10 +10,10 @@ parent_jobs: 'libvirt-glib-build' - autotools-syntax-check-job: parent_jobs: 'libvirt-dbus-build' - # CentOS 7 doesn't include Python 3, while the versions of flake8 - # and pyflakes currently available on FreeBSD (3.5.0 and 2.0.0 - # respectively) are not compatible + # flake8 and pyflakes versions currently available on FreeBSD + # (3.5.0 and 2.0.0 respectively) are not compatible.
... and this one don't match. Make sure they do. With that fixed, Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
Ubuntu is not a supported OS in jenkins. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- jenkins/projects/libvirt-dbus.yaml | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/jenkins/projects/libvirt-dbus.yaml b/jenkins/projects/libvirt-dbus.yaml index 1693d24..3accc5d 100644 --- a/jenkins/projects/libvirt-dbus.yaml +++ b/jenkins/projects/libvirt-dbus.yaml @@ -21,8 +21,7 @@ - libvirt-fedora-rawhide - autotools-check-job: parent_jobs: 'libvirt-dbus-syntax-check' - # Python 3 version in Ubuntu 16.04 and python3-pytest version - # in CentOS 7 are too old + # python3-pytest version in CentOS 7 is too old machines: - libvirt-debian-9 - libvirt-debian-10 -- 2.23.0
On Tue, 2019-10-08 at 09:58 +0200, Fabiano Fidêncio wrote:
+++ b/jenkins/projects/libvirt-dbus.yaml @@ -21,8 +21,7 @@ - libvirt-fedora-rawhide - autotools-check-job: parent_jobs: 'libvirt-dbus-syntax-check' - # Python 3 version in Ubuntu 16.04 and python3-pytest version - # in CentOS 7 are too old + # python3-pytest version in CentOS 7 is too old
We want the Ansible and Jenkins parts to be as close as possible, even when that means a comment or two are not accurate. NACK -- Andrea Bolognani / Red Hat / Virtualization
As libvirt-dbus has switched to using Meson instead of autotools, let's switch its jobs here. During the switch, `make check` and `make syntax-check` got merged into `ninja test`, which is reflected here by merging those two jobs. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> --- .../playbooks/build/projects/libvirt-dbus.yml | 26 ++++++------------- jenkins/projects/libvirt-dbus.yaml | 20 +++++--------- 2 files changed, 15 insertions(+), 31 deletions(-) diff --git a/guests/playbooks/build/projects/libvirt-dbus.yml b/guests/playbooks/build/projects/libvirt-dbus.yml index 7474d5f..f6b71e5 100644 --- a/guests/playbooks/build/projects/libvirt-dbus.yml +++ b/guests/playbooks/build/projects/libvirt-dbus.yml @@ -6,36 +6,26 @@ git_url: '{{ git_urls["libvirt-dbus"][git_remote] }}' - include: '{{ playbook_base }}/jobs/prepare.yml' -- include: '{{ playbook_base }}/jobs/autotools-build-job.yml' -- include: '{{ playbook_base }}/jobs/autotools-syntax-check-job.yml' +- include: '{{ playbook_base }}/jobs/meson-build-job.yml' +- include: '{{ playbook_base }}/jobs/meson-check-job.yml' vars: # flake8 and pyflakes versions available on FreeBSD (3.5.0 and 2.0.0 - # respectively) are not compatible + # respectively) are not compatible. + # Python3 version in Ubuntu 16.04 and python3-pytest version + # in CentOS 7 are too old machines: - - libvirt-centos-7 - libvirt-debian-9 - libvirt-debian-10 - libvirt-debian-sid - libvirt-fedora-29 - libvirt-fedora-30 - libvirt-fedora-rawhide - - libvirt-ubuntu-16 - libvirt-ubuntu-18 -- include: '{{ playbook_base }}/jobs/autotools-check-job.yml' +- include: '{{ playbook_base }}/jobs/meson-rpm-job.yml' vars: - # Python3 version in Ubuntu 16.04 and python3-pytest version - # in CentOS 7 are too old + # RPM build is still not possible on CentOS7 as it does not + # have the needed RPM macros for meson. machines: - - libvirt-debian-9 - - libvirt-debian-10 - - libvirt-debian-sid - libvirt-fedora-29 - libvirt-fedora-30 - libvirt-fedora-rawhide - - libvirt-freebsd-11 - - libvirt-freebsd-12 - - libvirt-freebsd-current - - libvirt-ubuntu-18 -- include: '{{ playbook_base }}/jobs/autotools-rpm-job.yml' - vars: - machines: '{{ rpm_machines }}' diff --git a/jenkins/projects/libvirt-dbus.yaml b/jenkins/projects/libvirt-dbus.yaml index 3accc5d..2637ff8 100644 --- a/jenkins/projects/libvirt-dbus.yaml +++ b/jenkins/projects/libvirt-dbus.yaml @@ -6,30 +6,24 @@ archive_format: xz git_url: '{git_urls[libvirt-dbus][default]}' jobs: - - autotools-build-job: + - meson-build-job: parent_jobs: 'libvirt-glib-build' - - autotools-syntax-check-job: + - meson-check-job: parent_jobs: 'libvirt-dbus-build' # flake8 and pyflakes versions currently available on FreeBSD # (3.5.0 and 2.0.0 respectively) are not compatible. + # pytest version is CentOS 7 is too old. machines: - - libvirt-centos-7 - libvirt-debian-9 - libvirt-debian-10 - libvirt-fedora-29 - libvirt-fedora-30 - libvirt-fedora-rawhide - - autotools-check-job: - parent_jobs: 'libvirt-dbus-syntax-check' - # python3-pytest version in CentOS 7 is too old + - meson-rpm-job: + parent_jobs: 'libvirt-dbus-check' + # RPM build is still not possible on CentOS7 as it does not + # have the needed RPM macros for meson. machines: - - libvirt-debian-9 - - libvirt-debian-10 - libvirt-fedora-29 - libvirt-fedora-30 - libvirt-fedora-rawhide - - libvirt-freebsd-11 - - libvirt-freebsd-12 - - autotools-rpm-job: - parent_jobs: 'libvirt-dbus-check' - machines: '{rpm_machines}' -- 2.23.0
On Tue, 2019-10-08 at 09:58 +0200, Fabiano Fidêncio wrote:
+++ b/guests/playbooks/build/projects/libvirt-dbus.yml vars: # flake8 and pyflakes versions available on FreeBSD (3.5.0 and 2.0.0 - # respectively) are not compatible + # respectively) are not compatible. + # Python3 version in Ubuntu 16.04 and python3-pytest version + # in CentOS 7 are too old
[...]
+++ b/jenkins/projects/libvirt-dbus.yaml @@ -6,30 +6,24 @@ + - meson-check-job: parent_jobs: 'libvirt-dbus-build' # flake8 and pyflakes versions currently available on FreeBSD # (3.5.0 and 2.0.0 respectively) are not compatible. + # pytest version is CentOS 7 is too old.
Make sure the comments match. With that fixed, and with the obvious caveat that we'll have to coordinate with Pavel so that this is merged at the same time as the libvirt-dbus patches, Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
participants (2)
-
Andrea Bolognani -
Fabiano Fidêncio