For example to prevent Xen being installed on any s390x xen: deb: libxen-dev Fedora: xen-devel s390x-default: Or the inverse to only install Xen on x86_64 on Debian, but allow it on all archs on Fedora xen: Fedora: xen-devel x86_64-deb: libxen-dev Note that the architecture specific matches are considered after all the non-architcture matches. The dockerfile generator is updated to apply arch filtering based on the host arch. The ansible playbook is not actually implementing support for non-x86_64 architectures in this commit. It is just hardcoding x86_64, which is enough to ensure the changes in the mappings.yml file are a no-op initially. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com&gt; --- guests/lcitool | 16 +++++++++- guests/playbooks/update/tasks/packages.yml | 32 +++++++++++++++++++ guests/vars/mappings.yml | 37 ++++++++++++++++++++-- 3 files changed, 81 insertions(+), 4 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index 88bc945..d762721 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -20,6 +20,7 @@ import argparse import fnmatch import json import os +import platform import random import string import subprocess @@ -79,6 +80,16 @@ class Util: return sorted(set(matches)) + @staticmethod + def get_native_arch(): + # Same canonicalization as libvirt virArchFromHost + arch = platform.machine() + if arch in ["i386", "i486", "i586"]: + arch = "i686" + if arch == "amd64": + arch = "x86_64" + return arch + class Config: @@ -302,6 +313,8 @@ class Application: self._inventory = Inventory() self._projects = Projects() + self._native_arch = Util.get_native_arch() + self._parser = argparse.ArgumentParser( conflict_handler="resolve", description="libvirt CI guest management tool", @@ -540,7 +553,8 @@ class Application: ) pkgs = {} - keys = ["default", package_format, os_name, os_full] + base_keys = ["default", package_format, os_name, os_full] + keys = base_keys + [self._native_arch + "-" + k for k in base_keys] # We need to add the base project manually here: the standard # machinery hides it because it's an implementation detail for project in projects + ["base"]: diff --git a/guests/playbooks/update/tasks/packages.yml b/guests/playbooks/update/tasks/packages.yml index 7fdfc45..01d4616 100644 --- a/guests/playbooks/update/tasks/packages.yml +++ b/guests/playbooks/update/tasks/packages.yml @@ -52,6 +52,38 @@ when: - mappings[item][os_name + os_version] is defined +- name: '{{ project }}: Look up mappings (default with arch)' + set_fact: + temp: '{{ temp|combine({ item: mappings[item]["x86_64" + "-" + "default"] }) }}' + with_items: + '{{ packages }}' + when: + - mappings[item]["x86_64" + "-" + "default"] is defined + +- name: '{{ project }}: Look up mappings (package format with arch)' + set_fact: + temp: '{{ temp|combine({ item: mappings[item]["x86_64" + "-" + package_format] }) }}' + with_items: + '{{ packages }}' + when: + - mappings[item]["x86_64" + "-" + package_format] is defined + +- name: '{{ project }}: Look up mappings (OS name with arch)' + set_fact: + temp: '{{ temp|combine({ item: mappings[item]["x86_64" + "-" + os_name] }) }}' + with_items: + '{{ packages }}' + when: + - mappings[item]["x86_64" + "-" + os_name] is defined + +- name: '{{ project }}: Look up mappings (OS version with arch)' + set_fact: + temp: '{{ temp|combine({ item: mappings[item]["x86_64" + "-" + os_name + os_version] }) }}' + with_items: + '{{ packages }}' + when: + - mappings[item]["x86_64" + "-" + os_name + os_version] is defined + - set_fact: flattened: [] diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index 8ff2f34..f856cda 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -7,9 +7,25 @@ # priority: # # - default -# - package format (deb, pkg, rpm) -# - OS name (CentOS, Debian, Fedora, FreeBSD, Ubuntu) -# - OS version (CentOS7, Debian9, FedoraRawhide, Ubuntu18 and so on) +# - package format +# - OS name +# - OS version +# - arch with default +# - arch with package format +# - arch with OS name +# - arch with OS version +# +# Valid package formats are +# - deb, pkg, rpm +# +# Valid OS names are: +# - CentOS, Debian, Fedora, FreeBSD, Ubuntu +# +# Valid OS versions are: +# - CentOS7, Debian9, FedoraRawhide, Ubuntu18 and so on +# +# The arch specific rules use a prefix "$ARCH-" where $ARCH +# is a libvirt arch name. # # So something like # @@ -27,6 +43,21 @@ # # will result in the 'ccache' package being installed everywhere except # for CentOS, where nothing will be installed. +# +# For example to prevent Xen being installed on any s390x +# +# xen: +# deb: libxen-dev +# Fedora: xen-devel +# s390x-default: +# +# Or the inverse to only install Xen on x86_64 only on debian +# based distros or Fedora +# +# xen: +# x86_64-deb: libxen-dev +# x86_64-Fedora: xen-devel +# mappings: -- 2.20.1

Both libnuma and xen are only available on a subset of architectures so need to be filtered accordingly. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com&gt; --- guests/vars/mappings.yml | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index f856cda..4d06b83 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -310,6 +310,8 @@ mappings: libnuma: deb: libnuma-dev rpm: numactl-devel + armv6l-deb: + armv7l-deb: libparted: deb: libparted-dev @@ -852,8 +854,10 @@ mappings: Debian8: xen: - deb: libxen-dev Fedora: xen-devel + x86_64-deb: libxen-dev + armv7l-deb: libxen-dev + aarch64-deb: libxen-dev xfsprogs: deb: xfslibs-dev -- 2.20.1

Debian's filesystem layout has a nice advantage over Fedora which is that it can install non-native packages in the main root filesystem. It is thus possible to prepare an x86_64 filesystem containing -dev packages for a foreign architecture, along with a GCC cross compiler. QEMU has used this technique to facilitate developer build testing of non-x86 architectures, since few people have access to physical hardware for most of these architectures. For the same reason it would be helpful to libvirt developers. This patch extends the 'dockerfile' command to 'lcitool' so that it accepts a '-x $ARCH' argument. $ lcitool dockerfile -x s390x libvirt-debian-9 libvirt This is only valid when using Debian OS versions. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com&gt; --- guests/lcitool | 103 ++++++++++++++++++++++++++++++++++++++- guests/vars/mappings.yml | 7 +++ 2 files changed, 108 insertions(+), 2 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index d762721..9b723bb 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -90,6 +90,47 @@ class Util: arch = "x86_64" return arch + @staticmethod + def convert_native_arch_to_abi(native_arch): + archmap = { + "aarch64": "aarch64-linux-gnu", + "armv6l": "arm-linux-gnueabi", + "armv7l": "arm-linux-gnueabihf", + "i686": "i686-linux-gnu", + "mips": "mips-linux-gnu", + "mipsel": "mipsel-linux-gnu", + "mips64el": "mips64el-linux-gnuabi64", + "ppc64el": "powerpc64le-linux-gnu", + "s390x": "s390x-linux-gnu", + "x86_64": "x86_64-linux-gnu", + } + if native_arch not in archmap: + raise Exception("Unsupported architecture {}".format(native_arch)) + return archmap[native_arch] + + @staticmethod + def convert_native_arch_to_deb_arch(native_arch): + archmap = { + "aarch64": "arm64", + "armv6l": "armel", + "armv7l": "armhf", + "i686": "i386", + "mips": "mips", + "mipsel": "mipsel", + "mips64el": "mips64el", + "ppc64el": "ppc64el", + "s390x": "s390x", + "x86_64": "amd64", + } + if native_arch not in archmap: + raise Exception("Unsupported architecture {}".format(native_arch)) + return archmap[native_arch] + + @staticmethod + def convert_native_arch_to_deb_cross_gcc(native_arch): + abi = Util.convert_native_arch_to_abi(native_arch) + return "gcc-" + abi + class Config: @@ -341,6 +382,12 @@ class Application: help="git revision to build (remote/branch)", ) + def add_cross_arch_arg(parser): + parser.add_argument( + "-x", "--cross-arch", + help="cross compiler architecture", + ) + installparser = subparsers.add_parser( "install", help="perform unattended host installation") installparser.set_defaults(func=self._action_install) @@ -377,6 +424,7 @@ class Application: add_hosts_arg(dockerfileparser) add_projects_arg(dockerfileparser) + add_cross_arch_arg(dockerfileparser) def _execute_playbook(self, playbook, hosts, projects, git_revision): base = Util.get_base() @@ -541,6 +589,12 @@ class Application: if package_format not in ["deb", "rpm"]: raise Error("Host {} doesn't support Dockerfiles".format(host)) + if args.cross_arch: + if os_name != "Debian": + raise Error("Cannot cross compile on {}".format(os_name)) + if args.cross_arch == self._native_arch: + raise Error("Cross arch {} should differ from native {}". + format(args.cross_arch, self._native_arch)) projects = self._projects.expand_pattern(args.projects) for project in projects: @@ -553,19 +607,36 @@ class Application: ) pkgs = {} + cross_pkgs = {} base_keys = ["default", package_format, os_name, os_full] - keys = base_keys + [self._native_arch + "-" + k for k in base_keys] + cross_keys = [] + if args.cross_arch: + keys = base_keys + [args.cross_arch + "-" + k for k in base_keys] + cross_keys = ["cross-policy-" + k for k in base_keys] + else: + keys = base_keys + [self._native_arch + "-" + k for k in base_keys] + # We need to add the base project manually here: the standard # machinery hides it because it's an implementation detail for project in projects + ["base"]: for package in self._projects.get_packages(project): + cross_policy = "native" + for key in cross_keys: + if key in mappings[package]: + cross_policy = mappings[package][key] + if cross_policy not in ["native", "foreign", "skip"]: + raise Error("Unexpected cross arch policy {} for {}", + cross_policy, package) + for key in keys: if key in mappings[package]: pkgs[package] = mappings[package][key] if package not in pkgs: continue - if pkgs[package] is None: + if cross_policy == "foreign" and pkgs[package] is not None: + cross_pkgs[package] = pkgs[package] + if pkgs[package] is None or cross_policy in ["skip", "foreign"]: del pkgs[package] print("FROM {}".format(facts["docker_base"])) @@ -573,6 +644,15 @@ class Application: varmap = {} varmap["pkgs"] = " \\\n ".join(sorted(set(pkgs.values()))) if package_format == "deb": + if args.cross_arch: + deb_arch = Util.convert_native_arch_to_deb_arch(args.cross_arch) + gcc = Util.convert_native_arch_to_deb_cross_gcc(args.cross_arch) + varmap["cross_arch"] = deb_arch + pkg_names = [p + ":" + deb_arch for p in cross_pkgs.values()] + pkg_names.append(gcc) + varmap["cross_pkgs"] = " \\\n ".join(sorted(set(pkg_names))) + varmap["cross_abi"] = Util.convert_native_arch_to_abi(args.cross_arch) + sys.stdout.write(textwrap.dedent(""" RUN export DEBIAN_FRONTEND=noninteractive && \\ apt-get update && \\ @@ -582,6 +662,25 @@ class Application: apt-get autoremove -y && \\ apt-get autoclean -y """).format(**varmap)) + if args.cross_arch: + # Intentionally a separate RUN command from the above + # so that the common packages of all cross-built images + # share a docker image layer. + sys.stdout.write(textwrap.dedent(""" + RUN export DEBIAN_FRONTEND=noninteractive && \\ + dpkg --add-architecture {cross_arch} && \\ + apt-get update && \\ + apt-get dist-upgrade -y && \\ + apt-get install --no-install-recommends -y \\ + {cross_pkgs} && \\ + apt-get autoremove -y && \\ + apt-get autoclean -y + + ENV ABI "{cross_abi}" + ENV CONFIGURE_OPTS "--host={cross_abi} \\ + --target={cross_abi}" + ENV PKG_CONFIG_LIBDIR "/usr/lib/{cross_abi}/pkgconfig" + """).format(**varmap)) elif package_format == "rpm": if os_name == "Fedora" and os_version == "Rawhide": sys.stdout.write(textwrap.dedent(""" diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index 4d06b83..d7f2b28 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -58,6 +58,13 @@ # x86_64-deb: libxen-dev # x86_64-Fedora: xen-devel # +# In parallel with this 'cross-arch-XXX:' entries can used to set the +# installation policy when setting up a cross-architecture build env, +# taking one of the values: +# +# - 'native': use the native architecture package (default if omitted) +# - 'foreign: use the foreign archtiecture package +# - 'skip': don't install the package mappings: -- 2.20.1

All the -dev packages in Debian should use the foreign arch when setting up cross-builds, rather than than native arch package. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com&gt; --- guests/vars/mappings.yml | 58 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 57 insertions(+), 1 deletion(-) diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index d7f2b28..0e69bb2 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -58,7 +58,7 @@ # x86_64-deb: libxen-dev # x86_64-Fedora: xen-devel # -# In parallel with this 'cross-arch-XXX:' entries can used to set the +# In parallel with this 'cross-policy-XXX:' entries can used to set the # installation policy when setting up a cross-architecture build env, # taking one of the values: # @@ -70,6 +70,7 @@ mappings: apparmor: deb: libapparmor-dev + cross-policy-Debian: foreign augeas: default: augeas @@ -90,6 +91,7 @@ mappings: deb: libavahi-client-dev pkg: avahi rpm: avahi-devel + cross-policy-Debian: foreign bash: default: bash @@ -116,6 +118,7 @@ mappings: deb: libsasl2-dev pkg: cyrus-sasl rpm: cyrus-sasl-devel + cross-policy-Debian: foreign dbus-daemon: default: dbus @@ -125,6 +128,7 @@ mappings: device-mapper: deb: libdevmapper-dev rpm: device-mapper-devel + cross-policy-Debian: foreign dnsmasq: default: dnsmasq @@ -133,6 +137,7 @@ mappings: dtrace: deb: systemtap-sdt-dev rpm: systemtap-sdt-devel + cross-policy-Debian: skip dwarves: default: dwarves @@ -154,6 +159,7 @@ mappings: deb: libfuse-dev pkg: fusefs-libs rpm: fuse-devel + cross-policy-Debian: foreign gcc: default: gcc @@ -169,14 +175,17 @@ mappings: deb: libglib2.0-dev pkg: glib rpm: glib2-devel + cross-policy-Debian: foreign glibc: deb: libc6-dev rpm: glibc-devel + cross-policy-Debian: foreign glibc-static: deb: libc6-dev rpm: glibc-static + cross-policy-Debian: foreign glusterfs: deb: libglusterfs-dev @@ -185,6 +194,7 @@ mappings: Debian9: glusterfs-common Ubuntu16: glusterfs-common Ubuntu18: glusterfs-common + cross-policy-Debian: foreign gnome-common: default: gnome-common @@ -193,6 +203,7 @@ mappings: deb: libgnutls28-dev pkg: gnutls rpm: gnutls-devel + cross-policy-Debian: foreign go: default: golang @@ -202,11 +213,13 @@ mappings: deb: libgirepository1.0-dev pkg: gobject-introspection rpm: gobject-introspection-devel + cross-policy-Debian: foreign gtk3: deb: libgtk-3-dev pkg: gtk3 rpm: gtk3-devel + cross-policy-Debian: foreign gtk-doc: default: gtk-doc @@ -221,6 +234,7 @@ mappings: deb: libgtk-vnc-2.0-dev pkg: gtk-vnc rpm: gtk-vnc2-devel + cross-policy-Debian: foreign hal: FreeBSD: hal @@ -253,31 +267,38 @@ mappings: deb: libjson-glib-dev pkg: json-glib rpm: json-glib-devel + cross-policy-Debian: foreign libacl: deb: libacl1-dev rpm: libacl-devel + cross-policy-Debian: foreign libarchive: deb: libarchive-dev pkg: libarchive rpm: libarchive-devel + cross-policy-Debian: foreign libattr: deb: libattr1-dev rpm: libattr-devel + cross-policy-Debian: foreign libaudit: deb: libaudit-dev rpm: audit-libs-devel + cross-policy-Debian: foreign libblkid: deb: libblkid-dev rpm: libblkid-devel + cross-policy-Debian: foreign libcap-ng: deb: libcap-ng-dev rpm: libcap-ng-devel + cross-policy-Debian: foreign libcmpiutil: rpm: libcmpiutil-devel @@ -286,82 +307,99 @@ mappings: deb: libconfig-dev pkg: libconfig rpm: libconfig-devel + cross-policy-Debian: foreign libcurl: deb: libcurl4-gnutls-dev pkg: curl rpm: libcurl-devel + cross-policy-Debian: foreign libdbus: deb: libdbus-1-dev pkg: dbus rpm: dbus-devel + cross-policy-Debian: foreign libgovirt: rpm: libgovirt-devel Debian: libgovirt-dev Debian8: + cross-policy-Debianian: foreign libiscsi: deb: libiscsi-dev rpm: libiscsi-devel + cross-policy-Debian: foreign libnl3: deb: libnl-3-dev rpm: libnl3-devel + cross-policy-Debian: foreign libnlroute3: deb: libnl-route-3-dev rpm: libnl3-devel + cross-policy-Debian: foreign libnuma: deb: libnuma-dev rpm: numactl-devel armv6l-deb: armv7l-deb: + cross-policy-Debian: foreign libparted: deb: libparted-dev rpm: parted-devel + cross-policy-Debian: foreign libpcap: deb: libpcap0.8-dev pkg: libpcap rpm: libpcap-devel + cross-policy-Debian: foreign libpciaccess: deb: libpciaccess-dev pkg: libpciaccess rpm: libpciaccess-devel + cross-policy-Debian: foreign librbd: deb: librbd-dev Fedora: librbd-devel CentOS7: librbd1-devel + cross-policy-Debian: foreign libselinux: deb: libselinux1-dev rpm: libselinux-devel + cross-policy-Debian: foreign libsoup: deb: libsoup2.4-dev pkg: libsoup rpm: libsoup-devel + cross-policy-Debian: foreign libssh: pkg: libssh rpm: libssh-devel Debian: libssh-gcrypt-dev Ubuntu: libssh-dev + cross-policy-Debian: foreign libssh2: deb: libssh2-1-dev pkg: libssh2 rpm: libssh2-devel + cross-policy-Debian: foreign libtirpc: deb: libtirpc-dev rpm: libtirpc-devel + cross-policy-Debian: foreign libtool: default: libtool @@ -373,21 +411,25 @@ mappings: libudev: deb: libudev-dev rpm: libudev-devel + cross-policy-Debian: foreign libuuid: deb: uuid-dev pkg: e2fsprogs-libuuid rpm: libuuid-devel + cross-policy-Debian: foreign libxml2: deb: libxml2-dev pkg: libxml2 rpm: libxml2-devel + cross-policy-Debian: foreign libxslt: deb: libxslt1-dev pkg: libxslt rpm: libxslt-devel + cross-policy-Debian: foreign lvm2: default: lvm2 @@ -563,6 +605,7 @@ mappings: netcf: deb: libnetcf-dev rpm: netcf-devel + cross-policy-Debian: skip numad: default: numad @@ -724,6 +767,7 @@ mappings: deb: python-dev pkg: python2 rpm: python2-devel + cross-policy-Debian: foreign python2-lxml: default: python-lxml @@ -748,6 +792,7 @@ mappings: deb: python3-dev pkg: python3 Fedora: python3-devel + cross-policy-Debian: foreign python3-gi: deb: python3-gi @@ -793,6 +838,7 @@ mappings: deb: libreadline-dev pkg: readline rpm: readline-devel + cross-policy-Debian: foreign rpcgen: deb: libc-dev-bin @@ -806,6 +852,7 @@ mappings: sanlock: deb: libsanlock-dev rpm: sanlock-devel + cross-policy-Debian: foreign screen: default: screen @@ -828,6 +875,7 @@ mappings: deb: libspice-client-gtk-3.0-dev pkg: spice-gtk rpm: spice-gtk3-devel + cross-policy-Debian: foreign sudo: default: sudo @@ -859,16 +907,19 @@ mappings: deb: wireshark-dev Fedora: wireshark-devel Debian8: + cross-policy-Debian: skip xen: Fedora: xen-devel x86_64-deb: libxen-dev armv7l-deb: libxen-dev aarch64-deb: libxen-dev + cross-policy-Debian: foreign xfsprogs: deb: xfslibs-dev rpm: xfsprogs-devel + cross-policy-Debian: foreign xmllint: default: libxml2 @@ -881,15 +932,18 @@ mappings: xz: deb: liblzma-dev rpm: xz-devel + cross-policy-Debian: foreign xz-static: deb: liblzma-dev Fedora: xz-static + cross-policy-Debian: foreign yajl: deb: libyajl-dev pkg: yajl rpm: yajl-devel + cross-policy-Debian: foreign zfs: default: zfs-fuse @@ -899,7 +953,9 @@ mappings: zlib: deb: zlib1g-dev rpm: zlib-devel + cross-policy-Debian: foreign zlib-static: deb: zlib1g-dev rpm: zlib-static + cross-policy-Debian: foreign -- 2.20.1