[libvirt] [jenkins-ci PATCH v6 0/4] Add support for cross compiling libvirt via Debian
Changed in v6: - Use a static mapping for arch info - Split out data file changes from code changes - Change arch suffix to a prefix Changed in v5: - Remove redundant formatter class - Update commit message - Update readme - Move misplaced patch chunk - Dont set "dest" in arg when not needed - Use metavar for subparser Changed in v4: - Pull in change to use argparse sub-parsers - Refactor way architecture specific package rules are stored to be in the main package mappings data Changed in v3: - Remove sheepdog more generally - Use .format() style printf - Split config to cross-build.yml - Make glusterfs name per-distro customized - Misc code style changes - Rename fields in cross-build.yml - Don't use crossbuild-essential packages Changed in v2: - Fix multiple package name mistakes - Modify lcitool to generate cross-arch docker files - Add --no-install-recommended flag to apt-get - Add DEBIAN_FRONTEND=noninteractive env to apt-get - Improve error reporting in lcitool - Add make rule for generating dockerfiles locally Daniel P. Berrangé (4): mappings: extend mapping to allow per-arch entries mappings: filter arch usage for libnuma / xen lcitool: support generating cross compiler dockerfiles mappings: mark packages using foreign arch for cross builds guests/lcitool | 117 ++++++++++++++++++++- guests/playbooks/update/tasks/packages.yml | 32 ++++++ guests/vars/mappings.yml | 106 ++++++++++++++++++- 3 files changed, 249 insertions(+), 6 deletions(-) -- 2.20.1
For example to prevent Xen being installed on any s390x xen: deb: libxen-dev Fedora: xen-devel s390x-default: Or the inverse to only install Xen on x86_64 on Debian, but allow it on all archs on Fedora xen: Fedora: xen-devel x86_64-deb: libxen-dev Note that the architecture specific matches are considered after all the non-architcture matches. The dockerfile generator is updated to apply arch filtering based on the host arch. The ansible playbook is not actually implementing support for non-x86_64 architectures in this commit. It is just hardcoding x86_64, which is enough to ensure the changes in the mappings.yml file are a no-op initially. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- guests/lcitool | 16 +++++++++- guests/playbooks/update/tasks/packages.yml | 32 +++++++++++++++++++ guests/vars/mappings.yml | 37 ++++++++++++++++++++-- 3 files changed, 81 insertions(+), 4 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index 88bc945..d762721 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -20,6 +20,7 @@ import argparse import fnmatch import json import os +import platform import random import string import subprocess @@ -79,6 +80,16 @@ class Util: return sorted(set(matches)) + @staticmethod + def get_native_arch(): + # Same canonicalization as libvirt virArchFromHost + arch = platform.machine() + if arch in ["i386", "i486", "i586"]: + arch = "i686" + if arch == "amd64": + arch = "x86_64" + return arch + class Config: @@ -302,6 +313,8 @@ class Application: self._inventory = Inventory() self._projects = Projects() + self._native_arch = Util.get_native_arch() + self._parser = argparse.ArgumentParser( conflict_handler="resolve", description="libvirt CI guest management tool", @@ -540,7 +553,8 @@ class Application: ) pkgs = {} - keys = ["default", package_format, os_name, os_full] + base_keys = ["default", package_format, os_name, os_full] + keys = base_keys + [self._native_arch + "-" + k for k in base_keys] # We need to add the base project manually here: the standard # machinery hides it because it's an implementation detail for project in projects + ["base"]: diff --git a/guests/playbooks/update/tasks/packages.yml b/guests/playbooks/update/tasks/packages.yml index 7fdfc45..01d4616 100644 --- a/guests/playbooks/update/tasks/packages.yml +++ b/guests/playbooks/update/tasks/packages.yml @@ -52,6 +52,38 @@ when: - mappings[item][os_name + os_version] is defined +- name: '{{ project }}: Look up mappings (default with arch)' + set_fact: + temp: '{{ temp|combine({ item: mappings[item]["x86_64" + "-" + "default"] }) }}' + with_items: + '{{ packages }}' + when: + - mappings[item]["x86_64" + "-" + "default"] is defined + +- name: '{{ project }}: Look up mappings (package format with arch)' + set_fact: + temp: '{{ temp|combine({ item: mappings[item]["x86_64" + "-" + package_format] }) }}' + with_items: + '{{ packages }}' + when: + - mappings[item]["x86_64" + "-" + package_format] is defined + +- name: '{{ project }}: Look up mappings (OS name with arch)' + set_fact: + temp: '{{ temp|combine({ item: mappings[item]["x86_64" + "-" + os_name] }) }}' + with_items: + '{{ packages }}' + when: + - mappings[item]["x86_64" + "-" + os_name] is defined + +- name: '{{ project }}: Look up mappings (OS version with arch)' + set_fact: + temp: '{{ temp|combine({ item: mappings[item]["x86_64" + "-" + os_name + os_version] }) }}' + with_items: + '{{ packages }}' + when: + - mappings[item]["x86_64" + "-" + os_name + os_version] is defined + - set_fact: flattened: [] diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index 8ff2f34..f856cda 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -7,9 +7,25 @@ # priority: # # - default -# - package format (deb, pkg, rpm) -# - OS name (CentOS, Debian, Fedora, FreeBSD, Ubuntu) -# - OS version (CentOS7, Debian9, FedoraRawhide, Ubuntu18 and so on) +# - package format +# - OS name +# - OS version +# - arch with default +# - arch with package format +# - arch with OS name +# - arch with OS version +# +# Valid package formats are +# - deb, pkg, rpm +# +# Valid OS names are: +# - CentOS, Debian, Fedora, FreeBSD, Ubuntu +# +# Valid OS versions are: +# - CentOS7, Debian9, FedoraRawhide, Ubuntu18 and so on +# +# The arch specific rules use a prefix "$ARCH-" where $ARCH +# is a libvirt arch name. # # So something like # @@ -27,6 +43,21 @@ # # will result in the 'ccache' package being installed everywhere except # for CentOS, where nothing will be installed. +# +# For example to prevent Xen being installed on any s390x +# +# xen: +# deb: libxen-dev +# Fedora: xen-devel +# s390x-default: +# +# Or the inverse to only install Xen on x86_64 only on debian +# based distros or Fedora +# +# xen: +# x86_64-deb: libxen-dev +# x86_64-Fedora: xen-devel +# mappings: -- 2.20.1
On Thu, 2019-02-28 at 15:53 +0000, Daniel P. Berrangé wrote:
For example to prevent Xen being installed on any s390x
xen: deb: libxen-dev Fedora: xen-devel s390x-default:
Or the inverse to only install Xen on x86_64 on Debian, but allow it on all archs on Fedora
xen: Fedora: xen-devel x86_64-deb: libxen-dev
It's not necessary to repeat the examples verbatim in the commit message, they're already in the diff. [...]
The ansible playbook is not actually implementing support for
s/ansible/Ansible/ [...]
+++ b/guests/playbooks/update/tasks/packages.yml @@ -52,6 +52,38 @@ when: - mappings[item][os_name + os_version] is defined
+- name: '{{ project }}: Look up mappings (default with arch)'
In the documentation you have used "arch with default" rather than "default with arch", so you might want to keep these messages consistent. [...]
@@ -27,6 +43,21 @@ # # will result in the 'ccache' package being installed everywhere except # for CentOS, where nothing will be installed. +# +# For example to prevent Xen being installed on any s390x
s/any //
+# +# xen: +# deb: libxen-dev +# Fedora: xen-devel +# s390x-default: +# +# Or the inverse to only install Xen on x86_64 only on debian +# based distros or Fedora
s/debian based/Debian-based/ With the above addressed, Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
Both libnuma and xen are only available on a subset of architectures so need to be filtered accordingly. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- guests/vars/mappings.yml | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index f856cda..4d06b83 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -310,6 +310,8 @@ mappings: libnuma: deb: libnuma-dev rpm: numactl-devel + armv6l-deb: + armv7l-deb: libparted: deb: libparted-dev @@ -852,8 +854,10 @@ mappings: Debian8: xen: - deb: libxen-dev Fedora: xen-devel + x86_64-deb: libxen-dev + armv7l-deb: libxen-dev + aarch64-deb: libxen-dev xfsprogs: deb: xfslibs-dev -- 2.20.1
On Thu, 2019-02-28 at 15:53 +0000, Daniel P. Berrangé wrote:
Both libnuma and xen are only available on a subset of architectures so need to be filtered accordingly.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- guests/vars/mappings.yml | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
Debian's filesystem layout has a nice advantage over Fedora which is that it can install non-native packages in the main root filesystem. It is thus possible to prepare an x86_64 filesystem containing -dev packages for a foreign architecture, along with a GCC cross compiler. QEMU has used this technique to facilitate developer build testing of non-x86 architectures, since few people have access to physical hardware for most of these architectures. For the same reason it would be helpful to libvirt developers. This patch extends the 'dockerfile' command to 'lcitool' so that it accepts a '-x $ARCH' argument. $ lcitool dockerfile -x s390x libvirt-debian-9 libvirt This is only valid when using Debian OS versions. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- guests/lcitool | 103 ++++++++++++++++++++++++++++++++++++++- guests/vars/mappings.yml | 7 +++ 2 files changed, 108 insertions(+), 2 deletions(-) diff --git a/guests/lcitool b/guests/lcitool index d762721..9b723bb 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -90,6 +90,47 @@ class Util: arch = "x86_64" return arch + @staticmethod + def convert_native_arch_to_abi(native_arch): + archmap = { + "aarch64": "aarch64-linux-gnu", + "armv6l": "arm-linux-gnueabi", + "armv7l": "arm-linux-gnueabihf", + "i686": "i686-linux-gnu", + "mips": "mips-linux-gnu", + "mipsel": "mipsel-linux-gnu", + "mips64el": "mips64el-linux-gnuabi64", + "ppc64el": "powerpc64le-linux-gnu", + "s390x": "s390x-linux-gnu", + "x86_64": "x86_64-linux-gnu", + } + if native_arch not in archmap: + raise Exception("Unsupported architecture {}".format(native_arch)) + return archmap[native_arch] + + @staticmethod + def convert_native_arch_to_deb_arch(native_arch): + archmap = { + "aarch64": "arm64", + "armv6l": "armel", + "armv7l": "armhf", + "i686": "i386", + "mips": "mips", + "mipsel": "mipsel", + "mips64el": "mips64el", + "ppc64el": "ppc64el", + "s390x": "s390x", + "x86_64": "amd64", + } + if native_arch not in archmap: + raise Exception("Unsupported architecture {}".format(native_arch)) + return archmap[native_arch] + + @staticmethod + def convert_native_arch_to_deb_cross_gcc(native_arch): + abi = Util.convert_native_arch_to_abi(native_arch) + return "gcc-" + abi + class Config: @@ -341,6 +382,12 @@ class Application: help="git revision to build (remote/branch)", ) + def add_cross_arch_arg(parser): + parser.add_argument( + "-x", "--cross-arch", + help="cross compiler architecture", + ) + installparser = subparsers.add_parser( "install", help="perform unattended host installation") installparser.set_defaults(func=self._action_install) @@ -377,6 +424,7 @@ class Application: add_hosts_arg(dockerfileparser) add_projects_arg(dockerfileparser) + add_cross_arch_arg(dockerfileparser) def _execute_playbook(self, playbook, hosts, projects, git_revision): base = Util.get_base() @@ -541,6 +589,12 @@ class Application: if package_format not in ["deb", "rpm"]: raise Error("Host {} doesn't support Dockerfiles".format(host)) + if args.cross_arch: + if os_name != "Debian": + raise Error("Cannot cross compile on {}".format(os_name)) + if args.cross_arch == self._native_arch: + raise Error("Cross arch {} should differ from native {}". + format(args.cross_arch, self._native_arch)) projects = self._projects.expand_pattern(args.projects) for project in projects: @@ -553,19 +607,36 @@ class Application: ) pkgs = {} + cross_pkgs = {} base_keys = ["default", package_format, os_name, os_full] - keys = base_keys + [self._native_arch + "-" + k for k in base_keys] + cross_keys = [] + if args.cross_arch: + keys = base_keys + [args.cross_arch + "-" + k for k in base_keys] + cross_keys = ["cross-policy-" + k for k in base_keys] + else: + keys = base_keys + [self._native_arch + "-" + k for k in base_keys] + # We need to add the base project manually here: the standard # machinery hides it because it's an implementation detail for project in projects + ["base"]: for package in self._projects.get_packages(project): + cross_policy = "native" + for key in cross_keys: + if key in mappings[package]: + cross_policy = mappings[package][key] + if cross_policy not in ["native", "foreign", "skip"]: + raise Error("Unexpected cross arch policy {} for {}", + cross_policy, package) + for key in keys: if key in mappings[package]: pkgs[package] = mappings[package][key] if package not in pkgs: continue - if pkgs[package] is None: + if cross_policy == "foreign" and pkgs[package] is not None: + cross_pkgs[package] = pkgs[package] + if pkgs[package] is None or cross_policy in ["skip", "foreign"]: del pkgs[package] print("FROM {}".format(facts["docker_base"])) @@ -573,6 +644,15 @@ class Application: varmap = {} varmap["pkgs"] = " \\\n ".join(sorted(set(pkgs.values()))) if package_format == "deb": + if args.cross_arch: + deb_arch = Util.convert_native_arch_to_deb_arch(args.cross_arch) + gcc = Util.convert_native_arch_to_deb_cross_gcc(args.cross_arch) + varmap["cross_arch"] = deb_arch + pkg_names = [p + ":" + deb_arch for p in cross_pkgs.values()] + pkg_names.append(gcc) + varmap["cross_pkgs"] = " \\\n ".join(sorted(set(pkg_names))) + varmap["cross_abi"] = Util.convert_native_arch_to_abi(args.cross_arch) + sys.stdout.write(textwrap.dedent(""" RUN export DEBIAN_FRONTEND=noninteractive && \\ apt-get update && \\ @@ -582,6 +662,25 @@ class Application: apt-get autoremove -y && \\ apt-get autoclean -y """).format(**varmap)) + if args.cross_arch: + # Intentionally a separate RUN command from the above + # so that the common packages of all cross-built images + # share a docker image layer. + sys.stdout.write(textwrap.dedent(""" + RUN export DEBIAN_FRONTEND=noninteractive && \\ + dpkg --add-architecture {cross_arch} && \\ + apt-get update && \\ + apt-get dist-upgrade -y && \\ + apt-get install --no-install-recommends -y \\ + {cross_pkgs} && \\ + apt-get autoremove -y && \\ + apt-get autoclean -y + + ENV ABI "{cross_abi}" + ENV CONFIGURE_OPTS "--host={cross_abi} \\ + --target={cross_abi}" + ENV PKG_CONFIG_LIBDIR "/usr/lib/{cross_abi}/pkgconfig" + """).format(**varmap)) elif package_format == "rpm": if os_name == "Fedora" and os_version == "Rawhide": sys.stdout.write(textwrap.dedent(""" diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index 4d06b83..d7f2b28 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -58,6 +58,13 @@ # x86_64-deb: libxen-dev # x86_64-Fedora: xen-devel # +# In parallel with this 'cross-arch-XXX:' entries can used to set the +# installation policy when setting up a cross-architecture build env, +# taking one of the values: +# +# - 'native': use the native architecture package (default if omitted) +# - 'foreign: use the foreign archtiecture package +# - 'skip': don't install the package mappings: -- 2.20.1
On Thu, 2019-02-28 at 15:53 +0000, Daniel P. Berrangé wrote: [...]
+ @staticmethod + def convert_native_arch_to_abi(native_arch):
I'd s/convert_// for all these small helper functions.
+ archmap = { + "aarch64": "aarch64-linux-gnu",
Some extra whitespace sneaked in, both here...
+ "armv6l": "arm-linux-gnueabi", + "armv7l": "arm-linux-gnueabihf", + "i686": "i686-linux-gnu",
... and here. [...]
+ @staticmethod + def convert_native_arch_to_deb_cross_gcc(native_arch): + abi = Util.convert_native_arch_to_abi(native_arch) + return "gcc-" + abi
This one we can open-code, I think. It's absolutely trivial and we use it only once. [...]
+ def add_cross_arch_arg(parser): + parser.add_argument( + "-x", "--cross-arch", + help="cross compiler architecture",
"target architecture for cross compilation"? [...]
+ # Intentionally a separate RUN command from the above + # so that the common packages of all cross-built images + # share a docker image layer.
s/docker/Docker/ [...]
+ ENV PKG_CONFIG_LIBDIR "/usr/lib/{cross_abi}/pkgconfig"
We discussed elsewhere how this can't work for i686, but I'm okay with merging this version of the patchset and figuring out a way to address that in a follow-up. [...]
+++ b/guests/vars/mappings.yml @@ -58,6 +58,13 @@ # x86_64-deb: libxen-dev # x86_64-Fedora: xen-devel # +# In parallel with this 'cross-arch-XXX:' entries can used to set the
This is now called 'cross-policy-XXX', and you have a hunk updating the documentation accordingly in the next patch. Please make sure it's it *this* patch instead :) With the above addressed, Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
All the -dev packages in Debian should use the foreign arch when setting up cross-builds, rather than than native arch package. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- guests/vars/mappings.yml | 58 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 57 insertions(+), 1 deletion(-) diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index d7f2b28..0e69bb2 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -58,7 +58,7 @@ # x86_64-deb: libxen-dev # x86_64-Fedora: xen-devel # -# In parallel with this 'cross-arch-XXX:' entries can used to set the +# In parallel with this 'cross-policy-XXX:' entries can used to set the # installation policy when setting up a cross-architecture build env, # taking one of the values: # @@ -70,6 +70,7 @@ mappings: apparmor: deb: libapparmor-dev + cross-policy-Debian: foreign augeas: default: augeas @@ -90,6 +91,7 @@ mappings: deb: libavahi-client-dev pkg: avahi rpm: avahi-devel + cross-policy-Debian: foreign bash: default: bash @@ -116,6 +118,7 @@ mappings: deb: libsasl2-dev pkg: cyrus-sasl rpm: cyrus-sasl-devel + cross-policy-Debian: foreign dbus-daemon: default: dbus @@ -125,6 +128,7 @@ mappings: device-mapper: deb: libdevmapper-dev rpm: device-mapper-devel + cross-policy-Debian: foreign dnsmasq: default: dnsmasq @@ -133,6 +137,7 @@ mappings: dtrace: deb: systemtap-sdt-dev rpm: systemtap-sdt-devel + cross-policy-Debian: skip dwarves: default: dwarves @@ -154,6 +159,7 @@ mappings: deb: libfuse-dev pkg: fusefs-libs rpm: fuse-devel + cross-policy-Debian: foreign gcc: default: gcc @@ -169,14 +175,17 @@ mappings: deb: libglib2.0-dev pkg: glib rpm: glib2-devel + cross-policy-Debian: foreign glibc: deb: libc6-dev rpm: glibc-devel + cross-policy-Debian: foreign glibc-static: deb: libc6-dev rpm: glibc-static + cross-policy-Debian: foreign glusterfs: deb: libglusterfs-dev @@ -185,6 +194,7 @@ mappings: Debian9: glusterfs-common Ubuntu16: glusterfs-common Ubuntu18: glusterfs-common + cross-policy-Debian: foreign gnome-common: default: gnome-common @@ -193,6 +203,7 @@ mappings: deb: libgnutls28-dev pkg: gnutls rpm: gnutls-devel + cross-policy-Debian: foreign go: default: golang @@ -202,11 +213,13 @@ mappings: deb: libgirepository1.0-dev pkg: gobject-introspection rpm: gobject-introspection-devel + cross-policy-Debian: foreign gtk3: deb: libgtk-3-dev pkg: gtk3 rpm: gtk3-devel + cross-policy-Debian: foreign gtk-doc: default: gtk-doc @@ -221,6 +234,7 @@ mappings: deb: libgtk-vnc-2.0-dev pkg: gtk-vnc rpm: gtk-vnc2-devel + cross-policy-Debian: foreign hal: FreeBSD: hal @@ -253,31 +267,38 @@ mappings: deb: libjson-glib-dev pkg: json-glib rpm: json-glib-devel + cross-policy-Debian: foreign libacl: deb: libacl1-dev rpm: libacl-devel + cross-policy-Debian: foreign libarchive: deb: libarchive-dev pkg: libarchive rpm: libarchive-devel + cross-policy-Debian: foreign libattr: deb: libattr1-dev rpm: libattr-devel + cross-policy-Debian: foreign libaudit: deb: libaudit-dev rpm: audit-libs-devel + cross-policy-Debian: foreign libblkid: deb: libblkid-dev rpm: libblkid-devel + cross-policy-Debian: foreign libcap-ng: deb: libcap-ng-dev rpm: libcap-ng-devel + cross-policy-Debian: foreign libcmpiutil: rpm: libcmpiutil-devel @@ -286,82 +307,99 @@ mappings: deb: libconfig-dev pkg: libconfig rpm: libconfig-devel + cross-policy-Debian: foreign libcurl: deb: libcurl4-gnutls-dev pkg: curl rpm: libcurl-devel + cross-policy-Debian: foreign libdbus: deb: libdbus-1-dev pkg: dbus rpm: dbus-devel + cross-policy-Debian: foreign libgovirt: rpm: libgovirt-devel Debian: libgovirt-dev Debian8: + cross-policy-Debianian: foreign libiscsi: deb: libiscsi-dev rpm: libiscsi-devel + cross-policy-Debian: foreign libnl3: deb: libnl-3-dev rpm: libnl3-devel + cross-policy-Debian: foreign libnlroute3: deb: libnl-route-3-dev rpm: libnl3-devel + cross-policy-Debian: foreign libnuma: deb: libnuma-dev rpm: numactl-devel armv6l-deb: armv7l-deb: + cross-policy-Debian: foreign libparted: deb: libparted-dev rpm: parted-devel + cross-policy-Debian: foreign libpcap: deb: libpcap0.8-dev pkg: libpcap rpm: libpcap-devel + cross-policy-Debian: foreign libpciaccess: deb: libpciaccess-dev pkg: libpciaccess rpm: libpciaccess-devel + cross-policy-Debian: foreign librbd: deb: librbd-dev Fedora: librbd-devel CentOS7: librbd1-devel + cross-policy-Debian: foreign libselinux: deb: libselinux1-dev rpm: libselinux-devel + cross-policy-Debian: foreign libsoup: deb: libsoup2.4-dev pkg: libsoup rpm: libsoup-devel + cross-policy-Debian: foreign libssh: pkg: libssh rpm: libssh-devel Debian: libssh-gcrypt-dev Ubuntu: libssh-dev + cross-policy-Debian: foreign libssh2: deb: libssh2-1-dev pkg: libssh2 rpm: libssh2-devel + cross-policy-Debian: foreign libtirpc: deb: libtirpc-dev rpm: libtirpc-devel + cross-policy-Debian: foreign libtool: default: libtool @@ -373,21 +411,25 @@ mappings: libudev: deb: libudev-dev rpm: libudev-devel + cross-policy-Debian: foreign libuuid: deb: uuid-dev pkg: e2fsprogs-libuuid rpm: libuuid-devel + cross-policy-Debian: foreign libxml2: deb: libxml2-dev pkg: libxml2 rpm: libxml2-devel + cross-policy-Debian: foreign libxslt: deb: libxslt1-dev pkg: libxslt rpm: libxslt-devel + cross-policy-Debian: foreign lvm2: default: lvm2 @@ -563,6 +605,7 @@ mappings: netcf: deb: libnetcf-dev rpm: netcf-devel + cross-policy-Debian: skip numad: default: numad @@ -724,6 +767,7 @@ mappings: deb: python-dev pkg: python2 rpm: python2-devel + cross-policy-Debian: foreign python2-lxml: default: python-lxml @@ -748,6 +792,7 @@ mappings: deb: python3-dev pkg: python3 Fedora: python3-devel + cross-policy-Debian: foreign python3-gi: deb: python3-gi @@ -793,6 +838,7 @@ mappings: deb: libreadline-dev pkg: readline rpm: readline-devel + cross-policy-Debian: foreign rpcgen: deb: libc-dev-bin @@ -806,6 +852,7 @@ mappings: sanlock: deb: libsanlock-dev rpm: sanlock-devel + cross-policy-Debian: foreign screen: default: screen @@ -828,6 +875,7 @@ mappings: deb: libspice-client-gtk-3.0-dev pkg: spice-gtk rpm: spice-gtk3-devel + cross-policy-Debian: foreign sudo: default: sudo @@ -859,16 +907,19 @@ mappings: deb: wireshark-dev Fedora: wireshark-devel Debian8: + cross-policy-Debian: skip xen: Fedora: xen-devel x86_64-deb: libxen-dev armv7l-deb: libxen-dev aarch64-deb: libxen-dev + cross-policy-Debian: foreign xfsprogs: deb: xfslibs-dev rpm: xfsprogs-devel + cross-policy-Debian: foreign xmllint: default: libxml2 @@ -881,15 +932,18 @@ mappings: xz: deb: liblzma-dev rpm: xz-devel + cross-policy-Debian: foreign xz-static: deb: liblzma-dev Fedora: xz-static + cross-policy-Debian: foreign yajl: deb: libyajl-dev pkg: yajl rpm: yajl-devel + cross-policy-Debian: foreign zfs: default: zfs-fuse @@ -899,7 +953,9 @@ mappings: zlib: deb: zlib1g-dev rpm: zlib-devel + cross-policy-Debian: foreign zlib-static: deb: zlib1g-dev rpm: zlib-static + cross-policy-Debian: foreign -- 2.20.1
On Thu, 2019-02-28 at 15:53 +0000, Daniel P. Berrangé wrote: [...]
apparmor: deb: libapparmor-dev + cross-policy-Debian: foreign
Maybe we should use 'cross-policy-deb' instead? It's true that we're only doing cross compilation on Debian at the moment, but if we wanted to extend that to Ubuntu at some point this information should still be accurate for the most part... [...]
libgovirt: rpm: libgovirt-devel Debian: libgovirt-dev Debian8: + cross-policy-Debianian: foreign
s/Debianian/Debian/ ;) Unless you agree with the above, of course. Either way, Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
On Fri, Mar 01, 2019 at 06:03:04PM +0100, Andrea Bolognani wrote:
On Thu, 2019-02-28 at 15:53 +0000, Daniel P. Berrangé wrote: [...]
apparmor: deb: libapparmor-dev + cross-policy-Debian: foreign
Maybe we should use 'cross-policy-deb' instead? It's true that we're only doing cross compilation on Debian at the moment, but if we wanted to extend that to Ubuntu at some point this information should still be accurate for the most part...
I went back & forth on this. Originally I had 'deb' for everything except the few packages which were excluded on Ubuntu, where I had used 'Debian'. So I made it consistent. I guess it doesn't actually harm to just use 'deb' for everything even if it doesn't exist on Ubuntu
[...]
libgovirt: rpm: libgovirt-devel Debian: libgovirt-dev Debian8: + cross-policy-Debianian: foreign
s/Debianian/Debian/ ;)
Opps, missed this as this package isn't used by libvirt, only virt-viewer
Unless you agree with the above, of course.
Either way,
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
-- Andrea Bolognani / Red Hat / Virtualization
Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Forgot to mention that while I tested with Debian 9, I'm unable to test on Sid right now since the cross toolchain is busted failing to find deps in the apt repos On Thu, Feb 28, 2019 at 03:53:15PM +0000, Daniel P. Berrangé wrote:
Changed in v6:
- Use a static mapping for arch info - Split out data file changes from code changes - Change arch suffix to a prefix
Changed in v5:
- Remove redundant formatter class - Update commit message - Update readme - Move misplaced patch chunk - Dont set "dest" in arg when not needed - Use metavar for subparser
Changed in v4:
- Pull in change to use argparse sub-parsers - Refactor way architecture specific package rules are stored to be in the main package mappings data
Changed in v3:
- Remove sheepdog more generally - Use .format() style printf - Split config to cross-build.yml - Make glusterfs name per-distro customized - Misc code style changes - Rename fields in cross-build.yml - Don't use crossbuild-essential packages
Changed in v2:
- Fix multiple package name mistakes - Modify lcitool to generate cross-arch docker files - Add --no-install-recommended flag to apt-get - Add DEBIAN_FRONTEND=noninteractive env to apt-get - Improve error reporting in lcitool - Add make rule for generating dockerfiles locally
Daniel P. Berrangé (4): mappings: extend mapping to allow per-arch entries mappings: filter arch usage for libnuma / xen lcitool: support generating cross compiler dockerfiles mappings: mark packages using foreign arch for cross builds
guests/lcitool | 117 ++++++++++++++++++++- guests/playbooks/update/tasks/packages.yml | 32 ++++++ guests/vars/mappings.yml | 106 ++++++++++++++++++- 3 files changed, 249 insertions(+), 6 deletions(-)
-- 2.20.1
Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
participants (2)
-
Andrea Bolognani -
Daniel P. Berrangé