[libvirt] [jenkins-ci PATCH v2 8/9] lcitool: support generating cross compiler dockerfiles

Debian's filesystem layout has a nice advantage over Fedora which is that it can install non-native RPMs in the main root filesystem. It is thus possible to prepare an x86_64 filesystem containing -dev packages for a foreign architecture, along with a GCC cross compiler. QEMU has used this technique to facilitate developer build testing of non-x86 architectures, since few people have access to physical hardware for most of these architectures. For the same reason it would be helpful to libvirt developers. This patch extends the 'dockerfile' command to 'lcitool' so that it accepts a '-x $ARCH' argument. $ lcitool -a dockerfile -h libvirt-debian-9 -p libvirt -x s390x This is only valid when using a 'deb' based distro. With the Debian 9 distro, this supports arm64, armel, armhf, mips, mipsel, mips64el, ppc64el, s390x, which are all the official archs that Debian maintainers currently build libvirt for. With Debian Sid, this is extended to include i386. Debian also builds libvirt on hppa, powerpcspe, sparc64 and x32 which are unofficial ports. Unfortunately it is not possible to reliably add foreign arch packages from the unofficial ports in parallel with those from the native arch, without hitting dependency problems from apt. When an architecture is given, any package name ending in '-dev' will be installed using that architecture variant, while all remaining packages will have their native variant installed. For various reasons (commented inline) a few blacklists are required on a per-arch basis. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com&gt; --- guests/host_vars/libvirt-debian-9/docker.yml | 57 +++++++++++++++ .../host_vars/libvirt-debian-sid/docker.yml | 62 +++++++++++++++++ guests/lcitool | 69 ++++++++++++++++--- 3 files changed, 179 insertions(+), 9 deletions(-) diff --git a/guests/host_vars/libvirt-debian-9/docker.yml b/guests/host_vars/libvirt-debian-9/docker.yml index 0b4ccee..0e613da 100644 --- a/guests/host_vars/libvirt-debian-9/docker.yml +++ b/guests/host_vars/libvirt-debian-9/docker.yml @@ -1,2 +1,59 @@ --- docker_base: debian:9 +cross_build: + blacklist: + # Doesn't properly resolve from foreign arch package + # to the native package of augeas-lenses + - libnetcf-dev + # Fails to resolve deps from foreign arch + - wireshark-dev + # dtrace tool doesn't know how to cross-compile + - systemtap-sdt-dev + arches: + arm64: + gcc-pkg-prefix: crossbuild-essential-arm64 + target-prefix: aarch64-linux-gnu + armel: + gcc-pkg-prefix: crossbuild-essential-armel + target-prefix: arm-linux-gnueabi + blacklist: + # Not built on this arch + - libxen-dev + # Arch does not support NUMA concept + - libnuma-dev + armhf: + gcc-pkg-prefix: crossbuild-essential-armhf + target-prefix: arm-linux-gnueabihf + blacklist: + # Arch does not support NUMA concept + - libnuma-dev + mips64el: + gcc-pkg-prefix: gcc-mips64el-linux-gnuabi64 + target-prefix: mips64el-linux-gnuabi64 + blacklist: + # Not built on this arch + - libxen-dev + mips: + gcc-pkg-prefix: gcc-mips-linux-gnu + target-prefix: mips-linux-gnu + blacklist: + # Not built on this arch + - libxen-dev + mipsel: + gcc-pkg-prefix: gcc-mipsel-linux-gnu + target-prefix: mipsel-linux-gnu + blacklist: + # Not built on this arch + - libxen-dev + ppc64el: + gcc-pkg-prefix: crossbuild-essential-ppc64el + target-prefix: powerpc64le-linux-gnu + blacklist: + # Not built on this arch + - libxen-dev + s390x: + gcc-pkg-prefix: gcc-multilib-s390x-linux-gnu + target-prefix: s390x-linux-gnu + blacklist: + # Not built on this arch + - libxen-dev diff --git a/guests/host_vars/libvirt-debian-sid/docker.yml b/guests/host_vars/libvirt-debian-sid/docker.yml index e20a37e..61ecc1f 100644 --- a/guests/host_vars/libvirt-debian-sid/docker.yml +++ b/guests/host_vars/libvirt-debian-sid/docker.yml @@ -1,2 +1,64 @@ --- docker_base: debian:sid +cross_build: + blacklist: + # Doesn't properly resolve from foreign arch package + # to the native package of augeas-lenses + - libnetcf-dev + # Fails to resolve deps from foreign arch + - wireshark-dev + # dtrace tool doesn't know how to cross-compile + - systemtap-sdt-dev + # appears to be dropped from the 'sid' tree + - sheepdog + arches: + arm64: + gcc-pkg-prefix: crossbuild-essential-arm64 + target-prefix: aarch64-linux-gnu + armel: + gcc-pkg-prefix: crossbuild-essential-armel + target-prefix: arm-linux-gnueabi + blacklist: + # Not built on this arch + - libxen-dev + # Arch does not support NUMA concept + - libnuma-dev + armhf: + gcc-pkg-prefix: crossbuild-essential-armhf + target-prefix: arm-linux-gnueabihf + blacklist: + # Arch does not support NUMA concept + - libnuma-dev + mips64el: + gcc-pkg-prefix: gcc-mips64el-linux-gnuabi64 + target-prefix: mips64el-linux-gnuabi64 + blacklist: + # Not built on this arch + - libxen-dev + mips: + gcc-pkg-prefix: gcc-mips-linux-gnu + target-prefix: mips-linux-gnu + blacklist: + # Not built on this arch + - libxen-dev + mipsel: + gcc-pkg-prefix: gcc-mipsel-linux-gnu + target-prefix: mipsel-linux-gnu + blacklist: + # Not built on this arch + - libxen-dev + ppc64el: + gcc-pkg-prefix: crossbuild-essential-ppc64el + target-prefix: powerpc64le-linux-gnu + blacklist: + # Not built on this arch + - libxen-dev + s390x: + gcc-pkg-prefix: gcc-multilib-s390x-linux-gnu + target-prefix: s390x-linux-gnu + blacklist: + # Not built on this arch + - libxen-dev + i386: + gcc-pkg-prefix: gcc-multilib-i686-linux-gnu + target-prefix: i686-linux-gnu diff --git a/guests/lcitool b/guests/lcitool index cd757eb..dc5741e 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -343,6 +343,11 @@ class Application: metavar="GIT_REVISION", help="git revision to build (remote/branch)", ) + self._parser.add_argument( + "-x", + metavar="CROSS-ARCH", + help="cross compiler architecture (dockerfile action only)", + ) def _execute_playbook(self, playbook, hosts, projects, git_revision): base = Util.get_base() @@ -402,15 +407,15 @@ class Application: except Exception: raise Error("Failed to run {} on '{}'".format(playbook, hosts)) - def _action_hosts(self, _hosts, _projects, _revision): + def _action_hosts(self, _hosts, _projects, _revision, _cross_arch): for host in self._inventory.expand_pattern("all"): print(host) - def _action_projects(self, _hosts, _projects, _revision): + def _action_projects(self, _hosts, _projects, _revision, _cross_arch): for project in self._projects.expand_pattern("all"): print(project) - def _action_install(self, hosts, _projects, _revision): + def _action_install(self, hosts, _projects, _revision, _cross_arch): base = Util.get_base() flavor = self._config.get_flavor() @@ -482,13 +487,13 @@ class Application: except Exception: raise Error("Failed to install '{}'".format(host)) - def _action_update(self, hosts, projects, git_revision): + def _action_update(self, hosts, projects, git_revision, _cross_arch): self._execute_playbook("update", hosts, projects, git_revision) - def _action_build(self, hosts, projects, git_revision): + def _action_build(self, hosts, projects, git_revision, _cross_arch): self._execute_playbook("build", hosts, projects, git_revision) - def _action_dockerfile(self, hosts, projects, _revision): + def _action_dockerfile(self, hosts, projects, _revision, cross_arch): mappings = self._projects.get_mappings() hosts = self._inventory.expand_pattern(hosts) @@ -501,10 +506,24 @@ class Application: os_name = facts["os_name"] os_version = facts["os_version"] os_full = os_name + os_version + blacklist = [] + cross_build_facts = None if package_format not in ["deb", "rpm"]: raise Error("Host {} doesn't support Dockerfiles".format(host)) + if cross_arch is not None: + if package_format != "deb": + raise Error("cross compilers only supported for debian packages") + if "cross_build" not in facts: + raise Error("cross compilers not supported for this host") + if cross_arch not in facts["cross_build"]["arches"]: + raise Error("unsupported cross compiler architecture, use one of {}".format( + ", ".join(facts["cross_build"]["arches"].keys()))) + cross_build_facts = facts["cross_build"]["arches"][cross_arch] + blacklist.extend(facts["cross_build"].get("blacklist", [])) + blacklist.extend(cross_build_facts.get("blacklist", [])) + projects = self._projects.expand_pattern(projects) for project in projects: if project not in facts["projects"]: @@ -531,18 +550,31 @@ class Application: temp[package] = mappings[package][os_full] pkgs = [] + cross_pkgs = [] for item in temp: pkgname = temp[item] if pkgname is None: continue - if pkgname not in pkgs: - pkgs.append(pkgname) + if pkgname in blacklist: + continue + if cross_arch and pkgname[-4:] == "-dev": + if pkgname not in cross_pkgs: + cross_pkgs.append(pkgname + ":" + cross_arch) + else: + if pkgname not in pkgs: + pkgs.append(pkgname) print("FROM {}".format(facts["docker_base"])) varmap = {} varmap["pkgs"] = "".join([" \\\n " + pkgname for pkgname in sorted(pkgs)]) + if cross_arch: + varmap["cross_arch"] = cross_arch + varmap["cross_pkgs"] = "".join([" \\\n " + pkgname + for pkgname in sorted(cross_pkgs)]) + varmap["cross_gcc"] = cross_build_facts["gcc-pkg-prefix"] + varmap["cross_prefix"] = cross_build_facts["target-prefix"] if package_format == "deb": sys.stdout.write(textwrap.dedent(""" RUN DEBIAN_FRONTEND=noninteractive && \\ @@ -554,6 +586,24 @@ class Application: apt-get autoclean -y \\ ) """) % varmap ) + if cross_arch: + # Intentionally a separate RUN command from the above + # so that the common packages of all cross-built images + # share a docker image layer + sys.stdout.write(textwrap.dedent(""" + RUN DEBIAN_FRONTEND=noninteractive && \\ + ( \\ + dpkg --add-architecture %(cross_arch)s && \\ + apt-get update && \\ + apt-get dist-upgrade -y && \\ + apt-get install --no-install-recommends -y %(cross_gcc)s %(cross_pkgs)s && \\ + apt-get autoremove -y && \\ + apt-get autoclean -y \\ + ) + ENV TARGET "%(cross_prefix)s" + ENV CONFIGURE_OPTS "--host=%(cross_prefix)s --target=%(cross_prefix)s" + ENV PKG_CONFIG_LIBDIR "/usr/lib/%(cross_prefix)s/pkgconfig" + """) % varmap ) elif package_format == "rpm": if os_name == "Fedora" and os_version == "Rawhide": sys.stdout.write(textwrap.dedent(""" @@ -577,11 +627,12 @@ class Application: hosts = cmdline.h projects = cmdline.p git_revision = cmdline.g + cross_arch = cmdline.x method = "_action_{}".format(action.replace("-", "_")) if hasattr(self, method): - getattr(self, method).__call__(hosts, projects, git_revision) + getattr(self, method).__call__(hosts, projects, git_revision, cross_arch) else: raise Error("Invalid action '{}'".format(action)) -- 2.20.1