[libvirt-glib PATCH] gitlab: introduce CI jobs testing git master & distro libvirt

The glib build needs to validate two axis - A variety of libvirt versions - Native and mingw We test a variety of libvirt versions by running a build against the distro provided libvirt packages. All that is then missing is a build against the latest libvirt git master, which only needs to be run on a single distro, for which CentOS 8 is picked as a stable long life base. For mingw, we only test against libvirt git master, using Fedora rawhide. It is expected that any issues against older libvirt will have already been caught by the native builds. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- .gitlab-ci.yml | 228 ++++++++++++++++++ ci/libvirt-centos-7.Dockerfile | 88 +++++++ ci/libvirt-centos-8.Dockerfile | 64 +++++ ci/libvirt-centos-stream.Dockerfile | 58 +++++ ci/libvirt-debian-10.Dockerfile | 58 +++++ ci/libvirt-debian-9.Dockerfile | 61 +++++ ci/libvirt-debian-sid.Dockerfile | 58 +++++ ci/libvirt-fedora-31.Dockerfile | 55 +++++ ci/libvirt-fedora-32.Dockerfile | 55 +++++ ...rt-fedora-rawhide-cross-mingw32.Dockerfile | 133 ++++++++++ ...rt-fedora-rawhide-cross-mingw64.Dockerfile | 133 ++++++++++ ci/libvirt-fedora-rawhide.Dockerfile | 56 +++++ ci/libvirt-opensuse-151.Dockerfile | 57 +++++ ci/libvirt-ubuntu-1804.Dockerfile | 61 +++++ ci/libvirt-ubuntu-2004.Dockerfile | 58 +++++ ci/refresh | 36 +++ 16 files changed, 1259 insertions(+) create mode 100644 ci/libvirt-centos-7.Dockerfile create mode 100644 ci/libvirt-centos-8.Dockerfile create mode 100644 ci/libvirt-centos-stream.Dockerfile create mode 100644 ci/libvirt-debian-10.Dockerfile create mode 100644 ci/libvirt-debian-9.Dockerfile create mode 100644 ci/libvirt-debian-sid.Dockerfile create mode 100644 ci/libvirt-fedora-31.Dockerfile create mode 100644 ci/libvirt-fedora-32.Dockerfile create mode 100644 ci/libvirt-fedora-rawhide-cross-mingw32.Dockerfile create mode 100644 ci/libvirt-fedora-rawhide-cross-mingw64.Dockerfile create mode 100644 ci/libvirt-fedora-rawhide.Dockerfile create mode 100644 ci/libvirt-opensuse-151.Dockerfile create mode 100644 ci/libvirt-ubuntu-1804.Dockerfile create mode 100644 ci/libvirt-ubuntu-2004.Dockerfile create mode 100755 ci/refresh diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 50dae92..da1e588 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,6 +1,97 @@ stages: - prebuild + - containers + - builds + - docs + +.container_job_template: &container_job_definition + image: docker:stable + stage: containers + services: + - docker:dind + before_script: + - export TAG="$CI_REGISTRY_IMAGE/ci-$NAME:latest" + - export COMMON_TAG="$CI_REGISTRY/libvirt/libvirt-glib/ci-$NAME:latest" + - docker info + - docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" + script: + - docker pull "$TAG" || docker pull "$COMMON_TAG" || true + - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" -f "ci/libvirt-$NAME.Dockerfile" ci + - docker push "$TAG" + after_script: + - docker logout + +.script_variables: &script_variables | + export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)" + export VROOT="$SCRATCH_DIR/vroot" + export CCACHE_BASEDIR="$(pwd)" + export CCACHE_DIR="$CCACHE_BASEDIR/ccache" + export CCACHE_MAXSIZE="500M" + export PATH="$CCACHE_WRAPPERSDIR:$VROOT/bin:$PATH" + export SCRATCH_DIR="/tmp/scratch" + export PKG_CONFIG_PATH="$VROOT/lib/pkgconfig" + +.git_native_build_job_template: &git_native_build_job_definition + image: $CI_REGISTRY_IMAGE/ci-$NAME:latest + stage: builds + before_script: + - *script_variables + - export LD_LIBRARY_PATH="$VROOT/lib" + script: + - pushd "$PWD" + - mkdir -p "$SCRATCH_DIR" + - cd "$SCRATCH_DIR" + - git clone --depth 1 https://gitlab.com/libvirt/libvirt.git + - mkdir libvirt/build + - cd libvirt/build + - ../autogen.sh --prefix="$VROOT" --without-libvirtd + - $MAKE install + - popd + - mkdir build + - cd build + - ../autogen.sh --prefix="$VROOT" + - $MAKE install + - $MAKE dist + - if test -x /usr/bin/rpmbuild && test "$RPM" != "skip" ; then rpmbuild --nodeps -ta libvirt-glib*.tar.gz ; fi + +.dist_native_build_job_template: &dist_native_build_job_definition + image: $CI_REGISTRY_IMAGE/ci-$NAME:latest + stage: builds + before_script: + - *script_variables + script: + - mkdir build + - cd build + - ../autogen.sh --prefix="$VROOT" + - $MAKE install + - $MAKE dist + - if test -x /usr/bin/rpmbuild && test "$RPM" != "skip" ; then rpmbuild -ta libvirt-glib*.tar.gz ; fi + +# Default cross build jobs that are always run +.git_cross_build_default_job_template: &git_cross_build_default_job_definition + image: $CI_REGISTRY_IMAGE/ci-$NAME:latest + stage: builds + cache: + paths: + - ccache/ + key: "$CI_JOB_NAME" + before_script: + - *script_variables + script: + - pushd "$PWD" + - mkdir -p "$SCRATCH_DIR" + - cd "$SCRATCH_DIR" + - git clone --depth 1 https://gitlab.com/libvirt/libvirt.git + - mkdir libvirt/build + - cd libvirt/build + - ../autogen.sh $CONFIGURE_OPTS --prefix="$VROOT" + - $MAKE install + - popd + - mkdir build + - cd build + - ../autogen.sh $CONFIGURE_OPTS || (cat config.log && exit 1) + - $MAKE # Check that all commits are signed-off for the DCO. # Skip on "libvirt" namespace, since we only need to run @@ -14,3 +105,140 @@ check-dco: except: variables: - $CI_PROJECT_NAMESPACE == 'libvirt' + +x64-centos-7-container: + <<: *container_job_definition + variables: + NAME: centos-7 + +x64-centos-8-container: + <<: *container_job_definition + variables: + NAME: centos-8 + +x64-debian-9-container: + <<: *container_job_definition + variables: + NAME: debian-9 + +x64-debian-10-container: + <<: *container_job_definition + variables: + NAME: debian-10 + +x64-debian-sid-container: + <<: *container_job_definition + variables: + NAME: debian-sid + +x64-fedora-31-container: + <<: *container_job_definition + variables: + NAME: fedora-31 + +x64-fedora-32-container: + <<: *container_job_definition + variables: + NAME: fedora-32 + +x64-fedora-rawhide-container: + <<: *container_job_definition + variables: + NAME: fedora-rawhide + +x64-opensuse-151-container: + <<: *container_job_definition + variables: + NAME: opensuse-151 + +x64-ubuntu-1804-container: + <<: *container_job_definition + variables: + NAME: ubuntu-1804 + +x64-ubuntu-2004-container: + <<: *container_job_definition + variables: + NAME: ubuntu-2004 + +mingw32-fedora-rawhide-container: + <<: *container_job_definition + variables: + NAME: fedora-rawhide-cross-mingw32 + +mingw64-fedora-rawhide-container: + <<: *container_job_definition + variables: + NAME: fedora-rawhide-cross-mingw64 + + + +x64-centos-8-git-build: + <<: *git_native_build_job_definition + variables: + NAME: centos-8 + +x64-centos-7-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: centos-7 + +x64-centos-8-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: centos-8 + +x64-debian-9-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: debian-9 + +x64-debian-10-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: debian-10 + +x64-debian-sid-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: debian-sid + +x64-fedora-31-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: fedora-31 + +x64-fedora-32-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: fedora-32 + +x64-fedora-rawhide-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: fedora-rawhide + +x64-opensuse-151-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: opensuse-151 + +x64-ubuntu-1804-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: ubuntu-1804 + +x64-ubuntu-2004-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: ubuntu-2004 + +mingw32-fedora-rawhide-git-build: + <<: *git_cross_build_default_job_definition + variables: + NAME: fedora-rawhide-cross-mingw32 + +mingw64-fedora-rawhide-git-build: + <<: *git_cross_build_default_job_definition + variables: + NAME: fedora-rawhide-cross-mingw64 diff --git a/ci/libvirt-centos-7.Dockerfile b/ci/libvirt-centos-7.Dockerfile new file mode 100644 index 0000000..ad6b262 --- /dev/null +++ b/ci/libvirt-centos-7.Dockerfile @@ -0,0 +1,88 @@ +FROM centos:7 + +RUN echo -e '[openvz]\n\ +name=OpenVZ addons\n\ +baseurl=https://download.openvz.org/virtuozzo/releases/openvz-7.0.11-235/x86_64/os/\n\ +enabled=1\n\ +gpgcheck=1\n\ +skip_if_unavailable=0\n\ +metadata_expire=6h\n\ +priority=90\n\ +includepkgs=libprl*' > /etc/yum.repos.d/openvz.repo && \ + echo -e '-----BEGIN PGP PUBLIC KEY BLOCK-----\n\ +Version: GnuPG v2.0.22 (GNU/Linux)\n\ +\n\ +mI0EVl80nQEEAKrEeyeTCwrzS9kYedZ/sAc/GUqlb81C7pA9SaR3fyck5mVw1Ogk\n\ +YdmNBPM2kY7QDxR9F0EpSpnxSCAXZXugsQ8KzZ0DRLVeBDQyGs9IGK5hI0zzxIil\n\ +BzfvIexLiQQhLy7YlIi8Jt/uUqKkW0pIMNMGcduY97VATtczpncpkmSzABEBAAG0\n\ +SFZpcnR1b3p6byBUZWFtIChHUEcga2V5IHNpZ25hdHVyZSBmb3IgcGFja2FnZXMp\n\ +IDxzZWN1cml0eUB2aXJ0dW96em8uY29tPoi5BBMBAgAjBQJWXzSdAhsDBwsJCAcD\n\ +AgEGFQgCCQoLBBYCAwECHgECF4AACgkQygt9GUTNrSruIgP/er70Eyo73A1gfrjv\n\ +oPUkyo4rslVRZu3qqCwoMFtJc/Z/UxWgEka1buorlcGLa6eO/EZ49c0n+KGa4Kvt\n\ +EUboIq0yEu5i0FyAj92ifm+hNhoAbGfm0cZ4/fD0oGr3l8OsQo4+iHX4xAPwFe7Y\n\ +zABuB8I1ZDZ4OIp5tDfTTuF2LT24jQRWXzSdAQQAog2Aqb+Ptl68O7cQhWLjVGkj\n\ +yyigZrdeReLx3HloKJPBeQ/kA6uvMJc/IYS3uppMWXv9v+QenS6uhP1TUJ2k9FvM\n\ +t94MQZfALN7Vpf8AF+UeWu4Ru+y4BNzcFhrPhIFNFChOR2QqW6FkgE57D9I177NC\n\ +oJMyrlNe8wcGa178An8AEQEAAYifBBgBAgAJBQJWXzSdAhsMAAoJEMoLfRlEza0q\n\ +bKwD/3+OFVIEXnIv5XgdGRNX5fHggsUN1bb8gva7HANRlKdd4LD8foDM3F/yv/3V\n\ +igG14D5EjKz56SaBDNgiI4++hOzb2M8jhAsR86jxkXFrrP1U3ZNRKg6av9DPFAPS\n\ +WEiJKtQrZDJloqtyi/mmRa1VsV7RYR0VPJjhK/R8EQ7Ysshy\n\ +=fRMg\n\ +-----END PGP PUBLIC KEY BLOCK-----' > /etc/pki/rpm-gpg/RPM-GPG-KEY-OpenVZ && \ + rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-OpenVZ && \ + yum install -y epel-release && \ + yum update -y && \ + yum install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-common \ + glibc-devel \ + gobject-introspection-devel \ + gtk-doc \ + libtool \ + libvirt-devel \ + libxml2-devel \ + lsof \ + make \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-pip \ + python3-setuptools \ + python3-wheel \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + yum autoremove -y && \ + yum clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +RUN pip3 install \ + meson==0.49.0 + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja-build" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-centos-8.Dockerfile b/ci/libvirt-centos-8.Dockerfile new file mode 100644 index 0000000..360cf08 --- /dev/null +++ b/ci/libvirt-centos-8.Dockerfile @@ -0,0 +1,64 @@ +FROM centos:8 + +RUN dnf install 'dnf-command(config-manager)' -y && \ + dnf config-manager --set-enabled -y PowerTools && \ + dnf install -y epel-release && \ + dnf update -y && \ + dnf install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + gnutls-devel \ + gobject-introspection-devel \ + gtk-doc \ + libnl3-devel \ + libtirpc-devel \ + libtool \ + libvirt-devel \ + libxml2 \ + libxml2-devel \ + libxslt \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-docutils \ + python3-setuptools \ + python3-wheel \ + rpcgen \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-centos-stream.Dockerfile b/ci/libvirt-centos-stream.Dockerfile new file mode 100644 index 0000000..6f6c830 --- /dev/null +++ b/ci/libvirt-centos-stream.Dockerfile @@ -0,0 +1,58 @@ +FROM centos:8 + +RUN dnf install -y centos-release-stream && \ + dnf install 'dnf-command(config-manager)' -y && \ + dnf config-manager --set-enabled -y Stream-PowerTools && \ + dnf install -y epel-release && \ + dnf update -y && \ + dnf install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + gobject-introspection-devel \ + gtk-doc \ + libtool \ + libvirt-devel \ + libxml2-devel \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-setuptools \ + python3-wheel \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-debian-10.Dockerfile b/ci/libvirt-debian-10.Dockerfile new file mode 100644 index 0000000..f55fe06 --- /dev/null +++ b/ci/libvirt-debian-10.Dockerfile @@ -0,0 +1,58 @@ +FROM debian:10 + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc6-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + valac \ + vim && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-debian-9.Dockerfile b/ci/libvirt-debian-9.Dockerfile new file mode 100644 index 0000000..182b3a6 --- /dev/null +++ b/ci/libvirt-debian-9.Dockerfile @@ -0,0 +1,61 @@ +FROM debian:9 + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc6-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-pip \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + valac \ + vim && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +RUN pip3 install \ + meson==0.49.0 + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-debian-sid.Dockerfile b/ci/libvirt-debian-sid.Dockerfile new file mode 100644 index 0000000..fbb1cb6 --- /dev/null +++ b/ci/libvirt-debian-sid.Dockerfile @@ -0,0 +1,58 @@ +FROM debian:sid + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc6-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + valac \ + vim && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-fedora-31.Dockerfile b/ci/libvirt-fedora-31.Dockerfile new file mode 100644 index 0000000..e23f6d5 --- /dev/null +++ b/ci/libvirt-fedora-31.Dockerfile @@ -0,0 +1,55 @@ +FROM fedora:31 + +RUN dnf update -y && \ + dnf install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + gobject-introspection-devel \ + gtk-doc \ + libtool \ + libvirt-devel \ + libxml2-devel \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-setuptools \ + python3-wheel \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-fedora-32.Dockerfile b/ci/libvirt-fedora-32.Dockerfile new file mode 100644 index 0000000..1da98c3 --- /dev/null +++ b/ci/libvirt-fedora-32.Dockerfile @@ -0,0 +1,55 @@ +FROM fedora:32 + +RUN dnf update -y && \ + dnf install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + gobject-introspection-devel \ + gtk-doc \ + libtool \ + libvirt-devel \ + libxml2-devel \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-setuptools \ + python3-wheel \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-fedora-rawhide-cross-mingw32.Dockerfile b/ci/libvirt-fedora-rawhide-cross-mingw32.Dockerfile new file mode 100644 index 0000000..c366c5f --- /dev/null +++ b/ci/libvirt-fedora-rawhide-cross-mingw32.Dockerfile @@ -0,0 +1,133 @@ +FROM fedora:rawhide + +RUN dnf update -y --nogpgcheck fedora-gpg-keys && \ + dnf update -y && \ + dnf install -y \ + audit-libs-devel \ + augeas \ + autoconf \ + automake \ + avahi-devel \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + cyrus-sasl-devel \ + dbus-devel \ + device-mapper-devel \ + dnsmasq \ + dwarves \ + ebtables \ + fuse-devel \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + glusterfs-api-devel \ + gnutls-devel \ + gobject-introspection-devel \ + gtk-doc \ + iproute \ + iproute-tc \ + iscsi-initiator-utils \ + kmod \ + libacl-devel \ + libattr-devel \ + libblkid-devel \ + libcap-ng-devel \ + libcurl-devel \ + libiscsi-devel \ + libnl3-devel \ + libpcap-devel \ + libpciaccess-devel \ + librbd-devel \ + libselinux-devel \ + libssh-devel \ + libssh2-devel \ + libtirpc-devel \ + libtool \ + libudev-devel \ + libwsman-devel \ + libxml2 \ + libxml2-devel \ + libxslt \ + lsof \ + lvm2 \ + make \ + meson \ + ncurses-devel \ + net-tools \ + netcf-devel \ + nfs-utils \ + ninja-build \ + numactl-devel \ + numad \ + parted \ + parted-devel \ + patch \ + perl \ + pkgconfig \ + polkit \ + python3 \ + python3-docutils \ + python3-flake8 \ + python3-setuptools \ + python3-wheel \ + qemu-img \ + radvd \ + readline-devel \ + rpcgen \ + rpm-build \ + sanlock-devel \ + screen \ + scrub \ + sheepdog \ + strace \ + sudo \ + systemtap-sdt-devel \ + vala \ + vim \ + wireshark-devel \ + xen-devel \ + xfsprogs-devel \ + yajl-devel \ + zfs-fuse && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-w64-mingw32-cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-w64-mingw32-$(basename /usr/bin/gcc) + +RUN dnf install -y \ + mingw32-curl \ + mingw32-dbus \ + mingw32-dlfcn \ + mingw32-gcc \ + mingw32-gettext \ + mingw32-glib2 \ + mingw32-gnutls \ + mingw32-libssh2 \ + mingw32-libxml2 \ + mingw32-openssl \ + mingw32-pkg-config \ + mingw32-portablexdr \ + mingw32-readline && \ + dnf clean all -y + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" + +ENV ABI "i686-w64-mingw32" +ENV CONFIGURE_OPTS "--host=i686-w64-mingw32" +ENV MESON_OPTS "--cross-file=/usr/share/mingw/toolchain-mingw32.meson" diff --git a/ci/libvirt-fedora-rawhide-cross-mingw64.Dockerfile b/ci/libvirt-fedora-rawhide-cross-mingw64.Dockerfile new file mode 100644 index 0000000..1c5c07c --- /dev/null +++ b/ci/libvirt-fedora-rawhide-cross-mingw64.Dockerfile @@ -0,0 +1,133 @@ +FROM fedora:rawhide + +RUN dnf update -y --nogpgcheck fedora-gpg-keys && \ + dnf update -y && \ + dnf install -y \ + audit-libs-devel \ + augeas \ + autoconf \ + automake \ + avahi-devel \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + cyrus-sasl-devel \ + dbus-devel \ + device-mapper-devel \ + dnsmasq \ + dwarves \ + ebtables \ + fuse-devel \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + glusterfs-api-devel \ + gnutls-devel \ + gobject-introspection-devel \ + gtk-doc \ + iproute \ + iproute-tc \ + iscsi-initiator-utils \ + kmod \ + libacl-devel \ + libattr-devel \ + libblkid-devel \ + libcap-ng-devel \ + libcurl-devel \ + libiscsi-devel \ + libnl3-devel \ + libpcap-devel \ + libpciaccess-devel \ + librbd-devel \ + libselinux-devel \ + libssh-devel \ + libssh2-devel \ + libtirpc-devel \ + libtool \ + libudev-devel \ + libwsman-devel \ + libxml2 \ + libxml2-devel \ + libxslt \ + lsof \ + lvm2 \ + make \ + meson \ + ncurses-devel \ + net-tools \ + netcf-devel \ + nfs-utils \ + ninja-build \ + numactl-devel \ + numad \ + parted \ + parted-devel \ + patch \ + perl \ + pkgconfig \ + polkit \ + python3 \ + python3-docutils \ + python3-flake8 \ + python3-setuptools \ + python3-wheel \ + qemu-img \ + radvd \ + readline-devel \ + rpcgen \ + rpm-build \ + sanlock-devel \ + screen \ + scrub \ + sheepdog \ + strace \ + sudo \ + systemtap-sdt-devel \ + vala \ + vim \ + wireshark-devel \ + xen-devel \ + xfsprogs-devel \ + yajl-devel \ + zfs-fuse && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/x86_64-w64-mingw32-cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/x86_64-w64-mingw32-$(basename /usr/bin/gcc) + +RUN dnf install -y \ + mingw64-curl \ + mingw64-dbus \ + mingw64-dlfcn \ + mingw64-gcc \ + mingw64-gettext \ + mingw64-glib2 \ + mingw64-gnutls \ + mingw64-libssh2 \ + mingw64-libxml2 \ + mingw64-openssl \ + mingw64-pkg-config \ + mingw64-portablexdr \ + mingw64-readline && \ + dnf clean all -y + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" + +ENV ABI "x86_64-w64-mingw32" +ENV CONFIGURE_OPTS "--host=x86_64-w64-mingw32" +ENV MESON_OPTS "--cross-file=/usr/share/mingw/toolchain-mingw64.meson" diff --git a/ci/libvirt-fedora-rawhide.Dockerfile b/ci/libvirt-fedora-rawhide.Dockerfile new file mode 100644 index 0000000..7017282 --- /dev/null +++ b/ci/libvirt-fedora-rawhide.Dockerfile @@ -0,0 +1,56 @@ +FROM fedora:rawhide + +RUN dnf update -y --nogpgcheck fedora-gpg-keys && \ + dnf update -y && \ + dnf install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + gobject-introspection-devel \ + gtk-doc \ + libtool \ + libvirt-devel \ + libxml2-devel \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-setuptools \ + python3-wheel \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-opensuse-151.Dockerfile b/ci/libvirt-opensuse-151.Dockerfile new file mode 100644 index 0000000..87237d4 --- /dev/null +++ b/ci/libvirt-opensuse-151.Dockerfile @@ -0,0 +1,57 @@ +FROM opensuse/leap:15.1 + +RUN zypper update -y && \ + zypper install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-locale \ + gobject-introspection-devel \ + gtk-doc \ + libtool \ + libvirt-devel \ + libxml2-devel \ + lsof \ + make \ + net-tools \ + ninja \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-pip \ + python3-setuptools \ + python3-wheel \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + zypper clean --all && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +RUN pip3 install \ + meson==0.49.0 + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-ubuntu-1804.Dockerfile b/ci/libvirt-ubuntu-1804.Dockerfile new file mode 100644 index 0000000..2b8bd41 --- /dev/null +++ b/ci/libvirt-ubuntu-1804.Dockerfile @@ -0,0 +1,61 @@ +FROM ubuntu:18.04 + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc6-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-pip \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + valac \ + vim && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +RUN pip3 install \ + meson==0.49.0 + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-ubuntu-2004.Dockerfile b/ci/libvirt-ubuntu-2004.Dockerfile new file mode 100644 index 0000000..38920f0 --- /dev/null +++ b/ci/libvirt-ubuntu-2004.Dockerfile @@ -0,0 +1,58 @@ +FROM ubuntu:20.04 + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc6-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + valac \ + vim && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/refresh b/ci/refresh new file mode 100755 index 0000000..dfbfa87 --- /dev/null +++ b/ci/refresh @@ -0,0 +1,36 @@ +#!/bin/sh + +if test -z "$1" +then + echo "syntax: $0 PATH-TO-LCITOOL" + exit 1 +fi + +LCITOOL=$1 + +if ! test -x "$LCITOOL" +then + echo "$LCITOOL is not executable" + exit 1 +fi + +HOSTS=$($LCITOOL hosts | grep -v freebsd) + +for host in $HOSTS +do + + if test "$host" = "libvirt-fedora-rawhide" + then + for cross in mingw32 mingw64 + do + $LCITOOL dockerfile $host libvirt,libvirt-glib --cross $cross >$host-cross-$cross.Dockerfile + done + fi + + if test "$host" = "libvirt-centos-8" + then + $LCITOOL dockerfile $host libvirt+minimal,libvirt+dist,libvirt-glib > $host.Dockerfile + else + $LCITOOL dockerfile $host libvirt+dist,libvirt-glib > $host.Dockerfile + fi +done -- 2.26.2

On Thu, 2020-06-11 at 17:42 +0100, Daniel P. Berrangé wrote:
ci/libvirt-centos-7.Dockerfile | 88 +++++++ ci/libvirt-centos-8.Dockerfile | 64 +++++ ci/libvirt-centos-stream.Dockerfile | 58 +++++ ci/libvirt-debian-10.Dockerfile | 58 +++++ ci/libvirt-debian-9.Dockerfile | 61 +++++ ci/libvirt-debian-sid.Dockerfile | 58 +++++ ci/libvirt-fedora-31.Dockerfile | 55 +++++ ci/libvirt-fedora-32.Dockerfile | 55 +++++ ...rt-fedora-rawhide-cross-mingw32.Dockerfile | 133 ++++++++++ ...rt-fedora-rawhide-cross-mingw64.Dockerfile | 133 ++++++++++ ci/libvirt-fedora-rawhide.Dockerfile | 56 +++++ ci/libvirt-opensuse-151.Dockerfile | 57 +++++ ci/libvirt-ubuntu-1804.Dockerfile | 61 +++++ ci/libvirt-ubuntu-2004.Dockerfile | 58 +++++ ci/refresh | 36 +++
Please put all the Dockerfiles in ci/containers, as is already the case for libvirt: the extra directory will ensure things remain tidy even after we roll out support for Cirrus CI to all projects. You're also missing the usual README.rst explaining how the Dockerfiles are generated. [...]
stages: - prebuild + - containers + - builds + - docs
The 'docs' stage is not used anywhere.
+.script_variables: &script_variables | + export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)" + export VROOT="$SCRATCH_DIR/vroot" + export CCACHE_BASEDIR="$(pwd)" + export CCACHE_DIR="$CCACHE_BASEDIR/ccache" + export CCACHE_MAXSIZE="500M" + export PATH="$CCACHE_WRAPPERSDIR:$VROOT/bin:$PATH" + export SCRATCH_DIR="/tmp/scratch" + export PKG_CONFIG_PATH="$VROOT/lib/pkgconfig"
You need to define $SCRATCH_DIR before $VROOT, otherwise the latter will get the wrong value. I also just realized that the way we set $CCACHE_BASEDIR might not work for the libvirt build that we perform as a prerequisite, and since we use the same paths across builds anyway there doesn't seem to be a point in setting it. So I suggest we have export SCRATCH_DIR="/tmp/scratch" export VROOT="$SCRATCH_DIR/vroot" export CCACHE_DIR="$SCRATCH_DIR/ccache"
+.git_native_build_job_template: &git_native_build_job_definition + image: $CI_REGISTRY_IMAGE/ci-$NAME:latest + stage: builds + before_script: + - *script_variables + - export LD_LIBRARY_PATH="$VROOT/lib"
You could set $LD_LIBRARY_PATH in script_variables without causing any issue for the dist-build jobs.
+.dist_native_build_job_template: &dist_native_build_job_definition + image: $CI_REGISTRY_IMAGE/ci-$NAME:latest + stage: builds + before_script: + - *script_variables
This job is missing GitLab CI cache support, as is the one above.
+ script: + - mkdir build + - cd build + - ../autogen.sh --prefix="$VROOT" + - $MAKE install + - $MAKE dist
Do we want distcheck here, or is the combination of dist plus building the RPM package (which effectively runs check inside the generated tarball) good enough?
+x64-centos-7-container: + <<: *container_job_definition + variables: + NAME: centos-7 + +x64-centos-8-container: + <<: *container_job_definition + variables: + NAME: centos-8
You have a Dockerfile for CentOS Stream, but you're not building the corresponding container...
+x64-centos-8-git-build: + <<: *git_native_build_job_definition + variables: + NAME: centos-8 + +x64-centos-7-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: centos-7 + +x64-centos-8-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: centos-8
... or running the corresponding build. -- Andrea Bolognani / Red Hat / Virtualization

On Wed, Jun 17, 2020 at 12:54:56PM +0200, Andrea Bolognani wrote:
On Thu, 2020-06-11 at 17:42 +0100, Daniel P. Berrangé wrote:
ci/libvirt-centos-7.Dockerfile | 88 +++++++ ci/libvirt-centos-8.Dockerfile | 64 +++++ ci/libvirt-centos-stream.Dockerfile | 58 +++++ ci/libvirt-debian-10.Dockerfile | 58 +++++ ci/libvirt-debian-9.Dockerfile | 61 +++++ ci/libvirt-debian-sid.Dockerfile | 58 +++++ ci/libvirt-fedora-31.Dockerfile | 55 +++++ ci/libvirt-fedora-32.Dockerfile | 55 +++++ ...rt-fedora-rawhide-cross-mingw32.Dockerfile | 133 ++++++++++ ...rt-fedora-rawhide-cross-mingw64.Dockerfile | 133 ++++++++++ ci/libvirt-fedora-rawhide.Dockerfile | 56 +++++ ci/libvirt-opensuse-151.Dockerfile | 57 +++++ ci/libvirt-ubuntu-1804.Dockerfile | 61 +++++ ci/libvirt-ubuntu-2004.Dockerfile | 58 +++++ ci/refresh | 36 +++
Please put all the Dockerfiles in ci/containers, as is already the case for libvirt: the extra directory will ensure things remain tidy even after we roll out support for Cirrus CI to all projects.
You're also missing the usual README.rst explaining how the Dockerfiles are generated.
[...]
stages: - prebuild + - containers + - builds + - docs
The 'docs' stage is not used anywhere.
+.script_variables: &script_variables | + export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)" + export VROOT="$SCRATCH_DIR/vroot" + export CCACHE_BASEDIR="$(pwd)" + export CCACHE_DIR="$CCACHE_BASEDIR/ccache" + export CCACHE_MAXSIZE="500M" + export PATH="$CCACHE_WRAPPERSDIR:$VROOT/bin:$PATH" + export SCRATCH_DIR="/tmp/scratch" + export PKG_CONFIG_PATH="$VROOT/lib/pkgconfig"
You need to define $SCRATCH_DIR before $VROOT, otherwise the latter will get the wrong value.
I also just realized that the way we set $CCACHE_BASEDIR might not work for the libvirt build that we perform as a prerequisite, and
There's no problem - $(pwd) expands at the time the variable is defined.
since we use the same paths across builds anyway there doesn't seem to be a point in setting it. So I suggest we have
export SCRATCH_DIR="/tmp/scratch" export VROOT="$SCRATCH_DIR/vroot" export CCACHE_DIR="$SCRATCH_DIR/ccache"
The ccache dir has to be a subdir of the source checkout for gitlab to cache it between jobs.
+ script: + - mkdir build + - cd build + - ../autogen.sh --prefix="$VROOT" + - $MAKE install + - $MAKE dist
Do we want distcheck here, or is the combination of dist plus building the RPM package (which effectively runs check inside the generated tarball) good enough?
I think its good enough. Also I want to switch to meson for glib asap. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Wed, 2020-06-17 at 14:35 +0100, Daniel P. Berrangé wrote:
On Wed, Jun 17, 2020 at 12:54:56PM +0200, Andrea Bolognani wrote:
On Thu, 2020-06-11 at 17:42 +0100, Daniel P. Berrangé wrote:
+.script_variables: &script_variables | + export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)" + export VROOT="$SCRATCH_DIR/vroot" + export CCACHE_BASEDIR="$(pwd)" + export CCACHE_DIR="$CCACHE_BASEDIR/ccache" + export CCACHE_MAXSIZE="500M" + export PATH="$CCACHE_WRAPPERSDIR:$VROOT/bin:$PATH" + export SCRATCH_DIR="/tmp/scratch" + export PKG_CONFIG_PATH="$VROOT/lib/pkgconfig"
You need to define $SCRATCH_DIR before $VROOT, otherwise the latter will get the wrong value.
I also just realized that the way we set $CCACHE_BASEDIR might not work for the libvirt build that we perform as a prerequisite, and
There's no problem - $(pwd) expands at the time the variable is defined.
The problem with $CCACHE_BASEDIR being set to $PWD is that the build that we do for libvirt happens *outside* of that directory, which I think messes up with the path rewriting and might result in libvirt builds not taking proper advantage of the cache. It's all a bit confusing, to be honest, so I could have gotten it wrong, but IIUC the idea is that if you have multiple built trees for the same project you'd be able to do something like $ cd ~/build/tree/one $ export CCACHE_BASEDIR=$PWD $ make $ cd ~/build/tree/two $ export CCACHE_BASEDIR=$PWD $ make and, thanks to $CCACHE_BASEDIR, ccache will be able to figure out that some of the files are perfect matches and will thus achieve better hit rates. This is not our case for a few reasons: * we only build each project once per container; * we set $CCACHE_BASEDIR once instead of once per build directory; * we build libvirt outside of $CCACHE_BASEDIR. So basically our current use of $CCACHE_BASEDIR is likely useless if not even somewhat harmful, and we should just get rid of it.
since we use the same paths across builds anyway there doesn't seem to be a point in setting it. So I suggest we have
export SCRATCH_DIR="/tmp/scratch" export VROOT="$SCRATCH_DIR/vroot" export CCACHE_DIR="$SCRATCH_DIR/ccache"
The ccache dir has to be a subdir of the source checkout for gitlab to cache it between jobs.
Right! I think you told me about this already, but then I forgot about it O:-) Keeping that in mind, we should have export CCACHE_DIR="$PWD/ccache" then.
+ script: + - mkdir build + - cd build + - ../autogen.sh --prefix="$VROOT" + - $MAKE install + - $MAKE dist
Do we want distcheck here, or is the combination of dist plus building the RPM package (which effectively runs check inside the generated tarball) good enough?
I think its good enough. Also I want to switch to meson for glib asap.
Okay. Be aware of the fact that, with the switch to Meson, we're going to need something like https://gitlab.com/libosinfo/libosinfo/-/blob/master/.gitlab-ci.yml#L6-13 because Meson, unlike autotools, is properly multiarch-aware. In fact, a better solution would probably be to bake the multilib string into the Dockerfile directly, similarly to what we already do for other multiarch-related information, instead of figuring it out dynamically at build time. I haven't had the time to look into implementing that yet, unfortunatley, but maybe we can get that in before libvirt-glib moves off autotools. -- Andrea Bolognani / Red Hat / Virtualization
participants (2)
-
Andrea Bolognani
-
Daniel P. Berrangé