It was in the build stage previously to let it run in parallel with other build jobs, but with the "needs" clause this is not required. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com&gt; --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index d1609c260d..24feb48c99 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -585,7 +585,7 @@ website: codestyle: - stage: builds + stage: sanity_checks image: $CI_REGISTRY_IMAGE/ci-opensuse-leap-152:latest needs: - x64-opensuse-leap-152-container -- 2.31.1

The sanitizer jobs run in ubuntu 20.04 containers and thus overlap with testing already done for the regular ubuntu 20.04 build job. Fold the sanitizer run for GCC into the regular build job and add a second ubuntu 20.04 build job for CLang sanitizers. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com&gt; --- .gitlab-ci.yml | 53 ++++++++++++++------------------------------------ 1 file changed, 15 insertions(+), 38 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 24feb48c99..567ff91c82 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -61,7 +61,7 @@ stages: before_script: - *script_variables script: - - meson setup build --werror || (cat build/meson-logs/meson-log.txt && exit 1) + - meson setup build --werror $MESON_ARGS || (cat build/meson-logs/meson-log.txt && exit 1) - meson dist -C build --no-tests - if test -x /usr/bin/rpmbuild && test "$RPM" != "skip"; then @@ -71,27 +71,6 @@ stages: meson test -C build --no-suite syntax-check --print-errorlogs; fi -.sanitizer_build_job: - stage: builds - image: $CI_REGISTRY_IMAGE/ci-ubuntu-2004:latest - needs: - - x64-ubuntu-2004-container - rules: - - if: "$TEMPORARILY_DISABLED" - allow_failure: true - - when: on_success - cache: - paths: - - ccache/ - key: "$CI_JOB_NAME" - before_script: - - *script_variables - script: - - meson build --werror -Ddocs=disabled -Db_lundef=false -Db_sanitize="$SANITIZER" - - ninja -C build; - - ninja -C build test; - variables: - UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1 # Jobs that we delegate to Cirrus CI because they require an operating # system other than Linux. These jobs will only run if the required @@ -418,6 +397,20 @@ x64-ubuntu-2004: - x64-ubuntu-2004-container variables: NAME: ubuntu-2004 + ASAN_OPTIONS: verify_asan_link_order=0 + MESON_ARGS: -Db_lundef=false -Db_sanitize=address,undefined + UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1 + +x64-ubuntu-2004-clang: + extends: .native_build_job + needs: + - x64-ubuntu-2004-container + variables: + NAME: ubuntu-2004 + CC: clang + MESON_ARGS: -Db_lundef=false -Db_sanitize=address,undefined + UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1 + x64-freebsd-12-build: extends: .cirrus_build_job @@ -545,22 +538,6 @@ mingw64-fedora-rawhide: NAME: fedora-rawhide CROSS: mingw64 -# Sanitizers - -sanitize-gcc: - extends: .sanitizer_build_job - variables: - ASAN_OPTIONS: verify_asan_link_order=0 - CC: gcc - SANITIZER: address,undefined - -sanitize-clang: - extends: .sanitizer_build_job - variables: - CC: clang - SANITIZER: address,undefined - - # This artifact published by this job is downloaded by libvirt.org to # be deployed to the web root: # https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=w... -- 2.31.1

This uses the command "lcitool manifest ci/manifest.yml" to re-generate all existing dockerfiles and gitlab CI config. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com&gt; --- .gitlab-ci.yml | 512 +-------------- ci/cirrus/freebsd-12.vars | 2 +- ci/cirrus/freebsd-13.vars | 2 +- ci/cirrus/freebsd-current.vars | 2 +- ci/cirrus/macos-11.vars | 2 +- ci/containers/centos-8.Dockerfile | 2 +- ci/containers/centos-stream-8.Dockerfile | 2 +- .../debian-10-cross-aarch64.Dockerfile | 2 +- .../debian-10-cross-armv6l.Dockerfile | 2 +- .../debian-10-cross-armv7l.Dockerfile | 2 +- ci/containers/debian-10-cross-i686.Dockerfile | 2 +- ci/containers/debian-10-cross-mips.Dockerfile | 2 +- .../debian-10-cross-mips64el.Dockerfile | 2 +- .../debian-10-cross-mipsel.Dockerfile | 2 +- .../debian-10-cross-ppc64le.Dockerfile | 2 +- .../debian-10-cross-s390x.Dockerfile | 2 +- ci/containers/debian-10.Dockerfile | 2 +- .../debian-sid-cross-aarch64.Dockerfile | 2 +- .../debian-sid-cross-armv6l.Dockerfile | 2 +- .../debian-sid-cross-armv7l.Dockerfile | 2 +- .../debian-sid-cross-i686.Dockerfile | 2 +- .../debian-sid-cross-mips64el.Dockerfile | 2 +- .../debian-sid-cross-mipsel.Dockerfile | 2 +- .../debian-sid-cross-ppc64le.Dockerfile | 2 +- .../debian-sid-cross-s390x.Dockerfile | 2 +- ci/containers/debian-sid.Dockerfile | 2 +- ci/containers/fedora-33.Dockerfile | 2 +- ci/containers/fedora-34.Dockerfile | 2 +- .../fedora-rawhide-cross-mingw32.Dockerfile | 2 +- .../fedora-rawhide-cross-mingw64.Dockerfile | 2 +- ci/containers/fedora-rawhide.Dockerfile | 2 +- ci/containers/opensuse-leap-152.Dockerfile | 2 +- ci/containers/opensuse-tumbleweed.Dockerfile | 2 +- ci/containers/ubuntu-1804.Dockerfile | 2 +- ci/containers/ubuntu-2004.Dockerfile | 2 +- ci/gitlab.yml | 602 ++++++++++++++++++ 36 files changed, 643 insertions(+), 539 deletions(-) create mode 100644 ci/gitlab.yml diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 567ff91c82..b396a1511d 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -14,42 +14,10 @@ stages: export VIR_TEST_VERBOSE="1" export VIR_TEST_DEBUG="1" -# Common templates - -.container_job: - image: docker:stable - stage: containers - needs: [] - services: - - name: registry.gitlab.com/libvirt/libvirt-ci/docker-dind:master - alias: docker - rules: - - if: "$TEMPORARILY_DISABLED" - allow_failure: true - - when: on_success - before_script: - - export TAG="$CI_REGISTRY_IMAGE/ci-$NAME:latest" - - export COMMON_TAG="$CI_REGISTRY/libvirt/libvirt/ci-$NAME:latest" - - docker info - - docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" - script: - - docker pull "$TAG" || docker pull "$COMMON_TAG" || true - - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" -f "ci/containers/$NAME.Dockerfile" ci/containers - - docker push "$TAG" - after_script: - - docker logout - -# We build many containers which can be useful to debug problems but are not -# needed for the pipeline itself to complete: those sometimes fail, and when -# that happens it's mostly because of temporary issues with Debian sid. We -# don't want those failures to affect the overall pipeline status -.container_optional_job: - extends: .container_job - allow_failure: true +include: '/ci/gitlab.yml' .native_build_job: - stage: builds - image: $CI_REGISTRY_IMAGE/ci-$NAME:latest + extends: .gitlab_native_build_job rules: - if: "$TEMPORARILY_DISABLED" allow_failure: true @@ -71,52 +39,8 @@ stages: meson test -C build --no-suite syntax-check --print-errorlogs; fi - -# Jobs that we delegate to Cirrus CI because they require an operating -# system other than Linux. These jobs will only run if the required -# setup has been performed on the GitLab account (see ci/README.rst). -# -# The Cirrus CI configuration is generated by replacing target-specific -# variables in a generic template: some of these variables are provided -# when the GitLab CI job is defined, others are taken from a shell -# snippet generated using lcitool. -# -# Note that the $PATH environment variable has to be treated with -# special care, because we can't just override it at the GitLab CI job -# definition level or we risk breaking it completely. -.cirrus_build_job: - stage: builds - image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master - needs: [] - script: - - source ci/cirrus/$NAME.vars - - sed -e "s|[@]CI_REPOSITORY_URL@|$CI_REPOSITORY_URL|g" - -e "s|[@]CI_COMMIT_REF_NAME@|$CI_COMMIT_REF_NAME|g" - -e "s|[@]CI_COMMIT_SHA@|$CI_COMMIT_SHA|g" - -e "s|[@]CIRRUS_VM_INSTANCE_TYPE@|$CIRRUS_VM_INSTANCE_TYPE|g" - -e "s|[@]CIRRUS_VM_IMAGE_SELECTOR@|$CIRRUS_VM_IMAGE_SELECTOR|g" - -e "s|[@]CIRRUS_VM_IMAGE_NAME@|$CIRRUS_VM_IMAGE_NAME|g" - -e "s|[@]UPDATE_COMMAND@|$UPDATE_COMMAND|g" - -e "s|[@]UPGRADE_COMMAND@|$UPGRADE_COMMAND|g" - -e "s|[@]INSTALL_COMMAND@|$INSTALL_COMMAND|g" - -e "s|[@]PATH@|$PATH_EXTRA${PATH_EXTRA:+:}\$PATH|g" - -e "s|[@]PKG_CONFIG_PATH@|$PKG_CONFIG_PATH|g" - -e "s|[@]PKGS@|$PKGS|g" - -e "s|[@]MAKE@|$MAKE|g" - -e "s|[@]PYTHON@|$PYTHON|g" - -e "s|[@]PIP3@|$PIP3|g" - -e "s|[@]PYPI_PKGS@|$PYPI_PKGS|g" - <ci/cirrus/build.yml >ci/cirrus/$NAME.yml - - cat ci/cirrus/$NAME.yml - - cirrus-run -v --show-build-log always ci/cirrus/$NAME.yml - rules: - - if: "$TEMPORARILY_DISABLED" - allow_failure: true - - if: "$CIRRUS_GITHUB_REPO && $CIRRUS_API_TOKEN" - .cross_build_job: - stage: builds - image: $CI_REGISTRY_IMAGE/ci-$NAME-cross-$CROSS:latest + extends: .gitlab_cross_build_job cache: paths: - ccache/ @@ -133,411 +57,6 @@ stages: - if test "$CROSS" = "i686" ; then meson test -C build --no-suite syntax-check --print-errorlogs ; fi -# Native container build jobs - -x64-centos-8-container: - extends: .container_job - variables: - NAME: centos-8 - -x64-centos-stream-8-container: - extends: .container_job - variables: - NAME: centos-stream-8 - -x64-debian-10-container: - extends: .container_job - variables: - NAME: debian-10 - -x64-debian-sid-container: - extends: .container_job - variables: - NAME: debian-sid - -x64-fedora-33-container: - extends: .container_job - variables: - NAME: fedora-33 - -x64-fedora-34-container: - extends: .container_job - variables: - NAME: fedora-34 - -x64-fedora-rawhide-container: - extends: .container_optional_job - variables: - NAME: fedora-rawhide - -x64-opensuse-leap-152-container: - extends: .container_job - variables: - NAME: opensuse-leap-152 - -x64-opensuse-tumbleweed-container: - extends: .container_job - variables: - NAME: opensuse-tumbleweed - -x64-ubuntu-1804-container: - extends: .container_job - variables: - NAME: ubuntu-1804 - -x64-ubuntu-2004-container: - extends: .container_job - variables: - NAME: ubuntu-2004 - - -# Cross-build containers build jobs - -aarch64-debian-10-container: - extends: .container_optional_job - variables: - NAME: debian-10-cross-aarch64 - -armv6l-debian-10-container: - extends: .container_job - variables: - NAME: debian-10-cross-armv6l - -armv7l-debian-10-container: - extends: .container_job - variables: - NAME: debian-10-cross-armv7l - -i686-debian-10-container: - extends: .container_optional_job - variables: - NAME: debian-10-cross-i686 - -mips-debian-10-container: - extends: .container_job - variables: - NAME: debian-10-cross-mips - -mips64el-debian-10-container: - extends: .container_optional_job - variables: - NAME: debian-10-cross-mips64el - -mipsel-debian-10-container: - extends: .container_job - variables: - NAME: debian-10-cross-mipsel - -ppc64le-debian-10-container: - extends: .container_job - variables: - NAME: debian-10-cross-ppc64le - -s390x-debian-10-container: - extends: .container_optional_job - variables: - NAME: debian-10-cross-s390x - -aarch64-debian-sid-container: - extends: .container_job - variables: - NAME: debian-sid-cross-aarch64 - -armv6l-debian-sid-container: - extends: .container_optional_job - variables: - NAME: debian-sid-cross-armv6l - -armv7l-debian-sid-container: - extends: .container_optional_job - variables: - NAME: debian-sid-cross-armv7l - -i686-debian-sid-container: - extends: .container_job - variables: - NAME: debian-sid-cross-i686 - -mips64el-debian-sid-container: - extends: .container_job - variables: - NAME: debian-sid-cross-mips64el - -mipsel-debian-sid-container: - extends: .container_optional_job - variables: - NAME: debian-sid-cross-mipsel - -ppc64le-debian-sid-container: - extends: .container_optional_job - variables: - NAME: debian-sid-cross-ppc64le - -s390x-debian-sid-container: - extends: .container_job - variables: - NAME: debian-sid-cross-s390x - -mingw32-fedora-rawhide-container: - extends: .container_optional_job - variables: - NAME: fedora-rawhide-cross-mingw32 - -mingw64-fedora-rawhide-container: - extends: .container_optional_job - variables: - NAME: fedora-rawhide-cross-mingw64 - - -# Native architecture build + test jobs - -x64-debian-10: - extends: .native_build_job - needs: - - x64-debian-10-container - variables: - NAME: debian-10 - -x64-debian-10-clang: - extends: .native_build_job - needs: - - x64-debian-10-container - variables: - NAME: debian-10 - CC: clang - -x64-debian-sid: - extends: .native_build_job - needs: - - x64-debian-sid-container - variables: - NAME: debian-sid - -x64-centos-8: - extends: .native_build_job - needs: - - x64-centos-8-container - variables: - NAME: centos-8 - RPM: skip - -x64-centos-8-clang: - extends: .native_build_job - needs: - - x64-centos-8-container - variables: - NAME: centos-8 - CC: clang - RPM: skip - -x64-centos-stream-8: - extends: .native_build_job - needs: - - x64-centos-stream-8-container - variables: - NAME: centos-stream-8 - RPM: skip - -x64-fedora-33: - extends: .native_build_job - needs: - - x64-fedora-33-container - variables: - NAME: fedora-33 - -x64-fedora-34: - extends: .native_build_job - needs: - - x64-fedora-34-container - variables: - NAME: fedora-34 - -x64-fedora-rawhide: - extends: .native_build_job - needs: - - x64-fedora-rawhide-container - variables: - NAME: fedora-rawhide - -x64-fedora-rawhide-clang: - extends: .native_build_job - needs: - - x64-fedora-rawhide-container - variables: - NAME: fedora-rawhide - CC: clang - RPM: skip - -x64-opensuse-leap-152: - extends: .native_build_job - needs: - - x64-opensuse-leap-152-container - variables: - NAME: opensuse-leap-152 - RPM: skip - -x64-opensuse-tumbleweed: - extends: .native_build_job - needs: - - x64-opensuse-tumbleweed-container - variables: - NAME: opensuse-tumbleweed - RPM: skip - -x64-ubuntu-1804: - extends: .native_build_job - needs: - - x64-ubuntu-1804-container - variables: - NAME: ubuntu-1804 - -x64-ubuntu-2004: - extends: .native_build_job - needs: - - x64-ubuntu-2004-container - variables: - NAME: ubuntu-2004 - ASAN_OPTIONS: verify_asan_link_order=0 - MESON_ARGS: -Db_lundef=false -Db_sanitize=address,undefined - UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1 - -x64-ubuntu-2004-clang: - extends: .native_build_job - needs: - - x64-ubuntu-2004-container - variables: - NAME: ubuntu-2004 - CC: clang - MESON_ARGS: -Db_lundef=false -Db_sanitize=address,undefined - UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1 - - -x64-freebsd-12-build: - extends: .cirrus_build_job - variables: - NAME: freebsd-12 - CIRRUS_VM_INSTANCE_TYPE: freebsd_instance - CIRRUS_VM_IMAGE_SELECTOR: image_family - CIRRUS_VM_IMAGE_NAME: freebsd-12-2 - UPDATE_COMMAND: pkg update - UPGRADE_COMMAND: pkg upgrade -y - INSTALL_COMMAND: pkg install -y - -x64-freebsd-13-build: - extends: .cirrus_build_job - variables: - NAME: freebsd-13 - CIRRUS_VM_INSTANCE_TYPE: freebsd_instance - CIRRUS_VM_IMAGE_SELECTOR: image_family - CIRRUS_VM_IMAGE_NAME: freebsd-13-0 - UPDATE_COMMAND: pkg update - UPGRADE_COMMAND: pkg upgrade -y - INSTALL_COMMAND: pkg install -y - -x64-macos-11-build: - extends: .cirrus_build_job - variables: - NAME: macos-11 - CIRRUS_VM_INSTANCE_TYPE: osx_instance - CIRRUS_VM_IMAGE_SELECTOR: image - CIRRUS_VM_IMAGE_NAME: big-sur-base - UPDATE_COMMAND: brew update - UPGRADE_COMMAND: brew upgrade - INSTALL_COMMAND: brew install - PATH_EXTRA: /usr/local/opt/ccache/libexec:/usr/local/opt/gettext/bin:/usr/local/opt/libpcap/bin:/usr/local/opt/libxslt/bin:/usr/local/opt/rpcgen/bin - PKG_CONFIG_PATH: /usr/local/opt/curl/lib/pkgconfig:/usr/local/opt/libpcap/lib/pkgconfig:/usr/local/opt/libxml2/lib/pkgconfig:/usr/local/opt/ncurses/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig - - -# Cross compiled build jobs - -aarch64-debian-sid: - extends: .cross_build_job - needs: - - aarch64-debian-sid-container - variables: - NAME: debian-sid - CROSS: aarch64 - -armv6l-debian-10: - extends: .cross_build_job - needs: - - armv6l-debian-10-container - variables: - NAME: debian-10 - CROSS: armv6l - -armv7l-debian-10: - extends: .cross_build_job - needs: - - armv7l-debian-10-container - variables: - NAME: debian-10 - CROSS: armv7l - -i686-debian-sid: - extends: .cross_build_job - needs: - - i686-debian-sid-container - variables: - NAME: debian-sid - CROSS: i686 - -mips-debian-10: - extends: .cross_build_job - needs: - - mips-debian-10-container - variables: - NAME: debian-10 - CROSS: mips - -mips64el-debian-sid: - extends: .cross_build_job - needs: - - mips64el-debian-sid-container - variables: - NAME: debian-sid - CROSS: mips64el - -mipsel-debian-10: - extends: .cross_build_job - needs: - - mipsel-debian-10-container - variables: - NAME: debian-10 - CROSS: mipsel - -ppc64le-debian-10: - extends: .cross_build_job - needs: - - ppc64le-debian-10-container - variables: - NAME: debian-10 - CROSS: ppc64le - -s390x-debian-sid: - extends: .cross_build_job - needs: - - s390x-debian-sid-container - variables: - NAME: debian-sid - CROSS: s390x - -mingw32-fedora-rawhide: - extends: .cross_build_job - needs: - - mingw32-fedora-rawhide-container - variables: - NAME: fedora-rawhide - CROSS: mingw32 - -mingw64-fedora-rawhide: - extends: .cross_build_job - needs: - - mingw64-fedora-rawhide-container - variables: - NAME: fedora-rawhide - CROSS: mingw64 - # This artifact published by this job is downloaded by libvirt.org to # be deployed to the web root: # https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=w... @@ -545,7 +64,7 @@ website: stage: builds image: $CI_REGISTRY_IMAGE/ci-centos-8:latest needs: - - x64-centos-8-container + - x86_64-centos-8-container before_script: - *script_variables script: @@ -565,7 +84,7 @@ codestyle: stage: sanity_checks image: $CI_REGISTRY_IMAGE/ci-opensuse-leap-152:latest needs: - - x64-opensuse-leap-152-container + - x86_64-opensuse-leap-152-container before_script: - *script_variables script: @@ -581,7 +100,7 @@ potfile: stage: builds image: $CI_REGISTRY_IMAGE/ci-centos-8:latest needs: - - x64-centos-8-container + - x86_64-centos-8-container rules: - if: "$CI_COMMIT_BRANCH == 'master'" before_script: @@ -599,28 +118,11 @@ potfile: paths: - libvirt.pot - -# Check that all commits are signed-off for the DCO. -# Skip on "libvirt" namespace, since we only need to run -# this test on developer's personal forks from which -# merge requests are submitted -check-dco: - stage: sanity_checks - needs: [] - image: registry.gitlab.com/libvirt/libvirt-ci/check-dco:master - script: - - /check-dco - rules: - - if: "$CI_PROJECT_NAMESPACE != 'libvirt'" - variables: - GIT_DEPTH: 1000 - - # Coverity job that is run only by schedules coverity: image: $CI_REGISTRY_IMAGE/ci-centos-8:latest needs: - - x64-centos-8-container + - x86_64-centos-8-container stage: builds script: - curl https://scan.coverity.com/download/linux64 --form project=$COVERITY_SCAN_PROJECT_NAME --form token=$COVERITY_SCAN_TOKEN -o /tmp/cov-analysis-linux64.tgz diff --git a/ci/cirrus/freebsd-12.vars b/ci/cirrus/freebsd-12.vars index 36bcc522dc..4318b255e9 100644 --- a/ci/cirrus/freebsd-12.vars +++ b/ci/cirrus/freebsd-12.vars @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool variables freebsd-12 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/cirrus/freebsd-13.vars b/ci/cirrus/freebsd-13.vars index 8a86f55237..4318b255e9 100644 --- a/ci/cirrus/freebsd-13.vars +++ b/ci/cirrus/freebsd-13.vars @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool variables freebsd-13 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/cirrus/freebsd-current.vars b/ci/cirrus/freebsd-current.vars index d308986384..4318b255e9 100644 --- a/ci/cirrus/freebsd-current.vars +++ b/ci/cirrus/freebsd-current.vars @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool variables freebsd-current libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/cirrus/macos-11.vars b/ci/cirrus/macos-11.vars index 1da29e9ed6..065d86aa45 100644 --- a/ci/cirrus/macos-11.vars +++ b/ci/cirrus/macos-11.vars @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool variables macos-11 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/centos-8.Dockerfile b/ci/containers/centos-8.Dockerfile index 2667e5f4f8..5ac1e45459 100644 --- a/ci/containers/centos-8.Dockerfile +++ b/ci/containers/centos-8.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile centos-8 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/centos-stream-8.Dockerfile b/ci/containers/centos-stream-8.Dockerfile index f89f7648fe..6baf576580 100644 --- a/ci/containers/centos-stream-8.Dockerfile +++ b/ci/containers/centos-stream-8.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile centos-stream-8 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-10-cross-aarch64.Dockerfile b/ci/containers/debian-10-cross-aarch64.Dockerfile index b81a3f123e..3f0010d081 100644 --- a/ci/containers/debian-10-cross-aarch64.Dockerfile +++ b/ci/containers/debian-10-cross-aarch64.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross aarch64 debian-10 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-10-cross-armv6l.Dockerfile b/ci/containers/debian-10-cross-armv6l.Dockerfile index dcfb73d83a..853844d3f1 100644 --- a/ci/containers/debian-10-cross-armv6l.Dockerfile +++ b/ci/containers/debian-10-cross-armv6l.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross armv6l debian-10 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-10-cross-armv7l.Dockerfile b/ci/containers/debian-10-cross-armv7l.Dockerfile index f125fc1789..4d00116849 100644 --- a/ci/containers/debian-10-cross-armv7l.Dockerfile +++ b/ci/containers/debian-10-cross-armv7l.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross armv7l debian-10 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-10-cross-i686.Dockerfile b/ci/containers/debian-10-cross-i686.Dockerfile index fe9f9e3a6d..9c50fe0674 100644 --- a/ci/containers/debian-10-cross-i686.Dockerfile +++ b/ci/containers/debian-10-cross-i686.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross i686 debian-10 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-10-cross-mips.Dockerfile b/ci/containers/debian-10-cross-mips.Dockerfile index c1b441de40..fbeb8db795 100644 --- a/ci/containers/debian-10-cross-mips.Dockerfile +++ b/ci/containers/debian-10-cross-mips.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross mips debian-10 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-10-cross-mips64el.Dockerfile b/ci/containers/debian-10-cross-mips64el.Dockerfile index bdce689148..5212a02999 100644 --- a/ci/containers/debian-10-cross-mips64el.Dockerfile +++ b/ci/containers/debian-10-cross-mips64el.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross mips64el debian-10 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-10-cross-mipsel.Dockerfile b/ci/containers/debian-10-cross-mipsel.Dockerfile index 0bfa1e0552..2e9e716b21 100644 --- a/ci/containers/debian-10-cross-mipsel.Dockerfile +++ b/ci/containers/debian-10-cross-mipsel.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross mipsel debian-10 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-10-cross-ppc64le.Dockerfile b/ci/containers/debian-10-cross-ppc64le.Dockerfile index 08cc080f8a..2f1a56a624 100644 --- a/ci/containers/debian-10-cross-ppc64le.Dockerfile +++ b/ci/containers/debian-10-cross-ppc64le.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross ppc64le debian-10 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-10-cross-s390x.Dockerfile b/ci/containers/debian-10-cross-s390x.Dockerfile index 62f0584129..d7b06ec383 100644 --- a/ci/containers/debian-10-cross-s390x.Dockerfile +++ b/ci/containers/debian-10-cross-s390x.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross s390x debian-10 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-10.Dockerfile b/ci/containers/debian-10.Dockerfile index 211442c673..633d9a972e 100644 --- a/ci/containers/debian-10.Dockerfile +++ b/ci/containers/debian-10.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile debian-10 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-sid-cross-aarch64.Dockerfile b/ci/containers/debian-sid-cross-aarch64.Dockerfile index 3dcba9038a..e481c63388 100644 --- a/ci/containers/debian-sid-cross-aarch64.Dockerfile +++ b/ci/containers/debian-sid-cross-aarch64.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross aarch64 debian-sid libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-sid-cross-armv6l.Dockerfile b/ci/containers/debian-sid-cross-armv6l.Dockerfile index 34255cd3b0..3f63b3e86f 100644 --- a/ci/containers/debian-sid-cross-armv6l.Dockerfile +++ b/ci/containers/debian-sid-cross-armv6l.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross armv6l debian-sid libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-sid-cross-armv7l.Dockerfile b/ci/containers/debian-sid-cross-armv7l.Dockerfile index 8ec54bdee1..55f527b207 100644 --- a/ci/containers/debian-sid-cross-armv7l.Dockerfile +++ b/ci/containers/debian-sid-cross-armv7l.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross armv7l debian-sid libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-sid-cross-i686.Dockerfile b/ci/containers/debian-sid-cross-i686.Dockerfile index 555e36d014..719547ab31 100644 --- a/ci/containers/debian-sid-cross-i686.Dockerfile +++ b/ci/containers/debian-sid-cross-i686.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross i686 debian-sid libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-sid-cross-mips64el.Dockerfile b/ci/containers/debian-sid-cross-mips64el.Dockerfile index 20478e1f02..e1fac818a1 100644 --- a/ci/containers/debian-sid-cross-mips64el.Dockerfile +++ b/ci/containers/debian-sid-cross-mips64el.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross mips64el debian-sid libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-sid-cross-mipsel.Dockerfile b/ci/containers/debian-sid-cross-mipsel.Dockerfile index abf0642bd6..3684fe3bf9 100644 --- a/ci/containers/debian-sid-cross-mipsel.Dockerfile +++ b/ci/containers/debian-sid-cross-mipsel.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross mipsel debian-sid libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-sid-cross-ppc64le.Dockerfile b/ci/containers/debian-sid-cross-ppc64le.Dockerfile index a339905329..1d1aa18e33 100644 --- a/ci/containers/debian-sid-cross-ppc64le.Dockerfile +++ b/ci/containers/debian-sid-cross-ppc64le.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross ppc64le debian-sid libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-sid-cross-s390x.Dockerfile b/ci/containers/debian-sid-cross-s390x.Dockerfile index c29bb57d88..5d2fdd6a1b 100644 --- a/ci/containers/debian-sid-cross-s390x.Dockerfile +++ b/ci/containers/debian-sid-cross-s390x.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross s390x debian-sid libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/debian-sid.Dockerfile b/ci/containers/debian-sid.Dockerfile index cb6a24cb50..d8667c5f1b 100644 --- a/ci/containers/debian-sid.Dockerfile +++ b/ci/containers/debian-sid.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile debian-sid libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/fedora-33.Dockerfile b/ci/containers/fedora-33.Dockerfile index 7350de493f..0025e66d6a 100644 --- a/ci/containers/fedora-33.Dockerfile +++ b/ci/containers/fedora-33.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile fedora-33 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/fedora-34.Dockerfile b/ci/containers/fedora-34.Dockerfile index 10a19a8f51..9be6efc7ba 100644 --- a/ci/containers/fedora-34.Dockerfile +++ b/ci/containers/fedora-34.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile fedora-34 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/fedora-rawhide-cross-mingw32.Dockerfile b/ci/containers/fedora-rawhide-cross-mingw32.Dockerfile index 48c4dd3642..6f00940347 100644 --- a/ci/containers/fedora-rawhide-cross-mingw32.Dockerfile +++ b/ci/containers/fedora-rawhide-cross-mingw32.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross mingw32 fedora-rawhide libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/fedora-rawhide-cross-mingw64.Dockerfile b/ci/containers/fedora-rawhide-cross-mingw64.Dockerfile index 38fb9a6aeb..48c2a755d3 100644 --- a/ci/containers/fedora-rawhide-cross-mingw64.Dockerfile +++ b/ci/containers/fedora-rawhide-cross-mingw64.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --cross mingw64 fedora-rawhide libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/fedora-rawhide.Dockerfile b/ci/containers/fedora-rawhide.Dockerfile index b36fb9343e..74e1f01483 100644 --- a/ci/containers/fedora-rawhide.Dockerfile +++ b/ci/containers/fedora-rawhide.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile fedora-rawhide libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/opensuse-leap-152.Dockerfile b/ci/containers/opensuse-leap-152.Dockerfile index abe3e57185..23f03ed482 100644 --- a/ci/containers/opensuse-leap-152.Dockerfile +++ b/ci/containers/opensuse-leap-152.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile opensuse-leap-152 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/opensuse-tumbleweed.Dockerfile b/ci/containers/opensuse-tumbleweed.Dockerfile index 18b136fe03..be2d6b6d25 100644 --- a/ci/containers/opensuse-tumbleweed.Dockerfile +++ b/ci/containers/opensuse-tumbleweed.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile opensuse-tumbleweed libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/ubuntu-1804.Dockerfile b/ci/containers/ubuntu-1804.Dockerfile index 21b73e62aa..6dcd965e65 100644 --- a/ci/containers/ubuntu-1804.Dockerfile +++ b/ci/containers/ubuntu-1804.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile ubuntu-1804 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/containers/ubuntu-2004.Dockerfile b/ci/containers/ubuntu-2004.Dockerfile index 5654ad003f..f86eb9aef5 100644 --- a/ci/containers/ubuntu-2004.Dockerfile +++ b/ci/containers/ubuntu-2004.Dockerfile @@ -1,6 +1,6 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile ubuntu-2004 libvirt +# $ lcitool manifest ci/manifest.yml # # https://gitlab.com/libvirt/libvirt-ci diff --git a/ci/gitlab.yml b/ci/gitlab.yml new file mode 100644 index 0000000000..6063ce8351 --- /dev/null +++ b/ci/gitlab.yml @@ -0,0 +1,602 @@ +# THIS FILE WAS AUTO-GENERATED +# +# $ lcitool manifest ci/manifest.yml +# +# https://gitlab.com/libvirt/libvirt-ci + + +.container_job: + image: docker:stable + stage: containers + needs: [] + services: + - name: registry.gitlab.com/libvirt/libvirt-ci/docker-dind:master + alias: docker + before_script: + - export TAG="$CI_REGISTRY_IMAGE/ci-$NAME:latest" + - export COMMON_TAG="$CI_REGISTRY/libvirt/libvirt/ci-$NAME:latest" + - docker info + - docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" + script: + - docker pull "$TAG" || docker pull "$COMMON_TAG" || true + - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" -f "ci/containers/$NAME.Dockerfile" ci/containers + - docker push "$TAG" + after_script: + - docker logout + + +.gitlab_native_build_job: + image: $CI_REGISTRY_IMAGE/ci-$NAME:latest + stage: builds + + +.gitlab_cross_build_job: + image: $CI_REGISTRY_IMAGE/ci-$NAME-cross-$CROSS:latest + stage: builds + + +.cirrus_build_job: + stage: builds + image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master + needs: [] + script: + - source ci/cirrus/$NAME.vars + - sed -e "s|[@]CI_REPOSITORY_URL@|$CI_REPOSITORY_URL|g" + -e "s|[@]CI_COMMIT_REF_NAME@|$CI_COMMIT_REF_NAME|g" + -e "s|[@]CI_COMMIT_SHA@|$CI_COMMIT_SHA|g" + -e "s|[@]CIRRUS_VM_INSTANCE_TYPE@|$CIRRUS_VM_INSTANCE_TYPE|g" + -e "s|[@]CIRRUS_VM_IMAGE_SELECTOR@|$CIRRUS_VM_IMAGE_SELECTOR|g" + -e "s|[@]CIRRUS_VM_IMAGE_NAME@|$CIRRUS_VM_IMAGE_NAME|g" + -e "s|[@]UPDATE_COMMAND@|$UPDATE_COMMAND|g" + -e "s|[@]UPGRADE_COMMAND@|$UPGRADE_COMMAND|g" + -e "s|[@]INSTALL_COMMAND@|$INSTALL_COMMAND|g" + -e "s|[@]PATH@|$PATH_EXTRA${PATH_EXTRA:+:}\$PATH|g" + -e "s|[@]PKG_CONFIG_PATH@|$PKG_CONFIG_PATH|g" + -e "s|[@]PKGS@|$PKGS|g" + -e "s|[@]MAKE@|$MAKE|g" + -e "s|[@]PYTHON@|$PYTHON|g" + -e "s|[@]PIP3@|$PIP3|g" + -e "s|[@]PYPI_PKGS@|$PYPI_PKGS|g" + <ci/cirrus/build.yml >ci/cirrus/$NAME.yml + - cat ci/cirrus/$NAME.yml + - cirrus-run -v --show-build-log always ci/cirrus/$NAME.yml + rules: + - if: "$CIRRUS_GITHUB_REPO && $CIRRUS_API_TOKEN" + + +check-dco: + stage: sanity_checks + needs: [] + image: registry.gitlab.com/libvirt/libvirt-ci/check-dco:master + script: + - /check-dco libvirt + except: + variables: + - $CI_PROJECT_NAMESPACE == 'libvirt' + variables: + GIT_DEPTH: 1000 + + +# Native container jobs + +x86_64-centos-8-container: + extends: .container_job + allow_failure: false + variables: + NAME: centos-8 + + +x86_64-centos-stream-8-container: + extends: .container_job + allow_failure: false + variables: + NAME: centos-stream-8 + + +x86_64-debian-10-container: + extends: .container_job + allow_failure: false + variables: + NAME: debian-10 + + +x86_64-debian-sid-container: + extends: .container_job + allow_failure: false + variables: + NAME: debian-sid + + +x86_64-fedora-33-container: + extends: .container_job + allow_failure: false + variables: + NAME: fedora-33 + + +x86_64-fedora-34-container: + extends: .container_job + allow_failure: false + variables: + NAME: fedora-34 + + +x86_64-fedora-rawhide-container: + extends: .container_job + allow_failure: true + variables: + NAME: fedora-rawhide + + +x86_64-opensuse-leap-152-container: + extends: .container_job + allow_failure: false + variables: + NAME: opensuse-leap-152 + + +x86_64-opensuse-tumbleweed-container: + extends: .container_job + allow_failure: false + variables: + NAME: opensuse-tumbleweed + + +x86_64-ubuntu-1804-container: + extends: .container_job + allow_failure: false + variables: + NAME: ubuntu-1804 + + +x86_64-ubuntu-2004-container: + extends: .container_job + allow_failure: false + variables: + NAME: ubuntu-2004 + + + +# Cross container jobs + +aarch64-debian-10-container: + extends: .container_job + allow_failure: true + variables: + NAME: debian-10-cross-aarch64 + + +armv6l-debian-10-container: + extends: .container_job + allow_failure: false + variables: + NAME: debian-10-cross-armv6l + + +armv7l-debian-10-container: + extends: .container_job + allow_failure: false + variables: + NAME: debian-10-cross-armv7l + + +i686-debian-10-container: + extends: .container_job + allow_failure: true + variables: + NAME: debian-10-cross-i686 + + +mips-debian-10-container: + extends: .container_job + allow_failure: false + variables: + NAME: debian-10-cross-mips + + +mips64el-debian-10-container: + extends: .container_job + allow_failure: true + variables: + NAME: debian-10-cross-mips64el + + +mipsel-debian-10-container: + extends: .container_job + allow_failure: false + variables: + NAME: debian-10-cross-mipsel + + +ppc64le-debian-10-container: + extends: .container_job + allow_failure: false + variables: + NAME: debian-10-cross-ppc64le + + +s390x-debian-10-container: + extends: .container_job + allow_failure: true + variables: + NAME: debian-10-cross-s390x + + +aarch64-debian-sid-container: + extends: .container_job + allow_failure: false + variables: + NAME: debian-sid-cross-aarch64 + + +armv6l-debian-sid-container: + extends: .container_job + allow_failure: true + variables: + NAME: debian-sid-cross-armv6l + + +armv7l-debian-sid-container: + extends: .container_job + allow_failure: true + variables: + NAME: debian-sid-cross-armv7l + + +i686-debian-sid-container: + extends: .container_job + allow_failure: false + variables: + NAME: debian-sid-cross-i686 + + +mips64el-debian-sid-container: + extends: .container_job + allow_failure: false + variables: + NAME: debian-sid-cross-mips64el + + +mipsel-debian-sid-container: + extends: .container_job + allow_failure: true + variables: + NAME: debian-sid-cross-mipsel + + +ppc64le-debian-sid-container: + extends: .container_job + allow_failure: true + variables: + NAME: debian-sid-cross-ppc64le + + +s390x-debian-sid-container: + extends: .container_job + allow_failure: false + variables: + NAME: debian-sid-cross-s390x + + +mingw32-fedora-rawhide-container: + extends: .container_job + allow_failure: true + variables: + NAME: fedora-rawhide-cross-mingw32 + + +mingw64-fedora-rawhide-container: + extends: .container_job + allow_failure: true + variables: + NAME: fedora-rawhide-cross-mingw64 + + +# Native build jobs + +x86_64-centos-8: + extends: .native_build_job + needs: + - x86_64-centos-8-container + allow_failure: false + variables: + NAME: centos-8 + RPM: skip + + +x86_64-centos-8-clang: + extends: .native_build_job + needs: + - x86_64-centos-8-container + allow_failure: false + variables: + NAME: centos-8 + RPM: skip + CC: clang + + +x86_64-centos-stream-8: + extends: .native_build_job + needs: + - x86_64-centos-stream-8-container + allow_failure: false + variables: + NAME: centos-stream-8 + RPM: skip + + +x86_64-debian-10: + extends: .native_build_job + needs: + - x86_64-debian-10-container + allow_failure: false + variables: + NAME: debian-10 + + +x86_64-debian-10-clang: + extends: .native_build_job + needs: + - x86_64-debian-10-container + allow_failure: false + variables: + NAME: debian-10 + + +x86_64-debian-sid: + extends: .native_build_job + needs: + - x86_64-debian-sid-container + allow_failure: false + variables: + NAME: debian-sid + + +x86_64-fedora-33: + extends: .native_build_job + needs: + - x86_64-fedora-33-container + allow_failure: false + variables: + NAME: fedora-33 + + +x86_64-fedora-34: + extends: .native_build_job + needs: + - x86_64-fedora-34-container + allow_failure: false + variables: + NAME: fedora-34 + + +x86_64-fedora-rawhide: + extends: .native_build_job + needs: + - x86_64-fedora-rawhide-container + allow_failure: true + variables: + NAME: fedora-rawhide + + +x86_64-fedora-rawhide-clang: + extends: .native_build_job + needs: + - x86_64-fedora-rawhide-container + allow_failure: true + variables: + NAME: fedora-rawhide + CC: clang + RPM: skip + + +x86_64-opensuse-leap-152: + extends: .native_build_job + needs: + - x86_64-opensuse-leap-152-container + allow_failure: false + variables: + NAME: opensuse-leap-152 + RPM: skip + + +x86_64-opensuse-tumbleweed: + extends: .native_build_job + needs: + - x86_64-opensuse-tumbleweed-container + allow_failure: false + variables: + NAME: opensuse-tumbleweed + RPM: skip + + +x86_64-ubuntu-1804: + extends: .native_build_job + needs: + - x86_64-ubuntu-1804-container + allow_failure: false + variables: + NAME: ubuntu-1804 + + +x86_64-ubuntu-2004: + extends: .native_build_job + needs: + - x86_64-ubuntu-2004-container + allow_failure: false + variables: + NAME: ubuntu-2004 + ASAN_OPTIONS: verify_asan_link_order=0 + MESON_ARGS: -Db_lundef=false -Db_sanitize=address,undefined + UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1 + + +x86_64-ubuntu-2004-clang: + extends: .native_build_job + needs: + - x86_64-ubuntu-2004-container + allow_failure: false + variables: + NAME: ubuntu-2004 + CC: clang + MESON_ARGS: -Db_lundef=false -Db_sanitize=address,undefined + UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1 + + + +# Cross build jobs + +armv6l-debian-10: + extends: .cross_build_job + needs: + - armv6l-debian-10-container + allow_failure: false + variables: + NAME: debian-10 + CROSS: armv6l + + +armv7l-debian-10: + extends: .cross_build_job + needs: + - armv7l-debian-10-container + allow_failure: false + variables: + NAME: debian-10 + CROSS: armv7l + + +mips-debian-10: + extends: .cross_build_job + needs: + - mips-debian-10-container + allow_failure: false + variables: + NAME: debian-10 + CROSS: mips + + +mipsel-debian-10: + extends: .cross_build_job + needs: + - mipsel-debian-10-container + allow_failure: false + variables: + NAME: debian-10 + CROSS: mipsel + + +ppc64le-debian-10: + extends: .cross_build_job + needs: + - ppc64le-debian-10-container + allow_failure: false + variables: + NAME: debian-10 + CROSS: ppc64le + + +aarch64-debian-sid: + extends: .cross_build_job + needs: + - aarch64-debian-sid-container + allow_failure: false + variables: + NAME: debian-sid + CROSS: aarch64 + + +i686-debian-sid: + extends: .cross_build_job + needs: + - i686-debian-sid-container + allow_failure: false + variables: + NAME: debian-sid + CROSS: i686 + + +mips64el-debian-sid: + extends: .cross_build_job + needs: + - mips64el-debian-sid-container + allow_failure: false + variables: + NAME: debian-sid + CROSS: mips64el + + +s390x-debian-sid: + extends: .cross_build_job + needs: + - s390x-debian-sid-container + allow_failure: false + variables: + NAME: debian-sid + CROSS: s390x + + +mingw32-fedora-rawhide: + extends: .cross_build_job + needs: + - mingw32-fedora-rawhide-container + allow_failure: true + variables: + NAME: fedora-rawhide + CROSS: mingw32 + + +mingw64-fedora-rawhide: + extends: .cross_build_job + needs: + - mingw64-fedora-rawhide-container + allow_failure: true + variables: + NAME: fedora-rawhide + CROSS: mingw64 + + +# Native cirrus build jobs + +x86_64-freebsd-12: + extends: .cirrus_build_job + needs: [] + allow_failure: false + variables: + NAME: freebsd-12 + CIRRUS_VM_INSTANCE_TYPE: freebsd_instance + CIRRUS_VM_IMAGE_SELECTOR: image_family + CIRRUS_VM_IMAGE_NAME: freebsd-12-2 + UPDATE_COMMAND: pkg update + UPGRADE_COMMAND: pkg upgrade -y + INSTALL_COMMAND: pkg install -y + + +x86_64-freebsd-13: + extends: .cirrus_build_job + needs: [] + allow_failure: false + variables: + NAME: freebsd-13 + CIRRUS_VM_INSTANCE_TYPE: freebsd_instance + CIRRUS_VM_IMAGE_SELECTOR: image_family + CIRRUS_VM_IMAGE_NAME: freebsd-13-0 + UPDATE_COMMAND: pkg update + UPGRADE_COMMAND: pkg upgrade -y + INSTALL_COMMAND: pkg install -y + + +x86_64-macos-11: + extends: .cirrus_build_job + needs: [] + allow_failure: false + variables: + NAME: macos-11 + CIRRUS_VM_INSTANCE_TYPE: osx_instance + CIRRUS_VM_IMAGE_SELECTOR: image + CIRRUS_VM_IMAGE_NAME: big-sur-base + UPDATE_COMMAND: brew update + UPGRADE_COMMAND: brew upgrade + INSTALL_COMMAND: brew install + PATH_EXTRA: /usr/local/opt/ccache/libexec:/usr/local/opt/gettext/bin:/usr/local/opt/libpcap/bin:/usr/local/opt/libxslt/bin:/usr/local/opt/rpcgen/bin + PKG_CONFIG_PATH: /usr/local/opt/curl/lib/pkgconfig:/usr/local/opt/libpcap/lib/pkgconfig:/usr/local/opt/libxml2/lib/pkgconfig:/usr/local/opt/ncurses/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig -- 2.31.1

We now use lcitool's manifest feature to generate files. The logic for checking for stale containers in the registry, however, is still relevant so that is propagated to a standalone command. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com&gt; --- ci/helper | 121 ++++------------------------------------------------- ci/util.py | 32 +++++++------- 2 files changed, 23 insertions(+), 130 deletions(-) diff --git a/ci/helper b/ci/helper index 441258f511..8b8d0f68cb 100755 --- a/ci/helper +++ b/ci/helper @@ -7,7 +7,6 @@ import argparse import os import pathlib import pty -import shutil import subprocess import sys import textwrap @@ -60,15 +59,6 @@ class Parser: help="additional arguments passed to ninja", ) - # Options that are common to all actions that use lcitool - lcitoolparser = argparse.ArgumentParser(add_help=False) - lcitoolparser.add_argument( - "--lcitool", - metavar="PATH", - default="lcitool", - help="path to lcitool binary", - ) - # Options that are common to actions communicating with a GitLab # instance gitlabparser = argparse.ArgumentParser(add_help=False) @@ -127,27 +117,14 @@ class Parser: ) listimagesparser.set_defaults(func=Application._action_list_images) - # refresh action - refreshparser = subparsers.add_parser( - "refresh", - help="refresh data generated with lcitool", - parents=[lcitoolparser, gitlabparser], + # check_stale action + check_staleparser = subparsers.add_parser( + "check-stale", + help="check for existence of stale images on the GitLab instance", + parents=[gitlabparser], formatter_class=argparse.ArgumentDefaultsHelpFormatter, ) - refreshparser.add_argument( - "--quiet", - action="store_true", - default=False, - help="refresh data silently" - ) - refreshparser.add_argument( - "--check-stale", - action="store", - choices=["yes", "no"], - default="yes", - help="check for existence of stale images on the GitLab instance" - ) - refreshparser.set_defaults(func=Application._action_refresh) + check_staleparser.set_defaults(func=Application._action_check_stale) def parse(self): return self._parser.parse_args() @@ -158,10 +135,6 @@ class Application: self._basedir = pathlib.Path(__file__).resolve().parent self._args = Parser().parse() - if self._args.action == "refresh": - if not shutil.which(self._args.lcitool): - sys.exit("error: 'lcitool' not installed") - def _make_run(self, target): args = [ "-C", @@ -194,84 +167,12 @@ class Application: output = self._lcitool_run(["targets"]) return output.splitlines() - def _generate_dockerfile(self, target, cross=None): - args = ["dockerfile", target, "libvirt"] - outdir = self._basedir.joinpath("containers") - outfile = f"{target}.Dockerfile" - - if cross: - args.extend(["--cross", cross]) - outfile = f"{target}-cross-{cross}.Dockerfile" - - outpath = outdir.joinpath(outfile) - if not self._args.quiet: - print(outpath) - - output = self._lcitool_run(args) - with open(outpath, "w") as f: - f.write(output) - - def _generate_vars(self, target): - args = ["variables", target, "libvirt"] - outdir = self._basedir.joinpath("cirrus") - outfile = f"{target}.vars" - - outpath = outdir.joinpath(outfile) - if not self._args.quiet: - print(outpath) - - output = self._lcitool_run(args) - with open(outpath, "w") as f: - f.write(output) - - def _refresh_containers(self): - debian_cross = [ - "aarch64", - "armv6l", - "armv7l", - "i686", - "mips", - "mips64el", - "mipsel", - "ppc64le", - "s390x", - ] - fedora_cross = [ - "mingw32", - "mingw64", - ] - - for target in self._lcitool_get_targets(): - if target.startswith("freebsd-") or target.startswith("macos-"): - continue - - self._generate_dockerfile(target) - - if target == "fedora-rawhide": - for cross in fedora_cross: - self._generate_dockerfile(target, cross) - - if target.startswith("debian-"): - for cross in debian_cross: - if target == "debian-sid" and cross == "mips": - continue - self._generate_dockerfile(target, cross) - - def _refresh_cirrus(self): - for target in self._lcitool_get_targets(): - if not (target.startswith("freebsd-") or target.startswith("macos-")): - continue - - self._generate_vars(target) - def _check_stale_images(self): namespace = self._args.namespace gitlab_uri = self._args.gitlab_uri registry_uri = util.get_registry_uri(namespace, gitlab_uri) - lcitool_targets = self._lcitool_get_targets() - stale_images = util.get_registry_stale_images(registry_uri, - lcitool_targets) + stale_images = util.get_registry_stale_images(registry_uri, self._basedir) if stale_images: spacing = "\n" + 4 * " " stale_fmt = [f"{k} (ID: {v})" for k, v in stale_images.items()] @@ -328,12 +229,8 @@ class Application: print("Available cross-compiler container images:\n") print(spacing + ("\n" + spacing).join(cross)) - def _action_refresh(self): - self._refresh_containers() - self._refresh_cirrus() - - if self._args.check_stale == "yes" and not self._args.quiet: - self._check_stale_images() + def _action_check_stale(self): + self._check_stale_images() def run(self): self._args.func(self) diff --git a/ci/util.py b/ci/util.py index 90d58454be..d8be6631eb 100644 --- a/ci/util.py +++ b/ci/util.py @@ -1,4 +1,5 @@ import json +import pathlib import urllib.request import urllib.parse @@ -40,42 +41,37 @@ def get_registry_images(uri: str) -> List[Dict]: return json.loads(r.read().decode()) -def get_image_distro(image_name: str) -> str: +def get_dockerfiles(base_dir) -> List: """ - Extract the name of the distro in the GitLab image registry name, e.g. - ci-debian-9-cross-mipsel --> debian-9 + List all container dockerfiles in the local directory. - :param image_name: name of the GitLab registry image - :return: distro name as a string + :return: list of dockerfile names """ - name_prefix = "ci-" - name_suffix = "-cross-" - distro = image_name[len(name_prefix):] + dkrs = [] + d = pathlib.Path(base_dir, "containers") + for f in d.iterdir(): + if f.suffix == ".Dockerfile": + dkrs.append(f.stem) + return dkrs - index = distro.find(name_suffix) - if index > 0: - distro = distro[:index] - return distro - - -def get_registry_stale_images(registry_uri: str, - supported_distros: List[str]) -> Dict[str, int]: +def get_registry_stale_images(registry_uri: str, base_dir: str) -> Dict[str, int]: """ Check the GitLab image registry for images that we no longer support and which should be deleted. :param uri: URI pointing to a GitLab instance's image registry - :param supported_distros: list of hosts supported by lcitool + :param base_dir: local repository base directory :return: dictionary formatted as: {<gitlab_image_name>: <gitlab_image_id>} """ + dockerfiles = get_dockerfiles(base_dir) images = get_registry_images(registry_uri) stale_images = {} for img in images: - if get_image_distro(img["name"]) not in supported_distros: + if img["name"][3:] not in dockerfiles: stale_images[img["name"]] = img["id"] return stale_images -- 2.31.1