[libvirt PATCH 0/4] Functional CI - GitLab enablement
RFC here [1] Since RFC: - renamed the stage to integration_tests - dropped both the test child pipeline as well as triggering a multi-project CI pipeline to build the Perl bindings and instead use the latest bindings artifacts we have available from the libvirt-perl project. -> basically now we only have a regular CI stage with no extra jobs that just spins up a bunch of private runners and uploads RPM artifacts into them and runs the TCK tests - replaced sed invocations to edit daemon logging settings with augtool - used a pattern for log filters from [2] - used the "$CI_JOB_STATUS" variable in the after_script to determine whether the main script failed so as not to try moving and collecting logs on successful jobs (the logs would have been published as artifacts on failures anyway) Demo pipeline: https://gitlab.com/eskultety/libvirt/-/pipelines/491838828 [1] https://listman.redhat.com/archives/libvir-list/2022-January/227947.html [2] https://libvirt.org/kbase/debuglogs.html#less-verbose-logging-for-qemu-vms Erik Skultety (4): ci: gitlab: Refresh gitlab.yml ci: manifest: Publish RPMs as artifacts on CentOS Stream and Fedoras gitlab-ci: Introduce new 'integration_tests' pipeline stage DO NOT MERGE: Demo a functional CI pipeline running Fedoras CentOS only .gitlab-ci.yml | 99 +------- ci/gitlab.yml | 606 ++------------------------------------------- ci/integration.yml | 99 ++++++++ ci/manifest.yml | 21 +- 4 files changed, 142 insertions(+), 683 deletions(-) create mode 100644 ci/integration.yml -- 2.34.1
Signed-off-by: Erik Skultety <eskultet@redhat.com> --- ci/gitlab.yml | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/ci/gitlab.yml b/ci/gitlab.yml index cc03a2fe49..9825624938 100644 --- a/ci/gitlab.yml +++ b/ci/gitlab.yml @@ -10,8 +10,7 @@ stage: containers needs: [] services: - - name: registry.gitlab.com/libvirt/libvirt-ci/docker-dind:master - alias: docker + - docker:dind before_script: - export TAG="$CI_REGISTRY_IMAGE/ci-$NAME:latest" - export COMMON_TAG="$CI_REGISTRY/libvirt/libvirt/ci-$NAME:latest" -- 2.34.1
We're already building libvirt in the containers already, if we publish the build in form of, say, RPMs, later stages of the pipeline can consume the RPMs instead of re-building libvirt from scratch. Signed-off-by: Erik Skultety <eskultet@redhat.com> --- .gitlab-ci.yml | 3 ++- ci/gitlab.yml | 16 ++++++++++++++++ ci/manifest.yml | 21 ++++++++++++++++++++- 3 files changed, 38 insertions(+), 2 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 24533a937a..d84ac31601 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -30,7 +30,8 @@ include: '/ci/gitlab.yml' - meson dist -C build --no-tests - if test -x /usr/bin/rpmbuild && test "$RPM" != "skip"; then - rpmbuild --nodeps -ta build/meson-dist/libvirt-*.tar.xz; + rpmbuild --clean --nodeps --define "_topdir $PWD/rpmbuild/" -ta build/meson-dist/libvirt-*.tar.xz; + mv rpmbuild/RPMS/x86_64/ libvirt-rpms/; else meson compile -C build; meson test -C build --no-suite syntax-check --print-errorlogs; diff --git a/ci/gitlab.yml b/ci/gitlab.yml index 9825624938..61f8ce03ef 100644 --- a/ci/gitlab.yml +++ b/ci/gitlab.yml @@ -406,6 +406,10 @@ x86_64-centos-stream-8: allow_failure: false variables: NAME: centos-stream-8 + artifacts: + expire_in: 1 day + paths: + - libvirt-rpms x86_64-centos-stream-9: @@ -415,6 +419,10 @@ x86_64-centos-stream-9: allow_failure: false variables: NAME: centos-stream-9 + artifacts: + expire_in: 1 day + paths: + - libvirt-rpms x86_64-debian-10: @@ -460,6 +468,10 @@ x86_64-fedora-34: allow_failure: false variables: NAME: fedora-34 + artifacts: + expire_in: 1 day + paths: + - libvirt-rpms x86_64-fedora-35: @@ -469,6 +481,10 @@ x86_64-fedora-35: allow_failure: false variables: NAME: fedora-35 + artifacts: + expire_in: 1 day + paths: + - libvirt-rpms x86_64-fedora-rawhide: diff --git a/ci/manifest.yml b/ci/manifest.yml index 87d923ae78..1873efa0ff 100644 --- a/ci/manifest.yml +++ b/ci/manifest.yml @@ -21,10 +21,19 @@ targets: centos-stream-8: jobs: - arch: x86_64 + artifacts: + expire_in: 1 day + paths: + - libvirt-rpms centos-stream-9: jobs: - arch: x86_64 + artifacts: + expire_in: 1 day + paths: + - libvirt-rpms + debian-10: jobs: - arch: x86_64 @@ -126,11 +135,21 @@ targets: - arch: s390x allow-failure: true - fedora-34: x86_64 + fedora-34: + jobs: + - arch: x86_64 + artifacts: + expire_in: 1 day + paths: + - libvirt-rpms fedora-35: jobs: - arch: x86_64 + artifacts: + expire_in: 1 day + paths: + - libvirt-rpms - arch: mingw32 allow-failure: true -- 2.34.1
This stage will download build artifacts from both the libvirt and libvirt-perl (multi-project CI) builds, install all them on the custom runners and configures libvirt debug logging on the runners prior to executing the actual test suite. In case of a failure, libvirt and Avocado logs will be saved and published as pipeline artifacts. Signed-off-by: Erik Skultety <eskultet@redhat.com> --- .gitlab-ci.yml | 5 ++- ci/integration.yml | 99 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 103 insertions(+), 1 deletion(-) create mode 100644 ci/integration.yml diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index d84ac31601..6b4c843055 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -4,6 +4,7 @@ variables: stages: - containers - builds + - integration_tests - sanity_checks .script_variables: &script_variables | @@ -14,7 +15,9 @@ stages: export VIR_TEST_VERBOSE="1" export VIR_TEST_DEBUG="1" -include: '/ci/gitlab.yml' +include: + - '/ci/gitlab.yml' + - '/ci/integration.yml' .native_build_job: extends: .gitlab_native_build_job diff --git a/ci/integration.yml b/ci/integration.yml new file mode 100644 index 0000000000..9a36ccc05e --- /dev/null +++ b/ci/integration.yml @@ -0,0 +1,99 @@ +.tests: + stage: integration_tests + before_script: + - mkdir "$SCRATCH_DIR" + - sudo dnf install -y libvirt-rpms/* libvirt-perl-rpms/* + - sudo pip3 install --prefix=/usr avocado-framework + - source /etc/os-release # in order to query the vendor-provided variables + - if test "$ID" = "centos" && test "$VERSION_ID" -lt 9 || + test "$ID" = "fedora" && test "$VERSION_ID" -lt 35; + then + DAEMONS="libvirtd virtlogd virtlockd"; + else + DAEMONS="virtproxyd virtqemud virtinterfaced virtsecretd virtstoraged virtnwfilterd virtnodedevd virtlogd virtlockd"; + fi + - for daemon in $DAEMONS; + do + LOG_OUTPUTS="1:file:/var/log/libvirt/${daemon}.log"; + LOG_FILTERS="3:remote 4:event 3:util.json 3:util.object 3:util.dbus 3:util.netlink 3:node_device 3:rpc 3:access 1:*"; + sudo augtool set /files/etc/libvirt/${daemon}.conf/log_filters "$LOG_FILTERS" &>/dev/null; + sudo augtool set /files/etc/libvirt/${daemon}.conf/log_outputs "$LOG_OUTPUTS" &>/dev/null; + sudo systemctl --quiet stop ${daemon}.service; + sudo systemctl restart ${daemon}.socket; + done + - sudo virsh net-start default &>/dev/null || true; + script: + - mkdir logs + - cd "$SCRATCH_DIR" + - git clone --depth 1 https://gitlab.com/libvirt/libvirt-tck.git + - cd libvirt-tck + - sudo avocado --config avocado.config run --job-results-dir "$SCRATCH_DIR"/avocado + after_script: + - test "$CI_JOB_STATUS" = "success" && exit 0; + - test -e "$SCRATCH_DIR"/avocado && sudo mv "$SCRATCH_DIR"/avocado/latest/test-results logs/avocado; + - sudo mv /var/log/libvirt logs/libvirt + - sudo chown -R $(whoami):$(whoami) logs + variables: + SCRATCH_DIR: "/tmp/scratch" + artifacts: + name: logs + paths: + - logs + when: on_failure + + +centos-stream-8-tests: + extends: .tests + needs: + - x86_64-centos-stream-8 + - project: libvirt/libvirt-perl + job: x86_64-centos-stream-8 + ref: master + artifacts: true + variables: + # needed by libvirt-gitlab-executor + DISTRO: centos-stream-8 + tags: + - redhat-vm-host + +centos-stream-9-tests: + extends: .tests + needs: + - x86_64-centos-stream-9 + - project: libvirt/libvirt-perl + job: x86_64-centos-stream-9 + ref: master + artifacts: true + variables: + # needed by libvirt-gitlab-executor + DISTRO: centos-stream-9 + tags: + - redhat-vm-host + +fedora-34-tests: + extends: .tests + needs: + - x86_64-fedora-34 + - project: libvirt/libvirt-perl + job: x86_64-fedora-34 + ref: master + artifacts: true + variables: + # needed by libvirt-gitlab-executor + DISTRO: fedora-34 + tags: + - redhat-vm-host + +fedora-35-tests: + extends: .tests + needs: + - x86_64-fedora-35 + - project: libvirt/libvirt-perl + job: x86_64-fedora-35 + ref: master + artifacts: true + variables: + # needed by libvirt-gitlab-executor + DISTRO: fedora-35 + tags: + - redhat-vm-host -- 2.34.1
This patch is solely to demonstrate the new CI integration_tests stage without having to go through and see all the other stages (cross builds, coverity, webpage) we have and so a bunch of other targets and template jobs were dropped just to decrease the information and optical noise. --- .gitlab-ci.yml | 91 -------- ci/gitlab.yml | 587 ------------------------------------------------- 2 files changed, 678 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 6b4c843055..b003ba95b5 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -40,94 +40,3 @@ include: meson test -C build --no-suite syntax-check --print-errorlogs; fi -.cross_build_job: - extends: .gitlab_cross_build_job - cache: - paths: - - ccache/ - key: "$CI_JOB_NAME" - before_script: - - *script_variables - - cat /packages.txt - script: - - meson setup build --werror $MESON_OPTS || (cat build/meson-logs/meson-log.txt && exit 1) - - meson compile -C build - - if test "$CROSS" = "i686" ; then meson test -C build --no-suite syntax-check --print-errorlogs ; fi - - -# This artifact published by this job is downloaded by libvirt.org to -# be deployed to the web root: -# https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=webs... -website: - stage: builds - image: $CI_REGISTRY_IMAGE/ci-almalinux-8:latest - needs: - - x86_64-almalinux-8-container - before_script: - - *script_variables - script: - - meson setup build --werror --prefix=$(pwd)/vroot || (cat build/meson-logs/meson-log.txt && exit 1) - - ninja -C build install-web - - mv vroot/share/doc/libvirt/html/ website - artifacts: - expose_as: 'Website' - name: 'website' - when: on_success - expire_in: 30 days - paths: - - website - - -codestyle: - stage: sanity_checks - image: $CI_REGISTRY_IMAGE/ci-opensuse-leap-152:latest - needs: - - x86_64-opensuse-leap-152-container - before_script: - - *script_variables - script: - - meson setup build --werror || (cat build/meson-logs/meson-log.txt && exit 1) - - ninja -C build libvirt-pot-dep - - meson test -C build --suite syntax-check --no-rebuild --print-errorlogs - - -# This artifact published by this job is downloaded to push to Weblate -# for translation usage: -# https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=potf... -potfile: - stage: builds - image: $CI_REGISTRY_IMAGE/ci-almalinux-8:latest - needs: - - x86_64-almalinux-8-container - rules: - - if: "$CI_COMMIT_BRANCH == 'master'" - before_script: - - *script_variables - script: - - meson setup build --werror || (cat build/meson-logs/meson-log.txt && exit 1) - - ninja -C build libvirt-pot-dep - - ninja -C build libvirt-pot - - cp po/libvirt.pot libvirt.pot - artifacts: - expose_as: 'Potfile' - name: 'potfile' - when: on_success - expire_in: 30 days - paths: - - libvirt.pot - -# Coverity job that is run only by schedules -coverity: - image: $CI_REGISTRY_IMAGE/ci-almalinux-8:latest - needs: - - x86_64-almalinux-8-container - stage: builds - script: - - curl https://scan.coverity.com/download/linux64 --form project=$COVERITY_SCAN_PROJECT_NAME --form token=$COVERITY_SCAN_TOKEN -o /tmp/cov-analysis-linux64.tgz - - tar xfz /tmp/cov-analysis-linux64.tgz - - meson setup build --werror || (cat build/meson-logs/meson-log.txt && exit 1) - - cov-analysis-linux64-*/bin/cov-build --dir cov-int meson compile -C build - - tar cfz cov-int.tar.gz cov-int - - curl https://scan.coverity.com/builds?project=$COVERITY_SCAN_PROJECT_NAME --form token=$COVERITY_SCAN_TOKEN --form email=$GITLAB_USER_EMAIL --form file=@cov-int.tar.gz --form version="$(git describe --tags)" --form description="$(git describe --tags) / $CI_COMMIT_TITLE / $CI_COMMIT_REF_NAME:$CI_PIPELINE_ID" - rules: - - if: "$CI_PIPELINE_SOURCE == 'schedule' && $COVERITY_SCAN_PROJECT_NAME && $COVERITY_SCAN_TOKEN" diff --git a/ci/gitlab.yml b/ci/gitlab.yml index 61f8ce03ef..0f578664be 100644 --- a/ci/gitlab.yml +++ b/ci/gitlab.yml @@ -34,36 +34,6 @@ stage: builds -.cirrus_build_job: - stage: builds - image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master - needs: [] - script: - - source ci/cirrus/$NAME.vars - - sed -e "s|[@]CI_REPOSITORY_URL@|$CI_REPOSITORY_URL|g" - -e "s|[@]CI_COMMIT_REF_NAME@|$CI_COMMIT_REF_NAME|g" - -e "s|[@]CI_COMMIT_SHA@|$CI_COMMIT_SHA|g" - -e "s|[@]CIRRUS_VM_INSTANCE_TYPE@|$CIRRUS_VM_INSTANCE_TYPE|g" - -e "s|[@]CIRRUS_VM_IMAGE_SELECTOR@|$CIRRUS_VM_IMAGE_SELECTOR|g" - -e "s|[@]CIRRUS_VM_IMAGE_NAME@|$CIRRUS_VM_IMAGE_NAME|g" - -e "s|[@]UPDATE_COMMAND@|$UPDATE_COMMAND|g" - -e "s|[@]UPGRADE_COMMAND@|$UPGRADE_COMMAND|g" - -e "s|[@]INSTALL_COMMAND@|$INSTALL_COMMAND|g" - -e "s|[@]PATH@|$PATH_EXTRA${PATH_EXTRA:+:}\$PATH|g" - -e "s|[@]PKG_CONFIG_PATH@|$PKG_CONFIG_PATH|g" - -e "s|[@]PKGS@|$PKGS|g" - -e "s|[@]MAKE@|$MAKE|g" - -e "s|[@]PYTHON@|$PYTHON|g" - -e "s|[@]PIP3@|$PIP3|g" - -e "s|[@]PYPI_PKGS@|$PYPI_PKGS|g" - -e "s|[@]XML_CATALOG_FILES@|$XML_CATALOG_FILES|g" - <ci/cirrus/build.yml >ci/cirrus/$NAME.yml - - cat ci/cirrus/$NAME.yml - - cirrus-run -v --show-build-log always ci/cirrus/$NAME.yml - rules: - - if: "$CIRRUS_GITHUB_REPO && $CIRRUS_API_TOKEN" - - check-dco: stage: sanity_checks needs: [] @@ -79,12 +49,6 @@ check-dco: # Native container jobs -x86_64-almalinux-8-container: - extends: .container_job - allow_failure: false - variables: - NAME: almalinux-8 - x86_64-centos-stream-8-container: extends: .container_job @@ -100,27 +64,6 @@ x86_64-centos-stream-9-container: NAME: centos-stream-9 -x86_64-debian-10-container: - extends: .container_job - allow_failure: false - variables: - NAME: debian-10 - - -x86_64-debian-11-container: - extends: .container_job - allow_failure: false - variables: - NAME: debian-11 - - -x86_64-debian-sid-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-sid - - x86_64-fedora-34-container: extends: .container_job allow_failure: false @@ -135,270 +78,8 @@ x86_64-fedora-35-container: NAME: fedora-35 -x86_64-fedora-rawhide-container: - extends: .container_job - allow_failure: true - variables: - NAME: fedora-rawhide - - -x86_64-opensuse-leap-152-container: - extends: .container_job - allow_failure: false - variables: - NAME: opensuse-leap-152 - - -x86_64-opensuse-tumbleweed-container: - extends: .container_job - allow_failure: true - variables: - NAME: opensuse-tumbleweed - - -x86_64-ubuntu-1804-container: - extends: .container_job - allow_failure: false - variables: - NAME: ubuntu-1804 - - -x86_64-ubuntu-2004-container: - extends: .container_job - allow_failure: false - variables: - NAME: ubuntu-2004 - - - -# Cross container jobs - -aarch64-debian-10-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-10-cross-aarch64 - - -armv6l-debian-10-container: - extends: .container_job - allow_failure: false - variables: - NAME: debian-10-cross-armv6l - - -armv7l-debian-10-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-10-cross-armv7l - - -i686-debian-10-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-10-cross-i686 - - -mips-debian-10-container: - extends: .container_job - allow_failure: false - variables: - NAME: debian-10-cross-mips - - -mips64el-debian-10-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-10-cross-mips64el - - -mipsel-debian-10-container: - extends: .container_job - allow_failure: false - variables: - NAME: debian-10-cross-mipsel - - -ppc64le-debian-10-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-10-cross-ppc64le - - -s390x-debian-10-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-10-cross-s390x - - -aarch64-debian-11-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-11-cross-aarch64 - - -armv6l-debian-11-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-11-cross-armv6l - - -armv7l-debian-11-container: - extends: .container_job - allow_failure: false - variables: - NAME: debian-11-cross-armv7l - - -i686-debian-11-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-11-cross-i686 - - -mips64el-debian-11-container: - extends: .container_job - allow_failure: false - variables: - NAME: debian-11-cross-mips64el - - -mipsel-debian-11-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-11-cross-mipsel - - -ppc64le-debian-11-container: - extends: .container_job - allow_failure: false - variables: - NAME: debian-11-cross-ppc64le - - -s390x-debian-11-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-11-cross-s390x - - -aarch64-debian-sid-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-sid-cross-aarch64 - - -armv6l-debian-sid-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-sid-cross-armv6l - - -armv7l-debian-sid-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-sid-cross-armv7l - - -i686-debian-sid-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-sid-cross-i686 - - -mips64el-debian-sid-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-sid-cross-mips64el - - -mipsel-debian-sid-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-sid-cross-mipsel - - -ppc64le-debian-sid-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-sid-cross-ppc64le - - -s390x-debian-sid-container: - extends: .container_job - allow_failure: true - variables: - NAME: debian-sid-cross-s390x - - -mingw32-fedora-35-container: - extends: .container_job - allow_failure: true - variables: - NAME: fedora-35-cross-mingw32 - - -mingw64-fedora-35-container: - extends: .container_job - allow_failure: false - variables: - NAME: fedora-35-cross-mingw64 - - -mingw32-fedora-rawhide-container: - extends: .container_job - allow_failure: true - variables: - NAME: fedora-rawhide-cross-mingw32 - - -mingw64-fedora-rawhide-container: - extends: .container_job - allow_failure: true - variables: - NAME: fedora-rawhide-cross-mingw64 - - # Native build jobs -x86_64-almalinux-8: - extends: .native_build_job - needs: - - x86_64-almalinux-8-container - allow_failure: false - variables: - NAME: almalinux-8 - RPM: skip - - -x86_64-almalinux-8-clang: - extends: .native_build_job - needs: - - x86_64-almalinux-8-container - allow_failure: false - variables: - CC: clang - NAME: almalinux-8 - RPM: skip - - x86_64-centos-stream-8: extends: .native_build_job needs: @@ -425,42 +106,6 @@ x86_64-centos-stream-9: - libvirt-rpms -x86_64-debian-10: - extends: .native_build_job - needs: - - x86_64-debian-10-container - allow_failure: false - variables: - NAME: debian-10 - - -x86_64-debian-11: - extends: .native_build_job - needs: - - x86_64-debian-11-container - allow_failure: false - variables: - NAME: debian-11 - - -x86_64-debian-11-clang: - extends: .native_build_job - needs: - - x86_64-debian-11-container - allow_failure: false - variables: - NAME: debian-11 - - -x86_64-debian-sid: - extends: .native_build_job - needs: - - x86_64-debian-sid-container - allow_failure: true - variables: - NAME: debian-sid - - x86_64-fedora-34: extends: .native_build_job needs: @@ -485,235 +130,3 @@ x86_64-fedora-35: expire_in: 1 day paths: - libvirt-rpms - - -x86_64-fedora-rawhide: - extends: .native_build_job - needs: - - x86_64-fedora-rawhide-container - allow_failure: true - variables: - NAME: fedora-rawhide - - -x86_64-fedora-rawhide-clang: - extends: .native_build_job - needs: - - x86_64-fedora-rawhide-container - allow_failure: true - variables: - CC: clang - NAME: fedora-rawhide - RPM: skip - - -x86_64-opensuse-leap-152: - extends: .native_build_job - needs: - - x86_64-opensuse-leap-152-container - allow_failure: false - variables: - NAME: opensuse-leap-152 - RPM: skip - - -x86_64-opensuse-tumbleweed: - extends: .native_build_job - needs: - - x86_64-opensuse-tumbleweed-container - allow_failure: true - variables: - NAME: opensuse-tumbleweed - RPM: skip - - -x86_64-ubuntu-1804: - extends: .native_build_job - needs: - - x86_64-ubuntu-1804-container - allow_failure: false - variables: - NAME: ubuntu-1804 - - -x86_64-ubuntu-2004: - extends: .native_build_job - needs: - - x86_64-ubuntu-2004-container - allow_failure: false - variables: - ASAN_OPTIONS: verify_asan_link_order=0 - MESON_ARGS: -Db_lundef=false -Db_sanitize=address,undefined - NAME: ubuntu-2004 - UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1 - - -x86_64-ubuntu-2004-clang: - extends: .native_build_job - needs: - - x86_64-ubuntu-2004-container - allow_failure: false - variables: - CC: clang - MESON_ARGS: -Db_lundef=false -Db_sanitize=address,undefined - NAME: ubuntu-2004 - UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1 - - - -# Cross build jobs - -armv6l-debian-10: - extends: .cross_build_job - needs: - - armv6l-debian-10-container - allow_failure: false - variables: - CROSS: armv6l - NAME: debian-10 - - -mips-debian-10: - extends: .cross_build_job - needs: - - mips-debian-10-container - allow_failure: false - variables: - CROSS: mips - NAME: debian-10 - - -mipsel-debian-10: - extends: .cross_build_job - needs: - - mipsel-debian-10-container - allow_failure: false - variables: - CROSS: mipsel - NAME: debian-10 - - -armv7l-debian-11: - extends: .cross_build_job - needs: - - armv7l-debian-11-container - allow_failure: false - variables: - CROSS: armv7l - NAME: debian-11 - - -mips64el-debian-11: - extends: .cross_build_job - needs: - - mips64el-debian-11-container - allow_failure: false - variables: - CROSS: mips64el - NAME: debian-11 - - -ppc64le-debian-11: - extends: .cross_build_job - needs: - - ppc64le-debian-11-container - allow_failure: false - variables: - CROSS: ppc64le - NAME: debian-11 - - -aarch64-debian-sid: - extends: .cross_build_job - needs: - - aarch64-debian-sid-container - allow_failure: true - variables: - CROSS: aarch64 - NAME: debian-sid - - -i686-debian-sid: - extends: .cross_build_job - needs: - - i686-debian-sid-container - allow_failure: true - variables: - CROSS: i686 - NAME: debian-sid - - -s390x-debian-sid: - extends: .cross_build_job - needs: - - s390x-debian-sid-container - allow_failure: true - variables: - CROSS: s390x - NAME: debian-sid - - -mingw64-fedora-35: - extends: .cross_build_job - needs: - - mingw64-fedora-35-container - allow_failure: false - variables: - CROSS: mingw64 - NAME: fedora-35 - - -mingw32-fedora-rawhide: - extends: .cross_build_job - needs: - - mingw32-fedora-rawhide-container - allow_failure: true - variables: - CROSS: mingw32 - NAME: fedora-rawhide - - -# Native cirrus build jobs - -x86_64-freebsd-12: - extends: .cirrus_build_job - needs: [] - allow_failure: false - variables: - CIRRUS_VM_IMAGE_NAME: freebsd-12-2 - CIRRUS_VM_IMAGE_SELECTOR: image_family - CIRRUS_VM_INSTANCE_TYPE: freebsd_instance - INSTALL_COMMAND: pkg install -y - NAME: freebsd-12 - UPDATE_COMMAND: pkg update - UPGRADE_COMMAND: pkg upgrade -y - - -x86_64-freebsd-13: - extends: .cirrus_build_job - needs: [] - allow_failure: false - variables: - CIRRUS_VM_IMAGE_NAME: freebsd-13-0 - CIRRUS_VM_IMAGE_SELECTOR: image_family - CIRRUS_VM_INSTANCE_TYPE: freebsd_instance - INSTALL_COMMAND: pkg install -y - NAME: freebsd-13 - UPDATE_COMMAND: pkg update - UPGRADE_COMMAND: pkg upgrade -y - - -x86_64-macos-11: - extends: .cirrus_build_job - needs: [] - allow_failure: false - variables: - CIRRUS_VM_IMAGE_NAME: big-sur-base - CIRRUS_VM_IMAGE_SELECTOR: image - CIRRUS_VM_INSTANCE_TYPE: osx_instance - INSTALL_COMMAND: brew install - NAME: macos-11 - PATH_EXTRA: /usr/local/opt/ccache/libexec:/usr/local/opt/gettext/bin:/usr/local/opt/libpcap/bin:/usr/local/opt/libxslt/bin:/usr/local/opt/rpcgen/bin - PKG_CONFIG_PATH: /usr/local/opt/curl/lib/pkgconfig:/usr/local/opt/libpcap/lib/pkgconfig:/usr/local/opt/libxml2/lib/pkgconfig:/usr/local/opt/ncurses/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig - UPDATE_COMMAND: brew update - UPGRADE_COMMAND: brew upgrade -- 2.34.1
On 3/15/22 11:14, Erik Skultety wrote:
RFC here [1]
Since RFC: - renamed the stage to integration_tests - dropped both the test child pipeline as well as triggering a multi-project CI pipeline to build the Perl bindings and instead use the latest bindings artifacts we have available from the libvirt-perl project. -> basically now we only have a regular CI stage with no extra jobs that just spins up a bunch of private runners and uploads RPM artifacts into them and runs the TCK tests - replaced sed invocations to edit daemon logging settings with augtool - used a pattern for log filters from [2] - used the "$CI_JOB_STATUS" variable in the after_script to determine whether the main script failed so as not to try moving and collecting logs on successful jobs (the logs would have been published as artifacts on failures anyway)
Demo pipeline: https://gitlab.com/eskultety/libvirt/-/pipelines/491838828
[1] https://listman.redhat.com/archives/libvir-list/2022-January/227947.html [2] https://libvirt.org/kbase/debuglogs.html#less-verbose-logging-for-qemu-vms
Erik Skultety (4): ci: gitlab: Refresh gitlab.yml ci: manifest: Publish RPMs as artifacts on CentOS Stream and Fedoras gitlab-ci: Introduce new 'integration_tests' pipeline stage DO NOT MERGE: Demo a functional CI pipeline running Fedoras CentOS only
.gitlab-ci.yml | 99 +------- ci/gitlab.yml | 606 ++------------------------------------------- ci/integration.yml | 99 ++++++++ ci/manifest.yml | 21 +- 4 files changed, 142 insertions(+), 683 deletions(-) create mode 100644 ci/integration.yml
Reviewed-by: Michal Privoznik <mprivozn@redhat.com> Michal
participants (2)
-
Erik Skultety -
Michal Prívozník