[libvirt PATCH 0/4] ci: reduce number of jobs in the pipeline
Come June 1st (aka very very soon) GitLab will reduce the CI minutes quota available from 2000 to 400 for public projects. The wallclock minutes for CI are the quota divided by a cost factor currently 0.008. IOW, after June 1st our allowance reduces from 250,000 wallclock minutes to 50,000. While I had intended that we join the OSS program, I'm unhappy with the financial liability the T&Cs require us to agree to for tha Usage stats show we're currently consuming 70-80,000 minutes a month, so we need to cut our usage by at least 30%. The main libvirt project is responsible for the vast majority of usage, so is the biggest quick win, but we need to economise across all our repos. With this series, we increase the number of jobs in the pipeline from 82 to 100, but we mark a great many of them as manual jobs, so they never run unless a user explicitly triggers them. So we have only 54 jobs executing instead of the original 82. The remaining 46 jobs are optional. This is a decent win in usage, but we probably need to cut a little more later to give us breathing space Daniel P. Berrangé (4): ci: refresh with lcitool manifest ci: disable native builds on certain distros ci: move Ubuntu GCC santizers build to 20.04 ci: eliminate many cross arch CI builds ci/gitlab/build-templates.yml | 17 ++- ci/gitlab/builds.yml | 246 ++++++++++++++++++++++++++++-- ci/gitlab/container-templates.yml | 3 + ci/gitlab/containers.yml | 50 +++--- ci/manifest.yml | 59 ++++--- 5 files changed, 329 insertions(+), 46 deletions(-) -- 2.36.1
This refreshes the containers bringing in new behaviour when builds/containers are disabled. Instead of deleting the job entirely, the job still exists but is set to be a manual job. It won't affect the pipeline result, but can be triggered by the developer if they wish to test a specific scenario. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- ci/gitlab/build-templates.yml | 17 ++- ci/gitlab/builds.yml | 216 ++++++++++++++++++++++++++++++ ci/gitlab/container-templates.yml | 3 + 3 files changed, 235 insertions(+), 1 deletion(-) diff --git a/ci/gitlab/build-templates.yml b/ci/gitlab/build-templates.yml index 446822d5d1..20d22914c2 100644 --- a/ci/gitlab/build-templates.yml +++ b/ci/gitlab/build-templates.yml @@ -8,11 +8,21 @@ .gitlab_native_build_job: image: $CI_REGISTRY_IMAGE/ci-$NAME:latest stage: builds + rules: + - if: '$LIBVIRT_JOB_OPTIONAL' + when: manual + allow_failure: true + - when: on_success .gitlab_cross_build_job: image: $CI_REGISTRY_IMAGE/ci-$NAME-cross-$CROSS:latest stage: builds + rules: + - if: '$LIBVIRT_JOB_OPTIONAL' + when: manual + allow_failure: true + - when: on_success .cirrus_build_job: @@ -42,4 +52,9 @@ - cat ci/cirrus/$NAME.yml - cirrus-run -v --show-build-log always ci/cirrus/$NAME.yml rules: - - if: "$CIRRUS_GITHUB_REPO && $CIRRUS_API_TOKEN" + - if: "$CIRRUS_GITHUB_REPO == null || $CIRRUS_API_TOKEN == null" + when: never + - if: '$LIBVIRT_JOB_OPTIONAL' + when: manual + allow_failure: true + - when: on_success diff --git a/ci/gitlab/builds.yml b/ci/gitlab/builds.yml index d136b0776e..e2efeb4f73 100644 --- a/ci/gitlab/builds.yml +++ b/ci/gitlab/builds.yml @@ -235,6 +235,18 @@ x86_64-ubuntu-2204-clang: # Cross build jobs +aarch64-debian-10: + extends: .cross_build_job + needs: + - job: aarch64-debian-10-container + optional: true + allow_failure: true + variables: + CROSS: aarch64 + LIBVIRT_JOB_OPTIONAL: 1 + NAME: debian-10 + + armv6l-debian-10: extends: .cross_build_job needs: @@ -246,6 +258,30 @@ armv6l-debian-10: NAME: debian-10 +armv7l-debian-10: + extends: .cross_build_job + needs: + - job: armv7l-debian-10-container + optional: true + allow_failure: true + variables: + CROSS: armv7l + LIBVIRT_JOB_OPTIONAL: 1 + NAME: debian-10 + + +i686-debian-10: + extends: .cross_build_job + needs: + - job: i686-debian-10-container + optional: true + allow_failure: true + variables: + CROSS: i686 + LIBVIRT_JOB_OPTIONAL: 1 + NAME: debian-10 + + mips-debian-10: extends: .cross_build_job needs: @@ -257,6 +293,18 @@ mips-debian-10: NAME: debian-10 +mips64el-debian-10: + extends: .cross_build_job + needs: + - job: mips64el-debian-10-container + optional: true + allow_failure: true + variables: + CROSS: mips64el + LIBVIRT_JOB_OPTIONAL: 1 + NAME: debian-10 + + mipsel-debian-10: extends: .cross_build_job needs: @@ -268,6 +316,54 @@ mipsel-debian-10: NAME: debian-10 +ppc64le-debian-10: + extends: .cross_build_job + needs: + - job: ppc64le-debian-10-container + optional: true + allow_failure: true + variables: + CROSS: ppc64le + LIBVIRT_JOB_OPTIONAL: 1 + NAME: debian-10 + + +s390x-debian-10: + extends: .cross_build_job + needs: + - job: s390x-debian-10-container + optional: true + allow_failure: true + variables: + CROSS: s390x + LIBVIRT_JOB_OPTIONAL: 1 + NAME: debian-10 + + +aarch64-debian-11: + extends: .cross_build_job + needs: + - job: aarch64-debian-11-container + optional: true + allow_failure: true + variables: + CROSS: aarch64 + LIBVIRT_JOB_OPTIONAL: 1 + NAME: debian-11 + + +armv6l-debian-11: + extends: .cross_build_job + needs: + - job: armv6l-debian-11-container + optional: true + allow_failure: true + variables: + CROSS: armv6l + LIBVIRT_JOB_OPTIONAL: 1 + NAME: debian-11 + + armv7l-debian-11: extends: .cross_build_job needs: @@ -279,6 +375,18 @@ armv7l-debian-11: NAME: debian-11 +i686-debian-11: + extends: .cross_build_job + needs: + - job: i686-debian-11-container + optional: true + allow_failure: true + variables: + CROSS: i686 + LIBVIRT_JOB_OPTIONAL: 1 + NAME: debian-11 + + mips64el-debian-11: extends: .cross_build_job needs: @@ -290,6 +398,18 @@ mips64el-debian-11: NAME: debian-11 +mipsel-debian-11: + extends: .cross_build_job + needs: + - job: mipsel-debian-11-container + optional: true + allow_failure: true + variables: + CROSS: mipsel + LIBVIRT_JOB_OPTIONAL: 1 + NAME: debian-11 + + ppc64le-debian-11: extends: .cross_build_job needs: @@ -301,6 +421,18 @@ ppc64le-debian-11: NAME: debian-11 +s390x-debian-11: + extends: .cross_build_job + needs: + - job: s390x-debian-11-container + optional: true + allow_failure: true + variables: + CROSS: s390x + LIBVIRT_JOB_OPTIONAL: 1 + NAME: debian-11 + + aarch64-debian-sid: extends: .cross_build_job needs: @@ -312,6 +444,30 @@ aarch64-debian-sid: NAME: debian-sid +armv6l-debian-sid: + extends: .cross_build_job + needs: + - job: armv6l-debian-sid-container + optional: true + allow_failure: true + variables: + CROSS: armv6l + LIBVIRT_JOB_OPTIONAL: 1 + NAME: debian-sid + + +armv7l-debian-sid: + extends: .cross_build_job + needs: + - job: armv7l-debian-sid-container + optional: true + allow_failure: true + variables: + CROSS: armv7l + LIBVIRT_JOB_OPTIONAL: 1 + NAME: debian-sid + + i686-debian-sid: extends: .cross_build_job needs: @@ -323,6 +479,42 @@ i686-debian-sid: NAME: debian-sid +mips64el-debian-sid: + extends: .cross_build_job + needs: + - job: mips64el-debian-sid-container + optional: true + allow_failure: true + variables: + CROSS: mips64el + LIBVIRT_JOB_OPTIONAL: 1 + NAME: debian-sid + + +mipsel-debian-sid: + extends: .cross_build_job + needs: + - job: mipsel-debian-sid-container + optional: true + allow_failure: true + variables: + CROSS: mipsel + LIBVIRT_JOB_OPTIONAL: 1 + NAME: debian-sid + + +ppc64le-debian-sid: + extends: .cross_build_job + needs: + - job: ppc64le-debian-sid-container + optional: true + allow_failure: true + variables: + CROSS: ppc64le + LIBVIRT_JOB_OPTIONAL: 1 + NAME: debian-sid + + s390x-debian-sid: extends: .cross_build_job needs: @@ -334,6 +526,18 @@ s390x-debian-sid: NAME: debian-sid +mingw32-fedora-36: + extends: .cross_build_job + needs: + - job: mingw32-fedora-36-container + optional: true + allow_failure: false + variables: + CROSS: mingw32 + LIBVIRT_JOB_OPTIONAL: 1 + NAME: fedora-36 + + mingw64-fedora-36: extends: .cross_build_job needs: @@ -356,6 +560,18 @@ mingw32-fedora-rawhide: NAME: fedora-rawhide +mingw64-fedora-rawhide: + extends: .cross_build_job + needs: + - job: mingw64-fedora-rawhide-container + optional: true + allow_failure: true + variables: + CROSS: mingw64 + LIBVIRT_JOB_OPTIONAL: 1 + NAME: fedora-rawhide + + # Native cirrus build jobs x86_64-freebsd-12: diff --git a/ci/gitlab/container-templates.yml b/ci/gitlab/container-templates.yml index 4cd28af8a9..c71d03c759 100644 --- a/ci/gitlab/container-templates.yml +++ b/ci/gitlab/container-templates.yml @@ -49,4 +49,7 @@ when: on_success - if: '$CI_PROJECT_NAMESPACE == "libvirt"' when: never + - if: '$LIBVIRT_JOB_OPTIONAL' + when: manual + allow_failure: true - when: on_success -- 2.36.1
Currently we do native builds on all distros that are covered by the support matrix. This reduces that such that we mostly only run builds on the newest (ie bleeding edge non-released) version and the oldest version. The effect is that cut out builds on the newest release version. This is acceptable, because that version is sandwiched between two versions we do still test, so unlikely to have failures not already identified by other jobs. This has the effect of disabling: - AlmaLinux 8 GCC - still has a CLang build and CentOS 8 Stream also gives coverage - Debian 11 - still has a Debian 10 and Sid build - Alpine 3.15 - still has a Alpine 3.14 and Edge build Ideally Fedora 35 would be disabled too, but we rely on that for the integration tests. The Ubuntu jobs will be handled in the next patch. The containers are still built since this is cheap-ish. The build jobs can also be triggered manually if desired. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- ci/gitlab/builds.yml | 3 +++ ci/manifest.yml | 7 ++++++- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/ci/gitlab/builds.yml b/ci/gitlab/builds.yml index e2efeb4f73..255d24b0e2 100644 --- a/ci/gitlab/builds.yml +++ b/ci/gitlab/builds.yml @@ -14,6 +14,7 @@ x86_64-almalinux-8: optional: true allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: almalinux-8 RPM: skip @@ -47,6 +48,7 @@ x86_64-alpine-315: optional: true allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: alpine-315 @@ -105,6 +107,7 @@ x86_64-debian-11: optional: true allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-11 diff --git a/ci/manifest.yml b/ci/manifest.yml index ad0ec9a53d..a825333048 100644 --- a/ci/manifest.yml +++ b/ci/manifest.yml @@ -9,6 +9,7 @@ targets: almalinux-8: jobs: - arch: x86_64 + builds: false variables: RPM: skip @@ -20,7 +21,10 @@ targets: alpine-314: x86_64 - alpine-315: x86_64 + alpine-315: + jobs: + - arch: x86_64 + builds: false alpine-edge: jobs: @@ -80,6 +84,7 @@ targets: debian-11: jobs: - arch: x86_64 + builds: false - arch: x86_64 suffix: -clang -- 2.36.1
We currently build on Ubuntu 22.04 twice, for GCC and CLang with santizers turned on. Moving the GCC santizers build to 20.04 lets us cull one of the 22.04 jobs Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- ci/gitlab/builds.yml | 7 ++++--- ci/manifest.yml | 9 ++++++--- 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/ci/gitlab/builds.yml b/ci/gitlab/builds.yml index 255d24b0e2..a1183abb8d 100644 --- a/ci/gitlab/builds.yml +++ b/ci/gitlab/builds.yml @@ -206,7 +206,10 @@ x86_64-ubuntu-2004: optional: true allow_failure: false variables: + ASAN_OPTIONS: verify_asan_link_order=0 + MESON_ARGS: -Db_lundef=false -Db_sanitize=address,undefined NAME: ubuntu-2004 + UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1 x86_64-ubuntu-2204: @@ -216,10 +219,8 @@ x86_64-ubuntu-2204: optional: true allow_failure: false variables: - ASAN_OPTIONS: verify_asan_link_order=0 - MESON_ARGS: -Db_lundef=false -Db_sanitize=address,undefined + LIBVIRT_JOB_OPTIONAL: 1 NAME: ubuntu-2204 - UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1 x86_64-ubuntu-2204-clang: diff --git a/ci/manifest.yml b/ci/manifest.yml index a825333048..97de2d7822 100644 --- a/ci/manifest.yml +++ b/ci/manifest.yml @@ -209,9 +209,7 @@ targets: PATH_EXTRA: /usr/local/opt/ccache/libexec:/usr/local/opt/gettext/bin:/usr/local/opt/libpcap/bin:/usr/local/opt/libxslt/bin:/usr/local/opt/rpcgen/bin PKG_CONFIG_PATH: /usr/local/opt/curl/lib/pkgconfig:/usr/local/opt/libpcap/lib/pkgconfig:/usr/local/opt/libxml2/lib/pkgconfig:/usr/local/opt/ncurses/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig - ubuntu-2004: x86_64 - - ubuntu-2204: + ubuntu-2004: jobs: - arch: x86_64 variables: @@ -219,6 +217,11 @@ targets: MESON_ARGS: -Db_lundef=false -Db_sanitize=address,undefined UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1 + ubuntu-2204: + jobs: + - arch: x86_64 + builds: false + - arch: x86_64 suffix: -clang variables: -- 2.36.1
We currently build cross-arch containers for all three Debian releases (10, 11, Sid), and do libvirt builds covering each arch. This is overkill in terms of the number of problems it identifies. The most important aspect of cross arch builds is to find problems with 32-bit builds and problems with big endian builds. With this in mind the cross arch jobs are altered as follows - Debian 10 - build the containers by default - build armv7 (32-bit) & s390x (big endian) - other arch builds manual - Debian 11 / Sid - container builds all optional - arch builds all optional Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- ci/gitlab/builds.yml | 44 ++++++++++++++++++++--------------- ci/gitlab/containers.yml | 50 ++++++++++++++++++++++++++-------------- ci/manifest.yml | 43 ++++++++++++++++++++++------------ 3 files changed, 87 insertions(+), 50 deletions(-) diff --git a/ci/gitlab/builds.yml b/ci/gitlab/builds.yml index a1183abb8d..62a6aa1db4 100644 --- a/ci/gitlab/builds.yml +++ b/ci/gitlab/builds.yml @@ -256,9 +256,10 @@ armv6l-debian-10: needs: - job: armv6l-debian-10-container optional: true - allow_failure: false + allow_failure: true variables: CROSS: armv6l + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-10 @@ -267,10 +268,9 @@ armv7l-debian-10: needs: - job: armv7l-debian-10-container optional: true - allow_failure: true + allow_failure: false variables: CROSS: armv7l - LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-10 @@ -291,9 +291,10 @@ mips-debian-10: needs: - job: mips-debian-10-container optional: true - allow_failure: false + allow_failure: true variables: CROSS: mips + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-10 @@ -314,9 +315,10 @@ mipsel-debian-10: needs: - job: mipsel-debian-10-container optional: true - allow_failure: false + allow_failure: true variables: CROSS: mipsel + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-10 @@ -349,7 +351,7 @@ aarch64-debian-11: needs: - job: aarch64-debian-11-container optional: true - allow_failure: true + allow_failure: false variables: CROSS: aarch64 LIBVIRT_JOB_OPTIONAL: 1 @@ -361,7 +363,7 @@ armv6l-debian-11: needs: - job: armv6l-debian-11-container optional: true - allow_failure: true + allow_failure: false variables: CROSS: armv6l LIBVIRT_JOB_OPTIONAL: 1 @@ -376,6 +378,7 @@ armv7l-debian-11: allow_failure: false variables: CROSS: armv7l + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-11 @@ -384,7 +387,7 @@ i686-debian-11: needs: - job: i686-debian-11-container optional: true - allow_failure: true + allow_failure: false variables: CROSS: i686 LIBVIRT_JOB_OPTIONAL: 1 @@ -399,6 +402,7 @@ mips64el-debian-11: allow_failure: false variables: CROSS: mips64el + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-11 @@ -407,7 +411,7 @@ mipsel-debian-11: needs: - job: mipsel-debian-11-container optional: true - allow_failure: true + allow_failure: false variables: CROSS: mipsel LIBVIRT_JOB_OPTIONAL: 1 @@ -422,6 +426,7 @@ ppc64le-debian-11: allow_failure: false variables: CROSS: ppc64le + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-11 @@ -430,7 +435,7 @@ s390x-debian-11: needs: - job: s390x-debian-11-container optional: true - allow_failure: true + allow_failure: false variables: CROSS: s390x LIBVIRT_JOB_OPTIONAL: 1 @@ -442,9 +447,10 @@ aarch64-debian-sid: needs: - job: aarch64-debian-sid-container optional: true - allow_failure: true + allow_failure: false variables: CROSS: aarch64 + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-sid @@ -453,7 +459,7 @@ armv6l-debian-sid: needs: - job: armv6l-debian-sid-container optional: true - allow_failure: true + allow_failure: false variables: CROSS: armv6l LIBVIRT_JOB_OPTIONAL: 1 @@ -465,7 +471,7 @@ armv7l-debian-sid: needs: - job: armv7l-debian-sid-container optional: true - allow_failure: true + allow_failure: false variables: CROSS: armv7l LIBVIRT_JOB_OPTIONAL: 1 @@ -477,9 +483,10 @@ i686-debian-sid: needs: - job: i686-debian-sid-container optional: true - allow_failure: true + allow_failure: false variables: CROSS: i686 + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-sid @@ -488,7 +495,7 @@ mips64el-debian-sid: needs: - job: mips64el-debian-sid-container optional: true - allow_failure: true + allow_failure: false variables: CROSS: mips64el LIBVIRT_JOB_OPTIONAL: 1 @@ -500,7 +507,7 @@ mipsel-debian-sid: needs: - job: mipsel-debian-sid-container optional: true - allow_failure: true + allow_failure: false variables: CROSS: mipsel LIBVIRT_JOB_OPTIONAL: 1 @@ -512,7 +519,7 @@ ppc64le-debian-sid: needs: - job: ppc64le-debian-sid-container optional: true - allow_failure: true + allow_failure: false variables: CROSS: ppc64le LIBVIRT_JOB_OPTIONAL: 1 @@ -524,9 +531,10 @@ s390x-debian-sid: needs: - job: s390x-debian-sid-container optional: true - allow_failure: true + allow_failure: false variables: CROSS: s390x + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-sid diff --git a/ci/gitlab/containers.yml b/ci/gitlab/containers.yml index 37c179c439..7cdf3eb9a0 100644 --- a/ci/gitlab/containers.yml +++ b/ci/gitlab/containers.yml @@ -131,14 +131,14 @@ aarch64-debian-10-container: armv6l-debian-10-container: extends: .container_job - allow_failure: false + allow_failure: true variables: NAME: debian-10-cross-armv6l armv7l-debian-10-container: extends: .container_job - allow_failure: true + allow_failure: false variables: NAME: debian-10-cross-armv7l @@ -152,7 +152,7 @@ i686-debian-10-container: mips-debian-10-container: extends: .container_job - allow_failure: false + allow_failure: true variables: NAME: debian-10-cross-mips @@ -166,7 +166,7 @@ mips64el-debian-10-container: mipsel-debian-10-container: extends: .container_job - allow_failure: false + allow_failure: true variables: NAME: debian-10-cross-mipsel @@ -187,15 +187,17 @@ s390x-debian-10-container: aarch64-debian-11-container: extends: .container_job - allow_failure: true + allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-11-cross-aarch64 armv6l-debian-11-container: extends: .container_job - allow_failure: true + allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-11-cross-armv6l @@ -203,13 +205,15 @@ armv7l-debian-11-container: extends: .container_job allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-11-cross-armv7l i686-debian-11-container: extends: .container_job - allow_failure: true + allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-11-cross-i686 @@ -217,13 +221,15 @@ mips64el-debian-11-container: extends: .container_job allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-11-cross-mips64el mipsel-debian-11-container: extends: .container_job - allow_failure: true + allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-11-cross-mipsel @@ -231,69 +237,79 @@ ppc64le-debian-11-container: extends: .container_job allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-11-cross-ppc64le s390x-debian-11-container: extends: .container_job - allow_failure: true + allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-11-cross-s390x aarch64-debian-sid-container: extends: .container_job - allow_failure: true + allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-sid-cross-aarch64 armv6l-debian-sid-container: extends: .container_job - allow_failure: true + allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-sid-cross-armv6l armv7l-debian-sid-container: extends: .container_job - allow_failure: true + allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-sid-cross-armv7l i686-debian-sid-container: extends: .container_job - allow_failure: true + allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-sid-cross-i686 mips64el-debian-sid-container: extends: .container_job - allow_failure: true + allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-sid-cross-mips64el mipsel-debian-sid-container: extends: .container_job - allow_failure: true + allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-sid-cross-mipsel ppc64le-debian-sid-container: extends: .container_job - allow_failure: true + allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-sid-cross-ppc64le s390x-debian-sid-container: extends: .container_job - allow_failure: true + allow_failure: false variables: + LIBVIRT_JOB_OPTIONAL: 1 NAME: debian-sid-cross-s390x diff --git a/ci/manifest.yml b/ci/manifest.yml index 97de2d7822..6e35107fdd 100644 --- a/ci/manifest.yml +++ b/ci/manifest.yml @@ -56,22 +56,26 @@ targets: builds: false - arch: armv6l - - - arch: armv7l allow-failure: true builds: false + - arch: armv7l + - arch: i686 allow-failure: true builds: false - arch: mips + allow-failure: true + builds: false - arch: mips64el allow-failure: true builds: false - arch: mipsel + allow-failure: true + builds: false - arch: ppc64le allow-failure: true @@ -90,29 +94,35 @@ targets: suffix: -clang - arch: aarch64 - allow-failure: true + containers: false builds: false - arch: armv6l - allow-failure: true + containers: false builds: false - arch: armv7l + containers: false + builds: false - arch: i686 - allow-failure: true + containers: false builds: false - arch: mips64el + containers: false + builds: false - arch: mipsel - allow-failure: true + containers: false builds: false - arch: ppc64le + containers: false + builds: false - arch: s390x - allow-failure: true + containers: false builds: false debian-sid: @@ -121,33 +131,36 @@ targets: allow-failure: true - arch: aarch64 - allow-failure: true + containers: false + builds: false - arch: armv6l - allow-failure: true + containers: false builds: false - arch: armv7l - allow-failure: true + containers: false builds: false - arch: i686 - allow-failure: true + containers: false + builds: false - arch: mips64el - allow-failure: true + containers: false builds: false - arch: mipsel - allow-failure: true + containers: false builds: false - arch: ppc64le - allow-failure: true + containers: false builds: false - arch: s390x - allow-failure: true + containers: false + builds: false fedora-35: jobs: -- 2.36.1
On Fri, May 27, 2022 at 10:06:16AM -0400, Daniel P. Berrangé wrote:
Come June 1st (aka very very soon) GitLab will reduce the CI minutes quota available from 2000 to 400 for public projects.
The wallclock minutes for CI are the quota divided by a cost factor currently 0.008. IOW, after June 1st our allowance reduces from 250,000 wallclock minutes to 50,000.
While I had intended that we join the OSS program, I'm unhappy with the financial liability the T&Cs require us to agree to for tha
Usage stats show we're currently consuming 70-80,000 minutes a month, so we need to cut our usage by at least 30%.
The main libvirt project is responsible for the vast majority of usage, so is the biggest quick win, but we need to economise across all our repos.
With this series, we increase the number of jobs in the pipeline from 82 to 100, but we mark a great many of them as manual jobs, so they never run unless a user explicitly triggers them.
So we have only 54 jobs executing instead of the original 82. The remaining 46 jobs are optional. This is a decent win in usage, but we probably need to cut a little more later to give us breathing space
I should have mentioned this is dependant on https://gitlab.com/libvirt/libvirt-ci/-/merge_requests/276 without that, instead of getting manual jobs, the jobs disappear entirely. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Fri, May 27, 2022 at 10:06:16AM -0400, Daniel P. Berrangé wrote:
Come June 1st (aka very very soon) GitLab will reduce the CI minutes quota available from 2000 to 400 for public projects.
The wallclock minutes for CI are the quota divided by a cost factor currently 0.008. IOW, after June 1st our allowance reduces from 250,000 wallclock minutes to 50,000.
While I had intended that we join the OSS program, I'm unhappy with the financial liability the T&Cs require us to agree to for tha
Usage stats show we're currently consuming 70-80,000 minutes a month, so we need to cut our usage by at least 30%.
The main libvirt project is responsible for the vast majority of usage, so is the biggest quick win, but we need to economise across all our repos.
With this series, we increase the number of jobs in the pipeline from 82 to 100, but we mark a great many of them as manual jobs, so they never run unless a user explicitly triggers them.
So we have only 54 jobs executing instead of the original 82. The remaining 46 jobs are optional. This is a decent win in usage, but we probably need to cut a little more later to give us breathing space
Daniel P. Berrangé (4): ci: refresh with lcitool manifest ci: disable native builds on certain distros ci: move Ubuntu GCC santizers build to 20.04 ci: eliminate many cross arch CI builds
I'm not entirely convinced this is the exact set of trade-offs we want, but the new limit is going to be active starting tomorrow so it's probably a good idea to merge this right away and tweak things further later, so Reviewed-by: Andrea Bolognani <abologna@redhat.com> Erik, are you okay with this plan? If so, we can push the patches. -- Andrea Bolognani / Red Hat / Virtualization
participants (2)
-
Andrea Bolognani -
Daniel P. Berrangé