9:15 a.m.
From time to time it happens that some of the distros for which we run
our
pipelines break (or the images we pull break). Because we don't have
dedicated
maintainers for the jobs/runners, we can only employ the best effort approach
wrt to making the pipeline green again when we're positive the problem doesn't
lie in libvirt.
For jobs that are broken for quite some time we should opt to disable them
unconditionally unless a volunteer dedicates the time to either fix it or the
matter gets fixed on its own in time (e.g. updated container images).
v1: https://www.redhat.com/archives/libvir-list/2021-January/msg00573.html
v2: https://www.redhat.com/archives/libvir-list/2021-January/msg00672.html
Since v2:
- renamed all the template jobs according to the review comments
- added 'rules' for container jobs as well
- fixed the env variable naming inconsistency that somehow made it to the
v1 patches
Erik Skultety (3):
gitlab-ci.yml: Replace template anchors with extends
gitlab-ci.yml: Convert only/except to the rules syntax
gitlab-ci.yml: Add an explicit env variable to mark a job as broken
.gitlab-ci.yml | 164 ++++++++++++++++++++++++++-----------------------
1 file changed, 86 insertions(+), 78 deletions(-)
--
2.29.2
9:15 a.m.
New subject: [libvirt PATCH v3 1/3] gitlab-ci.yml: Replace template anchors with extends
'extends' is slightly more readable and definitely more flexible in
terms of allowing includes of templates.
The main reason for this patch though is that the next patch converts
the 'only/except' syntax to the new (preferable) 'rules' syntax.
Variable anchors are still kept intact because the use case there is
different from regular template anchors.
Signed-off-by: Erik Skultety <eskultet(a)redhat.com>
---
.gitlab-ci.yml | 128 ++++++++++++++++++++++++-------------------------
1 file changed, 64 insertions(+), 64 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 2171905f8e..eefcef526b 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -14,7 +14,7 @@ stages:
# Common templates
-.container_job_template: &container_job_definition
+.container_job:
image: docker:stable
stage: containers
needs: []
@@ -36,11 +36,11 @@ stages:
# needed for the pipeline itself to complete: those sometimes fail, and when
# that happens it's mostly because of temporary issues with Debian sid. We
# don't want those failures to affect the overall pipeline status
-.container_optional_job_template: &container_optional_job_definition
- <<: *container_job_definition
+.container_optional_job:
+ extends: .container_job
allow_failure: true
-.native_build_job_template: &native_build_job_definition
+.native_build_job:
stage: builds
image: $CI_REGISTRY_IMAGE/ci-$NAME:latest
cache:
@@ -75,7 +75,7 @@ stages:
# Note that the $PATH environment variable has to be treated with
# special care, because we can't just override it at the GitLab CI job
# definition level or we risk breaking it completely.
-.cirrus_build_job_template: &cirrus_build_job_definition
+.cirrus_build_job:
stage: builds
image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master
needs: []
@@ -103,7 +103,7 @@ stages:
- $CIRRUS_GITHUB_REPO
- $CIRRUS_API_TOKEN
-.cross_build_default_job_template: &cross_build_job_definition
+.cross_build_job:
stage: builds
image: $CI_REGISTRY_IMAGE/ci-$NAME-cross-$CROSS:latest
cache:
@@ -121,57 +121,57 @@ stages:
# Native container build jobs
x64-centos-7-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: centos-7
x64-centos-8-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: centos-8
x64-centos-stream-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: centos-stream
x64-debian-10-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: debian-10
x64-debian-sid-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: debian-sid
x64-fedora-32-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: fedora-32
x64-fedora-33-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: fedora-33
x64-fedora-rawhide-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: fedora-rawhide
x64-opensuse-151-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: opensuse-151
x64-ubuntu-1804-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: ubuntu-1804
x64-ubuntu-2004-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: ubuntu-2004
@@ -179,97 +179,97 @@ x64-ubuntu-2004-container:
# Cross-build containers build jobs
aarch64-debian-10-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: debian-10-cross-aarch64
armv6l-debian-10-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: debian-10-cross-armv6l
armv7l-debian-10-container:
- <<: *container_optional_job_definition
+ extends: .container_optional_job
variables:
NAME: debian-10-cross-armv7l
i686-debian-10-container:
- <<: *container_optional_job_definition
+ extends: .container_optional_job
variables:
NAME: debian-10-cross-i686
mips-debian-10-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: debian-10-cross-mips
mips64el-debian-10-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: debian-10-cross-mips64el
mipsel-debian-10-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: debian-10-cross-mipsel
ppc64le-debian-10-container:
- <<: *container_optional_job_definition
+ extends: .container_optional_job
variables:
NAME: debian-10-cross-ppc64le
s390x-debian-10-container:
- <<: *container_optional_job_definition
+ extends: .container_optional_job
variables:
NAME: debian-10-cross-s390x
aarch64-debian-sid-container:
- <<: *container_optional_job_definition
+ extends: .container_optional_job
variables:
NAME: debian-sid-cross-aarch64
armv6l-debian-sid-container:
- <<: *container_optional_job_definition
+ extends: .container_optional_job
variables:
NAME: debian-sid-cross-armv6l
armv7l-debian-sid-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: debian-sid-cross-armv7l
i686-debian-sid-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: debian-sid-cross-i686
mips64el-debian-sid-container:
- <<: *container_optional_job_definition
+ extends: .container_optional_job
variables:
NAME: debian-sid-cross-mips64el
mipsel-debian-sid-container:
- <<: *container_optional_job_definition
+ extends: .container_optional_job
variables:
NAME: debian-sid-cross-mipsel
ppc64le-debian-sid-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: debian-sid-cross-ppc64le
s390x-debian-sid-container:
- <<: *container_optional_job_definition
+ extends: .container_optional_job
variables:
NAME: debian-sid-cross-s390x
mingw32-fedora-rawhide-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: fedora-rawhide-cross-mingw32
mingw64-fedora-rawhide-container:
- <<: *container_job_definition
+ extends: .container_job
variables:
NAME: fedora-rawhide-cross-mingw64
@@ -277,14 +277,14 @@ mingw64-fedora-rawhide-container:
# Native architecture build + test jobs
x64-debian-10:
- <<: *native_build_job_definition
+ extends: .native_build_job
needs:
- x64-debian-10-container
variables:
NAME: debian-10
x64-debian-10-clang:
- <<: *native_build_job_definition
+ extends: .native_build_job
needs:
- x64-debian-10-container
variables:
@@ -292,14 +292,14 @@ x64-debian-10-clang:
CC: clang
x64-debian-sid:
- <<: *native_build_job_definition
+ extends: .native_build_job
needs:
- x64-debian-sid-container
variables:
NAME: debian-sid
x64-centos-7:
- <<: *native_build_job_definition
+ extends: .native_build_job
needs:
- x64-centos-7-container
variables:
@@ -310,7 +310,7 @@ x64-centos-7:
RPM: skip
x64-centos-8:
- <<: *native_build_job_definition
+ extends: .native_build_job
needs:
- x64-centos-8-container
variables:
@@ -318,7 +318,7 @@ x64-centos-8:
RPM: skip
x64-centos-8-clang:
- <<: *native_build_job_definition
+ extends: .native_build_job
needs:
- x64-centos-8-container
variables:
@@ -327,7 +327,7 @@ x64-centos-8-clang:
RPM: skip
x64-centos-stream:
- <<: *native_build_job_definition
+ extends: .native_build_job
needs:
- x64-centos-stream-container
variables:
@@ -335,7 +335,7 @@ x64-centos-stream:
RPM: skip
x64-fedora-32:
- <<: *native_build_job_definition
+ extends: .native_build_job
needs:
- x64-fedora-32-container
variables:
@@ -343,21 +343,21 @@ x64-fedora-32:
RPM: skip
x64-fedora-33:
- <<: *native_build_job_definition
+ extends: .native_build_job
needs:
- x64-fedora-33-container
variables:
NAME: fedora-33
x64-fedora-rawhide:
- <<: *native_build_job_definition
+ extends: .native_build_job
needs:
- x64-fedora-rawhide-container
variables:
NAME: fedora-rawhide
x64-fedora-rawhide-clang:
- <<: *native_build_job_definition
+ extends: .native_build_job
needs:
- x64-fedora-rawhide-container
variables:
@@ -366,7 +366,7 @@ x64-fedora-rawhide-clang:
RPM: skip
x64-opensuse-151:
- <<: *native_build_job_definition
+ extends: .native_build_job
needs:
- x64-opensuse-151-container
variables:
@@ -374,21 +374,21 @@ x64-opensuse-151:
RPM: skip
x64-ubuntu-1804:
- <<: *native_build_job_definition
+ extends: .native_build_job
needs:
- x64-ubuntu-1804-container
variables:
NAME: ubuntu-1804
x64-ubuntu-2004:
- <<: *native_build_job_definition
+ extends: .native_build_job
needs:
- x64-ubuntu-2004-container
variables:
NAME: ubuntu-2004
x64-freebsd-11-build:
- <<: *cirrus_build_job_definition
+ extends: .cirrus_build_job
variables:
NAME: freebsd-11
CIRRUS_VM_INSTANCE_TYPE: freebsd_instance
@@ -397,7 +397,7 @@ x64-freebsd-11-build:
INSTALL_COMMAND: pkg install -y
x64-freebsd-12-build:
- <<: *cirrus_build_job_definition
+ extends: .cirrus_build_job
variables:
NAME: freebsd-12
CIRRUS_VM_INSTANCE_TYPE: freebsd_instance
@@ -406,7 +406,7 @@ x64-freebsd-12-build:
INSTALL_COMMAND: pkg install -y
x64-macos-1015-build:
- <<: *cirrus_build_job_definition
+ extends: .cirrus_build_job
variables:
NAME: macos-1015
CIRRUS_VM_INSTANCE_TYPE: osx_instance
@@ -420,7 +420,7 @@ x64-macos-1015-build:
# Cross compiled build jobs
armv6l-debian-10:
- <<: *cross_build_job_definition
+ extends: .cross_build_job
needs:
- armv6l-debian-10-container
variables:
@@ -428,7 +428,7 @@ armv6l-debian-10:
CROSS: armv6l
armv7l-debian-10:
- <<: *cross_build_job_definition
+ extends: .cross_build_job
needs:
- armv7l-debian-10-container
variables:
@@ -436,7 +436,7 @@ armv7l-debian-10:
CROSS: armv7l
mips64el-debian-10:
- <<: *cross_build_job_definition
+ extends: .cross_build_job
needs:
- mips64el-debian-10-container
variables:
@@ -444,7 +444,7 @@ mips64el-debian-10:
CROSS: mips64el
mips-debian-10:
- <<: *cross_build_job_definition
+ extends: .cross_build_job
needs:
- mips-debian-10-container
variables:
@@ -452,7 +452,7 @@ mips-debian-10:
CROSS: mips
aarch64-debian-10:
- <<: *cross_build_job_definition
+ extends: .cross_build_job
needs:
- aarch64-debian-10-container
variables:
@@ -460,7 +460,7 @@ aarch64-debian-10:
CROSS: aarch64
mipsel-debian-10:
- <<: *cross_build_job_definition
+ extends: .cross_build_job
needs:
- mipsel-debian-10-container
variables:
@@ -468,7 +468,7 @@ mipsel-debian-10:
CROSS: mipsel
s390x-debian-sid:
- <<: *cross_build_job_definition
+ extends: .cross_build_job
needs:
- s390x-debian-10-container
variables:
@@ -476,7 +476,7 @@ s390x-debian-sid:
CROSS: s390x
i686-debian-sid:
- <<: *cross_build_job_definition
+ extends: .cross_build_job
needs:
- i686-debian-sid-container
variables:
@@ -484,7 +484,7 @@ i686-debian-sid:
CROSS: i686
ppc64le-debian-sid:
- <<: *cross_build_job_definition
+ extends: .cross_build_job
needs:
- ppc64le-debian-10-container
variables:
@@ -492,7 +492,7 @@ ppc64le-debian-sid:
CROSS: ppc64le
mingw32-fedora-rawhide:
- <<: *cross_build_job_definition
+ extends: .cross_build_job
needs:
- mingw32-fedora-rawhide-container
variables:
@@ -500,7 +500,7 @@ mingw32-fedora-rawhide:
CROSS: mingw32
mingw64-fedora-rawhide:
- <<: *cross_build_job_definition
+ extends: .cross_build_job
needs:
- mingw64-fedora-rawhide-container
variables:
--
2.29.2
9:15 a.m.
New subject: [libvirt PATCH v3 2/3] gitlab-ci.yml: Convert only/except to the rules syntax
'rules' syntax replaces the only/except syntax with which it is
mutually exclusive. In some cases the 'rules' syntax is more readable
than the 'only/except' equivalent, in some cases it is not.
The idea behind this conversion is to introduce an explicit env variable
controlling the 'allow_failure' attribute which would then be attached
to a broken build job which would in turn result in a soft failure.
Such behaviour is not possible to achieve with the older 'only/except'
syntax.
Signed-off-by: Erik Skultety <eskultet(a)redhat.com>
---
.gitlab-ci.yml | 28 ++++++++++++++--------------
1 file changed, 14 insertions(+), 14 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index eefcef526b..7636544ee1 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -20,6 +20,8 @@ stages:
needs: []
services:
- docker:dind
+ rules:
+ - when: on_success
before_script:
- export TAG="$CI_REGISTRY_IMAGE/ci-$NAME:latest"
- export COMMON_TAG="$CI_REGISTRY/libvirt/libvirt/ci-$NAME:latest"
@@ -43,6 +45,8 @@ stages:
.native_build_job:
stage: builds
image: $CI_REGISTRY_IMAGE/ci-$NAME:latest
+ rules:
+ - when: on_success
cache:
paths:
- ccache/
@@ -98,10 +102,8 @@ stages:
<ci/cirrus/build.yml >ci/cirrus/$NAME.yml
- cat ci/cirrus/$NAME.yml
- cirrus-run -v --show-build-log always ci/cirrus/$NAME.yml
- only:
- variables:
- - $CIRRUS_GITHUB_REPO
- - $CIRRUS_API_TOKEN
+ rules:
+ - if: "$CIRRUS_GITHUB_REPO && $CIRRUS_API_TOKEN"
.cross_build_job:
stage: builds
@@ -110,6 +112,8 @@ stages:
paths:
- ccache/
key: "$CI_JOB_NAME"
+ rules:
+ - when: on_success
before_script:
- *script_variables
script:
@@ -552,8 +556,8 @@ potfile:
image: $CI_REGISTRY_IMAGE/ci-centos-8:latest
needs:
- x64-centos-8-container
- only:
- - master
+ rules:
+ - if: "$CI_COMMIT_BRANCH == 'master'"
before_script:
- *script_variables
script:
@@ -580,9 +584,8 @@ check-dco:
image: registry.gitlab.com/libvirt/libvirt-ci/check-dco:master
script:
- /check-dco
- except:
- variables:
- - $CI_PROJECT_NAMESPACE == 'libvirt'
+ rules:
+ - if: "$CI_PROJECT_NAMESPACE != 'libvirt'"
variables:
GIT_DEPTH: 1000
@@ -600,8 +603,5 @@ coverity:
- cov-analysis-linux64-*/bin/cov-build --dir cov-int ninja -C build
- tar cfz cov-int.tar.gz cov-int
- curl https://scan.coverity.com/builds?project=$COVERITY_SCAN_PROJECT_NAME --form
token=$COVERITY_SCAN_TOKEN --form email=$GITLAB_USER_EMAIL --form file=(a)cov-int.tar.gz
--form version="$(git describe --tags)" --form description="$(git describe
--tags) / $CI_COMMIT_TITLE / $CI_COMMIT_REF_NAME:$CI_PIPELINE_ID"
- only:
- refs:
- - schedules
- variables:
- - $COVERITY_SCAN_PROJECT_NAME && $COVERITY_SCAN_TOKEN
+ rules:
+ - if: "$CI_PIPELINE_SOURCE == 'schedule' &&
$COVERITY_SCAN_PROJECT_NAME && $COVERITY_SCAN_TOKEN"
--
2.29.2
9:15 a.m.
New subject: [libvirt PATCH v3 3/3] gitlab-ci.yml: Add an explicit env variable to mark a job as broken
Thanks to the 'rules' syntax, this will translate to
'allow_failure:true' and let the job fail but will not affect the rest
of the pipeline.
Signed-off-by: Erik Skultety <eskultet(a)redhat.com>
---
.gitlab-ci.yml | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 7636544ee1..673327fb9e 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -21,6 +21,8 @@ stages:
services:
- docker:dind
rules:
+ - if: "$TEMPORARILY_DISABLED"
+ allow_failure: true
- when: on_success
before_script:
- export TAG="$CI_REGISTRY_IMAGE/ci-$NAME:latest"
@@ -46,6 +48,8 @@ stages:
stage: builds
image: $CI_REGISTRY_IMAGE/ci-$NAME:latest
rules:
+ - if: "$TEMPORARILY_DISABLED"
+ allow_failure: true
- when: on_success
cache:
paths:
@@ -103,6 +107,8 @@ stages:
- cat ci/cirrus/$NAME.yml
- cirrus-run -v --show-build-log always ci/cirrus/$NAME.yml
rules:
+ - if: "$TEMPORARILY_DISABLED"
+ allow_failure: true
- if: "$CIRRUS_GITHUB_REPO && $CIRRUS_API_TOKEN"
.cross_build_job:
@@ -113,6 +119,8 @@ stages:
- ccache/
key: "$CI_JOB_NAME"
rules:
+ - if: "$TEMPORARILY_DISABLED"
+ allow_failure: true
- when: on_success
before_script:
- *script_variables
--
2.29.2
12:24 p.m.
On Thu, 2021-01-14 at 16:15 +0100, Erik Skultety wrote:
> From time to time it happens that some of the distros for which
we run our
pipelines break (or the images we pull break). Because we don't have dedicated
maintainers for the jobs/runners, we can only employ the best effort approach
wrt to making the pipeline green again when we're positive the problem doesn't
lie in libvirt.
For jobs that are broken for quite some time we should opt to disable them
unconditionally unless a volunteer dedicates the time to either fix it or the
matter gets fixed on its own in time (e.g. updated container images).
v1: https://www.redhat.com/archives/libvir-list/2021-January/msg00573.html
v2: https://www.redhat.com/archives/libvir-list/2021-January/msg00672.html
Since v2:
- renamed all the template jobs according to the review comments
- added 'rules' for container jobs as well
- fixed the env variable naming inconsistency that somehow made it to the
v1 patches
Erik Skultety (3):
gitlab-ci.yml: Replace template anchors with extends
gitlab-ci.yml: Convert only/except to the rules syntax
gitlab-ci.yml: Add an explicit env variable to mark a job as broken
.gitlab-ci.yml | 164 ++++++++++++++++++++++++++-----------------------
1 file changed, 86 insertions(+), 78 deletions(-)
Reviewed-by: Andrea Bolognani <abologna(a)redhat.com>
--
Andrea Bolognani / Red Hat / Virtualization
1377
days inactive
1377
days old
4 comments
2 participants
participants (2)
-
Andrea Bolognani -
Erik Skultety