[libvirt PATCH 0/5] gitlab: expand the CI job coverage
There are two main goals with this series - Introduce a minimal job building the website and publishing an artifact which can be deployed onto libvirt.org - Expanding CI jobs to get coverage closer to Travis/Jenkins The results of this series should be viewable on the pipeline results show by the "ci-docs" branch here: https://gitlab.com/berrange/libvirt/pipelines/124042722 Daniel P. Berrangé (5): gitlab: use CI for building website contents gitlab: reduce number of cross-build CI jobs gitlab: group jobs into stages gitlab: add several native CI jobs gitlab: rename the cross build jobs .gitlab-ci.yml | 105 +++++++++++++++++++++++++++++++++++-------------- 1 file changed, 75 insertions(+), 30 deletions(-) -- 2.24.1
Run the bare minimum build that is possible to create the docs. Ideally the '--without-remote' arg would be passed, but there are several bugs preventing a build from succeeding without the remote driver built. The generated website is published as an artifact and thus is browsable on build completion and can be downloaded as a zip file. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- .gitlab-ci.yml | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index ea49c6178b..6f7e0ce135 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -44,3 +44,26 @@ debian-sid-cross-i686: debian-sid-cross-mipsel: <<: *job_definition image: quay.io/libvirt/buildenv-libvirt-debian-sid-cross-mipsel:latest + +# This artifact published by this job is downloaded by libvirt.org to +# be deployed to the web root: +# https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=webs... +website: + script: + - mkdir build + - cd build + - ../autogen.sh $CONFIGURE_OPTS --prefix=$(pwd)/../vroot || (cat config.log && exit 1) + - make -j $(getconf _NPROCESSORS_ONLN) + - make -j $(getconf _NPROCESSORS_ONLN) install + - cd .. + - mv vroot/share/doc/libvirt/html/ website + image: quay.io/libvirt/buildenv-libvirt-fedora-31:latest + variables: + CONFIGURE_OPTS: --without-libvirtd --without-esx --without-hyperv --without-test --without-dtrace --without-openvz --without-vmware --without-attr --without-audit --without-blkid --without-bash-completion --without-capng --without-curl --without-dbus --without-firewalld --without-fuse --without-glusterfs --without-libiscsi --without-libssh --without-numactl --without-openwsman --without-pciaccess --without-readline --without-sanlock --without-sasl --without-selinux --without-ssh2 --without-udev + artifacts: + expose_as: 'Website' + name: 'website' + when: on_success + expire_in: 30 days + paths: + - website -- 2.24.1
We're going to add more build jobs to CI, and users have limited time granted on the shared CI runners. The number of cross-build jobs currently present is not sustainable, so cut it down to two interesting jobs to cover big endian and 32-bit platform variants. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- .gitlab-ci.yml | 37 ++++++------------------------------- 1 file changed, 6 insertions(+), 31 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 6f7e0ce135..b6a8db7881 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -5,29 +5,12 @@ - ../autogen.sh $CONFIGURE_OPTS || (cat config.log && exit 1) - make -j $(getconf _NPROCESSORS_ONLN) -# We could run every arch on every versions, but it is a little -# overkill. Instead we split jobs evenly across 9, 10 and sid -# to achieve reasonable cross-coverage. - -debian-9-cross-armv6l: - <<: *job_definition - image: quay.io/libvirt/buildenv-libvirt-debian-9-cross-armv6l:latest - -debian-9-cross-mips64el: - <<: *job_definition - image: quay.io/libvirt/buildenv-libvirt-debian-9-cross-mips64el:latest - -debian-9-cross-mips: - <<: *job_definition - image: quay.io/libvirt/buildenv-libvirt-debian-9-cross-mips:latest - -debian-10-cross-aarch64: - <<: *job_definition - image: quay.io/libvirt/buildenv-libvirt-debian-10-cross-aarch64:latest - -debian-10-cross-ppc64le: - <<: *job_definition - image: quay.io/libvirt/buildenv-libvirt-debian-10-cross-ppc64le:latest +# There are many possible cross-arch jobs we could do, but to preserve +# limited CI resource time allocated to users, we cut it down to two +# interesting variants. The default jobs are x86_64, which means 64-bit +# and little endian. We thus pick armv7l as an interesting 32-bit +# platform, and s390x as an interesting big endian platform. We split +# between Debian 10 and sid to help detect problems on the horizon. debian-10-cross-s390x: <<: *job_definition @@ -37,14 +20,6 @@ debian-sid-cross-armv7l: <<: *job_definition image: quay.io/libvirt/buildenv-libvirt-debian-sid-cross-armv7l:latest -debian-sid-cross-i686: - <<: *job_definition - image: quay.io/libvirt/buildenv-libvirt-debian-sid-cross-i686:latest - -debian-sid-cross-mipsel: - <<: *job_definition - image: quay.io/libvirt/buildenv-libvirt-debian-sid-cross-mipsel:latest - # This artifact published by this job is downloaded by libvirt.org to # be deployed to the web root: # https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=webs... -- 2.24.1
Within a stage all jobs run in parallel. Stages are ordered so later stages are only executed if previous stages succeeded. By using separate stages for the cross builds, we can avoid wasting CI resources if the relatively simple website build fails. Later we can avoid running cross builds, if the native build fails too. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- .gitlab-ci.yml | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index b6a8db7881..e28ec584ea 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,4 +1,9 @@ -.job_template: &job_definition +stages: + - website + - cross_build + +.cross_build_job_template: &cross_build_job_definition + stage: cross_build script: - mkdir build - cd build @@ -13,17 +18,18 @@ # between Debian 10 and sid to help detect problems on the horizon. debian-10-cross-s390x: - <<: *job_definition + <<: *cross_build_job_definition image: quay.io/libvirt/buildenv-libvirt-debian-10-cross-s390x:latest debian-sid-cross-armv7l: - <<: *job_definition + <<: *cross_build_job_definition image: quay.io/libvirt/buildenv-libvirt-debian-sid-cross-armv7l:latest # This artifact published by this job is downloaded by libvirt.org to # be deployed to the web root: # https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=webs... website: + stage: website script: - mkdir build - cd build -- 2.24.1
participants (1)
-
Daniel P. Berrangé