[libvirt] [jenkins-ci PATCH 0/5] Expand test matrix, plus cleanups
Andrea Bolognani (5): README: Update, expand and convert to Markdown jobs: Move check_env defaults jobs: Define $VIR_TEST_{DEBUG,EXPENSIVE} in default check_env jobs: Define $OSINFO_SYSTEM_DIR in default make_env projects: Expand test matrix README | 34 ------------------------------ README.markdown | 48 +++++++++++++++++++++++++++++++++++++++++++ jobs/autotools.yaml | 1 - jobs/defaults.yaml | 4 ++++ jobs/go.yaml | 1 - projects/libosinfo.yaml | 11 ++++------ projects/libvirt-glib.yaml | 4 ++++ projects/libvirt-go-xml.yaml | 4 ++++ projects/libvirt-go.yaml | 4 ++++ projects/libvirt-perl.yaml | 5 +++++ projects/libvirt-python.yaml | 4 ++++ projects/libvirt-sandbox.yaml | 2 ++ projects/libvirt-tck.yaml | 4 ++++ projects/libvirt.yaml | 15 ++++++-------- projects/osinfo-db-tools.yaml | 4 ++++ projects/osinfo-db.yaml | 3 +++ projects/virt-manager.yaml | 4 ++++ projects/virt-viewer.yaml | 4 ++++ 18 files changed, 104 insertions(+), 52 deletions(-) delete mode 100644 README create mode 100644 README.markdown -- 2.13.6
The documentation is not very extensive, plus the documented commands don't really work anymore now that the guests/ directory has been added. Move to Markdown while at it, since it's a much nicer format than whatever ad-hoc pseudo-markup we're using now. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- README | 34 ---------------------------------- README.markdown | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+), 34 deletions(-) delete mode 100644 README create mode 100644 README.markdown diff --git a/README b/README deleted file mode 100644 index 9871cfc..0000000 --- a/README +++ /dev/null @@ -1,34 +0,0 @@ - Libvirt Jenkins CI - ================== - -This package contains the libvirt Jenkins CI configuration tasks. - -This uses the jenkins job builder tool which can be installed - - $ pip install --user jenkins-job-builder - -The jobs/ directory contains general templates for defining -jobs for the different build systems (autotools, python distutils, -perl ExtUtils::MakeMaker, etc) - -The projects/ directory contains the per-project config which -activates the desired jobs and configures them if needed. - -It is possible to see the raw jenkins configuration using - - $ jenkins-jobs test -r . libvirt-glib-build - -Before updating the jobs on the server a file is needed to provide -the login details - - $ cat jenkins_jobs.ini - [jenkins] - user=XXX - password=XXX - url=https://ci.centos.org - - -To update the jenkins server with all jobs - - $ jenkins-jobs --conf jenkins_jobs.ini update -r . - diff --git a/README.markdown b/README.markdown new file mode 100644 index 0000000..391b970 --- /dev/null +++ b/README.markdown @@ -0,0 +1,48 @@ +Libvirt Jenkins CI +================== + +This repository contains jobs definitions for the libvirt Jenkins CI. + +They're supposed to be fed to the Jenkins Job Builder tool, which can +be installed either through your distribution's package manager, for +example using: + + # dnf install python3-jenkins-job-builder + +on Fedora, or straight from pip using: + + $ pip install --user jenkins-job-builder + +The `jobs/` directory contains general templates for defining jobs +for the different build systems, such as GNU `autotools`, Python's +`distutils`, Perl's `ExtUtils::MakeMaker` and so on. + +The `projects/` directory contains the per-project config which +activates the desired jobs and configures them if needed. + +It's possible to see the raw Jenkins configuration using either: + + $ jenkins-jobs test -x .git -x guests -r . + +to see al jobs, or: + + $ jenkins-jobs test -x .git -x guests -r . libvirt-master-build + +if you're interested in a single job. + +In order to apply the updated configuration on the server, you're +going to need a configuration file containing access information: + + $ cat jenkins.ini + [jenkins] + user=XXX + password=XXX + url=https://ci.centos.org + +To update the Jenkins server with all jobs, run: + + $ jenkins-jobs --conf jenkins.ini update -x .git -x guests -r . + +The `guests/` directory contains tools used to manage the CI guests +rather than the Jenkins server; they are documented separately in +`guests/README.markdown`. -- 2.13.6
We have a place specifically for defaults, so make use of it instead of setting the default in two separate places. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- jobs/autotools.yaml | 1 - jobs/defaults.yaml | 1 + jobs/go.yaml | 1 - 3 files changed, 1 insertion(+), 2 deletions(-) diff --git a/jobs/autotools.yaml b/jobs/autotools.yaml index 3e4fdc2..19f62be 100644 --- a/jobs/autotools.yaml +++ b/jobs/autotools.yaml @@ -95,7 +95,6 @@ name: '{name}-{branch}-check' project-type: matrix description: '{title} Check' - check_env: '' workspace: '{name}-{branch}' child-workspace: '.' block-downstream: true diff --git a/jobs/defaults.yaml b/jobs/defaults.yaml index 7180c13..0e628bb 100644 --- a/jobs/defaults.yaml +++ b/jobs/defaults.yaml @@ -9,5 +9,6 @@ if [ "$unamestr" = 'FreeBSD' ]; then MAKE='gmake' fi + check_env: | smp: 3 spam: ymankad@redhat.com libvirt-ci@redhat.com diff --git a/jobs/go.yaml b/jobs/go.yaml index e63846d..8eee6af 100644 --- a/jobs/go.yaml +++ b/jobs/go.yaml @@ -52,7 +52,6 @@ name: '{name}-{branch}-check' project-type: matrix description: '{title} Check' - check_env: '' workspace: '{name}-{branch}' child-workspace: '.' block-downstream: true -- 2.13.6
Althought the environment variables are only used when running libvirt's test suite, there is no harm in having it set in the environment at all times. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- jobs/defaults.yaml | 2 ++ projects/libvirt.yaml | 3 --- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/jobs/defaults.yaml b/jobs/defaults.yaml index 0e628bb..a403d0e 100644 --- a/jobs/defaults.yaml +++ b/jobs/defaults.yaml @@ -10,5 +10,7 @@ MAKE='gmake' fi check_env: | + export VIR_TEST_EXPENSIVE=1 + export VIR_TEST_DEBUG=2 smp: 3 spam: ymankad@redhat.com libvirt-ci@redhat.com diff --git a/projects/libvirt.yaml b/projects/libvirt.yaml index 1b31c1b..5617e17 100644 --- a/projects/libvirt.yaml +++ b/projects/libvirt.yaml @@ -34,9 +34,6 @@ - libvirt-fedora-rawhide - autotools-check-job: parent_jobs: 'libvirt-master-syntax-check' - check_env: | - export VIR_TEST_EXPENSIVE=1 - export VIR_TEST_DEBUG=2 - autotools-rpm-job: parent_jobs: 'libvirt-master-check' - autotools-mingw-job: -- 2.13.6
On Fri, Oct 27, 2017 at 09:48:13AM +0200, Andrea Bolognani wrote:
Althought the environment variables are only used when running libvirt's test suite, there is no harm in having it set in the environment at all times.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- jobs/defaults.yaml | 2 ++ projects/libvirt.yaml | 3 --- 2 files changed, 2 insertions(+), 3 deletions(-)
diff --git a/jobs/defaults.yaml b/jobs/defaults.yaml index 0e628bb..a403d0e 100644 --- a/jobs/defaults.yaml +++ b/jobs/defaults.yaml @@ -10,5 +10,7 @@ MAKE='gmake' fi check_env: | + export VIR_TEST_EXPENSIVE=1 + export VIR_TEST_DEBUG=2 smp: 3 spam: ymankad@redhat.com libvirt-ci@redhat.com diff --git a/projects/libvirt.yaml b/projects/libvirt.yaml index 1b31c1b..5617e17 100644 --- a/projects/libvirt.yaml +++ b/projects/libvirt.yaml @@ -34,9 +34,6 @@ - libvirt-fedora-rawhide - autotools-check-job: parent_jobs: 'libvirt-master-syntax-check' - check_env: | - export VIR_TEST_EXPENSIVE=1 - export VIR_TEST_DEBUG=2 - autotools-rpm-job: parent_jobs: 'libvirt-master-check' - autotools-mingw-job:
I don't really see the benefit in this change - it is a currently project specific env variable, in a project specific data file. Moving this into a env that is defined for all projects makes no sense. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Althought the environment variable is only used when building libosinfo, there is no harm in having it set in the environment at all times; moreover, hoisting it to the default make_env allows us to remove duplicated code used to set $MAKE in the libosinfo project. $OSINFO_SYSTEM_DIR itself depends on $VIRT_PREFIX, which along with a bunch of other environment variables is defined not at the job level but rather at the machine level, making it opaque to anyone except the Jenkins administrator: the plan is to move even more environment variables to the default make_env over time. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- jobs/defaults.yaml | 1 + projects/libosinfo.yaml | 7 ------- 2 files changed, 1 insertion(+), 7 deletions(-) diff --git a/jobs/defaults.yaml b/jobs/defaults.yaml index a403d0e..7aa5a30 100644 --- a/jobs/defaults.yaml +++ b/jobs/defaults.yaml @@ -9,6 +9,7 @@ if [ "$unamestr" = 'FreeBSD' ]; then MAKE='gmake' fi + export OSINFO_SYSTEM_DIR=$VIRT_PREFIX/share/osinfo check_env: | export VIR_TEST_EXPENSIVE=1 export VIR_TEST_DEBUG=2 diff --git a/projects/libosinfo.yaml b/projects/libosinfo.yaml index 77c0414..7fa5b48 100644 --- a/projects/libosinfo.yaml +++ b/projects/libosinfo.yaml @@ -7,13 +7,6 @@ - libvirt-fedora-26 - libvirt-fedora-rawhide title: libosinfo - make_env: | - MAKE='make' - unamestr=`uname` - if [ "$unamestr" = 'FreeBSD' ]; then - MAKE='gmake' - fi - export OSINFO_SYSTEM_DIR=$VIRT_PREFIX/share/osinfo jobs: - autotools-build-job: parent_jobs: 'osinfo-db-master-build' -- 2.13.6
On Fri, Oct 27, 2017 at 09:48:14AM +0200, Andrea Bolognani wrote:
Althought the environment variable is only used when building libosinfo, there is no harm in having it set in the environment at all times; moreover, hoisting it to the default make_env allows us to remove duplicated code used to set $MAKE in the libosinfo project.
$OSINFO_SYSTEM_DIR itself depends on $VIRT_PREFIX, which along with a bunch of other environment variables is defined not at the job level but rather at the machine level, making it opaque to anyone except the Jenkins administrator: the plan is to move even more environment variables to the default make_env over time.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- jobs/defaults.yaml | 1 + projects/libosinfo.yaml | 7 ------- 2 files changed, 1 insertion(+), 7 deletions(-)
diff --git a/jobs/defaults.yaml b/jobs/defaults.yaml index a403d0e..7aa5a30 100644 --- a/jobs/defaults.yaml +++ b/jobs/defaults.yaml @@ -9,6 +9,7 @@ if [ "$unamestr" = 'FreeBSD' ]; then MAKE='gmake' fi + export OSINFO_SYSTEM_DIR=$VIRT_PREFIX/share/osinfo check_env: | export VIR_TEST_EXPENSIVE=1 export VIR_TEST_DEBUG=2 diff --git a/projects/libosinfo.yaml b/projects/libosinfo.yaml index 77c0414..7fa5b48 100644 --- a/projects/libosinfo.yaml +++ b/projects/libosinfo.yaml @@ -7,13 +7,6 @@ - libvirt-fedora-26 - libvirt-fedora-rawhide title: libosinfo - make_env: | - MAKE='make' - unamestr=`uname` - if [ "$unamestr" = 'FreeBSD' ]; then - MAKE='gmake' - fi - export OSINFO_SYSTEM_DIR=$VIRT_PREFIX/share/osinfo jobs: - autotools-build-job: parent_jobs: 'osinfo-db-master-build'
Again, I don't really think this change is a benefit. Project specific env vars should only be defined against the project rules. If we want to eliminate the duplication, of the make variables, we should change the variables we set. ie, stop overriding 'make_env' here, and instead provide a general 'build_env' property that projects can set, that gets subs into the autotools.yaml file. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Fri, 2017-10-27 at 09:20 +0100, Daniel P. Berrange wrote:
title: libosinfo - make_env: | - MAKE='make' - unamestr=`uname` - if [ "$unamestr" = 'FreeBSD' ]; then - MAKE='gmake' - fi - export OSINFO_SYSTEM_DIR=$VIRT_PREFIX/share/osinfo jobs: - autotools-build-job: parent_jobs: 'osinfo-db-master-build'
Again, I don't really think this change is a benefit. Project specific env vars should only be defined against the project rules.
If we want to eliminate the duplication, of the make variables, we should change the variables we set. ie, stop overriding 'make_env' here, and instead provide a general 'build_env' property that projects can set, that gets subs into the autotools.yaml file.
Okay, I can do that. I'll post a v2 shortly. -- Andrea Bolognani / Red Hat / Virtualization
We're limiting builds to CentOS and Fedora, or even just Fedora, in a lot of cases where the project builds just fine on Debian and even FreeBSD. Let's build all the things on all the operating systems! Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- projects/libosinfo.yaml | 4 ++++ projects/libvirt-glib.yaml | 4 ++++ projects/libvirt-go-xml.yaml | 4 ++++ projects/libvirt-go.yaml | 4 ++++ projects/libvirt-perl.yaml | 5 +++++ projects/libvirt-python.yaml | 4 ++++ projects/libvirt-sandbox.yaml | 2 ++ projects/libvirt-tck.yaml | 4 ++++ projects/libvirt.yaml | 12 ++++++------ projects/osinfo-db-tools.yaml | 4 ++++ projects/osinfo-db.yaml | 3 +++ projects/virt-manager.yaml | 4 ++++ projects/virt-viewer.yaml | 4 ++++ 13 files changed, 52 insertions(+), 6 deletions(-) diff --git a/projects/libosinfo.yaml b/projects/libosinfo.yaml index 7fa5b48..a156b08 100644 --- a/projects/libosinfo.yaml +++ b/projects/libosinfo.yaml @@ -3,9 +3,13 @@ name: libosinfo machines: - libvirt-centos-7 + - libvirt-debian-8 + - libvirt-debian-9 - libvirt-fedora-25 - libvirt-fedora-26 - libvirt-fedora-rawhide + - libvirt-freebsd-10 + - libvirt-freebsd-11 title: libosinfo jobs: - autotools-build-job: diff --git a/projects/libvirt-glib.yaml b/projects/libvirt-glib.yaml index eba4646..e82dbcc 100644 --- a/projects/libvirt-glib.yaml +++ b/projects/libvirt-glib.yaml @@ -3,9 +3,13 @@ name: libvirt-glib machines: - libvirt-centos-7 + - libvirt-debian-8 + - libvirt-debian-9 - libvirt-fedora-25 - libvirt-fedora-26 - libvirt-fedora-rawhide + - libvirt-freebsd-10 + - libvirt-freebsd-11 title: Libvirt GLib jobs: - autotools-build-job: diff --git a/projects/libvirt-go-xml.yaml b/projects/libvirt-go-xml.yaml index ebe06fb..bd57d5f 100644 --- a/projects/libvirt-go-xml.yaml +++ b/projects/libvirt-go-xml.yaml @@ -3,9 +3,13 @@ name: libvirt-go-xml machines: - libvirt-centos-7 + - libvirt-debian-8 + - libvirt-debian-9 - libvirt-fedora-25 - libvirt-fedora-26 - libvirt-fedora-rawhide + - libvirt-freebsd-10 + - libvirt-freebsd-11 title: Libvirt Go XML jobs: - go-build-job: diff --git a/projects/libvirt-go.yaml b/projects/libvirt-go.yaml index 9ffdd0a..0fc7bec 100644 --- a/projects/libvirt-go.yaml +++ b/projects/libvirt-go.yaml @@ -3,9 +3,13 @@ name: libvirt-go machines: - libvirt-centos-7 + - libvirt-debian-8 + - libvirt-debian-9 - libvirt-fedora-25 - libvirt-fedora-26 - libvirt-fedora-rawhide + - libvirt-freebsd-10 + - libvirt-freebsd-11 title: Libvirt Go jobs: - go-build-job: diff --git a/projects/libvirt-perl.yaml b/projects/libvirt-perl.yaml index 7646e27..5c314ec 100644 --- a/projects/libvirt-perl.yaml +++ b/projects/libvirt-perl.yaml @@ -2,10 +2,15 @@ - project: name: libvirt-perl machines: + - libvirt-centos-6 - libvirt-centos-7 + - libvirt-debian-8 + - libvirt-debian-9 - libvirt-fedora-25 - libvirt-fedora-26 - libvirt-fedora-rawhide + - libvirt-freebsd-10 + - libvirt-freebsd-11 title: Libvirt Perl jobs: - perl-makemaker-build-job: diff --git a/projects/libvirt-python.yaml b/projects/libvirt-python.yaml index cae8ca7..de916aa 100644 --- a/projects/libvirt-python.yaml +++ b/projects/libvirt-python.yaml @@ -4,9 +4,13 @@ machines: - libvirt-centos-6 - libvirt-centos-7 + - libvirt-debian-8 + - libvirt-debian-9 - libvirt-fedora-25 - libvirt-fedora-26 - libvirt-fedora-rawhide + - libvirt-freebsd-10 + - libvirt-freebsd-11 title: Libvirt Python jobs: - python-distutils-build-job: diff --git a/projects/libvirt-sandbox.yaml b/projects/libvirt-sandbox.yaml index 2920084..9d6add8 100644 --- a/projects/libvirt-sandbox.yaml +++ b/projects/libvirt-sandbox.yaml @@ -2,6 +2,8 @@ - project: name: libvirt-sandbox machines: + - libvirt-centos-7 + - libvirt-debian-9 - libvirt-fedora-25 - libvirt-fedora-26 - libvirt-fedora-rawhide diff --git a/projects/libvirt-tck.yaml b/projects/libvirt-tck.yaml index ca72f6c..29340cb 100644 --- a/projects/libvirt-tck.yaml +++ b/projects/libvirt-tck.yaml @@ -2,9 +2,13 @@ - project: name: libvirt-tck machines: + - libvirt-debian-8 + - libvirt-debian-9 - libvirt-fedora-25 - libvirt-fedora-26 - libvirt-fedora-rawhide + - libvirt-freebsd-10 + - libvirt-freebsd-11 title: Libvirt TCK jobs: - perl-modulebuild-build-job: diff --git a/projects/libvirt.yaml b/projects/libvirt.yaml index 5617e17..a316913 100644 --- a/projects/libvirt.yaml +++ b/projects/libvirt.yaml @@ -4,6 +4,8 @@ machines: - libvirt-centos-6 - libvirt-centos-7 + - libvirt-debian-8 + - libvirt-debian-9 - libvirt-fedora-25 - libvirt-fedora-26 - libvirt-fedora-rawhide @@ -24,18 +26,16 @@ - libvirt-freebsd-11 - autotools-syntax-check-job: parent_jobs: 'libvirt-master-build' + - autotools-check-job: + parent_jobs: 'libvirt-master-syntax-check' + - autotools-rpm-job: + parent_jobs: 'libvirt-master-check' machines: - libvirt-centos-6 - libvirt-centos-7 - - libvirt-debian-8 - - libvirt-debian-9 - libvirt-fedora-25 - libvirt-fedora-26 - libvirt-fedora-rawhide - - autotools-check-job: - parent_jobs: 'libvirt-master-syntax-check' - - autotools-rpm-job: - parent_jobs: 'libvirt-master-check' - autotools-mingw-job: parent_jobs: machines: diff --git a/projects/osinfo-db-tools.yaml b/projects/osinfo-db-tools.yaml index 93931af..ca656ee 100644 --- a/projects/osinfo-db-tools.yaml +++ b/projects/osinfo-db-tools.yaml @@ -3,9 +3,13 @@ name: osinfo-db-tools machines: - libvirt-centos-7 + - libvirt-debian-8 + - libvirt-debian-9 - libvirt-fedora-25 - libvirt-fedora-26 - libvirt-fedora-rawhide + - libvirt-freebsd-10 + - libvirt-freebsd-11 title: osinfo database tools jobs: - autotools-build-job: diff --git a/projects/osinfo-db.yaml b/projects/osinfo-db.yaml index 83eb92f..9fec36a 100644 --- a/projects/osinfo-db.yaml +++ b/projects/osinfo-db.yaml @@ -3,9 +3,12 @@ name: osinfo-db machines: - libvirt-centos-7 + - libvirt-debian-9 - libvirt-fedora-25 - libvirt-fedora-26 - libvirt-fedora-rawhide + - libvirt-freebsd-10 + - libvirt-freebsd-11 title: osinfo database jobs: - generic-build-job: diff --git a/projects/virt-manager.yaml b/projects/virt-manager.yaml index a50e0ab..ecfd497 100644 --- a/projects/virt-manager.yaml +++ b/projects/virt-manager.yaml @@ -3,9 +3,13 @@ name: virt-manager machines: - libvirt-centos-7 + - libvirt-debian-8 + - libvirt-debian-9 - libvirt-fedora-25 - libvirt-fedora-26 - libvirt-fedora-rawhide + - libvirt-freebsd-10 + - libvirt-freebsd-11 title: Virtual Machine Manager jobs: - python-distutils-build-job: diff --git a/projects/virt-viewer.yaml b/projects/virt-viewer.yaml index e3ef04a..9726119 100644 --- a/projects/virt-viewer.yaml +++ b/projects/virt-viewer.yaml @@ -3,9 +3,13 @@ name: virt-viewer machines: - libvirt-centos-7 + - libvirt-debian-8 + - libvirt-debian-9 - libvirt-fedora-25 - libvirt-fedora-26 - libvirt-fedora-rawhide + - libvirt-freebsd-10 + - libvirt-freebsd-11 title: Virt Viewer jobs: - autotools-build-job: -- 2.13.6
participants (2)
-
Andrea Bolognani -
Daniel P. Berrange