12:26 p.m.
My own attempt at fixing libosinfo-master-rpm's failing status,
and then some.
This is an extension of Pavel's attempt[1] in a way; except of
course I think my approach is superior, otherwise I wouldn't have
bothered ;)
It also sets the groundwork for finally moving $VIRT_PREFIX and
friends out of the Jenkins server configuration, where they are
invisible to anyone but the CI admins, and into the git repository
so that they can be tracked alongside the jobs using them.
[1] https://www.redhat.com/archives/libvir-list/2017-November/msg00045.html
Andrea Bolognani (5):
jobs: Rename {make_env} to {global_env}
jobs: Include {global_env} everywhere
jobs: Rename {check_env} to {env}
jobs: Include {env} everywhere
jobs: Include $PATH definition in {global_env}
jobs/autotools.yaml | 17 ++++++++++++-----
jobs/defaults.yaml | 5 +++--
jobs/generic.yaml | 20 ++++++++++++++++----
jobs/go.yaml | 4 ++++
jobs/perl-makemaker.yaml | 9 ++++++---
jobs/perl-modulebuild.yaml | 6 ++++++
jobs/python-distutils.yaml | 6 ++++++
projects/libosinfo.yaml | 2 +-
projects/libvirt.yaml | 2 +-
projects/osinfo-db.yaml | 4 ----
10 files changed, 55 insertions(+), 20 deletions(-)
--
2.14.3
12:26 p.m.
New subject: [libvirt] [jenkins-ci PATCH 1/5] jobs: Rename {make_env} to {global_env}
Right now, this environment only contains definitions that are
useful when running make, but eventually it will be expanded and
include all sort of variables that are not specific to a single
project, eg. $VIRT_PREFIX and the like.
Signed-off-by: Andrea Bolognani <abologna(a)redhat.com>
---
jobs/autotools.yaml | 8 ++++----
jobs/defaults.yaml | 2 +-
jobs/perl-makemaker.yaml | 6 +++---
projects/osinfo-db.yaml | 4 ++--
4 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/jobs/autotools.yaml b/jobs/autotools.yaml
index 74378ce..a4a5a5b 100644
--- a/jobs/autotools.yaml
+++ b/jobs/autotools.yaml
@@ -40,7 +40,7 @@
values: '{obj:machines}'
builders:
- shell: |
- {make_env}
+ {global_env}
mkdir build
cd build
../autogen.sh --prefix=$VIRT_PREFIX {autogen_args}
@@ -81,7 +81,7 @@
values: '{obj:machines}'
builders:
- shell: |
- {make_env}
+ {global_env}
cd build
$MAKE -j{smp} syntax-check
publishers:
@@ -119,7 +119,7 @@
values: '{obj:machines}'
builders:
- shell: |
- {make_env}
+ {global_env}
{check_env}
cd build
if ! $MAKE -j{smp} check
@@ -163,7 +163,7 @@
values: '{obj:machines}'
builders:
- shell: |
- {make_env}
+ {global_env}
cd build
sed -i -e 's/BuildRequires: osinfo-db.*//' {name}.spec
sed -i -e 's/BuildRequires: libvirt.*devel.*//' {name}.spec
diff --git a/jobs/defaults.yaml b/jobs/defaults.yaml
index 0e628bb..e2a5b1a 100644
--- a/jobs/defaults.yaml
+++ b/jobs/defaults.yaml
@@ -3,7 +3,7 @@
name: global
branch: master
node: libvirt
- make_env: |
+ global_env: |
MAKE='make'
unamestr=`uname`
if [ "$unamestr" = 'FreeBSD' ]; then
diff --git a/jobs/perl-makemaker.yaml b/jobs/perl-makemaker.yaml
index 0a3227a..930d4ee 100644
--- a/jobs/perl-makemaker.yaml
+++ b/jobs/perl-makemaker.yaml
@@ -41,7 +41,7 @@
builders:
- shell: |
perl Makefile.PL PREFIX="$VIRT_PREFIX"
- {make_env}
+ {global_env}
$MAKE
$MAKE -j{smp} install
$MAKE -j{smp} manifest
@@ -82,7 +82,7 @@
values: '{obj:machines}'
builders:
- shell: |
- {make_env}
+ {global_env}
$MAKE -j{smp} test {test_args}
publishers:
- email:
@@ -120,7 +120,7 @@
values: '{obj:machines}'
builders:
- shell: |
- {make_env}
+ {global_env}
sed -i -e 's/BuildRequires: *perl-Sys-Virt.*//' *.spec
sed -i -e 's/BuildRequires: *libvirt.*devel.*//' *.spec
rm -f *.tar.{archive_format}
diff --git a/projects/osinfo-db.yaml b/projects/osinfo-db.yaml
index 6fa1895..9d70000 100644
--- a/projects/osinfo-db.yaml
+++ b/projects/osinfo-db.yaml
@@ -15,7 +15,7 @@
- generic-build-job:
parent_jobs: 'osinfo-db-tools-master-build'
command: |
- {make_env}
+ {global_env}
export PATH=$VIRT_PREFIX/bin:$PATH
$MAKE -j{smp} all
$MAKE install OSINFO_DB_TARGET="--system"
@@ -27,7 +27,7 @@
- libvirt-fedora-26
- libvirt-fedora-rawhide
command: |
- {make_env}
+ {global_env}
export PATH=$VIRT_PREFIX/bin:$PATH
rm -f *.tar.xz
$MAKE osinfo-db.spec
--
2.14.3
12:26 p.m.
New subject: [libvirt] [jenkins-ci PATCH 2/5] jobs: Include {global_env} everywhere
This environment will later include variables that are relevant to
more than make, so it should be included every time a shell command
is going to be executed.
Including {global_env} in the generic-*-job templates allows us to
drop the ad-hoc inclusions from the osinfo-db project.
Signed-off-by: Andrea Bolognani <abologna(a)redhat.com>
---
jobs/autotools.yaml | 2 ++
jobs/generic.yaml | 16 ++++++++++++----
jobs/go.yaml | 2 ++
jobs/perl-modulebuild.yaml | 3 +++
jobs/python-distutils.yaml | 3 +++
projects/osinfo-db.yaml | 2 --
6 files changed, 22 insertions(+), 6 deletions(-)
diff --git a/jobs/autotools.yaml b/jobs/autotools.yaml
index a4a5a5b..8a37d00 100644
--- a/jobs/autotools.yaml
+++ b/jobs/autotools.yaml
@@ -218,6 +218,7 @@
values: '{obj:machines}'
builders:
- shell: |
+ {global_env}
mkdir -p build32
cd build32
@@ -228,6 +229,7 @@
$MAKE -j{smp}
$MAKE -j{smp} install
- shell: |
+ {global_env}
mkdir -p build64
cd build64
diff --git a/jobs/generic.yaml b/jobs/generic.yaml
index 050e35d..20ef5da 100644
--- a/jobs/generic.yaml
+++ b/jobs/generic.yaml
@@ -39,7 +39,9 @@
type: slave
values: '{obj:machines}'
builders:
- - shell: '{command}'
+ - shell: |
+ {global_env}
+ {command}
publishers:
- email:
recipients: '{obj:spam}'
@@ -74,7 +76,9 @@
type: slave
values: '{obj:machines}'
builders:
- - shell: '{command}'
+ - shell: |
+ {global_env}
+ {command}
publishers:
- email:
recipients: '{obj:spam}'
@@ -109,7 +113,9 @@
type: slave
values: '{obj:machines}'
builders:
- - shell: '{command}'
+ - shell: |
+ {global_env}
+ {command}
publishers:
- email:
recipients: '{obj:spam}'
@@ -145,7 +151,9 @@
type: slave
values: '{obj:machines}'
builders:
- - shell: '{command}'
+ - shell: |
+ {global_env}
+ {command}
publishers:
- email:
recipients: '{obj:spam}'
diff --git a/jobs/go.yaml b/jobs/go.yaml
index 8eee6af..8764395 100644
--- a/jobs/go.yaml
+++ b/jobs/go.yaml
@@ -40,6 +40,7 @@
values: '{obj:machines}'
builders:
- shell: |
+ {global_env}
go build -v
publishers:
- email:
@@ -76,6 +77,7 @@
values: '{obj:machines}'
builders:
- shell: |
+ {global_env}
go test -tags api
publishers:
- email:
diff --git a/jobs/perl-modulebuild.yaml b/jobs/perl-modulebuild.yaml
index cb81de1..188b153 100644
--- a/jobs/perl-modulebuild.yaml
+++ b/jobs/perl-modulebuild.yaml
@@ -40,6 +40,7 @@
values: '{obj:machines}'
builders:
- shell: |
+ {global_env}
perl Build.PL install_base="$VIRT_PREFIX"
perl Build
perl Build manifest
@@ -80,6 +81,7 @@
values: '{obj:machines}'
builders:
- shell: |
+ {global_env}
perl Build test
publishers:
- email:
@@ -117,6 +119,7 @@
values: '{obj:machines}'
builders:
- shell: |
+ {global_env}
sed -i -e 's/BuildRequires: *perl-Sys-Virt.*//' *.spec
sed -i -e 's/BuildRequires: *perl(Sys::Virt).*//' *.spec
sed -i -e 's/BuildRequires: *libvirt.*devel.*//' *.spec
diff --git a/jobs/python-distutils.yaml b/jobs/python-distutils.yaml
index 7055d26..b0e3f5f 100644
--- a/jobs/python-distutils.yaml
+++ b/jobs/python-distutils.yaml
@@ -40,6 +40,7 @@
values: '{obj:machines}'
builders:
- shell: |
+ {global_env}
{command_pre_build}
python2 setup.py build
python2 setup.py install --prefix=$VIRT_PREFIX
@@ -79,6 +80,7 @@
values: '{obj:machines}'
builders:
- shell: |
+ {global_env}
python2 setup.py test
publishers:
- email:
@@ -115,6 +117,7 @@
values: '{obj:machines}'
builders:
- shell: |
+ {global_env}
sed -i -e 's/BuildRequires: libvirt.*devel.*//' *.spec.in
python2 setup.py rpm
publishers:
diff --git a/projects/osinfo-db.yaml b/projects/osinfo-db.yaml
index 9d70000..95ad845 100644
--- a/projects/osinfo-db.yaml
+++ b/projects/osinfo-db.yaml
@@ -15,7 +15,6 @@
- generic-build-job:
parent_jobs: 'osinfo-db-tools-master-build'
command: |
- {global_env}
export PATH=$VIRT_PREFIX/bin:$PATH
$MAKE -j{smp} all
$MAKE install OSINFO_DB_TARGET="--system"
@@ -27,7 +26,6 @@
- libvirt-fedora-26
- libvirt-fedora-rawhide
command: |
- {global_env}
export PATH=$VIRT_PREFIX/bin:$PATH
rm -f *.tar.xz
$MAKE osinfo-db.spec
--
2.14.3
12:26 p.m.
New subject: [libvirt] [jenkins-ci PATCH 3/5] jobs: Rename {check_env} to {env}
There's no reason to have an environment which is tied, at least
in name, to *-check-job only: having a local environment is
something that can be useful for any kind of job, so it's better
to use a neutral name.
Signed-off-by: Andrea Bolognani <abologna(a)redhat.com>
---
jobs/autotools.yaml | 2 +-
jobs/defaults.yaml | 2 +-
projects/libosinfo.yaml | 2 +-
projects/libvirt.yaml | 2 +-
4 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/jobs/autotools.yaml b/jobs/autotools.yaml
index 8a37d00..f6a2af0 100644
--- a/jobs/autotools.yaml
+++ b/jobs/autotools.yaml
@@ -120,7 +120,7 @@
builders:
- shell: |
{global_env}
- {check_env}
+ {env}
cd build
if ! $MAKE -j{smp} check
then
diff --git a/jobs/defaults.yaml b/jobs/defaults.yaml
index e2a5b1a..86e7100 100644
--- a/jobs/defaults.yaml
+++ b/jobs/defaults.yaml
@@ -9,6 +9,6 @@
if [ "$unamestr" = 'FreeBSD' ]; then
MAKE='gmake'
fi
- check_env: |
+ env: |
smp: 3
spam: ymankad(a)redhat.com libvirt-ci(a)redhat.com
diff --git a/projects/libosinfo.yaml b/projects/libosinfo.yaml
index 9ab2281..311ca71 100644
--- a/projects/libosinfo.yaml
+++ b/projects/libosinfo.yaml
@@ -11,7 +11,7 @@
- libvirt-freebsd-10
- libvirt-freebsd-11
title: libosinfo
- check_env: |
+ env: |
export OSINFO_SYSTEM_DIR=$VIRT_PREFIX/share/osinfo
jobs:
- autotools-build-job:
diff --git a/projects/libvirt.yaml b/projects/libvirt.yaml
index 5661066..52f8740 100644
--- a/projects/libvirt.yaml
+++ b/projects/libvirt.yaml
@@ -28,7 +28,7 @@
parent_jobs: 'libvirt-master-build'
- autotools-check-job:
parent_jobs: 'libvirt-master-syntax-check'
- check_env: |
+ env: |
export VIR_TEST_EXPENSIVE=1
export VIR_TEST_DEBUG=2
- autotools-rpm-job:
--
2.14.3
7:29 a.m.
New subject: [libvirt] [jenkins-ci PATCH 3/5] jobs: Rename {check_env} to {env}
On Fri, Nov 17, 2017 at 07:26:10PM +0100, Andrea Bolognani wrote:
There's no reason to have an environment which is tied, at least
in name, to *-check-job only: having a local environment is
something that can be useful for any kind of job, so it's better
to use a neutral name.
My proposal was "job_env" and you as a reply suggested "local_env".
I would probably prefer using "local_env" over "env" since it is
local environment as you've described it in commit message :).
Pavel
10:10 a.m.
New subject: [libvirt] [jenkins-ci PATCH 3/5] jobs: Rename {check_env} to {env}
On Mon, 2017-11-20 at 14:29 +0100, Pavel Hrdina wrote:
On Fri, Nov 17, 2017 at 07:26:10PM +0100, Andrea Bolognani wrote:
> There's no reason to have an environment which is tied, at least
> in name, to *-check-job only: having a local environment is
> something that can be useful for any kind of job, so it's better
> to use a neutral name.
My proposal was "job_env" and you as a reply suggested "local_env".
I would probably prefer using "local_env" over "env" since it is
local environment as you've described it in commit message :).
'env' looks more natural when defining projects, but 'local_env'
mirrors 'global_env' nicely and it's way more greppable, so I've
changed it and pushed :)
--
Andrea Bolognani / Red Hat / Virtualization
12:26 p.m.
New subject: [libvirt] [jenkins-ci PATCH 4/5] jobs: Include {env} everywhere
Any job might need to define a custom environment, so it should
be included everywhere to make it possible.
A side-effect of this change is that $OSINFO_SYSTEM_DIR will
be defined in libosinfo-master-rpm's environment, and the job
should stop failing.
Signed-off-by: Andrea Bolognani <abologna(a)redhat.com>
---
jobs/autotools.yaml | 5 +++++
jobs/generic.yaml | 4 ++++
jobs/go.yaml | 2 ++
jobs/perl-makemaker.yaml | 3 +++
jobs/perl-modulebuild.yaml | 3 +++
jobs/python-distutils.yaml | 3 +++
6 files changed, 20 insertions(+)
diff --git a/jobs/autotools.yaml b/jobs/autotools.yaml
index f6a2af0..d71d0ce 100644
--- a/jobs/autotools.yaml
+++ b/jobs/autotools.yaml
@@ -41,6 +41,7 @@
builders:
- shell: |
{global_env}
+ {env}
mkdir build
cd build
../autogen.sh --prefix=$VIRT_PREFIX {autogen_args}
@@ -82,6 +83,7 @@
builders:
- shell: |
{global_env}
+ {env}
cd build
$MAKE -j{smp} syntax-check
publishers:
@@ -164,6 +166,7 @@
builders:
- shell: |
{global_env}
+ {env}
cd build
sed -i -e 's/BuildRequires: osinfo-db.*//' {name}.spec
sed -i -e 's/BuildRequires: libvirt.*devel.*//' {name}.spec
@@ -219,6 +222,7 @@
builders:
- shell: |
{global_env}
+ {env}
mkdir -p build32
cd build32
@@ -230,6 +234,7 @@
$MAKE -j{smp} install
- shell: |
{global_env}
+ {env}
mkdir -p build64
cd build64
diff --git a/jobs/generic.yaml b/jobs/generic.yaml
index 20ef5da..9b4569f 100644
--- a/jobs/generic.yaml
+++ b/jobs/generic.yaml
@@ -41,6 +41,7 @@
builders:
- shell: |
{global_env}
+ {env}
{command}
publishers:
- email:
@@ -78,6 +79,7 @@
builders:
- shell: |
{global_env}
+ {env}
{command}
publishers:
- email:
@@ -115,6 +117,7 @@
builders:
- shell: |
{global_env}
+ {env}
{command}
publishers:
- email:
@@ -153,6 +156,7 @@
builders:
- shell: |
{global_env}
+ {env}
{command}
publishers:
- email:
diff --git a/jobs/go.yaml b/jobs/go.yaml
index 8764395..22d3459 100644
--- a/jobs/go.yaml
+++ b/jobs/go.yaml
@@ -41,6 +41,7 @@
builders:
- shell: |
{global_env}
+ {env}
go build -v
publishers:
- email:
@@ -78,6 +79,7 @@
builders:
- shell: |
{global_env}
+ {env}
go test -tags api
publishers:
- email:
diff --git a/jobs/perl-makemaker.yaml b/jobs/perl-makemaker.yaml
index 930d4ee..ea05b61 100644
--- a/jobs/perl-makemaker.yaml
+++ b/jobs/perl-makemaker.yaml
@@ -42,6 +42,7 @@
- shell: |
perl Makefile.PL PREFIX="$VIRT_PREFIX"
{global_env}
+ {env}
$MAKE
$MAKE -j{smp} install
$MAKE -j{smp} manifest
@@ -83,6 +84,7 @@
builders:
- shell: |
{global_env}
+ {env}
$MAKE -j{smp} test {test_args}
publishers:
- email:
@@ -121,6 +123,7 @@
builders:
- shell: |
{global_env}
+ {env}
sed -i -e 's/BuildRequires: *perl-Sys-Virt.*//' *.spec
sed -i -e 's/BuildRequires: *libvirt.*devel.*//' *.spec
rm -f *.tar.{archive_format}
diff --git a/jobs/perl-modulebuild.yaml b/jobs/perl-modulebuild.yaml
index 188b153..e556237 100644
--- a/jobs/perl-modulebuild.yaml
+++ b/jobs/perl-modulebuild.yaml
@@ -41,6 +41,7 @@
builders:
- shell: |
{global_env}
+ {env}
perl Build.PL install_base="$VIRT_PREFIX"
perl Build
perl Build manifest
@@ -82,6 +83,7 @@
builders:
- shell: |
{global_env}
+ {env}
perl Build test
publishers:
- email:
@@ -120,6 +122,7 @@
builders:
- shell: |
{global_env}
+ {env}
sed -i -e 's/BuildRequires: *perl-Sys-Virt.*//' *.spec
sed -i -e 's/BuildRequires: *perl(Sys::Virt).*//' *.spec
sed -i -e 's/BuildRequires: *libvirt.*devel.*//' *.spec
diff --git a/jobs/python-distutils.yaml b/jobs/python-distutils.yaml
index b0e3f5f..fc969ad 100644
--- a/jobs/python-distutils.yaml
+++ b/jobs/python-distutils.yaml
@@ -41,6 +41,7 @@
builders:
- shell: |
{global_env}
+ {env}
{command_pre_build}
python2 setup.py build
python2 setup.py install --prefix=$VIRT_PREFIX
@@ -81,6 +82,7 @@
builders:
- shell: |
{global_env}
+ {env}
python2 setup.py test
publishers:
- email:
@@ -118,6 +120,7 @@
builders:
- shell: |
{global_env}
+ {env}
sed -i -e 's/BuildRequires: libvirt.*devel.*//' *.spec.in
python2 setup.py rpm
publishers:
--
2.14.3
12:26 p.m.
New subject: [libvirt] [jenkins-ci PATCH 5/5] jobs: Include $PATH definition in {global_env}
$PATH is generic enough that it shouldn't be limited to the
osinfo-db project.
Signed-off-by: Andrea Bolognani <abologna(a)redhat.com>
---
jobs/defaults.yaml | 1 +
projects/osinfo-db.yaml | 2 --
2 files changed, 1 insertion(+), 2 deletions(-)
diff --git a/jobs/defaults.yaml b/jobs/defaults.yaml
index 86e7100..7b2c040 100644
--- a/jobs/defaults.yaml
+++ b/jobs/defaults.yaml
@@ -9,6 +9,7 @@
if [ "$unamestr" = 'FreeBSD' ]; then
MAKE='gmake'
fi
+ export PATH=$VIRT_PREFIX/bin:$PATH
env: |
smp: 3
spam: ymankad(a)redhat.com libvirt-ci(a)redhat.com
diff --git a/projects/osinfo-db.yaml b/projects/osinfo-db.yaml
index 95ad845..2a9cab2 100644
--- a/projects/osinfo-db.yaml
+++ b/projects/osinfo-db.yaml
@@ -15,7 +15,6 @@
- generic-build-job:
parent_jobs: 'osinfo-db-tools-master-build'
command: |
- export PATH=$VIRT_PREFIX/bin:$PATH
$MAKE -j{smp} all
$MAKE install OSINFO_DB_TARGET="--system"
- generic-rpm-job:
@@ -26,7 +25,6 @@
- libvirt-fedora-26
- libvirt-fedora-rawhide
command: |
- export PATH=$VIRT_PREFIX/bin:$PATH
rm -f *.tar.xz
$MAKE osinfo-db.spec
perl -i -p -e 's/BuildRequires: osinfo-db-tools.*//' osinfo-db.spec
--
2.14.3
7:31 a.m.
On Fri, Nov 17, 2017 at 07:26:07PM +0100, Andrea Bolognani wrote:
My own attempt at fixing libosinfo-master-rpm's failing status,
and then some.
This is an extension of Pavel's attempt[1] in a way; except of
course I think my approach is superior, otherwise I wouldn't have
bothered ;)
It also sets the groundwork for finally moving $VIRT_PREFIX and
friends out of the Jenkins server configuration, where they are
invisible to anyone but the CI admins, and into the git repository
so that they can be tracked alongside the jobs using them.
I've looked into the jenkins-job-builder documentation and was not able
to find a better way so we might use {global_env} for all the guest
specific environment variables.
[1] https://www.redhat.com/archives/libvir-list/2017-November/msg00045.html
Andrea Bolognani (5):
jobs: Rename {make_env} to {global_env}
jobs: Include {global_env} everywhere
jobs: Rename {check_env} to {env}
jobs: Include {env} everywhere
jobs: Include $PATH definition in {global_env}
See my note for PATCH 3.
Reviewed-by: Pavel Hrdina <phrdina(a)redhat.com>
2594
days inactive
2597
days old
8 comments
2 participants
participants (2)
-
Andrea Bolognani -
Pavel Hrdina