12:53 p.m.
The VZ driver in libvirt periodically gets broken by refactoring in
libvirt. This is not noticed either before or after merge because none
of our CI tests nor common developer build hosts includ the deps needed
for the VZ driver.
The OpenVZ project, however, does provide builds of the required
packages for RHEL-7. We can use these packages in our CentOS 7 CI VMs to
enable build testing of the VZ driver. This closes the only hole we have
in driver build coverage for CI.
Changed in v2:
- Enable GPG verification
- Use older repo avoid temporary deps problem
- Setup repos in dockerfile too.
Daniel P. Berrangé (3):
guests: add openvz repository on CentOS 7
guests: define mapping for the libprlsdk package
guests: add libprlsdk package to libvirt project
guests/lcitool | 22 ++++++++++++++++
guests/playbooks/update/tasks/base.yml | 25 +++++++++++++++++++
guests/playbooks/update/templates/openvz.key | 20 +++++++++++++++
.../playbooks/update/templates/openvz.repo.j2 | 9 +++++++
guests/vars/mappings.yml | 4 +++
guests/vars/projects/libvirt.yml | 1 +
6 files changed, 81 insertions(+)
create mode 100644 guests/playbooks/update/templates/openvz.key
create mode 100644 guests/playbooks/update/templates/openvz.repo.j2
--
2.23.0
12:53 p.m.
New subject: [libvirt] [jenkins-ci PATCH v2 1/3] guests: add openvz repository on CentOS 7
The OpenVZ site provides a yum repo built against RHEL-7 that includes
the prlsdk-devel RPM needed for the VZ driver. This repo has quite alot
of packages that replace stuff from standard RHEL repos, so the yum
config file is set to whitelist only the minimal RPMs we need to do
builds. Fortunately they have no deps which would cause replacement of
standard RHEL RPMs.
Note this does not use the latest OpenVZ repo link, since that currently
has broken dependencies present
Error: Package: libprlcommon-7.0.183-1.vz7.x86_64 (vz)
Requires: libjson-c.so.2(libjson-c.so.2)(64bit)
The Requires line ought to be
libjson-c.so.2()(64bit)
Once that's fixed we can switch to the latest repo link.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
guests/lcitool | 22 ++++++++++++++++
guests/playbooks/update/tasks/base.yml | 25 +++++++++++++++++++
guests/playbooks/update/templates/openvz.key | 20 +++++++++++++++
.../playbooks/update/templates/openvz.repo.j2 | 9 +++++++
4 files changed, 76 insertions(+)
create mode 100644 guests/playbooks/update/templates/openvz.key
create mode 100644 guests/playbooks/update/templates/openvz.repo.j2
diff --git a/guests/lcitool b/guests/lcitool
index d617beb..4f874b3 100755
--- a/guests/lcitool
+++ b/guests/lcitool
@@ -593,6 +593,18 @@ class Application:
self._execute_playbook("build", args.hosts, args.projects,
args.git_revision)
+ def _get_openvz_repo(self):
+ basedir = os.path.dirname(sys.argv[0])
+ repofile = os.path.join(basedir, "playbooks", "update",
"templates", "openvz.repo.j2")
+ with open(repofile, "r") as r:
+ return r.read().rstrip()
+
+ def _get_openvz_key(self):
+ basedir = os.path.dirname(sys.argv[0])
+ repofile = os.path.join(basedir, "playbooks", "update",
"templates", "openvz.key")
+ with open(repofile, "r") as r:
+ return r.read().rstrip()
+
def _action_dockerfile(self, args):
mappings = self._projects.get_mappings()
pip_mappings = self._projects.get_pip_mappings()
@@ -723,6 +735,16 @@ class Application:
{package_manager} clean all -y
""").format(**varmap))
elif os_name == "CentOS" and os_version == "7":
+ repo = self._get_openvz_repo()
+ repocmd = "\\n\\\n".join(repo.split("\n"))
+ key = self._get_openvz_key()
+ keycmd = "\\n\\\n".join(key.split("\n"))
+
+ sys.stdout.write(
+ "RUN echo -e '%s' > /etc/yum.repos.d/openvz.repo
&& \\\n" % repocmd +
+ " echo -e '%s' >
/etc/pki/rpm-gpg/RPM-GPG-KEY-OpenVZ && \\\n" % keycmd +
+ " rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-OpenVZ")
+
sys.stdout.write(textwrap.dedent("""
RUN {package_manager} update -y && \\
{package_manager} install -y epel-release && \\
diff --git a/guests/playbooks/update/tasks/base.yml
b/guests/playbooks/update/tasks/base.yml
index 3d83e78..e17b50b 100644
--- a/guests/playbooks/update/tasks/base.yml
+++ b/guests/playbooks/update/tasks/base.yml
@@ -13,6 +13,31 @@
package:
name: epel-release
state: latest
+
+- name: Create OpenVZ key
+ template:
+ src: '{{ playbook_base }}/templates/openvz.key'
+ dest: /etc/pki/rpm-gpg/RPM-GPG-KEY-OpenVZ
+ owner: root
+ group: root
+ when:
+ - os_name == 'CentOS'
+ - os_version == '7'
+
+- name: Import OpenVZ key
+ command: 'rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-OpenVZ'
+ args:
+ warn: no
+ when:
+ - os_name == 'CentOS'
+ - os_version == '7'
+
+- name: Enable OpenVZ repository
+ template:
+ src: '{{ playbook_base }}/templates/openvz.repo.j2'
+ dest: /etc/yum.repos.d/openvz.repo
+ owner: root
+ group: root
when:
- os_name == 'CentOS'
- os_version == '7'
diff --git a/guests/playbooks/update/templates/openvz.key
b/guests/playbooks/update/templates/openvz.key
new file mode 100644
index 0000000..b77a137
--- /dev/null
+++ b/guests/playbooks/update/templates/openvz.key
@@ -0,0 +1,20 @@
+-----BEGIN PGP PUBLIC KEY BLOCK-----
+Version: GnuPG v2.0.22 (GNU/Linux)
+
+mI0EVl80nQEEAKrEeyeTCwrzS9kYedZ/sAc/GUqlb81C7pA9SaR3fyck5mVw1Ogk
+YdmNBPM2kY7QDxR9F0EpSpnxSCAXZXugsQ8KzZ0DRLVeBDQyGs9IGK5hI0zzxIil
+BzfvIexLiQQhLy7YlIi8Jt/uUqKkW0pIMNMGcduY97VATtczpncpkmSzABEBAAG0
+SFZpcnR1b3p6byBUZWFtIChHUEcga2V5IHNpZ25hdHVyZSBmb3IgcGFja2FnZXMp
+IDxzZWN1cml0eUB2aXJ0dW96em8uY29tPoi5BBMBAgAjBQJWXzSdAhsDBwsJCAcD
+AgEGFQgCCQoLBBYCAwECHgECF4AACgkQygt9GUTNrSruIgP/er70Eyo73A1gfrjv
+oPUkyo4rslVRZu3qqCwoMFtJc/Z/UxWgEka1buorlcGLa6eO/EZ49c0n+KGa4Kvt
+EUboIq0yEu5i0FyAj92ifm+hNhoAbGfm0cZ4/fD0oGr3l8OsQo4+iHX4xAPwFe7Y
+zABuB8I1ZDZ4OIp5tDfTTuF2LT24jQRWXzSdAQQAog2Aqb+Ptl68O7cQhWLjVGkj
+yyigZrdeReLx3HloKJPBeQ/kA6uvMJc/IYS3uppMWXv9v+QenS6uhP1TUJ2k9FvM
+t94MQZfALN7Vpf8AF+UeWu4Ru+y4BNzcFhrPhIFNFChOR2QqW6FkgE57D9I177NC
+oJMyrlNe8wcGa178An8AEQEAAYifBBgBAgAJBQJWXzSdAhsMAAoJEMoLfRlEza0q
+bKwD/3+OFVIEXnIv5XgdGRNX5fHggsUN1bb8gva7HANRlKdd4LD8foDM3F/yv/3V
+igG14D5EjKz56SaBDNgiI4++hOzb2M8jhAsR86jxkXFrrP1U3ZNRKg6av9DPFAPS
+WEiJKtQrZDJloqtyi/mmRa1VsV7RYR0VPJjhK/R8EQ7Ysshy
+=fRMg
+-----END PGP PUBLIC KEY BLOCK-----
diff --git a/guests/playbooks/update/templates/openvz.repo.j2
b/guests/playbooks/update/templates/openvz.repo.j2
new file mode 100644
index 0000000..19a9546
--- /dev/null
+++ b/guests/playbooks/update/templates/openvz.repo.j2
@@ -0,0 +1,9 @@
+[openvz]
+name=OpenVZ addons
+baseurl=https://download.openvz.org/virtuozzo/releases/openvz-7.0.11-235/x86_64/os/
+enabled=1
+gpgcheck=1
+skip_if_unavailable=0
+metadata_expire=6h
+priority=90
+includepkgs=libprl*
--
2.23.0
4:40 a.m.
New subject: [libvirt] Broken OpenVZ RPM deps on CentOS 7 (Re: [jenkins-ci PATCH v2 1/3] guests: add openvz repository on) CentOS 7
CC'ing Nikolay on this to raise the issue of broken deps in the OpenVZ
repo for CentOS 7 & incorrectly documented GPG keys ...
On Fri, Dec 06, 2019 at 06:53:38PM +0000, Daniel P. Berrangé wrote:
The OpenVZ site provides a yum repo built against RHEL-7 that
includes
the prlsdk-devel RPM needed for the VZ driver. This repo has quite alot
of packages that replace stuff from standard RHEL repos, so the yum
config file is set to whitelist only the minimal RPMs we need to do
builds. Fortunately they have no deps which would cause replacement of
standard RHEL RPMs.
Note this does not use the latest OpenVZ repo link, since that currently
has broken dependencies present
Originally I was using this URL for yum:
https://download.openvz.org/virtuozzo/releases/7.0/x86_64/os/
Which results in this broken dep at install time:
Error: Package: libprlcommon-7.0.183-1.vz7.x86_64 (vz)
Requires: libjson-c.so.2(libjson-c.so.2)(64bit)
The Requires line ought to be
libjson-c.so.2()(64bit)
This appears to be a recent problem from the Dec 4th release of
openvz-7.0.12-283 - the previous openvz-7.0.11-235 has correctly
resolving deps.
The other issue that I forgot to mention is that the GPG keys used for
signing the RPMs on download.openvz.org are incorrectly / misleadingly
documented.
In the README at:
https://download.openvz.org/
it documents & links to https://download.openvz.org/RPM-GPG-Key-OpenVZ
saying this is used to sign RPMs on download.openvz.org
This doc is repeated at https://wiki.openvz.org/Package_signatures
That key has key ID a7a1d4b6 as identified as
"OpenVZ Project <security(a)openvz.org>"
This documentation is all wrong though, as this key is not used
to sign the RPMs for CentOS7 at least
The RPMs in
https://download.openvz.org/virtuozzo/releases/7.0/x86_64/os/
at signed by key with ID 44cdad2a. It took me a long time to find
this key, but eventually I discovered a link to it from
https://docs.virtuozzo.com/keys/
Section 2, 2. Virtuozzo 7, Virtuozzo Automator 7, and Virtuozzo
PowerPanel Signing Key
https://docs.virtuozzo.com/keys/VIRTUOZZO_GPG_KEY
which identifies itself as "Virtuozzo Team (GPG key signature
for packages) <security(a)virtuozzo.com>"
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
9:12 a.m.
New subject: [libvirt] [jenkins-ci PATCH v2 1/3] guests: add openvz repository on CentOS 7
On Fri, 2019-12-06 at 18:53 +0000, Daniel P. Berrangé wrote:
+++ b/guests/lcitool
+ def _get_openvz_repo(self):
+ basedir = os.path.dirname(sys.argv[0])
+ repofile = os.path.join(basedir, "playbooks", "update",
"templates", "openvz.repo.j2")
This should be
base = Util.get_base()
repofile = os.path.join(base, ...)
+ def _get_openvz_key(self):
+ basedir = os.path.dirname(sys.argv[0])
+ repofile = os.path.join(basedir, "playbooks", "update",
"templates", "openvz.key")
Same here, except you probably want to call it keyfile instead of
repofile.
@@ -723,6 +735,16 @@ class Application:
elif os_name == "CentOS" and os_version == "7":
+ repo = self._get_openvz_repo()
+ repocmd = "\\n\\\n".join(repo.split("\n"))
+ key = self._get_openvz_key()
+ keycmd = "\\n\\\n".join(key.split("\n"))
+
+ sys.stdout.write(
+ "RUN echo -e '%s' > /etc/yum.repos.d/openvz.repo
&& \\\n" % repocmd +
+ " echo -e '%s' >
/etc/pki/rpm-gpg/RPM-GPG-KEY-OpenVZ && \\\n" % keycmd +
+ " rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-OpenVZ")
This is different from what's right above and below it for, as far
as I can tell, no good reason.
You can make it nicer and more consistent like
repo = self._get_openvz_repo()
key = self._get_openvz_key()
varmap["repo"] = "\\n\\\n".join(repo.split("\n"))
varmap["key"] = "\\n\\\n".join(key.split("\n"))
sys.stdout.write(textwrap.dedent("""
RUN echo -e '{repo}' > /etc/yum.repos.d/openvz.repo && \\
echo -e '{key}' > /etc/pki/rpm-gpg/RPM-GPG-KEY-OpenVZ && \\
rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-OpenVZ
""").format(**varmap))
sys.stdout.write(textwrap.dedent("""
RUN {package_manager} update -y && \\
{package_manager} install -y epel-release && \\
{package_manager} install -y {pkgs} && \\
{package_manager} autoremove -y && \\
{package_manager} clean all -y
""").format(**varmap))
or even merge the two RUN statements to reduce the number of layers
that will end up in the resulting image.
Everything else looks good, so with the above changed
Reviewed-by: Andrea Bolognani <abologna(a)redhat.com>
--
Andrea Bolognani / Red Hat / Virtualization
12:53 p.m.
New subject: [libvirt] [jenkins-ci PATCH v2 2/3] guests: define mapping for the libprlsdk package
Add a mapping exclusively for CentOS 7 to pull in the libprlsdk package,
since other distros don't have it available at this time.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
guests/vars/mappings.yml | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml
index f5dab6a..ce1294c 100644
--- a/guests/vars/mappings.yml
+++ b/guests/vars/mappings.yml
@@ -366,6 +366,10 @@ mappings:
rpm: libpciaccess-devel
cross-policy-deb: foreign
+ libprlsdk:
+ default:
+ CentOS7: libprlsdk-devel
+
librbd:
deb: librbd-dev
Fedora: librbd-devel
--
2.23.0
9:12 a.m.
New subject: [libvirt] [jenkins-ci PATCH v2 2/3] guests: define mapping for the libprlsdk package
On Fri, 2019-12-06 at 18:53 +0000, Daniel P. Berrangé wrote:
Add a mapping exclusively for CentOS 7 to pull in the libprlsdk
package,
since other distros don't have it available at this time.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
guests/vars/mappings.yml | 4 ++++
1 file changed, 4 insertions(+)
Reviewed-by: Andrea Bolognani <abologna(a)redhat.com>
--
Andrea Bolognani / Red Hat / Virtualization
12:53 p.m.
New subject: [libvirt] [jenkins-ci PATCH v2 3/3] guests: add libprlsdk package to libvirt project
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
guests/vars/projects/libvirt.yml | 1 +
1 file changed, 1 insertion(+)
diff --git a/guests/vars/projects/libvirt.yml b/guests/vars/projects/libvirt.yml
index 780a5aa..1efa846 100644
--- a/guests/vars/projects/libvirt.yml
+++ b/guests/vars/projects/libvirt.yml
@@ -31,6 +31,7 @@ packages:
- libparted
- libpcap
- libpciaccess
+ - libprlsdk
- librbd
- libselinux
- libssh
--
2.23.0
9:13 a.m.
New subject: [libvirt] [jenkins-ci PATCH v2 3/3] guests: add libprlsdk package to libvirt project
On Fri, 2019-12-06 at 18:53 +0000, Daniel P. Berrangé wrote:
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
guests/vars/projects/libvirt.yml | 1 +
1 file changed, 1 insertion(+)
Reviewed-by: Andrea Bolognani <abologna(a)redhat.com>
--
Andrea Bolognani / Red Hat / Virtualization
1808
days inactive
1813
days old
7 comments
2 participants
participants (2)
-
Andrea Bolognani -
Daniel P. Berrangé