October 2019 - Devel - libvirt List Archives

[libvirt] [PATCH v4 00/12] remove repetition of URI path validation
by Daniel Henrique Barboza 03 Oct '19

03 Oct '19

This is a code repetition that I crossed a few times, then I noticed that Cole Robinson suggested a solution for it in the wiki. Here it is. changes from v3: - patch 8: fix the exception logic, move the code formatting to patch 1 - patch 9: use lowcase 'qemu' - patch 12: (optional) test case I created to aid in patch 8 logic changes from v2: - use a boolean to determine 'QEMU' and 'vbox' case to avoid block repetition (patch 8) - avoid 80+ chars lines in all patches changes from v1: - handle QEMU and vbox cases separately inside the validation function v3: https://www.redhat.com/archives/libvir-list/2019-September/msg01122.html v2: https://www.redhat.com/archives/libvir-list/2019-September/msg01007.html v1: https://www.redhat.com/archives/libvir-list/2019-September/msg00983.html Daniel Henrique Barboza (12): src/driver.c: add virConnectValidateURIPath() interface_backend_netcf.c: use virConnectValidateURIPath() interface_backend_udev.c: use virConnectValidateURIPath() bridge_driver.c: virConnectValidateURIPath() node_device_driver.c: use virConnectValidateURIPath() secret_driver.c: use virConnectValidateURIPath() storage_driver.c: use virConnectValidateURIPath() driver.c: change URI validation to handle QEMU and vbox case qemu_driver.c: use virConnectValidateURIPath() vbox_common.c: use virConnectValidateURIPath() vbox_driver.c: use virConnectValidateURIPath() tests: add a test for driver.c:virConnectValidateURIPath() src/driver.c | 38 +++++ src/driver.h | 4 + src/interface/interface_backend_netcf.c | 19 +-- src/interface/interface_backend_udev.c | 19 +-- src/libvirt_private.syms | 1 + src/network/bridge_driver.c | 19 +-- src/node_device/node_device_driver.c | 19 +-- src/qemu/qemu_driver.c | 20 +-- src/secret/secret_driver.c | 19 +-- src/storage/storage_driver.c | 19 +-- src/vbox/vbox_common.c | 16 +- src/vbox/vbox_driver.c | 16 +- tests/Makefile.am | 7 +- tests/virdriverconnvalidatetest.c | 186 ++++++++++++++++++++++++ 14 files changed, 267 insertions(+), 135 deletions(-) create mode 100644 tests/virdriverconnvalidatetest.c -- 2.21.0

2 16

[libvirt] [PATCH 1/1] src/driver.c: remove duplicated code in virGetConnect* functions
by Daniel Henrique Barboza 03 Oct '19

03 Oct '19

All the 6 virGetConnect* functions in driver.c shares the same code base. This patch creates a new static function virGetConnectGeneric() that contains the common code to be used with all other virGetConnect*. Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com> --- CC'ing Cole Robinson since he reviewed similar patches a few days ago. src/driver.c | 100 +++++++++++++-------------------------------------- 1 file changed, 25 insertions(+), 75 deletions(-) diff --git a/src/driver.c b/src/driver.c index ed2d943ddf..8a4bc8ff66 100644 --- a/src/driver.c +++ b/src/driver.c @@ -29,6 +29,7 @@ #include "virfile.h" #include "virlog.h" #include "virmodule.h" +#include "virstring.h" #include "virthread.h" #include "configmake.h" @@ -96,112 +97,61 @@ virConnectCacheOnceInit(void) VIR_ONCE_GLOBAL_INIT(virConnectCache); -virConnectPtr virGetConnectInterface(void) +static virConnectPtr +virGetConnectGeneric(virThreadLocal thread, const char *name) { virConnectPtr conn; if (virConnectCacheInitialize() < 0) return NULL; - conn = virThreadLocalGet(&connectInterface); + conn = virThreadLocalGet(&thread); + if (conn) { - VIR_DEBUG("Return cached interface connection %p", conn); + VIR_DEBUG("Return cached %s connection %p", name, conn); virObjectRef(conn); } else { - conn = virConnectOpen(geteuid() == 0 ? "interface:///system" : "interface:///session"); - VIR_DEBUG("Opened new interface connection %p", conn); + VIR_AUTOFREE(char *) uri = NULL; + const char *uriPath = geteuid() == 0 ? "/system" : "/session"; + + if (virAsprintf(&uri, "%s//%s", name, uriPath) < 0) + return NULL; + + conn = virConnectOpen(uri); + VIR_DEBUG("Opened new %s connection %p", name, conn); } return conn; } -virConnectPtr virGetConnectNetwork(void) -{ - virConnectPtr conn; - if (virConnectCacheInitialize() < 0) - return NULL; +virConnectPtr virGetConnectInterface(void) +{ + return virGetConnectGeneric(connectInterface, "interface"); +} - conn = virThreadLocalGet(&connectNetwork); - if (conn) { - VIR_DEBUG("Return cached network connection %p", conn); - virObjectRef(conn); - } else { - conn = virConnectOpen(geteuid() == 0 ? "network:///system" : "network:///session"); - VIR_DEBUG("Opened new network connection %p", conn); - } - return conn; +virConnectPtr virGetConnectNetwork(void) +{ + return virGetConnectGeneric(connectNetwork, "network"); } virConnectPtr virGetConnectNWFilter(void) { - virConnectPtr conn; - - if (virConnectCacheInitialize() < 0) - return NULL; - - conn = virThreadLocalGet(&connectNWFilter); - if (conn) { - VIR_DEBUG("Return cached nwfilter connection %p", conn); - virObjectRef(conn); - } else { - conn = virConnectOpen(geteuid() == 0 ? "nwfilter:///system" : "nwfilter:///session"); - VIR_DEBUG("Opened new nwfilter connection %p", conn); - } - return conn; + return virGetConnectGeneric(connectNWFilter, "nwfilter"); } virConnectPtr virGetConnectNodeDev(void) { - virConnectPtr conn; - - if (virConnectCacheInitialize() < 0) - return NULL; - - conn = virThreadLocalGet(&connectNodeDev); - if (conn) { - VIR_DEBUG("Return cached nodedev connection %p", conn); - virObjectRef(conn); - } else { - conn = virConnectOpen(geteuid() == 0 ? "nodedev:///system" : "nodedev:///session"); - VIR_DEBUG("Opened new nodedev connection %p", conn); - } - return conn; + return virGetConnectGeneric(connectNodeDev, "nodedev"); } virConnectPtr virGetConnectSecret(void) { - virConnectPtr conn; - - if (virConnectCacheInitialize() < 0) - return NULL; - - conn = virThreadLocalGet(&connectSecret); - if (conn) { - VIR_DEBUG("Return cached secret connection %p", conn); - virObjectRef(conn); - } else { - conn = virConnectOpen(geteuid() == 0 ? "secret:///system" : "secret:///session"); - VIR_DEBUG("Opened new secret connection %p", conn); - } - return conn; + return virGetConnectGeneric(connectSecret, "secret"); } virConnectPtr virGetConnectStorage(void) { - virConnectPtr conn; - - if (virConnectCacheInitialize() < 0) - return NULL; - - conn = virThreadLocalGet(&connectStorage); - if (conn) { - VIR_DEBUG("Return cached storage connection %p", conn); - virObjectRef(conn); - } else { - conn = virConnectOpen(geteuid() == 0 ? "storage:///system" : "storage:///session"); - VIR_DEBUG("Opened new storage connection %p", conn); - } - return conn; + return virGetConnectGeneric(connectStorage, "storage"); } -- 2.21.0

2 1

[libvirt] [jenkins-ci PATCH] Fix target list for osinfo-db builds
by Andrea Bolognani 03 Oct '19

03 Oct '19

Commit 570143fc41fa removed all unsuitable Debian-based distributions, but failed to account for the fact that CentOS 7 doesn't have a Meson version suitable to build osinfo-db-tools and as such can't possibly build osinfo-db. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- Pushed as a CI fix. guests/playbooks/build/projects/osinfo-db.yml | 25 ++++++++++--------- jenkins/projects/osinfo-db.yaml | 21 ++++++++-------- 2 files changed, 24 insertions(+), 22 deletions(-) diff --git a/guests/playbooks/build/projects/osinfo-db.yml b/guests/playbooks/build/projects/osinfo-db.yml index 872b448..1952827 100644 --- a/guests/playbooks/build/projects/osinfo-db.yml +++ b/guests/playbooks/build/projects/osinfo-db.yml @@ -1,7 +1,15 @@ --- - set_fact: name: osinfo-db - machines: '{{ all_machines }}' + machines: + - libvirt-debian-10 + - libvirt-debian-sid + - libvirt-fedora-29 + - libvirt-fedora-30 + - libvirt-fedora-rawhide + - libvirt-freebsd-11 + - libvirt-freebsd-12 + - libvirt-freebsd-current archive_format: xz git_url: '{{ git_urls["osinfo-db"][git_remote] }}' @@ -13,21 +21,14 @@ $MAKE install OSINFO_DB_TARGET="--system" - include: '{{ playbook_base }}/jobs/generic-check-job.yml' vars: - # osinfo-db tests are Python 3 only, so they can't be ran on CentOS 7; - machines: - - libvirt-debian-10 - - libvirt-debian-sid - - libvirt-fedora-29 - - libvirt-fedora-30 - - libvirt-fedora-rawhide - - libvirt-freebsd-11 - - libvirt-freebsd-12 - - libvirt-freebsd-current command: | $MAKE check - include: '{{ playbook_base }}/jobs/generic-rpm-job.yml' vars: - machines: '{{ rpm_machines }}' + machines: + - libvirt-fedora-29 + - libvirt-fedora-30 + - libvirt-fedora-rawhide command: | {{ strip_buildrequires }} rpmbuild --clean --define "_topdir `pwd`/rpmbuild" --define "_sourcedir `pwd`" -ba osinfo-db.spec diff --git a/jenkins/projects/osinfo-db.yaml b/jenkins/projects/osinfo-db.yaml index 37c1d84..7074fd1 100644 --- a/jenkins/projects/osinfo-db.yaml +++ b/jenkins/projects/osinfo-db.yaml @@ -1,7 +1,13 @@ --- - project: name: osinfo-db - machines: '{all_machines}' + machines: + - libvirt-debian-10 + - libvirt-fedora-29 + - libvirt-fedora-30 + - libvirt-fedora-rawhide + - libvirt-freebsd-11 + - libvirt-freebsd-12 title: osinfo database archive_format: xz git_url: '{git_urls[osinfo-db][default]}' @@ -13,19 +19,14 @@ $MAKE install OSINFO_DB_TARGET="--system" - generic-check-job: parent_jobs: 'osinfo-db-build' - # osinfo-db tests are Python 3 only, so they can't be ran on CentOS 7; - machines: - - libvirt-debian-10 - - libvirt-fedora-29 - - libvirt-fedora-30 - - libvirt-fedora-rawhide - - libvirt-freebsd-11 - - libvirt-freebsd-12 command: | $MAKE check - generic-rpm-job: parent_jobs: 'osinfo-db-check' - machines: '{rpm_machines}' + machines: + - libvirt-fedora-29 + - libvirt-fedora-30 + - libvirt-fedora-rawhide command: | {strip_buildrequires} rpmbuild --clean --define "_topdir `pwd`/rpmbuild" --define "_sourcedir `pwd`" -ba osinfo-db.spec -- 2.21.0

1 0

[libvirt] [jenkins-ci PATCH 0/2] Fix project and target list
by Andrea Bolognani 03 Oct '19

03 Oct '19

Andrea Bolognani (2): guests: Fix project list Fix target list for builds guests/host_vars/libvirt-centos-7/main.yml | 1 - guests/host_vars/libvirt-debian-9/main.yml | 2 -- guests/host_vars/libvirt-ubuntu-16/main.yml | 1 - guests/host_vars/libvirt-ubuntu-18/main.yml | 2 -- guests/playbooks/build/projects/osinfo-db.yml | 3 --- guests/playbooks/build/projects/virt-manager.yml | 4 ---- jenkins/projects/osinfo-db.yaml | 1 - jenkins/projects/virt-manager.yaml | 2 -- 8 files changed, 16 deletions(-) -- 2.21.0

2 3

[libvirt] [PATCH v2 0/2] make attaching disk partition to VM illegal
by Pavel Mores 02 Oct '19

02 Oct '19

This version addresses comments and objections to the original submission. Pavel Mores (2): qemu: make attaching disk partition to VM illegal remove a now redundant call to virDiskNameToIndex() src/qemu/qemu_command.c | 6 ----- src/qemu/qemu_domain.c | 15 +++++++++++ .../disk-attaching-partition-nosupport.xml | 27 +++++++++++++++++++ tests/qemuxml2argvtest.c | 1 + 4 files changed, 43 insertions(+), 6 deletions(-) create mode 100644 tests/qemuxml2argvdata/disk-attaching-partition-nosupport.xml -- 2.21.0

3 8

[libvirt] [PATCH 1/1] admin_server: use VIR_AUTOFREE() in adminClientGetInfo string
by Daniel Henrique Barboza 02 Oct '19

02 Oct '19

Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com> --- Trivial change. Found it while reviewing a patch from Daniel yesterday. Didn't found any other instances in this file that can VIR_AUTO*. src/admin/admin_server.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/admin/admin_server.c b/src/admin/admin_server.c index 0d6091937d..dc67dd044f 100644 --- a/src/admin/admin_server.c +++ b/src/admin/admin_server.c @@ -218,7 +218,7 @@ adminClientGetInfo(virNetServerClientPtr client, int ret = -1; int maxparams = 0; bool readonly; - char *sock_addr = NULL; + VIR_AUTOFREE(char *) sock_addr = NULL; const char *attr = NULL; virTypedParameterPtr tmpparams = NULL; virIdentityPtr identity = NULL; @@ -313,7 +313,6 @@ adminClientGetInfo(virNetServerClientPtr client, if (tmpparams) virTypedParamsFree(tmpparams, *nparams); virObjectUnref(identity); - VIR_FREE(sock_addr); return ret; } -- 2.21.0

4 9

[libvirt] [PATCH v2 0/6] rewrite virt-host-validate to be data driven, using Go & YAML
by Daniel P. Berrangé 02 Oct '19

02 Oct '19

This is a followup to a previous PoC patch I submitted a month ago: https://www.redhat.com/archives/libvir-list/2019-September/msg00036.html The commit messages in the individual patches given quite a bit of detail, so I'll keep this cover letter brief. In my previous posting I was unhappy with the implications for the RPM packaging, and was considering having this as a separate source repo & RPM. On further investigation such an approach would not in fact solve the RPM packaging problem, because we would still not be using a pure go build toolchain, as we have data files that need installing in the right place. This forced me to actually address the RPM packaging problems that Fedora had with Go when used from a build tool like make or meson. After alot of debugging I finally got a viable solution merged into the Fedora go-rpm-macros package: https://pagure.io/go-rpm-macros/c/67b4fbbbfce0986ac46cd1329bf85a18ea7a43d2 commit 67b4fbbbfce0986ac46cd1329bf85a18ea7a43d2 Author: Daniel P. Berrangé <berrange(a)redhat.com> Date: Wed Sep 18 16:49:58 2019 +0100 macros: define a %gobuildflags macro Using the %gobuild macro is fine for a project where the go code is the only thing being built, and can be built directly by invoking the Go toolchain from RPM. In more complex cases though, the Go code is just a small part of the project and the Go toolchain is invoked by a build system such as make (possibly automake), or meson. In such a case we need to be able to tell this build system what flags to pass to the compiler. The %gobuildflags macros services this purpose allowing a RPM spec todo GOBUILDFLAGS="%gobuildflags" %configure or %make GOBUILDFLAGS="%gobuildflags" Ideally the %gobuild macro would in turn reference the %gobuildflags macro, but that does not appear possible given the semantics around quote expansion and escaping across RPM and shell. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> As a result in this series, we're now fully integrated into the RPM build, on Fedora at least. I've not checked what approach RHEL takes for Go, whether it requires separate RPM for each 3rd party dep, or prefers bundling. Either way though, we can deal with the problem now. The other obvious change is that this is now a patch series, to make it easier to review the code in managable chunks. The really big difference though is that I replaced the use of XML data files with YAML data files. This was done with the aim of making the data more human friendly. XML is really optimized for machines, not humans, so writing the data files was not pretty. YAML is optimized for human readability, and is actually even easier to consume in Go than the XML was, so its a double win. Finally, we also add new checks at the end for the various CPU hardware side channel mitigations, and report whether SMT/HT is unsafe or not (any Intel host is basically unsafe before Icelake). Daniel P. Berrangé (6): build: introduce logic for using golang in libvirt tools: introduce a data driven impl of virt-host-validate tools: define YAML rules for virt-host-validate checks tools: switch to build the new virt-host-validate impl tools: delete the old virt-host-validate impl tools: make virt-host-validate check CPU vulnerabilities configure.ac | 1 + libvirt.spec.in | 35 +- m4/virt-golang.m4 | 46 ++ m4/virt-host-validate.m4 | 8 +- po/POTFILES | 5 - tools/Makefile.am | 76 +-- tools/host-validate/go.mod | 10 + tools/host-validate/go.sum | 9 + tools/host-validate/main.go | 98 +++ tools/host-validate/pkg/engine.go | 481 ++++++++++++++ tools/host-validate/pkg/facts.go | 585 ++++++++++++++++++ .../pkg/facts_test.go} | 36 +- tools/host-validate/rules/builtin.yaml | 20 + tools/host-validate/rules/cpu.yaml | 50 ++ tools/host-validate/rules/freebsd-kernel.yaml | 77 +++ tools/host-validate/rules/linux-acpi.yaml | 39 ++ tools/host-validate/rules/linux-cgroups.yaml | 470 ++++++++++++++ .../rules/linux-cpu-hardware-flaws.yaml | 165 +++++ tools/host-validate/rules/linux-cpu.yaml | 134 ++++ tools/host-validate/rules/linux-devices.yaml | 71 +++ tools/host-validate/rules/linux-iommu.yaml | 113 ++++ .../host-validate/rules/linux-namespaces.yaml | 119 ++++ tools/host-validate/rules/linux-pci.yaml | 10 + tools/virt-host-validate-bhyve.c | 77 --- tools/virt-host-validate-common.c | 419 ------------- tools/virt-host-validate-common.h | 85 --- tools/virt-host-validate-lxc.c | 87 --- tools/virt-host-validate-lxc.h | 24 - tools/virt-host-validate-qemu.c | 116 ---- tools/virt-host-validate-qemu.h | 24 - tools/virt-host-validate.c | 152 ----- tools/virt-host-validate.pod | 12 +- 32 files changed, 2609 insertions(+), 1045 deletions(-) create mode 100644 m4/virt-golang.m4 create mode 100644 tools/host-validate/go.mod create mode 100644 tools/host-validate/go.sum create mode 100644 tools/host-validate/main.go create mode 100644 tools/host-validate/pkg/engine.go create mode 100644 tools/host-validate/pkg/facts.go rename tools/{virt-host-validate-bhyve.h => host-validate/pkg/facts_test.go} (52%) create mode 100644 tools/host-validate/rules/builtin.yaml create mode 100644 tools/host-validate/rules/cpu.yaml create mode 100644 tools/host-validate/rules/freebsd-kernel.yaml create mode 100644 tools/host-validate/rules/linux-acpi.yaml create mode 100644 tools/host-validate/rules/linux-cgroups.yaml create mode 100644 tools/host-validate/rules/linux-cpu-hardware-flaws.yaml create mode 100644 tools/host-validate/rules/linux-cpu.yaml create mode 100644 tools/host-validate/rules/linux-devices.yaml create mode 100644 tools/host-validate/rules/linux-iommu.yaml create mode 100644 tools/host-validate/rules/linux-namespaces.yaml create mode 100644 tools/host-validate/rules/linux-pci.yaml delete mode 100644 tools/virt-host-validate-bhyve.c delete mode 100644 tools/virt-host-validate-common.c delete mode 100644 tools/virt-host-validate-common.h delete mode 100644 tools/virt-host-validate-lxc.c delete mode 100644 tools/virt-host-validate-lxc.h delete mode 100644 tools/virt-host-validate-qemu.c delete mode 100644 tools/virt-host-validate-qemu.h delete mode 100644 tools/virt-host-validate.c -- 2.21.0

5 25

[libvirt] [PATCH] docs: document that C & Python are the preferred languages
by Daniel P. Berrangé 01 Oct '19

01 Oct '19

Blacklist Perl and Shell code in favour of Python for sake of readability and portability. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- docs/hacking.html.in | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/docs/hacking.html.in b/docs/hacking.html.in index 90376968c3..8572eb65a6 100644 --- a/docs/hacking.html.in +++ b/docs/hacking.html.in @@ -343,6 +343,36 @@ Richard Jones' guide to working with open source projects</a>. </p> + <h2><a id="lang">Language Usage</a></h2> + + <p> + The libvirt repository makes use of a large number of programming + languages. There is a general desire to phase out some of the + existing languages used to reduce the knowledge burden on + developers, and facilitate introduction of new languages in + the future. + </p> + + <p> + The preferred languages at this time are + </p> + + <ul> + <li>C - for the main libvirt codebase. Dialect supported by + GCC/CLang only.</li> + <li>Python - for supporting build scripts / tools. Code must + run with both version 2.7 and 3.x at this time.</li> + </ul> + + <p> + Languages that should not be used for any new contributions. + </p> + + <ul> + <li>Perl - build scripts must be written in Python instead.</li> + <li>Shell - build scripts must be written in Python instead.</li> + </ul> + <h2><a id="tooling">Tooling</a></h2> <p> -- 2.21.0

5 11

[libvirt] Libvirt release ...
by Daniel Veillard 01 Oct '19

01 Oct '19

Seems I totally forgot to work on the release in the last week. I suggest to enter freeze in a few hours, then have RC2 on Thursday and the release this Saturday ! Daniel -- Daniel Veillard | Red Hat Developers Tools http://developer.redhat.com/ veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

1 0