April 2019 - Devel - libvirt List Archives

[libvirt] vcpupin reports bogus vcpu affinities
by Allen, John 15 Apr '19

15 Apr '19

For pinned vcpus, vcpupin will report inaccurate affinity values on machines with high core counts (256 cores in my case). The problem is produced as follows: $ virsh vcpupin myguest 0 4 $ virsh vcpupin myguest 0 VCPU CPU Affinity --------------------------- 0 4,192,194,196-197 Running taskset on the qemu threads shows the correct affinity, so this seems to be a reporting problem. Strangely, the value "192" is significant. If I pin a cpu greater than 192, the problem no longer appears. I believe the cause of the problem in my case is that in this case in src/conf/domain_conf.c:virDomainDefGetVcpuPinInfoHelper: ... if (vcpu && vcpu->cpumask) bitmap = vcpu->cpumask; ... vcpu->cpumask is "shortened" in that it is only long enough to contain the last set bit in the mask. However, when we go to copy the mask to the buffer that is returned, we use the masklen passed to the function which is the "full" masklen with a bit for each cpu. So it seems virBitmapToDataBuf copies some extra data past the end of the bitmask. Why the "192" value is always set and I typically see similar bogus bits set is still unknown. What is the function meant to assume in this case? Is it sane to assume that the bitmask is the full length of the buffer here and it's the responsibility of the setter of vcpu->cpumask to provide the length of the bitmap we're expecting? Or should we assume that we may receive a shortened bitmask here and expand the bitmask before copying to the buffer? -John

2 2

[libvirt] [PATCH] Don't hardcode list of git submodules
by Andrea Bolognani 15 Apr '19

15 Apr '19

Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- Makefile.ci | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile.ci b/Makefile.ci index 738da53cc8..864948dc96 100644 --- a/Makefile.ci +++ b/Makefile.ci @@ -44,7 +44,7 @@ CI_CONFIGURE_ARGS = # Avoid pulling submodules over the network by locally # cloning them -CI_SUBMODULES = .gnulib src/keycodemapdb +CI_SUBMODULES = $(shell git submodules | awk '{ print $2 }') # Location of the Docker images we're going to pull # Can be useful to overridde to use a locally built -- 2.20.1

3 4

[libvirt] [PATCH 0/5] Add support for setting the emulator scheduler parameters
by Martin Kletzander 15 Apr '19

15 Apr '19

I/O threads and vCPU threads already support setting schedulers, but until now it was impossible to do so for the main QEMU thread (emulator thread in the libvirt naming). This is, however, requested for some very specific scenarios, for example when vCPU threads are running at such priority that could starve the main thread. Martin Kletzander (5): conf: Parse common scheduler attributes in separate function conf: Format thread IDs optionally docs: Mention iothreadsched element in the docs and reword conf: Add support for emulatorsched qemu: Add support for emulatorsched docs/formatdomain.html.in | 25 +++-- docs/news.xml | 12 +++ docs/schemas/domaincommon.rng | 5 + src/conf/domain_conf.c | 136 +++++++++++++++++------- src/conf/domain_conf.h | 1 + src/qemu/qemu_process.c | 2 +- tests/genericxml2xmlindata/cputune.xml | 37 +++++++ tests/genericxml2xmloutdata/cputune.xml | 40 +++++++ tests/genericxml2xmltest.c | 2 + 9 files changed, 212 insertions(+), 48 deletions(-) create mode 100644 tests/genericxml2xmlindata/cputune.xml create mode 100644 tests/genericxml2xmloutdata/cputune.xml -- 2.21.0

1 6

[libvirt] [keycodemapdb PATCH] keymap-gen: Name sections in pod output
by Andrea Bolognani 15 Apr '19

15 Apr '19

Properly formatted man pages have at least a NAME section, which tools like apropos(1) parse and index. The current implementation of PodLanguageGenerator doesn't generate this section, and even if the user were to try and work around the limitation by passing --varname=NAME, there would still be nothing in the section to actually parse. Change the generator so that varname will be used as the contents of the NAME section, add a DESCRIPTION section in which values are listed. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- Dan said on IRC he would accept keycodemapdb patches sent to the mailing list of any project using the module, so here we are :) I need this to fix generation of virkey{code,name}-* man pages anyway. tools/keymap-gen | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/tools/keymap-gen b/tools/keymap-gen index 7965d37..78676a9 100755 --- a/tools/keymap-gen +++ b/tools/keymap-gen @@ -873,7 +873,11 @@ class PodLanguageGenerator(LanguageDocGenerator): print("#") def _array_start_name_doc(self, varname, namemap): - print("=head1 %s" % varname) + print("=head1 NAME") + print("") + print("%s" % varname) + print("") + print("=head1 DESCRIPTION") print("") print("List of %s key code names, with corresponding key code values" % namemap) print("") @@ -881,7 +885,11 @@ class PodLanguageGenerator(LanguageDocGenerator): print("") def _array_start_code_doc(self, varname, codemap, namemap): - print("=head1 %s" % varname) + print("=head1 NAME") + print("") + print("%s" % varname) + print("") + print("=head1 DESCRIPTION") print("") print("List of %s key code values, with corresponding %s key code names" % (codemap, namemap)) print("") -- 2.20.1

2 1

[libvirt] [PATCH] qemu: Don't cache microcode version
by Jiri Denemark 15 Apr '19

15 Apr '19

My earlier commit be46f61326 was incomplete. It removed caching of microcode version in the CPU driver, which means the capabilities XML will see the correct microcode version. But it is also cached in the QEMU capabilities cache where it is used to detect whether we need to reprobe QEMU. By missing the second place, the original commit be46f61326 made the situation even worse since libvirt would report correct microcode version while still using the old host CPU model (visible in domain capabilities XML). Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- src/qemu/qemu_capabilities.c | 12 ++++++++---- src/qemu/qemu_capabilities.h | 3 +-- src/qemu/qemu_driver.c | 9 +-------- tests/testutilsqemu.c | 2 +- 4 files changed, 11 insertions(+), 15 deletions(-) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index da90cd666e..a74dc84056 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -4542,7 +4542,7 @@ virQEMUCapsNewData(const char *binary, priv->libDir, priv->runUid, priv->runGid, - priv->microcodeVersion, + virHostCPUGetMicrocodeVersion(), priv->kernelVersion); } @@ -4625,8 +4625,7 @@ virFileCachePtr virQEMUCapsCacheNew(const char *libDir, const char *cacheDir, uid_t runUid, - gid_t runGid, - unsigned int microcodeVersion) + gid_t runGid) { char *capsCacheDir = NULL; virFileCachePtr cache = NULL; @@ -4650,7 +4649,6 @@ virQEMUCapsCacheNew(const char *libDir, priv->runUid = runUid; priv->runGid = runGid; - priv->microcodeVersion = microcodeVersion; priv->kvmUsable = VIR_TRISTATE_BOOL_ABSENT; if (uname(&uts) == 0 && @@ -4672,8 +4670,11 @@ virQEMUCapsPtr virQEMUCapsCacheLookup(virFileCachePtr cache, const char *binary) { + virQEMUCapsCachePrivPtr priv = virFileCacheGetPriv(cache); virQEMUCapsPtr ret = NULL; + priv->microcodeVersion = virHostCPUGetMicrocodeVersion(); + ret = virFileCacheLookup(cache, binary); VIR_DEBUG("Returning caps %p for %s", ret, binary); @@ -4727,6 +4728,7 @@ virQEMUCapsPtr virQEMUCapsCacheLookupByArch(virFileCachePtr cache, virArch arch) { + virQEMUCapsCachePrivPtr priv = virFileCacheGetPriv(cache); virQEMUCapsPtr ret = NULL; const char *binaryFilters[] = { "qemu-system-", @@ -4739,6 +4741,8 @@ virQEMUCapsCacheLookupByArch(virFileCachePtr cache, size_t i; size_t j; + priv->microcodeVersion = virHostCPUGetMicrocodeVersion(); + for (i = 0; i < ARRAY_CARDINALITY(binaryFilters); i++) { for (j = 0; j < ARRAY_CARDINALITY(archs); j++) { struct virQEMUCapsSearchData data = { diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h index fd7bec7589..8f5656a89b 100644 --- a/src/qemu/qemu_capabilities.h +++ b/src/qemu/qemu_capabilities.h @@ -590,8 +590,7 @@ void virQEMUCapsFilterByMachineType(virQEMUCapsPtr qemuCaps, virFileCachePtr virQEMUCapsCacheNew(const char *libDir, const char *cacheDir, uid_t uid, - gid_t gid, - unsigned int microcodeVersion); + gid_t gid); virQEMUCapsPtr virQEMUCapsCacheLookup(virFileCachePtr cache, const char *binary); virQEMUCapsPtr virQEMUCapsCacheLookupCopy(virFileCachePtr cache, diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 7e334d0d8d..4510b0ce60 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -584,8 +584,6 @@ qemuStateInitialize(bool privileged, char *hugepagePath = NULL; char *memoryBackingPath = NULL; size_t i; - virCPUDefPtr hostCPU = NULL; - unsigned int microcodeVersion = 0; if (VIR_ALLOC(qemu_driver) < 0) return -1; @@ -808,15 +806,10 @@ qemuStateInitialize(bool privileged, run_gid = cfg->group; } - if ((hostCPU = virCPUProbeHost(virArchFromHost()))) - microcodeVersion = hostCPU->microcodeVersion; - virCPUDefFree(hostCPU); - qemu_driver->qemuCapsCache = virQEMUCapsCacheNew(cfg->libDir, cfg->cacheDir, run_uid, - run_gid, - microcodeVersion); + run_gid); if (!qemu_driver->qemuCapsCache) goto error; diff --git a/tests/testutilsqemu.c b/tests/testutilsqemu.c index 3ee8a0c6c6..763f511bcf 100644 --- a/tests/testutilsqemu.c +++ b/tests/testutilsqemu.c @@ -740,7 +740,7 @@ int qemuTestDriverInit(virQEMUDriver *driver) /* Using /dev/null for libDir and cacheDir automatically produces errors * upon attempt to use any of them */ - driver->qemuCapsCache = virQEMUCapsCacheNew("/dev/null", "/dev/null", 0, 0, 0); + driver->qemuCapsCache = virQEMUCapsCacheNew("/dev/null", "/dev/null", 0, 0); if (!driver->qemuCapsCache) goto error; -- 2.21.0

2 1

[libvirt] Cleaning up libvirt.git tags
by Andrea Bolognani 15 Apr '19

15 Apr '19

We have four types of tags in libvirt.git right now: * vX.Y.Z: we have one of these per release, starting with v0.1.0 from April 2006 and going all the way to v5.2.0 from April 2019 (but see below). Most of them[1] are PGP-signed. * LIBVIR(T)_X_Y_Z: these point to the same commit as the ones above, but the last one we created is for v0.6.5 from July 2009; not a single one is PGP-signed. * CVE-YYYY-*: these point to the commit fixing the corresponding vulnerability. We seem to have stopped creating them in 2017, even though we definitely have had to deal with CVEs since ;) * LIBXEN_FIRST_COMMIT: one-of-a-kind tag pointing to the second ever (!) libvirt (!) commit. Of course the project was still called libxen back then, but this is the only surviving tag to reference the old name. Interestingly[2] enough, a few releases seem to have partially or completely slipped through the cracks: version commit tag tarball --------- -------------- ----- --------- v0.1.2 | 567b42ce6a07 | no | no v0.1.5 | 786e029cd743 | no | yes v0.4.0 | 6cb028991705 | no | yes v0.4.3 | 7db4c905d745 | no | yes v0.4.5 | 9d3d43436eac | no | yes Note that I stopped checking at v0.6.5, so there might actually be more. Anyway, since the LIBVIR(T)_X_Y_Z tags are a strict subset of the vX.Y.Z tags, and we have abandoned that naming scheme a decade ago, all they're doing right now is clutter the output of 'git tag' and I would suggest getting rid of them; same goes for that lonely, lonely LIBXEN_FIRST_COMMIT tag, which serves no real purpose other than reminding us that the project was named differently for, like, an entire month. As for the missing release commits, I see no harm in creating them retroactively for completeness' sake, but if nobody can be bothered doing that I'll also fully understand :) The CVE tags... I really don't have an opinion there. Thoughts? [1] Though not all of them: I'm pointing my finger at you, Cole! :P [2] Well, to me at least. -- Andrea Bolognani / Red Hat / Virtualization

3 4

[libvirt] [PATCH for-QEMU-4.1] Declare -realtime as deprecated
by Thomas Huth 15 Apr '19

15 Apr '19

The old -realtime mlock=on|off parameter does exactly the same as the new -overcommit mem-lock=on|off parameter. Additionally, "-realtime" does not activate any additional "realtime" capabilities as the name might indicate. We should avoid to confuse the users this way, so let's deprecate the old -realtime option. Signed-off-by: Thomas Huth <thuth(a)redhat.com> --- Deprecation has already been suggested last year: https://patchwork.kernel.org/patch/10480963/#22026215 ... but apparently we forgot to really do it. qemu-deprecated.texi | 5 +++++ vl.c | 2 ++ 2 files changed, 7 insertions(+) diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi index 2219386769..2c45204f3f 100644 --- a/qemu-deprecated.texi +++ b/qemu-deprecated.texi @@ -72,6 +72,11 @@ backend settings instead of environment variables. To ease migration to the new format, the ``-audiodev-help'' option can be used to convert the current values of the environment variables to ``-audiodev'' options. +@subsection -realtime (since 4.1) + +The @code{-realtime mlock=on|off} argument has been replaced by the +@code{-overcommit mem-lock=on|off} argument. + @section QEMU Machine Protocol (QMP) commands @subsection block-dirty-bitmap-add "autoload" parameter (since 2.12.0) diff --git a/vl.c b/vl.c index c696ad2a13..d6a2779b70 100644 --- a/vl.c +++ b/vl.c @@ -3912,6 +3912,8 @@ int main(int argc, char **argv, char **envp) } break; case QEMU_OPTION_realtime: + warn_report("'-realtime mlock=...' is deprecated, please use " + "'-overcommit mem-lock=...' instead"); opts = qemu_opts_parse_noisily(qemu_find_opts("realtime"), optarg, false); if (!opts) { -- 2.21.0

5 4

[libvirt] [PATCH] examples: domtop: fix build error
by Bjoern Walk 15 Apr '19

15 Apr '19

GCC version 8.2.1 has problems seeing the the correct flow of control, so manually initialize the temporary variable. ../../examples/domtop/domtop.c: In function 'print_cpu_usage': ../../examples/domtop/domtop.c:242:28: error: 'pos' may be used uninitialized in this function [-Werror=maybe-uninitialized] usage = (now_params[pos].value.ul - then_params[pos].value.ul) / cc1: all warnings being treated as errors Signed-off-by: Bjoern Walk <bwalk(a)linux.ibm.com> --- examples/domtop/domtop.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/domtop/domtop.c b/examples/domtop/domtop.c index c9b51aed..e463e287 100644 --- a/examples/domtop/domtop.c +++ b/examples/domtop/domtop.c @@ -206,7 +206,7 @@ print_cpu_usage(size_t cpu, } for (i = 0; i < ncpus; i++) { - size_t pos; + size_t pos = 0; double usage; /* check if the vCPU is in the maps */ -- 2.19.1

1 1

[libvirt] [PATCH] examples: Initialize @pos in domtop.c
by Martin Kletzander 15 Apr '19

15 Apr '19

This is a zero-cost workaround for a bug in GCC 8.3.0 which causes the compilation to fail, because the compiler thinks that the value might be used uninitialized even though it clearly cannot be. Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- This is technically a fix for a build-breaker, but I'm sending it for review as it is not that big of a deal it being in examples/ examples/domtop/domtop.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/domtop/domtop.c b/examples/domtop/domtop.c index c9b51aed510d..e463e2873f42 100644 --- a/examples/domtop/domtop.c +++ b/examples/domtop/domtop.c @@ -206,7 +206,7 @@ print_cpu_usage(size_t cpu, } for (i = 0; i < ncpus; i++) { - size_t pos; + size_t pos = 0; double usage; /* check if the vCPU is in the maps */ -- 2.21.0

2 2

[libvirt] [PATCH v2 0/2] network: only reload firewall after firewalld is finished restarting
by Laine Stump 15 Apr '19

15 Apr '19

Explanation in Patch 2/2 Changes in v2: * don't (erroneously) check the fields of the message if it's a "Reload" message; only do that if it's a NameOwnerChanged Message. * merge virDBusMessageRead() and virDBusMessageDecode(), since they are now identical (new Patch 1/2) Laine Stump (2): util: eliminate duplicate function virDBusMessageRead network: only reload firewall after firewalld is finished restarting src/libvirt_private.syms | 1 - src/network/bridge_driver.c | 30 +++++++++++++++++---- src/util/virdbus.c | 53 ++++++++++++------------------------- src/util/virdbus.h | 4 +-- src/util/virdbuspriv.h | 4 --- src/util/virfirewalld.c | 8 +++--- src/util/virpolkit.c | 12 ++++----- src/util/virsystemd.c | 6 ++--- tests/virpolkittest.c | 24 ++++++++--------- 9 files changed, 69 insertions(+), 73 deletions(-) -- 2.20.1

2 4