- Devel - libvirt List Archives

[libvirt] [PATCH] cpu_conf: Fix XPath for parsing TSC frequency
by Jiri Denemark 05 Jun '19

05 Jun '19

Due to this bug the following command would fail on any host where TSC frequency can be probed: $ virsh capabilities | virsh cpu-baseline /dev/stdin error: unsupported configuration: Invalid TSC frequency https://bugzilla.redhat.com/show_bug.cgi?id=1641702 Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- src/conf/cpu_conf.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/conf/cpu_conf.c b/src/conf/cpu_conf.c index dc46e7f57a..825df88246 100644 --- a/src/conf/cpu_conf.c +++ b/src/conf/cpu_conf.c @@ -417,8 +417,8 @@ virCPUDefParseXML(xmlXPathContextPtr ctxt, if (VIR_ALLOC(tsc) < 0) goto cleanup; - if (virXPathULongLong("./counter[@name='tsc']/@frequency", ctxt, - &tsc->frequency) < 0) { + if (virXPathULongLong("string(./counter[@name='tsc']/@frequency)", + ctxt, &tsc->frequency) < 0) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("Invalid TSC frequency")); goto cleanup; -- 2.21.0

2 1

[libvirt] [PATCH 0/3] virsh: undefine: Clarify things around snapshots
by Peter Krempa 05 Jun '19

05 Jun '19

Peter Krempa (3): virsh: undefine: Clarify help string for --snapshots-metadata virsh: undefine: Rename --delete-snapshots to --delete-storage-volume-snapshots virsh: undefine: Clarify that --delete-storage-volume-snapshots causes failures tools/virsh-domain.c | 6 +++++- tools/virsh.pod | 10 ++++++---- 2 files changed, 11 insertions(+), 5 deletions(-) -- 2.21.0

3 6

[libvirt] QMP; unsigned 64-bit ints; JSON standards compliance
by Daniel P. Berrangé 05 Jun '19

05 Jun '19

The QEMU QMP service is based on JSON which is nice because that is a widely supported "standard" data format..... ....except QEMU's implementation (and indeed most impls) are not strictly standards compliant. Specifically the problem is around representing 64-bit integers, whether signed or unsigned. The JSON standard declares that largest integer is 2^53-1 and the likewise the smallest is -(2^53-1): http://www.ecma-international.org/ecma-262/6.0/index.html#sec-number.max_sa… A crazy limit inherited from its javascript origins IIUC. QEMU, and indeed many applications, want to handle 64-bit integers. The C JSON library impls have traditionally mapped integers to the data type 'long long int' which gives a min/max of -(2^63) / 2^63-1. QEMU however /really/ needs 64-bit unsigned integers, ie a max 2^64-1. Libvirt has historically used the YAJL library which uses 'long long int' and thus can't officially go beyond 2^63-1 values. Fortunately it lets libvirt get at the raw json string, so libvirt can re-parse the value to get an 'unsigned long long'. We recently tried to switch to Jansson because YAJL has a dead upstream for many years and countless unanswered bugs & patches. Unfortunately we forgot about this need for 2^64-1 max, and Jansson also uses 'long long int' and raises a fatal parse error for unsigned 64-bit values above 2^63-1. It also provides no backdoor for libvirt todo its own integer parsing. Thus we had to abort our switch to jansson as it broke parsing QEMU's JSON: https://bugzilla.redhat.com/show_bug.cgi?id=1614569 Other JSON libraries we've investigated have similar problems. I imagine the same may well be true of non-C based JOSN impls, though I've not investigated in any detail. Essentially libvirt is stuck with either using the dead YAJL library forever, or writing its own JSON parser (most likely copying QEMU's JSON code into libvirt's git). This feels like a very unappealing situation to be in as not being able to use a JSON library of our choice is loosing one of the key benefits of using a standard data format. Thus I'd like to see a solution to this to allow QMP to be reliably consumed by any JSON library that exists. I can think of some options: 1. Encode unsigned 64-bit integers as signed 64-bit integers. This follows the example that most C libraries map JSON ints to 'long long int'. This is still relying on undefined behaviour as apps don't need to support > 2^53-1. Apps would need to cast back to 'unsigned long long' for those QMP fields they know are supposed to be unsigned. 2. Encode all 64-bit integers as a pair of 32-bit integers. This is fully compliant with the JSON spec as each half is fully within the declared limits. App has to split or assemble the 2 pieces from/to a signed/unsigned 64-bit int as needed. 3. Encode all 64-bit integers as strings The application has todo all parsing/formatting client side. None of these changes are backwards compatible, so I doubt we could make the change transparently in QMP. Instead we would have to have a QMP greeting message capability where the client can request enablement of the enhanced integer handling. Any of the three options above would likely work for libvirt, but I would have a slight preference for either 2 or 3, so that we become 100% standards compliant. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

5 22

[libvirt] [PATCH] tests: Fix parentheses order in an assignment-comparison conditional
by Erik Skultety 04 Jun '19

04 Jun '19

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1717090 Signed-off-by: Erik Skultety <eskultet(a)redhat.com> --- Pushed as trivial. tests/testutilsqemu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/testutilsqemu.c b/tests/testutilsqemu.c index bce847ce5e..9ac9f9bd39 100644 --- a/tests/testutilsqemu.c +++ b/tests/testutilsqemu.c @@ -918,7 +918,7 @@ testQemuCapsIterate(const char *suffix, if (virDirOpen(&dir, TEST_QEMU_CAPS_PATH) < 0) goto cleanup; - while ((rc = virDirRead(dir, &ent, TEST_QEMU_CAPS_PATH) > 0)) { + while ((rc = virDirRead(dir, &ent, TEST_QEMU_CAPS_PATH)) > 0) { char *tmp = ent->d_name; char *base = NULL; char *archName = NULL; -- 2.20.1

1 0

[libvirt] [PATCH 0/9] Grab modify job for changing domain XML
by Michal Privoznik 04 Jun '19

04 Jun '19

This is an alternative proposal to: https://www.redhat.com/archives/libvir-list/2019-May/msg00830.html The problem I'm trying to fix is described here: https://www.redhat.com/archives/libvir-list/2019-May/msg00810.html Michal Prívozník (9): virDomainObjListAddLocked: Drop useless @cleanup label virDomainObjListAddObjLocked: Don't expect vm->def to be set virDomainObjListAddLocked: Set vm->def only in success path virDomainObjIsActive: Allow vm->def to be NULL virDomainObjListAdd: Leave def assigning as an exercise for caller qemu: Allow vm->def == NULL in job control APIs qemu: Grab modify job for changing domain XML lxc: Grab modify job for changing domain XML libxl: Grab modify job for changing domain XML src/bhyve/bhyve_driver.c | 10 +++++--- src/conf/domain_conf.h | 2 +- src/conf/virdomainobjlist.c | 48 ++++++++++++++---------------------- src/conf/virdomainobjlist.h | 3 +-- src/libxl/libxl_domain.c | 3 ++- src/libxl/libxl_driver.c | 48 ++++++++++++++++++++++++------------ src/libxl/libxl_migration.c | 14 +++++------ src/lxc/lxc_domain.c | 3 ++- src/lxc/lxc_driver.c | 23 +++++++++++------ src/openvz/openvz_conf.c | 12 ++++----- src/openvz/openvz_driver.c | 17 +++++++------ src/qemu/qemu_domain.c | 30 ++++++++++++++--------- src/qemu/qemu_driver.c | 49 ++++++++++++++++++++++++++----------- src/qemu/qemu_migration.c | 13 +++++++--- src/test/test_driver.c | 21 +++++++++------- src/vmware/vmware_conf.c | 4 +-- src/vmware/vmware_driver.c | 9 +++---- src/vz/vz_sdk.c | 4 ++- 18 files changed, 183 insertions(+), 130 deletions(-) -- 2.21.0

2 20

[libvirt] [PATCH 0/4] qemu: Fix qemuProcessInitCpuAffinity()
by Andrea Bolognani 04 Jun '19

04 Jun '19

See detailed explanation in the commit message for patch 1/4 and the corresponding bug report. Andrea Bolognani (4): util: Introduce virBitmapUnion() fixup? util: Optimize virBitmapUnion() util: Introduce virNumaNodesetToCPUset() qemu: Fix qemuProcessInitCpuAffinity() src/libvirt_private.syms | 2 ++ src/qemu/qemu_process.c | 7 ++++- src/util/virbitmap.c | 32 +++++++++++++++++++++++ src/util/virbitmap.h | 4 +++ src/util/virnuma.c | 55 ++++++++++++++++++++++++++++++++++++++++ src/util/virnuma.h | 2 ++ tests/virbitmaptest.c | 37 +++++++++++++++++++++++++++ 7 files changed, 138 insertions(+), 1 deletion(-) -- 2.21.0

3 16

[libvirt] [PATCH] util: Propagate numad failures correctly
by Andrea Bolognani 04 Jun '19

04 Jun '19

Right now, if numad fails, we raise an error but return an empty string to the caller instead of a NULL pointer, which means processing will continue and the user will see # virsh start guest error: Failed to start domain guest error: invalid argument: Failed to parse bitmap '' instead of a more reasonable # virsh start guest error: Failed to start domain guest error: operation failed: Failed to query numad for the advisory nodeset Make sure the user gets a better error message. https://bugzilla.redhat.com/show_bug.cgi?id=1716387 Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- src/util/virnuma.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/util/virnuma.c b/src/util/virnuma.c index dd3fb7519e..fee0dfcd5a 100644 --- a/src/util/virnuma.c +++ b/src/util/virnuma.c @@ -66,10 +66,12 @@ virNumaGetAutoPlacementAdvice(unsigned short vcpus, virCommandSetOutputBuffer(cmd, &output); - if (virCommandRun(cmd, NULL) < 0) - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + if (virCommandRun(cmd, NULL) < 0) { + virReportError(VIR_ERR_OPERATION_FAILED, "%s", _("Failed to query numad for the " "advisory nodeset")); + VIR_FREE(output); + } return output; } -- 2.21.0

3 2

[libvirt] [PATCH] news: Drop empty section
by Andrea Bolognani 03 Jun '19

03 Jun '19

Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- Pushed as trivial. docs/news.xml | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/news.xml b/docs/news.xml index 86f5ffb3f6..f95dfdf3b4 100644 --- a/docs/news.xml +++ b/docs/news.xml @@ -56,8 +56,6 @@ </section> </release> <release version="v5.4.0" date="2019-06-03"> - <section title="New features"> - </section> <section title="Security"> <change> <summary> -- 2.21.0

1 0

[libvirt] [PATCH v2 0/7] Check TSC frequency before starting QEMU
by Jiri Denemark 03 Jun '19

03 Jun '19

Version 2: - new patches (1 and 2) fixing qemuargv2xmltest, which was probing host CPU When migrating a domain with invtsc CPU feature enabled, the TSC frequency of the destination host must match the frequency used when the domain was started on the source host or the destination host has to support TSC scaling. If the frequencies do not match and the destination host does not support TSC scaling, QEMU will fail to set the right TSC frequency when starting vCPUs on the destination and thus migration will fail. However, this is quite late since both host might have spent significant time transferring memory and perhaps even storage data. By adding the check to libvirt we can let migration fail before any data starts to be sent over. If for some reason libvirt is unable to detect the host's TSC frequency or scaling support, we'll just let QEMU try and the migration will either succeed or fail later. Luckily, we mandate TSC frequency to be explicitly set in the domain XML to even allow migration of domains with invtsc. We can just check whether the requested frequency is compatible with the current host before starting QEMU. And to let libvirt client decide whether it should even start the migration to a specific host, this series adds host's TSC frequency and scaling support to the host CPU capabilities XML. https://bugzilla.redhat.com/show_bug.cgi?id=1641702 Jiri Denemark (7): qemu: Make virQEMUCapsProbeHostCPUForEmulator more generic qemuargv2xmltest: Use mocked virQEMUCapsProbeHostCPU util: Add virHostCPUGetTscInfo conf: Report TSC frequency in host CPU capabilities cpu_x86: Fix placement of *CheckFeature functions cpu_x86: Probe TSC frequency and scaling support qemu: Check TSC frequency before starting QEMU src/conf/cpu_conf.c | 48 +++++++++++++++++++++ src/conf/cpu_conf.h | 2 + src/cpu/cpu_x86.c | 81 ++++++++++++++++++++---------------- src/qemu/qemu_capabilities.c | 14 +++---- src/qemu/qemu_capspriv.h | 5 +-- src/qemu/qemu_process.c | 53 +++++++++++++++++++++++ src/util/virhostcpu.c | 71 +++++++++++++++++++++++++++++++ src/util/virhostcpu.h | 11 +++++ tests/Makefile.am | 3 +- tests/qemuargv2xmltest.c | 3 +- tests/qemucpumock.c | 5 +-- 11 files changed, 243 insertions(+), 53 deletions(-) -- 2.21.0

2 17

[libvirt] Release of libvirt-5.4.0
by Daniel Veillard 03 Jun '19

03 Jun '19

It's out ! The release is tagged in git, and I provided signed tarball and source rpms to the usual place: https://libvirt.org/sources/ I also cut off a 5.4.0 release of the python bindings but code is same a 5.3.0 one, you can find signed tarball and source rpms at: https://libvirt.org/sources/python/ Main theme of this release is security, there is a set of advisory covered so users are invited to update, along with some improvements and bug fixes. Security: - cpu: Introduce support for the md-clear CPUID bit This bit is set when microcode provides the mechanism to invoke a flush of various exploitable CPU buffers by invoking the x86 VERW instruction. CVE-2018-12126, CVE-2018-12127, CVE-2018-12130, CVE-2019-11091. - Restrict user access to virt-admin, virtlogd and virtlockd The intended users for these facilities are the root user and the libvirtd service respectively, but these restrictions were not enforced correctly. CVE-2019-10132. Improvements: - test driver: Expand API coverage Several APIs that were missing from the test driver have now been implemented. - Avoid unnecessary static linking Most binaries shipped as part of libvirt, for example virtlogd and libvirt_iohelper, were embedding parts of the library even though they also linked against the libvirt.so dynamic library. This is no longer the case, which results in both the disk and memory footprint being reduced. - qemu: Report stat-htlb-pgalloc and stat-htlb-pgfail balloon stats These stats have been introduced in QEMU 3.0. Bug fixes: - qemu: Fix emulator scheduler support Setting the scheduler for QEMU's main thread before QEMU had a chance to start up other threads was misleading as it would affect other threads (vCPU and I/O) as well. In some particular situations this could also lead to an error when the thread for vCPU #0 was being moved to its cpu,cpuacct cgroup. This was fixed so that the scheduler for the main thread is set after QEMU starts. - apparmor: Allow hotplug of vhost-scsi devices Thanks everybody who contributed to this release, be it with bug reports, patches, reviews, docs ... Enjoy ! Daniel -- Daniel Veillard | Red Hat Developers Tools http://developer.redhat.com/ veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

1 0