- Devel - libvirt List Archives

[libvirt] [PATCH] qemu: qemuDomainChangeNet: don't overwrite device info when pci addr is missing
by Katerina Koukiou 01 Aug '18

01 Aug '18

When trying to update an interface's rom settings with an device XML that is missing the PCI addr element, all new rom settings where not applied. https://bugzilla.redhat.com/show_bug.cgi?id=1599513 Signed-off-by: Katerina Koukiou <kkoukiou(a)redhat.com> --- Not sure why we chose to overwrite the whole info before though, I hope that this doesn't cause side problems. src/qemu/qemu_hotplug.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 1488f0a7c2..f45192b1d3 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -3445,17 +3445,14 @@ qemuDomainChangeNet(virQEMUDriverPtr driver, goto cleanup; } - /* info: if newdev->info is empty, fill it in from olddev, - * otherwise verify that it matches - nothing is allowed to - * change. (There is no helper function to do this, so - * individually check the few feidls of virDomainDeviceInfo that - * are relevant in this case). + /* info: if newdev->info.addr.pci is empty, fill it in from olddev, + * otherwise verify that it matches. */ if (!virDomainDeviceAddressIsValid(&newdev->info, - VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI) && - virDomainDeviceInfoCopy(&newdev->info, &olddev->info) < 0) { - goto cleanup; + VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI)) { + newdev->info.addr.pci = olddev->info.addr.pci; } + if (!virPCIDeviceAddressEqual(&olddev->info.addr.pci, &newdev->info.addr.pci)) { virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", -- 2.17.1

2 4

[libvirt] [PATCH RFC 0/2] ESX: Fixing SetAutoStart
by Marcos Paulo de Souza 01 Aug '18

01 Aug '18

Hi guys, while doing some random tests using the ESX driver, I faced a crash, and these two patches are trying to solve. I don't know if there is a better way to solve it, since ESX driver *generated* code to alloc and free some *generated* structs, but, please let me know if you have a better idea. Thanks, Marcos Paulo de Souza (2): esx: Do not crash SetAutoStart by double free esx: Fix SetAutoStart invalid pointer free src/esx/esx_driver.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) -- 2.17.1

2 3

[libvirt] [PATCH] util: avoid symbol clash between json libraries
by Daniel P. Berrangé 31 Jul '18

31 Jul '18

The jansson and json-glib libraries both export symbols with a json_ name prefix and json_object_iter_next() clashes between them. Unfortunately json_glib is linked in by GTK, so any app using GTK and libvirt will get a clash, resulting in SEGV. This also affects the NSS module provided by libvirt Instead of directly linking to jansson, use dlopen() with the RTLD_LOCAL flag which allows us to hide the symbols from the application that loads libvirt or the NSS module. Some preprocessor black magic and wrapper functions are used to redirect calls into the dlopen resolved symbols. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- libvirt.spec.in | 2 + src/Makefile.am | 3 + src/util/Makefile.inc.am | 3 +- src/util/virjson.c | 9 +- src/util/virjsoncompat.c | 253 +++++++++++++++++++++++++++++++++++++++ src/util/virjsoncompat.h | 86 +++++++++++++ 6 files changed, 354 insertions(+), 2 deletions(-) create mode 100644 src/util/virjsoncompat.c create mode 100644 src/util/virjsoncompat.h diff --git a/libvirt.spec.in b/libvirt.spec.in index 4113579e47..cfe7ab8a09 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -898,6 +898,8 @@ Requires: ncurses Requires: gettext # Needed by virt-pki-validate script. Requires: gnutls-utils +# We dlopen() it so need an explicit dep +Requires: libjansson.so.4()(64bit) %if %{with_bash_completion} Requires: %{name}-bash-completion = %{version}-%{release} %endif diff --git a/src/Makefile.am b/src/Makefile.am index 83263e69e5..59ae7a2e79 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -690,6 +690,7 @@ libvirt_setuid_rpc_client_la_SOURCES = \ util/virhashcode.c \ util/virhostcpu.c \ util/virjson.c \ + util/virjsoncompat.c \ util/virlog.c \ util/virobject.c \ util/virpidfile.c \ @@ -961,6 +962,8 @@ libvirt_nss_la_SOURCES = \ util/virhashcode.h \ util/virjson.c \ util/virjson.h \ + util/virjsoncompat.c \ + util/virjsoncompat.h \ util/virkmod.c \ util/virkmod.h \ util/virlease.c \ diff --git a/src/util/Makefile.inc.am b/src/util/Makefile.inc.am index 71b2b93c2d..8ef9ee1dfa 100644 --- a/src/util/Makefile.inc.am +++ b/src/util/Makefile.inc.am @@ -86,6 +86,8 @@ UTIL_SOURCES = \ util/viriscsi.h \ util/virjson.c \ util/virjson.h \ + util/virjsoncompat.c \ + util/virjsoncompat.h \ util/virkeycode.c \ util/virkeycode.h \ util/virkeyfile.c \ @@ -264,7 +266,6 @@ libvirt_util_la_CFLAGS = \ $(NULL) libvirt_util_la_LIBADD = \ $(CAPNG_LIBS) \ - $(JANSSON_LIBS) \ $(LIBNL_LIBS) \ $(THREAD_LIBS) \ $(AUDIT_LIBS) \ diff --git a/src/util/virjson.c b/src/util/virjson.c index 01a387b2f7..5bab662cd3 100644 --- a/src/util/virjson.c +++ b/src/util/virjson.c @@ -1437,7 +1437,8 @@ virJSONValueCopy(const virJSONValue *in) #if WITH_JANSSON -# include <jansson.h> + +# include "virjsoncompat.h" static virJSONValuePtr virJSONValueFromJansson(json_t *json) @@ -1524,6 +1525,9 @@ virJSONValueFromString(const char *jsonstring) size_t flags = JSON_REJECT_DUPLICATES | JSON_DECODE_ANY; + if (virJSONInitialize() < 0) + return NULL; + if (!(json = json_loads(jsonstring, flags, &error))) { virReportError(VIR_ERR_INTERNAL_ERROR, _("failed to parse JSON %d:%d: %s"), @@ -1630,6 +1634,9 @@ virJSONValueToString(virJSONValuePtr object, json_t *json; char *str = NULL; + if (virJSONInitialize() < 0) + return NULL; + if (pretty) flags |= JSON_INDENT(2); else diff --git a/src/util/virjsoncompat.c b/src/util/virjsoncompat.c new file mode 100644 index 0000000000..c317e50c32 --- /dev/null +++ b/src/util/virjsoncompat.c @@ -0,0 +1,253 @@ +/* + * virjsoncompat.c: JSON object parsing/formatting + * + * Copyright (C) 2018 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + */ + +#include <config.h> + +#include "virthread.h" +#include "virerror.h" +#define VIR_JSON_COMPAT_IMPL +#include "virjsoncompat.h" + +#define VIR_FROM_THIS VIR_FROM_NONE + +#if WITH_JANSSON + +#include <dlfcn.h> + +json_t *(*json_array_ptr)(void); +int (*json_array_append_new_ptr)(json_t *array, json_t *value); +json_t *(*json_array_get_ptr)(const json_t *array, size_t index); +size_t (*json_array_size_ptr)(const json_t *array); +void (*json_delete_ptr)(json_t *json); +char *(*json_dumps_ptr)(const json_t *json, size_t flags); +json_t *(*json_false_ptr)(void); +json_t *(*json_integer_ptr)(json_int_t value); +json_int_t (*json_integer_value_ptr)(const json_t *integer); +json_t *(*json_loads_ptr)(const char *input, size_t flags, json_error_t *error); +json_t *(*json_null_ptr)(void); +json_t *(*json_object_ptr)(void); +void *(*json_object_iter_ptr)(json_t *object); +const char *(*json_object_iter_key_ptr)(void *iter); +void *(*json_object_iter_next_ptr)(json_t *object, void *iter); +json_t *(*json_object_iter_value_ptr)(void *iter); +void *(*json_object_key_to_iter_ptr)(const char *key); +int (*json_object_set_new_ptr)(json_t *object, const char *key, json_t *value); +json_t *(*json_real_ptr)(double value); +double (*json_real_value_ptr)(const json_t *real); +json_t *(*json_string_ptr)(const char *value); +const char *(*json_string_value_ptr)(const json_t *string); +json_t *(*json_true_ptr)(void); + + +static int virJSONJanssonOnceInit(void) +{ + void *handle = dlopen("libjansson.so.4", RTLD_LAZY|RTLD_LOCAL|RTLD_DEEPBIND|RTLD_NODELETE); + if (!handle) { + virReportError(VIR_ERR_NO_SUPPORT, + _("libjansson.so.4 JSON library not available: %s"), dlerror()); + return -1; + } + +#define LOAD(name) \ + do { \ + if (!(name ## _ptr = dlsym(handle, #name))) { \ + virReportError(VIR_ERR_NO_SUPPORT, \ + _("missing symbol '%s' in libjansson.so.4: %s"), #name, dlerror()); \ + goto error; \ + } \ + fprintf(stderr, "Resolve %s to %p\n", #name, name ## _ptr); \ + } while (0) + + LOAD(json_array); + LOAD(json_array_append_new); + LOAD(json_array_get); + LOAD(json_array_size); + LOAD(json_delete); + LOAD(json_dumps); + LOAD(json_false); + LOAD(json_integer); + LOAD(json_integer_value); + LOAD(json_loads); + LOAD(json_null); + LOAD(json_object); + LOAD(json_object_iter); + LOAD(json_object_iter_key); + LOAD(json_object_iter_next); + LOAD(json_object_iter_value); + LOAD(json_object_key_to_iter); + LOAD(json_object_set_new); + LOAD(json_real); + LOAD(json_real_value); + LOAD(json_string); + LOAD(json_string_value); + LOAD(json_true); + + return 0; + + error: + return -1; +} + +VIR_ONCE_GLOBAL_INIT(virJSONJansson); + +int virJSONInitialize(void) { + return virJSONJanssonInitialize(); +} + +json_t *json_array_impl(void) +{ + return json_array_ptr(); +} + + +int json_array_append_new_impl(json_t *array, json_t *value) +{ + return json_array_append_new_ptr(array, value); +} + + +json_t *json_array_get_impl(const json_t *array, size_t index) +{ + return json_array_get_ptr(array, index); +} + + +size_t json_array_size_impl(const json_t *array) +{ + return json_array_size_ptr(array); +} + + +void json_delete_impl(json_t *json) +{ + return json_delete_ptr(json); +} + + +char *json_dumps_impl(const json_t *json, size_t flags) +{ + return json_dumps_ptr(json, flags); +} + + +json_t *json_false_impl(void) +{ + return json_false_ptr(); +} + + +json_t *json_integer_impl(json_int_t value) +{ + return json_integer_ptr(value); +} + + +json_int_t json_integer_value_impl(const json_t *integer) +{ + return json_integer_value_ptr(integer); +} + + +json_t *json_loads_impl(const char *input, size_t flags, json_error_t *error) +{ + return json_loads_ptr(input, flags, error); +} + + +json_t *json_null_impl(void) +{ + return json_null_ptr(); +} + + +json_t *json_object_impl(void) +{ + return json_object_ptr(); +} + + +void *json_object_iter_impl(json_t *object) +{ + return json_object_iter_ptr(object); +} + + +const char *json_object_iter_key_impl(void *iter) +{ + return json_object_iter_key_ptr(iter); +} + + +void *json_object_iter_next_impl(json_t *object, void *iter) +{ + return json_object_iter_next_ptr(object, iter); +} + + +json_t *json_object_iter_value_impl(void *iter) +{ + return json_object_iter_value_ptr(iter); +} + + +void *json_object_key_to_iter_impl(const char *key) +{ + return json_object_key_to_iter_ptr(key); +} + + +int json_object_set_new_impl(json_t *object, const char *key, json_t *value) +{ + return json_object_set_new_ptr(object, key, value); +} + + +json_t *json_real_impl(double value) +{ + return json_real_ptr(value); +} + + +double json_real_value_impl(const json_t *real) +{ + return json_real_value_ptr(real); +} + + +json_t *json_string_impl(const char *value) +{ + return json_string_ptr(value); +} + + +const char *json_string_value_impl(const json_t *string) +{ + return json_string_value_ptr(string); +} + + +json_t *json_true_impl(void) +{ + return json_true_ptr(); +} + +#endif + diff --git a/src/util/virjsoncompat.h b/src/util/virjsoncompat.h new file mode 100644 index 0000000000..c3963a5259 --- /dev/null +++ b/src/util/virjsoncompat.h @@ -0,0 +1,86 @@ +/* + * virjson.h: JSON object parsing/formatting + * + * Copyright (C) 2018 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + */ + + +#ifndef __VIR_JSON_COMPAT_H_ +# define __VIR_JSON_COMPAT_H_ + +#ifndef VIR_JSON_COMPAT_IMPL + +#define json_array json_array_impl +#define json_array_append_new json_array_append_new_impl +#define json_array_get json_array_get_impl +#define json_array_size json_array_size_impl +#define json_delete json_delete_impl +#define json_dumps json_dumps_impl +#define json_false json_false_impl +#define json_integer json_integer_impl +#define json_integer_value json_integer_value_impl +#define json_loads json_loads_impl +#define json_null json_null_impl +#define json_object json_object_impl +#define json_object_iter json_object_iter_impl +#define json_object_iter_key json_object_iter_key_impl +#define json_object_iter_next json_object_iter_next_impl +#define json_object_iter_value json_object_iter_value_impl +#define json_object_key_to_iter json_object_key_to_iter_impl +#define json_object_set_new json_object_set_new_impl +#define json_real json_real_impl +#define json_real_value json_real_value_impl +#define json_string json_string_impl +#define json_string_value json_string_value_impl +#define json_true json_true_impl + +#include <jansson.h> + +#else + +#include <jansson.h> + +json_t *json_array_impl(void); +int json_array_append_new_impl(json_t *array, json_t *value); +json_t *json_array_get_impl(const json_t *array, size_t index); +size_t json_array_size_impl(const json_t *array); +void json_delete_impl(json_t *json); +char *json_dumps_impl(const json_t *json, size_t flags); +json_t *json_false_impl(void); +json_t *json_integer_impl(json_int_t value); +json_int_t json_integer_value_impl(const json_t *integer); +json_t *json_loads_impl(const char *input, size_t flags, json_error_t *error); +json_t *json_null_impl(void); +json_t *json_object_impl(void); +void *json_object_iter_impl(json_t *object); +const char *json_object_iter_key_impl(void *iter); +void *json_object_iter_next_impl(json_t *object, void *iter); +json_t *json_object_iter_value_impl(void *iter); +void *json_object_key_to_iter_impl(const char *key); +int json_object_set_new_impl(json_t *object, const char *key, json_t *value); +json_t *json_real_impl(double value); +double json_real_value_impl(const json_t *real); +json_t *json_string_impl(const char *value); +const char *json_string_value_impl(const json_t *string); +json_t *json_true_impl(void); + +#endif + +int virJSONInitialize(void); + +#endif -- 2.17.1

6 9

[libvirt] [PATCH v2 RESEND 00/12] PCI passthrough support on s390
by Yi Min Zhao 31 Jul '18

31 Jul '18

Abstract ======== The PCI representation in QEMU has recently been extended for S390 allowing configuration of zPCI attributes like uid (user-defined identifier) and fid (PCI function identifier). The details can be found here: https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg07262.html To support the new zPCI feature of the S390 platform, two new XML attributes, @uid and @fid, are introduced for device addresses of type 'pci', i.e.: <hostdev mode='subsystem' type='pci'> <driver name='vfio'/> <source> <address domain='0x0001' bus='0x00' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' uid='0x0003' fid='0x00000027'/> </hostdev> uid and fid are optional attributes. If they are defined by the user, unique values within the guest domain must be used. If they are not specified and the architecture requires them, they are automatically generated with non-conflicting values. Current implementation is the most seamless one for the user as it unites the address specific data of a PCI device on one XML element. It could accommodate both specifying our special parameters (uid and fid) and re-using standard statements (domain, bus, slot and function) for PCI devices. User can still specify bus/slot/function for the virtualized PCI devices in the XML. Thus uid/fid act as an extension to the PCI address and are stored in a new structure 'virZPCIDeviceAddress' which is a member of common PCI Address structure. Additionally, two hashtables are used for assignment and reservation of uid/fid. In support of extending the PCI address, a new PCI address extension flag is introduced. This extension flag allows is not only dedicated for the S390 platform but also other architectures needing certain extensions to PCI address space. Code Base ========= commit in master: 767f9e1449b1a36111532847f0c62dc758263c42 qemu: validate: Enforce compile time switch type checking for videos Change Log ========== v1->v2: 1. Separate test commit and merge testcases into corresponding commits that introduce the functionalities firstly. 2. Spare some checks for zpci device. 3. Add vsock and controller support. 4. Add uin32 type schema. 5. Rename zpciuid and zpcifid to zpci_uid and zpci_fid. 6. Always return multibus support on S390. Yi Min Zhao (12): conf: Add definitions for 'uid' and 'fid' PCI address attributes qemu: Introduce zPCI capability conf: Introduce a new PCI address extension flag qemu: Enable PCI multi bus for S390 guests qemu: Auto add pci-root for s390/s390x guests conf: Introduce address caching for PCI extensions conf: Introduce parser, formatter for uid and fid conf: Allocate/release 'uid' and 'fid' in PCI address qemu: Generate and use zPCI device in QEMU command line qemu: Add hotpluging support for PCI devices on S390 guests docs: Add 'uid' and 'fid' information news: Update news for PCI address extension attributes docs/formatdomain.html.in | 9 +- docs/news.xml | 11 + docs/schemas/basictypes.rng | 31 ++ docs/schemas/domaincommon.rng | 1 + src/conf/device_conf.c | 73 +++++ src/conf/device_conf.h | 1 + src/conf/domain_addr.c | 346 +++++++++++++++++++++ src/conf/domain_addr.h | 29 ++ src/conf/domain_conf.c | 6 + src/libvirt_private.syms | 4 + src/qemu/qemu_capabilities.c | 6 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 115 +++++++ src/qemu/qemu_command.h | 4 + src/qemu/qemu_domain.c | 1 + src/qemu/qemu_domain_address.c | 181 ++++++++++- src/qemu/qemu_hotplug.c | 182 ++++++++++- src/util/virpci.h | 13 + tests/qemucapabilitiesdata/caps_2.10.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.11.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.12.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.7.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.8.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.s390x.xml | 1 + tests/qemuxml2argvdata/disk-virtio-s390-zpci.args | 27 ++ tests/qemuxml2argvdata/disk-virtio-s390-zpci.xml | 17 + .../hostdev-vfio-zpci-autogenerate.args | 26 ++ .../hostdev-vfio-zpci-autogenerate.xml | 18 ++ .../hostdev-vfio-zpci-boundaries.args | 30 ++ .../hostdev-vfio-zpci-boundaries.xml | 26 ++ .../hostdev-vfio-zpci-multidomain-many.args | 40 +++ .../hostdev-vfio-zpci-multidomain-many.xml | 67 ++++ tests/qemuxml2argvdata/hostdev-vfio-zpci.args | 26 ++ tests/qemuxml2argvdata/hostdev-vfio-zpci.xml | 19 ++ tests/qemuxml2argvtest.c | 14 + tests/qemuxml2xmloutdata/disk-virtio-s390-zpci.xml | 29 ++ tests/qemuxml2xmloutdata/hostdev-vfio-zpci.xml | 30 ++ tests/qemuxml2xmltest.c | 3 + 38 files changed, 1377 insertions(+), 15 deletions(-) create mode 100644 tests/qemuxml2argvdata/disk-virtio-s390-zpci.args create mode 100644 tests/qemuxml2argvdata/disk-virtio-s390-zpci.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-multidomain-many.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-multidomain-many.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci.xml create mode 100644 tests/qemuxml2xmloutdata/disk-virtio-s390-zpci.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-vfio-zpci.xml -- Yi Min

7 52

[libvirt] [PATCH] qemu: monitor: Fix incrementing of 'nstats' in qemuMonitorJSONBlockStatsCollectData
by Peter Krempa 31 Jul '18

31 Jul '18

commit 8d9ca6cdb3a5 refactored qemuMonitorJSONBlockStatsCollectData so that the number of stats is passed back via a pointer. The commit failed to fix the macro which increments the number of stats to increment the actual pointee. Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- src/qemu/qemu_monitor_json.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c index 9acf62e0bb..2921f110a9 100644 --- a/src/qemu/qemu_monitor_json.c +++ b/src/qemu/qemu_monitor_json.c @@ -2299,7 +2299,7 @@ qemuMonitorJSONBlockStatsCollectData(virJSONValuePtr dev, #define QEMU_MONITOR_BLOCK_STAT_GET(NAME, VAR, MANDATORY) \ if (MANDATORY || virJSONValueObjectHasKey(stats, NAME)) { \ - nstats++; \ + (*nstats)++; \ if (virJSONValueObjectGetNumberLong(stats, NAME, &VAR) < 0) { \ virReportError(VIR_ERR_INTERNAL_ERROR, \ _("cannot read %s statistic"), NAME); \ -- 2.16.2

2 1

[libvirt] [PATCH v2] util: avoid symbol clash between json libraries
by Daniel P. Berrangé 31 Jul '18

31 Jul '18

The jansson and json-glib libraries both export symbols with a json_ name prefix and json_object_iter_next() clashes between them. Unfortunately json_glib is linked in by GTK, so any app using GTK and libvirt will get a clash, resulting in SEGV. This also affects the NSS module provided by libvirt Instead of directly linking to jansson, use dlopen() with the RTLD_LOCAL flag which allows us to hide the symbols from the application that loads libvirt or the NSS module. Some preprocessor black magic and wrapper functions are used to redirect calls into the dlopen resolved symbols. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- libvirt.spec.in | 2 + src/Makefile.am | 5 +- src/util/Makefile.inc.am | 3 +- src/util/virjson.c | 9 +- src/util/virjsoncompat.c | 274 +++++++++++++++++++++++++++++++++++++++ src/util/virjsoncompat.h | 88 +++++++++++++ 6 files changed, 377 insertions(+), 4 deletions(-) create mode 100644 src/util/virjsoncompat.c create mode 100644 src/util/virjsoncompat.h diff --git a/libvirt.spec.in b/libvirt.spec.in index 4113579e47..19ae55cdaf 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -898,6 +898,8 @@ Requires: ncurses Requires: gettext # Needed by virt-pki-validate script. Requires: gnutls-utils +# We dlopen(libjansson.so.4), so need an explicit dep +Requires: jansson %if %{with_bash_completion} Requires: %{name}-bash-completion = %{version}-%{release} %endif diff --git a/src/Makefile.am b/src/Makefile.am index 83263e69e5..4b6366bb4c 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -552,7 +552,6 @@ libvirt_admin_la_CFLAGS += \ libvirt_admin_la_LIBADD += \ $(CAPNG_LIBS) \ - $(JANSSON_LIBS) \ $(DEVMAPPER_LIBS) \ $(LIBXML_LIBS) \ $(SSH2_LIBS) \ @@ -690,6 +689,7 @@ libvirt_setuid_rpc_client_la_SOURCES = \ util/virhashcode.c \ util/virhostcpu.c \ util/virjson.c \ + util/virjsoncompat.c \ util/virlog.c \ util/virobject.c \ util/virpidfile.c \ @@ -961,6 +961,8 @@ libvirt_nss_la_SOURCES = \ util/virhashcode.h \ util/virjson.c \ util/virjson.h \ + util/virjsoncompat.c \ + util/virjsoncompat.h \ util/virkmod.c \ util/virkmod.h \ util/virlease.c \ @@ -1001,7 +1003,6 @@ libvirt_nss_la_LDFLAGS = \ $(NULL) libvirt_nss_la_LIBADD = \ - $(JANSSON_LIBS) \ $(NULL) endif WITH_NSS diff --git a/src/util/Makefile.inc.am b/src/util/Makefile.inc.am index 71b2b93c2d..8ef9ee1dfa 100644 --- a/src/util/Makefile.inc.am +++ b/src/util/Makefile.inc.am @@ -86,6 +86,8 @@ UTIL_SOURCES = \ util/viriscsi.h \ util/virjson.c \ util/virjson.h \ + util/virjsoncompat.c \ + util/virjsoncompat.h \ util/virkeycode.c \ util/virkeycode.h \ util/virkeyfile.c \ @@ -264,7 +266,6 @@ libvirt_util_la_CFLAGS = \ $(NULL) libvirt_util_la_LIBADD = \ $(CAPNG_LIBS) \ - $(JANSSON_LIBS) \ $(LIBNL_LIBS) \ $(THREAD_LIBS) \ $(AUDIT_LIBS) \ diff --git a/src/util/virjson.c b/src/util/virjson.c index 01a387b2f7..5bab662cd3 100644 --- a/src/util/virjson.c +++ b/src/util/virjson.c @@ -1437,7 +1437,8 @@ virJSONValueCopy(const virJSONValue *in) #if WITH_JANSSON -# include <jansson.h> + +# include "virjsoncompat.h" static virJSONValuePtr virJSONValueFromJansson(json_t *json) @@ -1524,6 +1525,9 @@ virJSONValueFromString(const char *jsonstring) size_t flags = JSON_REJECT_DUPLICATES | JSON_DECODE_ANY; + if (virJSONInitialize() < 0) + return NULL; + if (!(json = json_loads(jsonstring, flags, &error))) { virReportError(VIR_ERR_INTERNAL_ERROR, _("failed to parse JSON %d:%d: %s"), @@ -1630,6 +1634,9 @@ virJSONValueToString(virJSONValuePtr object, json_t *json; char *str = NULL; + if (virJSONInitialize() < 0) + return NULL; + if (pretty) flags |= JSON_INDENT(2); else diff --git a/src/util/virjsoncompat.c b/src/util/virjsoncompat.c new file mode 100644 index 0000000000..6c853e9bb5 --- /dev/null +++ b/src/util/virjsoncompat.c @@ -0,0 +1,274 @@ +/* + * virjsoncompat.c: JSON object parsing/formatting + * + * Copyright (C) 2018 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + */ + +#include <config.h> + +#include "virthread.h" +#include "virerror.h" +#define VIR_JSON_COMPAT_IMPL +#include "virjsoncompat.h" + +#define VIR_FROM_THIS VIR_FROM_NONE + +#if WITH_JANSSON + +# include <dlfcn.h> + +json_t *(*json_array_ptr)(void); +int (*json_array_append_new_ptr)(json_t *array, json_t *value); +json_t *(*json_array_get_ptr)(const json_t *array, size_t index); +size_t (*json_array_size_ptr)(const json_t *array); +void (*json_delete_ptr)(json_t *json); +char *(*json_dumps_ptr)(const json_t *json, size_t flags); +json_t *(*json_false_ptr)(void); +json_t *(*json_integer_ptr)(json_int_t value); +json_int_t (*json_integer_value_ptr)(const json_t *integer); +json_t *(*json_loads_ptr)(const char *input, size_t flags, json_error_t *error); +json_t *(*json_null_ptr)(void); +json_t *(*json_object_ptr)(void); +void *(*json_object_iter_ptr)(json_t *object); +const char *(*json_object_iter_key_ptr)(void *iter); +void *(*json_object_iter_next_ptr)(json_t *object, void *iter); +json_t *(*json_object_iter_value_ptr)(void *iter); +void *(*json_object_key_to_iter_ptr)(const char *key); +int (*json_object_set_new_ptr)(json_t *object, const char *key, json_t *value); +json_t *(*json_real_ptr)(double value); +double (*json_real_value_ptr)(const json_t *real); +json_t *(*json_string_ptr)(const char *value); +const char *(*json_string_value_ptr)(const json_t *string); +json_t *(*json_true_ptr)(void); + + +static int +virJSONJanssonOnceInit(void) +{ + void *handle = dlopen("libjansson.so.4", RTLD_LAZY|RTLD_LOCAL|RTLD_NODELETE); + if (!handle) { + virReportError(VIR_ERR_NO_SUPPORT, + _("libjansson.so.4 JSON library not available: %s"), dlerror()); + return -1; + } + +# define LOAD(name) \ + do { \ + if (!(name ## _ptr = dlsym(handle, #name))) { \ + virReportError(VIR_ERR_NO_SUPPORT, \ + _("missing symbol '%s' in libjansson.so.4: %s"), #name, dlerror()); \ + return -1; \ + } \ + } while (0) + + LOAD(json_array); + LOAD(json_array_append_new); + LOAD(json_array_get); + LOAD(json_array_size); + LOAD(json_delete); + LOAD(json_dumps); + LOAD(json_false); + LOAD(json_integer); + LOAD(json_integer_value); + LOAD(json_loads); + LOAD(json_null); + LOAD(json_object); + LOAD(json_object_iter); + LOAD(json_object_iter_key); + LOAD(json_object_iter_next); + LOAD(json_object_iter_value); + LOAD(json_object_key_to_iter); + LOAD(json_object_set_new); + LOAD(json_real); + LOAD(json_real_value); + LOAD(json_string); + LOAD(json_string_value); + LOAD(json_true); + + return 0; +} + +VIR_ONCE_GLOBAL_INIT(virJSONJansson); + +int +virJSONInitialize(void) +{ + return virJSONJanssonInitialize(); +} + +json_t * +json_array_impl(void) +{ + return json_array_ptr(); +} + + +int +json_array_append_new_impl(json_t *array, json_t *value) +{ + return json_array_append_new_ptr(array, value); +} + + +json_t * +json_array_get_impl(const json_t *array, size_t index) +{ + return json_array_get_ptr(array, index); +} + + +size_t +json_array_size_impl(const json_t *array) +{ + return json_array_size_ptr(array); +} + + +void +json_delete_impl(json_t *json) +{ + return json_delete_ptr(json); +} + + +char * +json_dumps_impl(const json_t *json, size_t flags) +{ + return json_dumps_ptr(json, flags); +} + + +json_t * +json_false_impl(void) +{ + return json_false_ptr(); +} + + +json_t * +json_integer_impl(json_int_t value) +{ + return json_integer_ptr(value); +} + + +json_int_t +json_integer_value_impl(const json_t *integer) +{ + return json_integer_value_ptr(integer); +} + + +json_t * +json_loads_impl(const char *input, size_t flags, json_error_t *error) +{ + return json_loads_ptr(input, flags, error); +} + + +json_t * +json_null_impl(void) +{ + return json_null_ptr(); +} + + +json_t * +json_object_impl(void) +{ + return json_object_ptr(); +} + + +void * +json_object_iter_impl(json_t *object) +{ + return json_object_iter_ptr(object); +} + + +const char * +json_object_iter_key_impl(void *iter) +{ + return json_object_iter_key_ptr(iter); +} + + +void * +json_object_iter_next_impl(json_t *object, void *iter) +{ + return json_object_iter_next_ptr(object, iter); +} + + +json_t * +json_object_iter_value_impl(void *iter) +{ + return json_object_iter_value_ptr(iter); +} + + +void * +json_object_key_to_iter_impl(const char *key) +{ + return json_object_key_to_iter_ptr(key); +} + + +int +json_object_set_new_impl(json_t *object, const char *key, json_t *value) +{ + return json_object_set_new_ptr(object, key, value); +} + + +json_t * +json_real_impl(double value) +{ + return json_real_ptr(value); +} + + +double +json_real_value_impl(const json_t *real) +{ + return json_real_value_ptr(real); +} + + +json_t * +json_string_impl(const char *value) +{ + return json_string_ptr(value); +} + + +const char * +json_string_value_impl(const json_t *string) +{ + return json_string_value_ptr(string); +} + + +json_t * +json_true_impl(void) +{ + return json_true_ptr(); +} + +#endif /* WITH_JANSSON */ diff --git a/src/util/virjsoncompat.h b/src/util/virjsoncompat.h new file mode 100644 index 0000000000..d9b7765a37 --- /dev/null +++ b/src/util/virjsoncompat.h @@ -0,0 +1,88 @@ +/* + * virjsoncompat.h: JSON object parsing/formatting + * + * Copyright (C) 2018 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + */ + + +#ifndef __VIR_JSON_COMPAT_H_ +# define __VIR_JSON_COMPAT_H_ + +# if WITH_JANSSON +# ifndef VIR_JSON_COMPAT_IMPL + +# define json_array json_array_impl +# define json_array_append_new json_array_append_new_impl +# define json_array_get json_array_get_impl +# define json_array_size json_array_size_impl +# define json_delete json_delete_impl +# define json_dumps json_dumps_impl +# define json_false json_false_impl +# define json_integer json_integer_impl +# define json_integer_value json_integer_value_impl +# define json_loads json_loads_impl +# define json_null json_null_impl +# define json_object json_object_impl +# define json_object_iter json_object_iter_impl +# define json_object_iter_key json_object_iter_key_impl +# define json_object_iter_next json_object_iter_next_impl +# define json_object_iter_value json_object_iter_value_impl +# define json_object_key_to_iter json_object_key_to_iter_impl +# define json_object_set_new json_object_set_new_impl +# define json_real json_real_impl +# define json_real_value json_real_value_impl +# define json_string json_string_impl +# define json_string_value json_string_value_impl +# define json_true json_true_impl + +# endif /* ! VIR_JSON_COMPAT_IMPL */ + +# include <jansson.h> + +# ifdef VIR_JSON_COMPAT_IMPL + +json_t *json_array_impl(void); +int json_array_append_new_impl(json_t *array, json_t *value); +json_t *json_array_get_impl(const json_t *array, size_t index); +size_t json_array_size_impl(const json_t *array); +void json_delete_impl(json_t *json); +char *json_dumps_impl(const json_t *json, size_t flags); +json_t *json_false_impl(void); +json_t *json_integer_impl(json_int_t value); +json_int_t json_integer_value_impl(const json_t *integer); +json_t *json_loads_impl(const char *input, size_t flags, json_error_t *error); +json_t *json_null_impl(void); +json_t *json_object_impl(void); +void *json_object_iter_impl(json_t *object); +const char *json_object_iter_key_impl(void *iter); +void *json_object_iter_next_impl(json_t *object, void *iter); +json_t *json_object_iter_value_impl(void *iter); +void *json_object_key_to_iter_impl(const char *key); +int json_object_set_new_impl(json_t *object, const char *key, json_t *value); +json_t *json_real_impl(double value); +double json_real_value_impl(const json_t *real); +json_t *json_string_impl(const char *value); +const char *json_string_value_impl(const json_t *string); +json_t *json_true_impl(void); + +# endif /* VIR_JSON_COMPAT_IMPL */ +# endif /* WITH_JANSSON */ + +int virJSONInitialize(void); + +#endif /* __VIR_JSON_COMPAT_H_ */ -- 2.17.1

3 2

[libvirt] [RFC PATCH 00/10] Revert the switch to Jansson
by Ján Tomko 31 Jul '18

31 Jul '18

As reported by Andrea Bolognani (Thanks, Andrea!), since the switch to jansson applications using libvirt_nss crash due to a function name collision with json-glib: https://www.redhat.com/archives/libvir-list/2018-July/msg02044.html Revert the changes until we come up with a better solution. Ján Tomko (10): Revert "tests: also skip qemuagenttest with old jansson" Revert "m4: Introduce STABLE_ORDERING_JANSSON" Revert "build: require Jansson if QEMU driver is enabled" Revert "build: switch --with-qemu default from yes to check" Revert "Remove virJSONValueNewStringLen" Revert "build: remove references to WITH_YAJL for SETUID_RPC_CLIENT" Revert "Remove functions using yajl" Revert "Switch from yajl to Jansson" Revert "build: undef WITH_JANSSON for SETUID_RPC_CLIENT" Revert "build: add --with-jansson" config-post.h | 3 +- configure.ac | 3 - libvirt.spec.in | 4 +- m4/virt-driver-qemu.m4 | 9 +- m4/virt-jansson.m4 | 32 -- m4/virt-nss.m4 | 4 +- m4/virt-yajl.m4 | 27 +- src/Makefile.am | 8 +- src/libvirt_private.syms | 1 + src/qemu/qemu_driver.c | 2 +- src/util/Makefile.inc.am | 4 +- src/util/virjson.c | 616 ++++++++++++++++++++++++------- src/util/virjson.h | 1 + tests/Makefile.am | 12 +- tests/cputest.c | 16 +- tests/libxlxml2domconfigtest.c | 4 +- tests/qemuagenttest.c | 4 +- tests/qemublocktest.c | 6 - tests/qemucapabilitiestest.c | 7 +- tests/qemucaps2xmltest.c | 2 +- tests/qemucommandutiltest.c | 7 +- tests/qemuhotplugtest.c | 7 +- tests/qemumigparamsdata/empty.json | 4 +- tests/qemumigparamsdata/unsupported.json | 4 +- tests/qemumigparamstest.c | 7 +- tests/qemumonitorjsontest.c | 7 +- tests/virjsontest.c | 5 - tests/virmacmaptest.c | 5 - tests/virmacmaptestdata/empty.json | 4 +- tests/virmocklibxl.c | 4 +- tests/virnetdaemontest.c | 7 +- tests/virstoragetest.c | 4 +- 32 files changed, 555 insertions(+), 275 deletions(-) delete mode 100644 m4/virt-jansson.m4 -- 2.16.1

1 10

[libvirt] [PATCH v4 0/4] iscsi-direct: first part
by clem＠lse.epita.fr 31 Jul '18

31 Jul '18

From: Clementine Hayat <clem(a)lse.epita.fr> Hello, This is the implementation of the iscsi-direct backend storage pool version 4. v1: https://www.redhat.com/archives/libvir-list/2018-July/msg00918.html v2: https://www.redhat.com/archives/libvir-list/2018-July/msg01528.html v3: https://www.redhat.com/archives/libvir-list/2018-July/msg02076.html Best Regards, -- Clementine Hayat Clementine Hayat (4): configure: Introduce libiscsi in build system storage: Introduce iscsi_direct pool type storage: Implement iscsi_direct pool backend news: add storage pool iscsi-direct configure.ac | 9 +- docs/news.xml | 9 + docs/schemas/storagepool.rng | 35 ++ docs/storage.html.in | 30 ++ m4/virt-libiscsi.m4 | 30 ++ m4/virt-storage-iscsi-direct.m4 | 44 ++ src/conf/domain_conf.c | 1 + src/conf/storage_conf.c | 22 +- src/conf/storage_conf.h | 1 + src/conf/virstorageobj.c | 2 + src/storage/Makefile.inc.am | 24 + src/storage/storage_backend.c | 6 + src/storage/storage_backend_iscsi_direct.c | 452 ++++++++++++++++++ src/storage/storage_backend_iscsi_direct.h | 6 + src/storage/storage_driver.c | 1 + .../pool-iscsi-direct-auth.xml | 14 + .../pool-iscsi-direct.xml | 11 + .../pool-iscsi-direct-auth.xml | 17 + .../pool-iscsi-direct.xml | 14 + tests/storagepoolxml2xmltest.c | 2 + tools/virsh-pool.c | 3 + 21 files changed, 727 insertions(+), 6 deletions(-) create mode 100644 m4/virt-libiscsi.m4 create mode 100644 m4/virt-storage-iscsi-direct.m4 create mode 100644 src/storage/storage_backend_iscsi_direct.c create mode 100644 src/storage/storage_backend_iscsi_direct.h create mode 100644 tests/storagepoolxml2xmlin/pool-iscsi-direct-auth.xml create mode 100644 tests/storagepoolxml2xmlin/pool-iscsi-direct.xml create mode 100644 tests/storagepoolxml2xmlout/pool-iscsi-direct-auth.xml create mode 100644 tests/storagepoolxml2xmlout/pool-iscsi-direct.xml -- 2.18.0

1 4

[libvirt] [PATCH 00/11] Add checks to ensure a name isn't all whitespace
by John Ferlan 31 Jul '18

31 Jul '18

https://bugzilla.redhat.com/show_bug.cgi?id=1107420 As unusual as the case may be, having a name of all white space has been allowed. This leads to the obvious problem of how a future usage would be able to "utilize" that resource since it's not "simple" to determine what combination of spaces and tabs are being used for the name. So add code to various drivers in order to inhibit this for those resources that would consider using a name for some sort of lookup type reference. For a majority of drivers, the schema validation also allowed the name; however, a couple (secrets and nwfilters) the virt-xml-validate would fail. Still this wouldn't necessarily stop someone - just slow them down. So for those the test xml file must use the *-invalid.xml type syntax to inhibit virschematest from looking at the schema. Similar validation for node devices and interfaces is not necessary since those are referencing system named elements which wouldn't have the oddly named resource. Also, while it could be checked, avoid the check for Storage Volumes as there's no way to "inhibit" in the parse logic algorithm. This is not for 4.6.0, but figured I'd get it on list for consideration for 4.7.0. Yes, it's quite unusual and one has to wonder who thinks up this type of situation, but still in the long run we should attempt to avoid it anyway. I've only added the extra checking for qemu and lxc domains, leaving others to those maintainers to choose to utilize. John Ferlan (11): conf: Add @flags to Storage Pool Def processing conf: Disallow new storage pools to use all white space as name conf: Add @flags to Network Def processing conf: Disallow new networks to use all white space as name conf: Disallow new qemu,lxc domains to use all white space as name conf: Split and rename secretXMLParseNode conf: Add @flags to Secret Def processing conf: Disallow new secrets to use all white space as usage id conf: Add @flags to NWFilter Def processing conf: Disallow new nwfilters to use all white space as name conf: Disallow new snapshots to use all white space as name src/conf/domain_conf.c | 11 ++- src/conf/domain_conf.h | 4 + src/conf/network_conf.c | 34 +++++--- src/conf/network_conf.h | 19 ++++- src/conf/nwfilter_conf.c | 32 +++++--- src/conf/nwfilter_conf.h | 16 +++- src/conf/secret_conf.c | 79 +++++++++++++------ src/conf/secret_conf.h | 16 +++- src/conf/snapshot_conf.c | 7 ++ src/conf/snapshot_conf.h | 1 + src/conf/storage_conf.c | 32 +++++--- src/conf/storage_conf.h | 19 ++++- src/conf/virnetworkobj.c | 4 +- src/conf/virnwfilterobj.c | 2 +- src/conf/virsecretobj.c | 2 +- src/conf/virstorageobj.c | 4 +- src/esx/esx_network_driver.c | 2 +- src/lxc/lxc_driver.c | 6 +- src/network/bridge_driver.c | 6 +- src/nwfilter/nwfilter_driver.c | 3 +- src/phyp/phyp_driver.c | 2 +- src/qemu/qemu_driver.c | 9 ++- src/qemu/qemu_process.c | 2 +- src/secret/secret_driver.c | 3 +- src/storage/storage_driver.c | 6 +- src/test/test_driver.c | 12 +-- src/vbox/vbox_network.c | 2 +- .../name_whitespace.xml | 3 + tests/domainsnapshotxml2xmltest.c | 38 ++++++--- tests/networkxml2conftest.c | 2 +- tests/networkxml2firewalltest.c | 2 +- .../network-whitespace-name.xml | 6 ++ tests/networkxml2xmltest.c | 4 +- tests/networkxml2xmlupdatetest.c | 2 +- tests/nwfilterxml2firewalltest.c | 2 +- .../name-whitespace-invalid.xml | 4 + tests/nwfilterxml2xmltest.c | 7 +- tests/qemuxml2argvdata/name-whitespace.xml | 29 +++++++ tests/qemuxml2argvtest.c | 5 +- .../usage-whitespace-invalid.xml | 7 ++ tests/secretxml2xmltest.c | 30 +++++-- tests/storagebackendsheepdogtest.c | 4 +- .../pool-dir-whitespace-name.xml | 18 +++++ tests/storagepoolxml2xmltest.c | 45 ++++++++--- tests/storagevolxml2argvtest.c | 4 +- tests/storagevolxml2xmltest.c | 2 +- 46 files changed, 418 insertions(+), 131 deletions(-) create mode 100644 tests/domainsnapshotxml2xmlin/name_whitespace.xml create mode 100644 tests/networkxml2xmlin/network-whitespace-name.xml create mode 100644 tests/nwfilterxml2xmlin/name-whitespace-invalid.xml create mode 100644 tests/qemuxml2argvdata/name-whitespace.xml create mode 100644 tests/secretxml2xmlin/usage-whitespace-invalid.xml create mode 100644 tests/storagepoolxml2xmlin/pool-dir-whitespace-name.xml -- 2.17.1

2 13

[libvirt] [PATCH RFC 00/39] qemu: Add support for -blockdev
by Peter Krempa 31 Jul '18

31 Jul '18

This series adds support for starting and hotplug of disks with -blockdev/blockdev-add. Blockjobs are not supported and thus the last patch should not be applied yet as some refactoring of the jobs is required. At the beginning of the series there are a few cleanup patches which may be pushed even at this point. The main reason this is in RFC state is that block stats reporting does not work. The following command: {"execute":"query-blockstats","arguments":{"query-nodes":true}} Returns no reasonable data: { "stats": { "flush_total_time_ns": 0, "wr_highest_offset": 0, "wr_total_time_ns": 0, "failed_wr_operations": 0, "failed_rd_operations": 0, "wr_merged": 0, "wr_bytes": 0, "timed_stats": [ ], "failed_flush_operations": 0, "account_invalid": false, "rd_total_time_ns": 0, "flush_operations": 0, "wr_operations": 0, "rd_merged": 0, "rd_bytes": 0, "invalid_flush_operations": 0, "account_failed": false, "rd_operations": 0, "invalid_wr_operations": 0, "invalid_rd_operations": 0 }, "node-name": "libvirt-7-storage" }, { "parent": { "stats": { "flush_total_time_ns": 0, "wr_highest_offset": 0, "wr_total_time_ns": 0, "failed_wr_operations": 0, "failed_rd_operations": 0, "wr_merged": 0, "wr_bytes": 0, "timed_stats": [ ], "failed_flush_operations": 0, "account_invalid": false, "rd_total_time_ns": 0, "flush_operations": 0, "wr_operations": 0, "rd_merged": 0, "rd_bytes": 0, "invalid_flush_operations": 0, "account_failed": false, "rd_operations": 0, "invalid_wr_operations": 0, "invalid_rd_operations": 0 }, "node-name": "libvirt-7-storage" }, "stats": { "flush_total_time_ns": 0, "wr_highest_offset": 0, "wr_total_time_ns": 0, "failed_wr_operations": 0, "failed_rd_operations": 0, "wr_merged": 0, "wr_bytes": 0, "timed_stats": [ ], "failed_flush_operations": 0, "account_invalid": false, "rd_total_time_ns": 0, "flush_operations": 0, "wr_operations": 0, "rd_merged": 0, "rd_bytes": 0, "invalid_flush_operations": 0, "account_failed": false, "rd_operations": 0, "invalid_wr_operations": 0, "invalid_rd_operations": 0 }, "node-name": "libvirt-7-format" }, the 'libvirt-7-storage' and 'libvirt-7-format' nodes represent the ISO backing the CDROM used to boot the VM so reads were executed. In the old approach when we use -drive and query-nodes is false the output looks like this: { "device": "drive-ide0-0-0", "parent": { "stats": { "flush_total_time_ns": 0, "wr_highest_offset": 0, "wr_total_time_ns": 0, "failed_wr_operations": 0, "failed_rd_operations": 0, "wr_merged": 0, "wr_bytes": 0, "timed_stats": [ ], "failed_flush_operations": 0, "account_invalid": false, "rd_total_time_ns": 0, "flush_operations": 0, "wr_operations": 0, "rd_merged": 0, "rd_bytes": 0, "invalid_flush_operations": 0, "account_failed": false, "rd_operations": 0, "invalid_wr_operations": 0, "invalid_rd_operations": 0 }, "node-name": "#block080" }, "stats": { "flush_total_time_ns": 0, "wr_highest_offset": 0, "wr_total_time_ns": 0, "failed_wr_operations": 0, "failed_rd_operations": 0, "wr_merged": 0, "wr_bytes": 0, "timed_stats": [ ], "failed_flush_operations": 0, "account_invalid": true, "rd_total_time_ns": 204236271, "flush_operations": 0, "wr_operations": 0, "rd_merged": 0, "rd_bytes": 30046628, "invalid_flush_operations": 0, "account_failed": true, "idle_time_ns": 18766797619, "rd_operations": 14680, "invalid_wr_operations": 0, "invalid_rd_operations": 0 }, "node-name": "#block152" }, I also get all zeroes when I use 'query-nodes' true on a machine started with -drive. Without the stats we'd not achieve feature parity unfortunately. Kevin, could you please have a look? Peter Krempa (39): qemu: monitor: Reuse qemuMonitorJSONQueryBlock in qemuMonitorJSONBlockIoThrottleInfo qemu: monitor: Allow using 'id' instead of 'device' for 'block_set_io_throttle' qemu: monitor: Allow using 'qdev' instead of 'device' for getting disk throttling tests: qemu: Drop disk from hostdev-mdev tests tests: qemuxml2argv: Fork CAPS_LATEST test cases for 'blockdev' tests: qemu: Add test data for backing chains and indexes qemu: hotplug: Don't generate alias when detaching disk util: virqemu: Simplify debugging if building QOM object with missing args qemu: caps: Add capability for using the blockdev infrastructure qemu: process: clear QEMU_CAPS_BLOCKDEV for VMs where we can't support it qemu: domain: Don't redetect backing chain when using -blockdev qemu: process: Don't detect nodenames when we support -blockdev conf: domain: Format out user provided backing chains in XML qemu: domain: Add infrastructure to generate block node names conf: Implement private data formatting and parsing for disks conf: Allow formatting and parsing of 'index' for disk source image qemu: Use proper backingIndex when reporting stats for backing chain qemu: Add field to store QDEV path of a disk in private data qemu: alias: Generate QDEV name of the block backend for disks qemu: domain: Add field for storing node name for copy-on-read qemu: proces: assign node names for user defined backing chains qemu: block: Add generator for the 'copy-on-read' blockdev driver qemu: domain: Prepare qemuDomainDiskGetBackendAlias for -blockdev qemu: command: format disk source commandline for -blockdev qemu: command: Add helper to check if disk throttling is enabled qemu: process: Setup disk io throttling for -blockdev qemu: driver: Use QOM backend name for disk IO throttling APIs qemu: hotplug: Prepare for blockdev-add/blockdev-del with backing chains qemu: monitor: Add APIs for cdrom tray handling for -blockdev qemu: hotplug: Implement removable media change for -blockdev qemu: monitor: Prepare query-block calls for dropping of -drive qemu: Use QOM path with query-block when using -blockdev qemu: monitor: Add API to retrieve blockstats by nodenames qemu: monitor: Add APIs for refreshing disk capacity when using -blockdev qemu: driver: Don't pass 'virDomainDiskDefPtr' to qemuDomainGetStatsOneBlock qemu: driver: Allow using blockdev with qemuDomainBlocksStatsGather qemu: monitor: Extract 'write-threshold' automatically for -blockdev qemu: driver: Prepare qemuDomainGetStatsBlock (bulk disk stats) for -blockdev DO NOT APPLY: Enable QEMU_CAPS_BLOCKDEV if 'copy-on-read' is supported docs/formatdomain.html.in | 7 +- docs/schemas/domaincommon.rng | 19 ++ src/conf/domain_conf.c | 89 +++++- src/conf/domain_conf.h | 7 + src/qemu/qemu_alias.c | 86 +++-- src/qemu/qemu_alias.h | 3 +- src/qemu/qemu_block.c | 22 ++ src/qemu/qemu_block.h | 2 + src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 94 +++++- src/qemu/qemu_command.h | 3 + src/qemu/qemu_domain.c | 173 ++++++++++- src/qemu/qemu_domain.h | 9 + src/qemu/qemu_driver.c | 244 ++++++++++----- src/qemu/qemu_hotplug.c | 175 +++++++++-- src/qemu/qemu_monitor.c | 108 ++++++- src/qemu/qemu_monitor.h | 36 ++- src/qemu/qemu_monitor_json.c | 346 ++++++++++++++++++--- src/qemu/qemu_monitor_json.h | 33 +- src/qemu/qemu_process.c | 84 ++++- src/util/virqemu.c | 5 +- tests/qemucapabilitiesdata/caps_3.0.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_3.0.0.x86_64.xml | 1 + tests/qemumonitorjsontest.c | 14 +- tests/qemustatusxml2xmldata/modern-in.xml | 6 + tests/qemuxml2argvdata/disk-aio.x86_64-2.12.0.args | 37 +++ tests/qemuxml2argvdata/disk-aio.x86_64-latest.args | 19 +- .../disk-backing-chains-index.x86_64-2.12.0.args | 1 + .../disk-backing-chains-index.x86_64-latest.args | 1 + .../qemuxml2argvdata/disk-backing-chains-index.xml | 145 +++++++++ .../disk-backing-chains-noindex.x86_64-2.12.0.args | 58 ++++ .../disk-backing-chains-noindex.x86_64-latest.args | 163 ++++++++++ .../disk-backing-chains-noindex.xml | 145 +++++++++ .../qemuxml2argvdata/disk-cache.x86_64-2.12.0.args | 50 +++ .../qemuxml2argvdata/disk-cache.x86_64-latest.args | 50 ++- .../disk-cdrom-network.x86_64-2.12.0.args | 41 +++ .../disk-cdrom-network.x86_64-latest.args | 32 +- .../disk-cdrom-tray.x86_64-2.12.0.args | 39 +++ .../disk-cdrom-tray.x86_64-latest.args | 24 +- .../qemuxml2argvdata/disk-cdrom.x86_64-2.12.0.args | 35 +++ .../qemuxml2argvdata/disk-cdrom.x86_64-latest.args | 17 +- .../disk-copy_on_read.x86_64-2.12.0.args | 41 +++ .../disk-copy_on_read.x86_64-latest.args | 19 +- .../disk-detect-zeroes.x86_64-2.12.0.args | 37 +++ .../disk-detect-zeroes.x86_64-latest.args | 17 +- .../disk-error-policy.x86_64-2.12.0.args | 41 +++ .../disk-error-policy.x86_64-latest.args | 30 +- .../disk-floppy.x86_64-2.12.0.args | 35 +++ .../disk-network-gluster.x86_64-2.12.0.args | 44 +++ .../disk-network-gluster.x86_64-latest.args | 32 +- .../disk-network-iscsi.x86_64-2.12.0.args | 63 ++++ .../disk-network-iscsi.x86_64-latest.args | 58 ++-- .../disk-network-nbd.x86_64-2.12.0.args | 46 +++ .../disk-network-nbd.x86_64-latest.args | 41 ++- .../disk-network-rbd.x86_64-2.12.0.args | 61 ++++ .../disk-network-rbd.x86_64-latest.args | 67 ++-- .../disk-network-sheepdog.x86_64-2.12.0.args | 35 +++ .../disk-network-sheepdog.x86_64-latest.args | 16 +- .../disk-network-source-auth.x86_64-2.12.0.args | 47 +++ .../disk-network-source-auth.x86_64-latest.args | 30 +- .../disk-network-tlsx509.x86_64-2.12.0.args | 59 ++++ .../disk-network-tlsx509.x86_64-latest.args | 61 ++-- .../disk-readonly-disk.x86_64-2.12.0.args | 34 ++ .../disk-readonly-disk.x86_64-latest.args | 14 +- .../disk-shared.x86_64-2.12.0.args | 37 +++ .../disk-shared.x86_64-latest.args | 18 +- ...isk-virtio-scsi-reservations.x86_64-2.12.0.args | 43 +++ ...isk-virtio-scsi-reservations.x86_64-latest.args | 20 +- .../floppy-drive-fat.x86_64-2.12.0.args | 33 ++ .../hostdev-mdev-display-missing-graphics.xml | 6 - ...v-display-spice-egl-headless.x86_64-latest.args | 2 - .../hostdev-mdev-display-spice-egl-headless.xml | 6 - ...ev-mdev-display-spice-opengl.x86_64-latest.args | 2 - .../hostdev-mdev-display-spice-opengl.xml | 6 - ...dev-display-vnc-egl-headless.x86_64-latest.args | 2 - .../hostdev-mdev-display-vnc-egl-headless.xml | 6 - .../hostdev-mdev-display-vnc.x86_64-latest.args | 2 - .../qemuxml2argvdata/hostdev-mdev-display-vnc.xml | 6 - tests/qemuxml2argvdata/hostdev-mdev-display.xml | 6 - .../hostdev-mdev-invalid-target-address.xml | 5 - .../qemuxml2argvdata/hostdev-mdev-precreated.args | 2 - tests/qemuxml2argvdata/hostdev-mdev-precreated.xml | 6 - .../hostdev-mdev-src-address-invalid.xml | 6 - tests/qemuxml2argvtest.c | 24 ++ .../disk-backing-chains-inactive.xml | 35 +++ .../disk-backing-chains-index-active.xml | 156 ++++++++++ .../disk-backing-chains-index-inactive.xml | 156 ++++++++++ .../disk-backing-chains-noindex-active.xml | 156 ++++++++++ .../disk-backing-chains-noindex-inactive.xml | 156 ++++++++++ tests/qemuxml2xmloutdata/disk-mirror-inactive.xml | 4 + .../disk-mirror-old-inactive.xml | 4 + tests/qemuxml2xmloutdata/hostdev-mdev-display.xml | 6 - .../qemuxml2xmloutdata/hostdev-mdev-precreated.xml | 6 - tests/qemuxml2xmltest.c | 2 + 95 files changed, 3781 insertions(+), 493 deletions(-) create mode 100644 tests/qemuxml2argvdata/disk-aio.x86_64-2.12.0.args create mode 120000 tests/qemuxml2argvdata/disk-backing-chains-index.x86_64-2.12.0.args create mode 120000 tests/qemuxml2argvdata/disk-backing-chains-index.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-backing-chains-index.xml create mode 100644 tests/qemuxml2argvdata/disk-backing-chains-noindex.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/disk-backing-chains-noindex.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-backing-chains-noindex.xml create mode 100644 tests/qemuxml2argvdata/disk-cache.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/disk-cdrom-network.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/disk-cdrom-tray.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/disk-cdrom.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/disk-copy_on_read.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/disk-detect-zeroes.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/disk-error-policy.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/disk-floppy.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/disk-network-gluster.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/disk-network-iscsi.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/disk-network-nbd.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/disk-network-rbd.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/disk-network-sheepdog.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/disk-network-source-auth.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/disk-network-tlsx509.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/disk-readonly-disk.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/disk-shared.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/disk-virtio-scsi-reservations.x86_64-2.12.0.args create mode 100644 tests/qemuxml2argvdata/floppy-drive-fat.x86_64-2.12.0.args create mode 100644 tests/qemuxml2xmloutdata/disk-backing-chains-index-active.xml create mode 100644 tests/qemuxml2xmloutdata/disk-backing-chains-index-inactive.xml create mode 100644 tests/qemuxml2xmloutdata/disk-backing-chains-noindex-active.xml create mode 100644 tests/qemuxml2xmloutdata/disk-backing-chains-noindex-inactive.xml -- 2.16.2

4 57