- Devel - libvirt List Archives

[libvirt] [PATCH 00/27] qemu: add support for modern vcpu hotplug and unplug
by Peter Krempa 17 Aug '16

17 Aug '16

This adds support for vcpu hotplug and unplug using device_add and device_del. Patches 1 and 2 are already mostly acked. Peter Krempa (27): qemu: monitor: Return structures from qemuMonitorGetCPUInfo qemu: monitor: Return struct from qemuMonitor(Text|Json)QueryCPUs qemu: caps: Add capability for query-hotpluggable-cpus command qemu: Forbid config when topology based cpu count doesn't match the config qemu: capabilities: Extract availability of new cpu hotplug for machine types qemu: monitor: Extract QOM path from query-cpus reply qemu: monitor: Add support for calling query-hotpluggable-cpus qemu: monitor: Add algorithm for combining query-(hotpluggable-)-cpus data tests: Add test infrastructure for qemuMonitorGetCPUInfo tests: cpu-hotplug: Add data for ppc64 platform including hotplug tests: cpu-hotplug: Add data for ppc64 out-of-order hotplug tests: cpu-hotplug: Add data for ppc64 without threads enabled qemu: domain: Extract cpu-hotplug related data qemu: domain: Prepare for VCPUs vanishing while libvirt is not running qemu: setcpus: Report better errors qemu: setcpus: Move NUMA setup code into the block checking live def qemu: setvcpus: Extract setting of maximum vcpu count util: Extract and rename qemuDomainDelCgroupForThread to virCgroupDelThread conf: Add XML for individual vCPU hotplug qemu: migration: Prepare for non-contiguous vcpu configurations qemu: command: Add helper to convert vcpu definition to JSON props qemu: process: Copy final vcpu order information into the vcpu definition qemu: command: Add support for sparse vcpu topologies qemu: Use mondern vcpu hotplug approach if possible qemu: hotplug: Allow marking unplugged devices by alias qemu: hotplug: Add support for VCPU unplug RFC: qemu: hotplug: Add DEVICE_(ADDED|DELETED) events for vcpus docs/formatdomain.html.in | 18 ++ docs/schemas/domaincommon.rng | 25 ++ src/conf/domain_conf.c | 151 ++++++++- src/conf/domain_conf.h | 6 + src/libvirt_private.syms | 1 + src/qemu/qemu_capabilities.c | 31 +- src/qemu/qemu_capabilities.h | 3 + src/qemu/qemu_command.c | 50 ++- src/qemu/qemu_command.h | 3 + src/qemu/qemu_domain.c | 303 ++++++++++++++---- src/qemu/qemu_domain.h | 19 +- src/qemu/qemu_driver.c | 354 +++++++++++++-------- src/qemu/qemu_hotplug.c | 134 +++++++- src/qemu/qemu_hotplug.h | 7 + src/qemu/qemu_migration.c | 16 +- src/qemu/qemu_monitor.c | 260 ++++++++++++++- src/qemu/qemu_monitor.h | 58 +++- src/qemu/qemu_monitor_json.c | 266 +++++++++++++--- src/qemu/qemu_monitor_json.h | 8 +- src/qemu/qemu_monitor_text.c | 39 ++- src/qemu/qemu_monitor_text.h | 3 +- src/qemu/qemu_process.c | 182 ++++++++++- src/util/vircgroup.c | 20 ++ src/util/vircgroup.h | 4 + .../generic-vcpus-individual.xml | 23 ++ tests/genericxml2xmltest.c | 2 + tests/qemucapabilitiesdata/caps_2.7.0.x86_64.xml | 55 ++-- .../qemumonitorjson-cpuinfo-ppc64-basic-cpus.json | 77 +++++ ...emumonitorjson-cpuinfo-ppc64-basic-hotplug.json | 27 ++ .../qemumonitorjson-cpuinfo-ppc64-basic.data | 40 +++ ...mumonitorjson-cpuinfo-ppc64-hotplug-1-cpus.json | 149 +++++++++ ...onitorjson-cpuinfo-ppc64-hotplug-1-hotplug.json | 28 ++ .../qemumonitorjson-cpuinfo-ppc64-hotplug-1.data | 51 +++ ...mumonitorjson-cpuinfo-ppc64-hotplug-2-cpus.json | 221 +++++++++++++ ...onitorjson-cpuinfo-ppc64-hotplug-2-hotplug.json | 29 ++ .../qemumonitorjson-cpuinfo-ppc64-hotplug-2.data | 62 ++++ ...mumonitorjson-cpuinfo-ppc64-hotplug-3-cpus.json | 149 +++++++++ ...onitorjson-cpuinfo-ppc64-hotplug-3-hotplug.json | 28 ++ .../qemumonitorjson-cpuinfo-ppc64-hotplug-3.data | 51 +++ ...mumonitorjson-cpuinfo-ppc64-hotplug-4-cpus.json | 221 +++++++++++++ ...onitorjson-cpuinfo-ppc64-hotplug-4-hotplug.json | 29 ++ .../qemumonitorjson-cpuinfo-ppc64-hotplug-4.data | 62 ++++ ...umonitorjson-cpuinfo-ppc64-no-threads-cpus.json | 77 +++++ ...nitorjson-cpuinfo-ppc64-no-threads-hotplug.json | 125 ++++++++ .../qemumonitorjson-cpuinfo-ppc64-no-threads.data | 72 +++++ ...nitorjson-cpuinfo-x86-basic-pluggable-cpus.json | 50 +++ ...orjson-cpuinfo-x86-basic-pluggable-hotplug.json | 82 +++++ ...emumonitorjson-cpuinfo-x86-basic-pluggable.data | 39 +++ tests/qemumonitorjsontest.c | 183 ++++++++++- .../qemuxml2argv-cpu-hotplug-startup.args | 20 ++ .../qemuxml2argv-cpu-hotplug-startup.xml | 29 ++ tests/qemuxml2argvtest.c | 2 + tests/testutils.c | 4 +- 53 files changed, 3638 insertions(+), 310 deletions(-) create mode 100644 tests/genericxml2xmlindata/generic-vcpus-individual.xml create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-basic-cpus.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-basic-hotplug.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-basic.data create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-1-cpus.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-1-hotplug.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-1.data create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-2-cpus.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-2-hotplug.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-2.data create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-3-cpus.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-3-hotplug.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-3.data create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-4-cpus.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-4-hotplug.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-4.data create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-no-threads-cpus.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-no-threads-hotplug.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-no-threads.data create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-x86-basic-pluggable-cpus.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-x86-basic-pluggable-hotplug.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-x86-basic-pluggable.data create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-cpu-hotplug-startup.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-cpu-hotplug-startup.xml -- 2.9.2

4 53

[libvirt] [PATCH] qemu: Fix the command line generation for rbd auth using aes secrets
by John Ferlan 17 Aug '16

17 Aug '16

https://bugzilla.redhat.com/show_bug.cgi?id=1182074 Since libvirt still uses a legacy qemu arg format to add a disk, the manner in which the 'password-secret' argument is passed to qemu needs to change to prepend a 'file.' If in the future, usage of the more modern disk format, then the prepended 'file.' can be removed. Fix based on Jim Fehlig <jfehlig(a)suse.com> posting and subsequent upstream list followups, see: http://www.redhat.com/archives/libvir-list/2016-August/msg00777.html for details. Signed-off-by: John Ferlan <jferlan(a)redhat.com> --- Jim - if you want me to change the authorship, just let me know. NB: Rather than waiting for a new bz, I've reused the bz that was used for the original patches/support. src/qemu/qemu_command.c | 7 ++++++- .../qemuxml2argv-disk-drive-network-rbd-auth-AES.args | 2 +- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index ebedaef..a6dea6a 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -1287,7 +1287,12 @@ qemuBuildDriveSourceStr(virDomainDiskDefPtr disk, virBufferAddLit(buf, ","); if (secinfo && secinfo->type == VIR_DOMAIN_SECRET_INFO_TYPE_AES) { - virBufferAsprintf(buf, "password-secret=%s,", + /* NB: If libvirt starts using the more modern option based + * syntax to build the command line (e.g., "-drive driver=rbd, + * filename=%s,...") instead of the legacy model (e.g."-drive + * file=%s,..."), then the "file." prefix can be removed + */ + virBufferAsprintf(buf, "file.password-secret=%s,", secinfo->s.aes.alias); } diff --git a/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-rbd-auth-AES.args b/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-rbd-auth-AES.args index 5034bb7..07d01b6 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-rbd-auth-AES.args +++ b/tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-rbd-auth-AES.args @@ -26,7 +26,7 @@ data=9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1,\ keyid=masterKey0,iv=AAECAwQFBgcICQoLDA0ODw==,format=base64 \ -drive 'file=rbd:pool/image:id=myname:auth_supported=cephx\;none:\ mon_host=mon1.example.org\:6321\;mon2.example.org\:6322\;mon3.example.org\:\ -6322,password-secret=virtio-disk0-secret0,format=raw,if=none,\ +6322,file.password-secret=virtio-disk0-secret0,format=raw,if=none,\ id=drive-virtio-disk0' \ -device virtio-blk-pci,bus=pci.0,addr=0x3,drive=drive-virtio-disk0,\ id=virtio-disk0 -- 2.7.4

2 2

[libvirt] [PATCH] qemu_driver: update comments for qemuDomainSaveInternal
by Chen Hanxiao 17 Aug '16

17 Aug '16

From: Chen Hanxiao <chenhanxiao(a)gmail.com> s/libvirt.c/libvirt-domain.c Signed-off-by: Chen Hanxiao <chenhanxiao(a)gmail.com> --- src/qemu/qemu_driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 5a7733c..54e2770 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -3177,7 +3177,7 @@ qemuDomainSaveInternal(virQEMUDriverPtr driver, virDomainPtr dom, } } - /* libvirt.c already guaranteed these two flags are exclusive. */ + /* libvirt-domain.c already guaranteed these two flags are exclusive. */ if (flags & VIR_DOMAIN_SAVE_RUNNING) was_running = true; else if (flags & VIR_DOMAIN_SAVE_PAUSED) -- 1.8.3.1

2 1

[libvirt] [PATCH python 0/2] Python binding for node device update event API
by Jovanka Gulicoska 17 Aug '16

17 Aug '16

Jovanka Gulicoska (2): Add support for node device update callback event-test: support node device update callback examples/event-test.py | 4 ++++ libvirt-override-virConnect.py | 10 ++++++++ libvirt-override.c | 54 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 68 insertions(+) -- 2.7.4

2 3

[libvirt] [PATCH v3 1/4] libxl: support serial list
by Bob Liu 17 Aug '16

17 Aug '16

Add support for multi serial devices, after this patch virsh can be used to connect different serial devices of running domains. E.g. vish # console <xxx> --devname serial<xxx> Note: This depends on a xen/libxl bug fix to have libxl_console_get_tty(...) correctly returning the tty path (as opposed to always returning the first one). [0] https://lists.xen.org/archives/html/xen-devel/2016-08/msg00438.html Signed-off-by: Bob Liu <bob.liu(a)oracle.com> --- v3: Comments from Jim. v2: Add #ifdef LIBXL_HAVE_BUILDINFO_SERIAL_LIST. --- src/libxl/libxl_conf.c | 23 ++++++++++++++++++++--- src/libxl/libxl_domain.c | 29 ++++++++++++++++++++++++++--- src/libxl/libxl_driver.c | 17 +++++++++-------- 3 files changed, 55 insertions(+), 14 deletions(-) diff --git a/src/libxl/libxl_conf.c b/src/libxl/libxl_conf.c index 146e08a..32db975 100644 --- a/src/libxl/libxl_conf.c +++ b/src/libxl/libxl_conf.c @@ -431,14 +431,31 @@ libxlMakeDomBuildInfo(virDomainDefPtr def, } if (def->nserials) { - if (def->nserials > 1) { + if (def->nserials == 1) { + if (libxlMakeChrdevStr(def->serials[0], &b_info->u.hvm.serial) < + 0) + return -1; + } else { +#ifdef LIBXL_HAVE_BUILDINFO_SERIAL_LIST + if (VIR_ALLOC_N(b_info->u.hvm.serial_list, def->nserials + 1) < + 0) + return -1; + for (i = 0; i < def->nserials; i++) { + if (libxlMakeChrdevStr(def->serials[i], + &b_info->u.hvm.serial_list[i]) < 0) + { + libxl_string_list_dispose(&b_info->u.hvm.serial_list); + return -1; + } + } + b_info->u.hvm.serial_list[i] = NULL; +#else virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("Only one serial device is supported by libxl")); return -1; +#endif } - if (libxlMakeChrdevStr(def->serials[0], &b_info->u.hvm.serial) < 0) - return -1; } if (def->nparallels) { diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c index 0e26b91..f529a2e 100644 --- a/src/libxl/libxl_domain.c +++ b/src/libxl/libxl_domain.c @@ -960,18 +960,20 @@ libxlConsoleCallback(libxl_ctx *ctx, libxl_event *ev, void *for_callback) { virDomainObjPtr vm = for_callback; size_t i; + virDomainChrDefPtr chr; + char *console = NULL; + int ret; virObjectLock(vm); for (i = 0; i < vm->def->nconsoles; i++) { - virDomainChrDefPtr chr = vm->def->consoles[i]; + chr = vm->def->consoles[i]; + if (i == 0 && chr->targetType == VIR_DOMAIN_CHR_CONSOLE_TARGET_TYPE_SERIAL) chr = vm->def->serials[0]; if (chr->source.type == VIR_DOMAIN_CHR_TYPE_PTY) { libxl_console_type console_type; - char *console = NULL; - int ret; console_type = (chr->deviceType == VIR_DOMAIN_CHR_DEVICE_TYPE_SERIAL ? @@ -989,6 +991,27 @@ libxlConsoleCallback(libxl_ctx *ctx, libxl_event *ev, void *for_callback) VIR_FREE(console); } } + for (i = 0; i < vm->def->nserials; i++) { + chr = vm->def->serials[i]; + + ignore_value(virAsprintf(&chr->info.alias, "serial%zd", i)); + if (chr->source.type == VIR_DOMAIN_CHR_TYPE_PTY) { + if (chr->source.data.file.path) + continue; + ret = libxl_console_get_tty(ctx, ev->domid, + chr->target.port, + LIBXL_CONSOLE_TYPE_SERIAL, + &console); + if (!ret) { + VIR_FREE(chr->source.data.file.path); + if (console && console[0] != '\0') { + ignore_value(VIR_STRDUP(chr->source.data.file.path, + console)); + } + } + VIR_FREE(console); + } + } virObjectUnlock(vm); libxl_event_free(ctx, ev); } diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c index f153f69..a34eb02 100644 --- a/src/libxl/libxl_driver.c +++ b/src/libxl/libxl_driver.c @@ -4450,13 +4450,6 @@ libxlDomainOpenConsole(virDomainPtr dom, virCheckFlags(VIR_DOMAIN_CONSOLE_FORCE, -1); - if (dev_name) { - /* XXX support device aliases in future */ - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("Named device aliases are not supported")); - goto cleanup; - } - if (!(vm = libxlDomObjFromDomain(dom))) goto cleanup; @@ -4472,8 +4465,16 @@ libxlDomainOpenConsole(virDomainPtr dom, } priv = vm->privateData; + if (dev_name) { + size_t i; - if (vm->def->nconsoles) { + for (i = 0; !chr && i < vm->def->nserials; i++) { + if (STREQ(dev_name, vm->def->serials[i]->info.alias)) { + chr = vm->def->serials[i]; + break; + } + } + } else if (vm->def->nconsoles) { chr = vm->def->consoles[0]; if (chr->targetType == VIR_DOMAIN_CHR_CONSOLE_TARGET_TYPE_SERIAL) chr = vm->def->serials[0]; -- 2.6.5

2 5

[libvirt] [PATCH 0/5] graphics cleanup and bug fix
by Pavel Hrdina 17 Aug '16

17 Aug '16

Pavel Hrdina (5): qemu_process: graphics: ref driver config only in function where it is used qemu_process: graphics: extract port allocation into function qemu_process: graphics: extract for loop out of qemuProcessGraphicsReservePorts qemu_process: graphics: reserve port only if listen type is address or network qemu_process: graphics: setup listen types before ports are reserved/allocated src/qemu/qemu_process.c | 174 ++++++++++++++++++++++++++++++------------------ 1 file changed, 108 insertions(+), 66 deletions(-) -- 2.9.2

2 10

[libvirt] [PATCH v2] conf: Provide error on undefined vcpusched entry
by John Ferlan 17 Aug '16

17 Aug '16

Modify virDomainDefGetVcpuSched to emit an error message if virDomainDefGetVcpu returns NULL meaning the vcpu could not be found. Prior to commit id '9cc931f0b' the error message would have been issued in virDomainDefGetVcpu. Signed-off-by: John Ferlan <jferlan(a)redhat.com> --- v1: http://www.redhat.com/archives/libvir-list/2016-August/msg00754.html Wasn't fully clear from the v1 discussion whether my proposal for vcpusched was acceptible or not, so I figured I'd generate a v2 and of course the example... Using a domain without <vcpusched> in a <cputune>, if there's: <vcpu placement='static' current='2'>4</vcpu> in the domain XML and one uses virsh edit to add: <vcpusched vcpus='7-8' scheduler='idle'/> then the generic error message is displayed. This only occurs as of v2.1 where the message was removed from virDomainDefGetVcpu. src/conf/domain_conf.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 9037304..14d4f7d 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -1480,8 +1480,12 @@ virDomainDefGetVcpuSched(virDomainDefPtr def, { virDomainVcpuDefPtr vcpuinfo; - if (!(vcpuinfo = virDomainDefGetVcpu(def, vcpu))) + if (!(vcpuinfo = virDomainDefGetVcpu(def, vcpu))) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("vCPU '%u' is not present in domain definition"), + vcpu); return NULL; + } return &vcpuinfo->sched; } -- 2.7.4

2 1

[libvirt] qemu: Bug: NUMA memory policy ignored in some situations
by Sam Bobroff 17 Aug '16

17 Aug '16

Hi all, I've found that if I create a QEMU guest that: (a) is backed by host hugepages and (b) specifies a host NUMA memory placement policy ... then the NUMA policy (b) is ignored. It's even possible to start the guest if the policy specifies that memory must come from non-existant host nodes. Is this a bug? (I ask because I suspect that it is a bug, and that it can't be fixed trivially due to migration considerations. Is that true?) A few more details: I do (a) using this in the guest XML: <memoryBacking> <hugepages/> </memoryBacking> And (b) using: <numatune> <memory mode='strict' nodeset='1'> </numatune> If the XML contains <cpu><numa><cell ...> and <numatune><memnode ...> to bind guest NUMA nodes to host NUMA nodes then the policy is followed and this behaviour isn't seen. The guest can't be started if the policy specifies invalid host nodes. (Strangely, if the host doesn't have the cpuset cgroup mount set up or otherwise lacks cgroup support, then the policy is not ignored. This appears to be due to a fallback path in libvirt so it seems like NUMA memory policy is intended to work in this situation.) Cheers, Sam.

1 0

[libvirt] Libvirt gitstats
by Michal Privoznik 17 Aug '16

17 Aug '16

Dear list, I've set up a small script running on my Raspberry that produces following stats: https://mprivozn.fedorapeople.org/libvirt.gitstats/index.html It's polling git every 3 hours (which is about time it takes to generate the stats there :D). Don't expect interactive stats regeneration, but some graphs are interesting. Michal

2 1

[libvirt] [PATCH 0/2] Restore error messages for {iothread|vcpu}sched id out of range.
by John Ferlan 17 Aug '16

17 Aug '16

These patches will restore error messages indicating the vcpu or iothread range(s) for vcpusched or iothreadsched are invalid rather than just quietly ignoring. If someone for some unknown reason had hand edited their domain prior to starting libvirt, they'll at least get a message indicating that "some" domain has an invaliv vcpu/iothread value. NB: Removal of error message occurred in the following releases: iothreadsched: Removed in v1.3.2 vcpusched: Removed in V2.1 John Ferlan (2): conf: Provide error on undefined iothreadsched entry conf: Provide error on undefined vcpusched entry src/conf/domain_conf.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) -- 2.7.4

2 6