[libvirt] [PATCHv3 00/12] Add support for memory hotplug

Peter Krempa

17 Mar 2015 17 Mar '15

10:19 a.m.

This version includes review feedback changes from John and also fixes the <memory> element documentation and code that calculates it to support possible non-NUMA configs with memory hotplug if any hypervisor would support that in the future. Peter Krempa (12): qemu: monitor: Don't leak @props with non-JSON in qemuMonitorAddObject libxl: Refactor logic in domain post parse callback conf: Add support for parsing and formatting max memory and slot count qemu: Implement setup of memory hotplug parameters conf: Add device address type for dimm devices conf: Add interface to parse and format memory device information qemu: memdev: Add infrastructure to load memory device information qemu: migration: Forbid migration with memory modules lacking info qemu: add support for memory devices qemu: conf: Add support for memory device cold(un)plug qemu: Implement memory device hotplug qemu: Implement memory device hotunplug docs/formatdomain.html.in | 112 +++- docs/schemas/domaincommon.rng | 76 +++ src/bhyve/bhyve_domain.c | 9 +- src/conf/domain_conf.c | 576 ++++++++++++++++++++- src/conf/domain_conf.h | 59 +++ src/libvirt_private.syms | 7 + src/libxl/libxl_domain.c | 15 +- src/lxc/lxc_domain.c | 8 + src/openvz/openvz_driver.c | 14 +- src/parallels/parallels_driver.c | 6 +- src/phyp/phyp_driver.c | 6 +- src/qemu/qemu_command.c | 166 +++++- src/qemu/qemu_command.h | 15 + src/qemu/qemu_domain.c | 80 +++ src/qemu/qemu_domain.h | 5 + src/qemu/qemu_driver.c | 29 ++ src/qemu/qemu_hotplug.c | 187 +++++++ src/qemu/qemu_hotplug.h | 6 + src/qemu/qemu_migration.c | 14 + src/qemu/qemu_monitor.c | 48 +- src/qemu/qemu_monitor.h | 14 + src/qemu/qemu_monitor_json.c | 122 +++++ src/qemu/qemu_monitor_json.h | 5 + src/qemu/qemu_process.c | 4 + src/uml/uml_driver.c | 9 +- src/vbox/vbox_common.c | 6 +- src/vmware/vmware_driver.c | 6 +- src/vmx/vmx.c | 6 +- src/xen/xen_driver.c | 7 + src/xenapi/xenapi_driver.c | 9 +- tests/domainschemadata/maxMemory.xml | 19 + .../qemuxml2argv-memory-hotplug-dimm.args | 11 + .../qemuxml2argv-memory-hotplug-dimm.xml | 50 ++ .../qemuxml2argv-memory-hotplug-nonuma.xml | 22 + .../qemuxml2argv-memory-hotplug.args | 6 + .../qemuxml2argv-memory-hotplug.xml | 34 ++ tests/qemuxml2argvtest.c | 6 + tests/qemuxml2xmltest.c | 4 + 38 files changed, 1747 insertions(+), 31 deletions(-) create mode 100644 tests/domainschemadata/maxMemory.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-dimm.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-dimm.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-nonuma.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug.xml -- 2.2.2

Show replies by date

Peter Krempa

17 Mar 17 Mar

10:19 a.m.

New subject: [libvirt] [PATCHv3 01/12] qemu: monitor: Don't leak @props with non-JSON in qemuMonitorAddObject

The function comment states that @props is always consumed, even on failure. This was not true with the failure if the monitor is not using QMP. --- This patch is new in the series. src/qemu/qemu_monitor.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index 18f866f..9b86695 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -3245,11 +3245,13 @@ qemuMonitorAddObject(qemuMonitorPtr mon, mon, type, objalias, props); int ret = -1; - if (mon->json) + if (mon->json) { ret = qemuMonitorJSONAddObject(mon, type, objalias, props); - else + } else { + virJSONValueFree(props); virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", _("object adding requires JSON monitor")); + } return ret; } -- 2.2.2

Peter Krempa

10:19 a.m.

New subject: [libvirt] [PATCHv3 02/12] libxl: Refactor logic in domain post parse callback

With the current control flow the post parse callback returned success right away for fully virtualized VMs. To allow adding additional checks into the post parse callback tweak the conditions so that the function doesn't return early except for error cases. To clarify the original piece of code borrow the wording from the commit message for the patch that introduced the code. --- This patch is new in the series. src/libxl/libxl_domain.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c index 407a9bd..611ccf4 100644 --- a/src/libxl/libxl_domain.c +++ b/src/libxl/libxl_domain.c @@ -546,10 +546,9 @@ libxlDomainDefPostParse(virDomainDefPtr def, virCapsPtr caps ATTRIBUTE_UNUSED, void *opaque ATTRIBUTE_UNUSED) { - if (STREQ(def->os.type, "hvm")) - return 0; - - if (def->nconsoles == 0) { + /* Xen PV domains always have a PV console, so add one to the domain config + * via post-parse callback if not explicitly specified in the XML. */ + if (STRNEQ(def->os.type, "hvm") && def->nconsoles == 0) { virDomainChrDefPtr chrdef; if (!(chrdef = virDomainChrDefNew())) -- 2.2.2

Peter Krempa

10:19 a.m.

New subject: [libvirt] [PATCHv3 03/12] conf: Add support for parsing and formatting max memory and slot count

Add a XML element that will allow to specify maximum supportable memory and the count of memory slots to use with memory hotplug. To avoid possible confusion and misuse of the new element this patch also explicitly forbids the use of the maxMemory setting in individual drivers's post parse callbacks. This limitation will be lifted when the support is implemented. --- Notes: Version 3: - tweaked docs according to suggestions Version 2: - changed version in docs to 1.2.14 - fixed typo in code and in commit message - simplified condition when formatting the <maxMemory> element - rebased on top of the changes to the refactor series Version 2: - changed version in docs to 1.2.14 - fixed typo in code and in commit message - simplified condition when formatting the <maxMemory> element - rebased on top of the changes to the refactor series docs/formatdomain.html.in | 20 +++++++++++ docs/schemas/domaincommon.rng | 8 +++++ src/bhyve/bhyve_domain.c | 4 +++ src/conf/domain_conf.c | 66 ++++++++++++++++++++++++++++++++++++ src/conf/domain_conf.h | 7 ++++ src/libvirt_private.syms | 1 + src/libxl/libxl_domain.c | 5 +++ src/lxc/lxc_domain.c | 4 +++ src/openvz/openvz_driver.c | 11 ++++-- src/parallels/parallels_driver.c | 6 +++- src/phyp/phyp_driver.c | 6 +++- src/qemu/qemu_domain.c | 4 +++ src/uml/uml_driver.c | 6 +++- src/vbox/vbox_common.c | 6 +++- src/vmware/vmware_driver.c | 6 +++- src/vmx/vmx.c | 6 +++- src/xen/xen_driver.c | 4 +++ src/xenapi/xenapi_driver.c | 6 +++- tests/domainschemadata/maxMemory.xml | 19 +++++++++++ 19 files changed, 186 insertions(+), 9 deletions(-) create mode 100644 tests/domainschemadata/maxMemory.xml diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 8d98915..57caef0 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -675,6 +675,7 @@ <pre> <domain> ... + <maxMemory slots='16' unit='KiB'>1524288</maxMemory> <memory unit='KiB'>524288</memory> <currentMemory unit='KiB'>524288</currentMemory> ... @@ -708,6 +709,25 @@ <code>unit</code> since 0.9.11, <code>dumpCore</code> since 0.10.2 (QEMU only)</dd> + <dt><code>maxMemory</code></dt> + <dd>The run time maximum memory allocation of the guest. The initial + memory specified by either the <code><memory></code> element or + the NUMA cell size configuration can be increased by hot-plugging of + memory to the limit specified by this element. + + The <code>unit</code> attribute behaves the same as for + <code><memory></code>. + + The <code>slots</code> attribute specifies the number of slots + available for adding memory to the guest. The bounds are hypervisor + specific. + + Note that due to alignment of the memory chunks added via memory + hotplug the full size allocation specified by this element may be + impossible to achieve. + Since 1.2.14 + </dd> + <dt><code>currentMemory</code></dt> <dd>The actual allocation of memory for the guest. This value can be less than the maximum allocation, to allow for ballooning diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index ebd9299..a0dee17 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -593,6 +593,14 @@ </element> </optional> <optional> + <element name="maxMemory"> + <ref name="scaledInteger"/> + <attribute name="slots"> + <ref name="unsignedInt"/> + </attribute> + </element> + </optional> + <optional> <element name="currentMemory"> <ref name='scaledInteger'/> </element> diff --git a/src/bhyve/bhyve_domain.c b/src/bhyve/bhyve_domain.c index ecb1758..25ef852 100644 --- a/src/bhyve/bhyve_domain.c +++ b/src/bhyve/bhyve_domain.c @@ -67,6 +67,10 @@ bhyveDomainDefPostParse(virDomainDefPtr def, VIR_DOMAIN_CONTROLLER_MODEL_PCI_ROOT) < 0) return -1; + /* memory hotplug tunables are not supported by this driver */ + if (virDomainDefCheckUnsupportedMemoryHotplug(def) < 0) + return -1; + return 0; } diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index c75b543..dc15d95 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -980,6 +980,27 @@ virDomainBlkioDeviceParseXML(xmlNodePtr root, } +/** + * virDomainDefCheckUnsupportedMemoryHotplug: + * @def: domain definition + * + * Returns -1 if the domain definition would enable memory hotplug via the + * <maxMemory> tunable and reports an error. Otherwise returns 0. + */ +int +virDomainDefCheckUnsupportedMemoryHotplug(virDomainDefPtr def) +{ + /* memory hotplug tunables are not supported by this driver */ + if (def->mem.max_memory > 0 || def->mem.memory_slots > 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("memory hotplug tunables <maxMemory> are not " + "supported by this hypervisor driver")); + return -1; + } + + return 0; +} + static void virDomainObjListDataFree(void *payload, const void *name ATTRIBUTE_UNUSED) @@ -3237,6 +3258,22 @@ virDomainDefPostParseInternal(virDomainDefPtr def, def->mem.cur_balloon = virDomainDefGetMemoryActual(def); } + if ((def->mem.max_memory || def->mem.memory_slots) && + !(def->mem.max_memory && def->mem.memory_slots)) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("both maximum memory size and " + "memory slot count must be specified")); + return -1; + } + + if (def->mem.max_memory && + def->mem.max_memory < virDomainDefGetMemoryActual(def)) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("maximum memory size must be equal or greater than " + "the actual memory size")); + return -1; + } + /* * Some really crazy backcompat stuff for consoles * @@ -13263,6 +13300,16 @@ virDomainDefParseXML(xmlDocPtr xml, &def->mem.cur_balloon, false, true) < 0) goto error; + if (virDomainParseMemory("./maxMemory[1]", NULL, ctxt, + &def->mem.max_memory, false, false) < 0) + goto error; + + if (virXPathUInt("string(./maxMemory[1]/@slots)", ctxt, &def->mem.memory_slots) == -2) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("Failed to parse memory slot count")); + goto error; + } + /* and info about it */ if ((tmp = virXPathString("string(./memory[1]/@dumpCore)", ctxt)) && (def->mem.dump_core = virTristateSwitchTypeFromString(tmp)) <= 0) { @@ -16189,6 +16236,19 @@ virDomainDefCheckABIStability(virDomainDefPtr src, if (!virDomainNumaCheckABIStability(src->numa, dst->numa)) goto error; + if (src->mem.memory_slots != dst->mem.memory_slots) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target domain memory slots count '%u' doesn't match source '%u"), + dst->mem.memory_slots, src->mem.memory_slots); + goto error; + } + if (src->mem.max_memory != dst->mem.max_memory) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target maximum memory size '%llu' doesn't match source '%llu'"), + dst->mem.max_memory, src->mem.max_memory); + goto error; + } + if (src->vcpus != dst->vcpus) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("Target domain vCPU count %d does not match source %d"), @@ -19878,6 +19938,12 @@ virDomainDefFormatInternal(virDomainDefPtr def, xmlIndentTreeOutput = oldIndentTreeOutput; } + if (def->mem.max_memory) { + virBufferAsprintf(buf, + "<maxMemory slots='%u' unit='KiB'>%llu</maxMemory>\n", + def->mem.memory_slots, def->mem.max_memory); + } + virBufferAddLit(buf, "<memory"); if (def->mem.dump_core) virBufferAsprintf(buf, " dumpCore='%s'", diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 4eb7742..469fc0c 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -2042,6 +2042,10 @@ struct _virDomainMemtune { virDomainHugePagePtr hugepages; size_t nhugepages; + /* maximum supported memory for a guest, for hotplugging */ + unsigned long long max_memory; /* in kibibytes */ + unsigned int memory_slots; /* maximum count of RAM memory slots */ + bool nosharepages; bool locked; int dump_core; /* enum virTristateSwitch */ @@ -2337,6 +2341,9 @@ virDomainObjPtr virDomainObjListFindByName(virDomainObjListPtr doms, bool virDomainObjTaint(virDomainObjPtr obj, virDomainTaintFlags taint); + +int virDomainDefCheckUnsupportedMemoryHotplug(virDomainDefPtr def); + void virDomainPanicDefFree(virDomainPanicDefPtr panic); void virDomainResourceDefFree(virDomainResourceDefPtr resource); void virDomainGraphicsDefFree(virDomainGraphicsDefPtr def); diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 1fb42ac..0988bc3 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -187,6 +187,7 @@ virDomainCpuPlacementModeTypeFromString; virDomainCpuPlacementModeTypeToString; virDomainDefAddImplicitControllers; virDomainDefCheckABIStability; +virDomainDefCheckUnsupportedMemoryHotplug; virDomainDefClearCCWAddresses; virDomainDefClearDeviceAliases; virDomainDefClearPCIAddresses; diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c index 611ccf4..3843ae0 100644 --- a/src/libxl/libxl_domain.c +++ b/src/libxl/libxl_domain.c @@ -567,6 +567,11 @@ libxlDomainDefPostParse(virDomainDefPtr def, def->nconsoles = 1; def->consoles[0] = chrdef; } + + /* memory hotplug tunables are not supported by this driver */ + if (virDomainDefCheckUnsupportedMemoryHotplug(def) < 0) + return -1; + return 0; } diff --git a/src/lxc/lxc_domain.c b/src/lxc/lxc_domain.c index 55f5b49..1367b0c 100644 --- a/src/lxc/lxc_domain.c +++ b/src/lxc/lxc_domain.c @@ -94,6 +94,10 @@ virLXCDomainDefPostParse(virDomainDefPtr def, !(def->emulator = virDomainDefGetDefaultEmulator(def, caps))) return -1; + /* memory hotplug tunables are not supported by this driver */ + if (virDomainDefCheckUnsupportedMemoryHotplug(def) < 0) + return -1; + return 0; } diff --git a/src/openvz/openvz_driver.c b/src/openvz/openvz_driver.c index 71b0471..055670a 100644 --- a/src/openvz/openvz_driver.c +++ b/src/openvz/openvz_driver.c @@ -96,8 +96,15 @@ openvzDomainDefPostParse(virDomainDefPtr def, void *opaque ATTRIBUTE_UNUSED) { /* fill the init path */ - if (STREQ(def->os.type, "exe") && !def->os.init) - return VIR_STRDUP(def->os.init, "/sbin/init") < 0 ? -1 : 0; + if (STREQ(def->os.type, "exe") && !def->os.init) { + if (VIR_STRDUP(def->os.init, "/sbin/init") < 0) + return -1; + } + + /* memory hotplug tunables are not supported by this driver */ + if (virDomainDefCheckUnsupportedMemoryHotplug(def) < 0) + return -1; + return 0; } diff --git a/src/parallels/parallels_driver.c b/src/parallels/parallels_driver.c index bf29a96..62f17aa 100644 --- a/src/parallels/parallels_driver.c +++ b/src/parallels/parallels_driver.c @@ -161,10 +161,14 @@ parallelsConnectGetCapabilities(virConnectPtr conn) } static int -parallelsDomainDefPostParse(virDomainDefPtr def ATTRIBUTE_UNUSED, +parallelsDomainDefPostParse(virDomainDefPtr def, virCapsPtr caps ATTRIBUTE_UNUSED, void *opaque ATTRIBUTE_UNUSED) { + /* memory hotplug tunables are not supported by this driver */ + if (virDomainDefCheckUnsupportedMemoryHotplug(def) < 0) + return -1; + return 0; } diff --git a/src/phyp/phyp_driver.c b/src/phyp/phyp_driver.c index 60a47ad..f4db2e0 100644 --- a/src/phyp/phyp_driver.c +++ b/src/phyp/phyp_driver.c @@ -1094,10 +1094,14 @@ openSSHSession(virConnectPtr conn, virConnectAuthPtr auth, static int -phypDomainDefPostParse(virDomainDefPtr def ATTRIBUTE_UNUSED, +phypDomainDefPostParse(virDomainDefPtr def, virCapsPtr caps ATTRIBUTE_UNUSED, void *opaque ATTRIBUTE_UNUSED) { + /* memory hotplug tunables are not supported by this driver */ + if (virDomainDefCheckUnsupportedMemoryHotplug(def) < 0) + return -1; + return 0; } diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 2eacef2..6484bb5 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -1052,6 +1052,10 @@ qemuDomainDefPostParse(virDomainDefPtr def, VIR_DOMAIN_INPUT_BUS_USB) < 0) return -1; + /* memory hotplug tunables are not supported by this driver */ + if (virDomainDefCheckUnsupportedMemoryHotplug(def) < 0) + return -1; + return 0; } diff --git a/src/uml/uml_driver.c b/src/uml/uml_driver.c index 27731f2..bdfc12e 100644 --- a/src/uml/uml_driver.c +++ b/src/uml/uml_driver.c @@ -444,10 +444,14 @@ umlDomainDeviceDefPostParse(virDomainDeviceDefPtr dev, static int -umlDomainDefPostParse(virDomainDefPtr def ATTRIBUTE_UNUSED, +umlDomainDefPostParse(virDomainDefPtr def, virCapsPtr caps ATTRIBUTE_UNUSED, void *opaque ATTRIBUTE_UNUSED) { + /* memory hotplug tunables are not supported by this driver */ + if (virDomainDefCheckUnsupportedMemoryHotplug(def) < 0) + return -1; + return 0; } diff --git a/src/vbox/vbox_common.c b/src/vbox/vbox_common.c index 0fb53aa..cad77f7 100644 --- a/src/vbox/vbox_common.c +++ b/src/vbox/vbox_common.c @@ -249,10 +249,14 @@ static char *vboxGenerateMediumName(PRUint32 storageBus, } static int -vboxDomainDefPostParse(virDomainDefPtr def ATTRIBUTE_UNUSED, +vboxDomainDefPostParse(virDomainDefPtr def, virCapsPtr caps ATTRIBUTE_UNUSED, void *opaque ATTRIBUTE_UNUSED) { + /* memory hotplug tunables are not supported by this driver */ + if (virDomainDefCheckUnsupportedMemoryHotplug(def) < 0) + return -1; + return 0; } diff --git a/src/vmware/vmware_driver.c b/src/vmware/vmware_driver.c index 36f992b..3382994 100644 --- a/src/vmware/vmware_driver.c +++ b/src/vmware/vmware_driver.c @@ -83,10 +83,14 @@ vmwareDataFreeFunc(void *data) } static int -vmwareDomainDefPostParse(virDomainDefPtr def ATTRIBUTE_UNUSED, +vmwareDomainDefPostParse(virDomainDefPtr def, virCapsPtr caps ATTRIBUTE_UNUSED, void *opaque ATTRIBUTE_UNUSED) { + /* memory hotplug tunables are not supported by this driver */ + if (virDomainDefCheckUnsupportedMemoryHotplug(def) < 0) + return -1; + return 0; } diff --git a/src/vmx/vmx.c b/src/vmx/vmx.c index 8cbf4d8..8b81436 100644 --- a/src/vmx/vmx.c +++ b/src/vmx/vmx.c @@ -524,10 +524,14 @@ VIR_ENUM_IMPL(virVMXControllerModelSCSI, VIR_DOMAIN_CONTROLLER_MODEL_SCSI_LAST, * Helpers */ static int -vmxDomainDefPostParse(virDomainDefPtr def ATTRIBUTE_UNUSED, +vmxDomainDefPostParse(virDomainDefPtr def, virCapsPtr caps ATTRIBUTE_UNUSED, void *opaque ATTRIBUTE_UNUSED) { + /* memory hotplug tunables are not supported by this driver */ + if (virDomainDefCheckUnsupportedMemoryHotplug(def) < 0) + return -1; + return 0; } diff --git a/src/xen/xen_driver.c b/src/xen/xen_driver.c index 7577881..b104243 100644 --- a/src/xen/xen_driver.c +++ b/src/xen/xen_driver.c @@ -388,6 +388,10 @@ xenDomainDefPostParse(virDomainDefPtr def, def->memballoon = memballoon; } + /* memory hotplug tunables are not supported by this driver */ + if (virDomainDefCheckUnsupportedMemoryHotplug(def) < 0) + return -1; + return 0; } diff --git a/src/xenapi/xenapi_driver.c b/src/xenapi/xenapi_driver.c index affc153..3134441 100644 --- a/src/xenapi/xenapi_driver.c +++ b/src/xenapi/xenapi_driver.c @@ -70,10 +70,14 @@ xenapiDomainDeviceDefPostParse(virDomainDeviceDefPtr dev, static int -xenapiDomainDefPostParse(virDomainDefPtr def ATTRIBUTE_UNUSED, +xenapiDomainDefPostParse(virDomainDefPtr def, virCapsPtr caps ATTRIBUTE_UNUSED, void *opaque ATTRIBUTE_UNUSED) { + /* memory hotplug tunables are not supported by this driver */ + if (virDomainDefCheckUnsupportedMemoryHotplug(def) < 0) + return -1; + return 0; } diff --git a/tests/domainschemadata/maxMemory.xml b/tests/domainschemadata/maxMemory.xml new file mode 100644 index 0000000..df2e3d8 --- /dev/null +++ b/tests/domainschemadata/maxMemory.xml @@ -0,0 +1,19 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <maxMemory slots='9' unit='KiB'>1233456789</maxMemory> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu</emulator> + </devices> +</domain> -- 2.2.2

Peter Krempa

10:19 a.m.

New subject: [libvirt] [PATCHv3 04/12] qemu: Implement setup of memory hotplug parameters

To enable memory hotplug the maximum memory size and slot count need to be specified. As qemu supports now other units than mebibytes when specifying memory, use the new interface in this case. --- Notes: Version 2: - simplified condition for enabling the max_memory formatting - tweaked test code to use the NONE macro instead of 0 docs/formatdomain.html.in | 2 +- src/qemu/qemu_command.c | 34 ++++++++++++++++++---- src/qemu/qemu_domain.c | 8 ++--- .../qemuxml2argv-memory-hotplug-nonuma.xml | 22 ++++++++++++++ .../qemuxml2argv-memory-hotplug.args | 6 ++++ .../qemuxml2argv-memory-hotplug.xml | 34 ++++++++++++++++++++++ tests/qemuxml2argvtest.c | 4 +++ tests/qemuxml2xmltest.c | 3 ++ 8 files changed, 102 insertions(+), 11 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-nonuma.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug.xml diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 57caef0..c167e1c 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -725,7 +725,7 @@ Note that due to alignment of the memory chunks added via memory hotplug the full size allocation specified by this element may be impossible to achieve. - Since 1.2.14 + Since 1.2.14 supported by the QEMU driver. </dd> <dt><code>currentMemory</code></dt> diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 02105c3..ef8feeb 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -8489,13 +8489,35 @@ qemuBuildCommandLine(virConnectPtr conn, if (qemuDomainAlignMemorySizes(def) < 0) goto error; - /* Set '-m MB' based on maxmem, because the lower 'memory' limit - * is set post-startup using the balloon driver. If balloon driver - * is not supported, then they're out of luck anyway. Update the - * XML to reflect our rounding. - */ virCommandAddArg(cmd, "-m"); - virCommandAddArgFormat(cmd, "%llu", virDomainDefGetMemoryInitial(def) / 1024); + + if (def->mem.max_memory) { + if (!virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_PC_DIMM)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("memory hotplug isn't supported by this QEMU binary")); + goto error; + } + + /* due to guest support, qemu would silently enable NUMA with one node + * once the memory hotplug backend is enabled. To avoid possible + * confusion we will enforce user originated numa configuration along + * with memory hotplug. */ + if (virDomainNumaGetNodeCount(def->numa) == 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("At least one numa node has to be configured when " + "enabling memory hotplug")); + goto error; + } + + /* Use the 'k' suffix to let qemu handle the units */ + virCommandAddArgFormat(cmd, "size=%lluk,slots=%u,maxmem=%lluk", + virDomainDefGetMemoryInitial(def), + def->mem.memory_slots, + def->mem.max_memory); + + } else { + virCommandAddArgFormat(cmd, "%llu", virDomainDefGetMemoryInitial(def) / 1024); + } if (def->mem.nhugepages && !virDomainNumaGetNodeCount(def->numa)) { const long system_page_size = virGetSystemPageSizeKB(); diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 6484bb5..3223752 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -1052,10 +1052,6 @@ qemuDomainDefPostParse(virDomainDefPtr def, VIR_DOMAIN_INPUT_BUS_USB) < 0) return -1; - /* memory hotplug tunables are not supported by this driver */ - if (virDomainDefCheckUnsupportedMemoryHotplug(def) < 0) - return -1; - return 0; } @@ -2898,5 +2894,9 @@ qemuDomainAlignMemorySizes(virDomainDefPtr def) mem = virDomainDefGetMemoryInitial(def); virDomainDefSetMemoryInitial(def, VIR_ROUND_UP(mem, 1024)); + /* Align maximum memory size. QEMU requires rounding to next 4KiB block. + * We'll take the "traditional" path and round it to 1MiB*/ + def->mem.max_memory = VIR_ROUND_UP(def->mem.max_memory, 1024); + return 0; } diff --git a/tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-nonuma.xml b/tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-nonuma.xml new file mode 100644 index 0000000..5c807ed --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-nonuma.xml @@ -0,0 +1,22 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <maxMemory slots='9' unit='KiB'>1233456789</maxMemory> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu</emulator> + <controller type='usb' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug.args b/tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug.args new file mode 100644 index 0000000..6c26586 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug.args @@ -0,0 +1,6 @@ +LC_ALL=C PATH=/bin HOME=/home/test USER=test LOGNAME=test QEMU_AUDIO_DRV=none \ +/usr/bin/qemu -S -M pc \ +-m size=219136k,slots=16,maxmem=1099511627776k \ +-smp 2 -numa node,nodeid=0,cpus=0-1,mem=214 \ +-nographic -monitor unix:/tmp/test-monitor,server,nowait -no-acpi -boot c \ +-usb -hda /dev/HostVG/QEMUGuest1 -net none -serial none -parallel none diff --git a/tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug.xml b/tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug.xml new file mode 100644 index 0000000..567a662 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug.xml @@ -0,0 +1,34 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <maxMemory slots='16' unit='KiB'>1099511627776</maxMemory> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static' cpuset='0-1'>2</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <cpu> + <topology sockets='2' cores='1' threads='1'/> + <numa> + <cell id='0' cpus='0-1' memory='219136' unit='KiB'/> + </numa> + </cpu> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='hda' bus='ide'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='ide' index='0'/> + <controller type='usb' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index fcf5218..387b349 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1543,6 +1543,10 @@ mymain(void) DO_TEST_PARSE_ERROR("shmem-msi-only", NONE); DO_TEST("cpu-host-passthrough-features", QEMU_CAPS_KVM, QEMU_CAPS_CPU_HOST); + DO_TEST_FAILURE("memory-hotplug-nonuma", QEMU_CAPS_DEVICE_PC_DIMM); + DO_TEST_FAILURE("memory-hotplug", NONE); + DO_TEST("memory-hotplug", QEMU_CAPS_DEVICE_PC_DIMM, QEMU_CAPS_NUMA); + virObjectUnref(driver.config); virObjectUnref(driver.caps); virObjectUnref(driver.xmlopt); diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index 7755ea3..ed462b8 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -427,6 +427,9 @@ mymain(void) DO_TEST("smbios"); DO_TEST("aarch64-aavmf-virtio-mmio"); + DO_TEST("memory-hotplug"); + DO_TEST("memory-hotplug-nonuma"); + virObjectUnref(driver.caps); virObjectUnref(driver.xmlopt); -- 2.2.2

Peter Krempa

10:19 a.m.

New subject: [libvirt] [PATCHv3 05/12] conf: Add device address type for dimm devices

Dimm devices are described by the slot and base address. Add a new address type to be able to describe such address. --- Notes: Version 2: - drop 'ACPI' from everything docs/schemas/domaincommon.rng | 18 +++++++++++ src/conf/domain_conf.c | 74 ++++++++++++++++++++++++++++++++++++++++++- src/conf/domain_conf.h | 9 ++++++ 3 files changed, 100 insertions(+), 1 deletion(-) diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index a0dee17..1f4df8e 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -4030,6 +4030,18 @@ </attribute> </optional> </define> + <define name="dimmaddress"> + <optional> + <attribute name="slot"> + <ref name="unsignedInt"/> + </attribute> + </optional> + <optional> + <attribute name="base"> + <ref name="hexuint"/> + </attribute> + </optional> + </define> <define name="devices"> <element name="devices"> <interleave> @@ -4449,6 +4461,12 @@ <value>virtio-mmio</value> </attribute> </group> + <group> + <attribute name="type"> + <value>dimm</value> + </attribute> + <ref name="dimmaddress"/> + </group> </choice> </element> </define> diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index dc15d95..2d765ba 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -249,7 +249,8 @@ VIR_ENUM_IMPL(virDomainDeviceAddress, VIR_DOMAIN_DEVICE_ADDRESS_TYPE_LAST, "virtio-s390", "ccw", "virtio-mmio", - "isa") + "isa", + "dimm") VIR_ENUM_IMPL(virDomainDiskDevice, VIR_DOMAIN_DISK_DEVICE_LAST, "disk", @@ -2832,6 +2833,11 @@ virDomainDeviceInfoAddressIsEqual(const virDomainDeviceInfo *a, if (memcmp(&a->addr.isa, &b->addr.isa, sizeof(a->addr.isa))) return false; break; + + case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_DIMM: + if (memcmp(&a->addr.dimm, &b->addr.dimm, sizeof(a->addr.dimm))) + return false; + break; } return true; @@ -3701,6 +3707,12 @@ virDomainDeviceInfoFormat(virBufferPtr buf, virBufferAsprintf(buf, " irq='0x%x'", info->addr.isa.irq); break; + case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_DIMM: + virBufferAsprintf(buf, " slot='%u'", info->addr.dimm.slot); + virBufferAsprintf(buf, " base='0x%llx'", info->addr.dimm.base); + + break; + case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_VIRTIO_S390: case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE: case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_LAST: @@ -4070,6 +4082,41 @@ virDomainDeviceISAAddressParseXML(xmlNodePtr node, return ret; } + +static int +virDomainDeviceDimmAddressParseXML(xmlNodePtr node, + virDomainDeviceDimmAddressPtr addr) +{ + int ret = -1; + char *tmp = NULL; + + if (!(tmp = virXMLPropString(node, "slot")) || + virStrToLong_uip(tmp, NULL, 10, &addr->slot) < 0) { + virReportError(VIR_ERR_XML_ERROR, + _("invalid or missing dimm slot id '%s'"), + NULLSTR(tmp)); + goto cleanup; + } + VIR_FREE(tmp); + + if (!(tmp = virXMLPropString(node, "base")) || + virStrToLong_ullp(tmp, NULL, 16, &addr->base) < 0) { + virReportError(VIR_ERR_XML_ERROR, + _("invalid or missing dimm base address '%s'"), + NULLSTR(tmp)); + goto cleanup; + } + VIR_FREE(tmp); + + ret = 0; + + cleanup: + VIR_FREE(tmp); + + return ret; +} + + /* Parse the XML definition for a device address * @param node XML nodeset to parse for device address definition */ @@ -4211,6 +4258,11 @@ virDomainDeviceInfoParseXML(xmlNodePtr node, _("virtio-s390 bus doesn't have an address")); goto cleanup; + case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_DIMM: + if (virDomainDeviceDimmAddressParseXML(address, &info->addr.dimm) < 0) + goto cleanup; + break; + case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE: case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_LAST: break; @@ -15408,6 +15460,26 @@ virDomainDeviceInfoCheckABIStability(virDomainDeviceInfoPtr src, } break; + case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_DIMM: + if (src->addr.dimm.slot != dst->addr.dimm.slot) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target device dimm slot %u does not match " + "source %u"), + dst->addr.dimm.slot, + src->addr.dimm.slot); + return false; + } + + if (src->addr.dimm.base != dst->addr.dimm.base) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target device dimm base addres '%llx' does " + "not match source '%llx'"), + dst->addr.dimm.base, + src->addr.dimm.base); + return false; + } + break; + case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_USB: case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_SPAPRVIO: case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_VIRTIO_S390: diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 469fc0c..e7376ed 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -235,6 +235,7 @@ typedef enum { VIR_DOMAIN_DEVICE_ADDRESS_TYPE_CCW, VIR_DOMAIN_DEVICE_ADDRESS_TYPE_VIRTIO_MMIO, VIR_DOMAIN_DEVICE_ADDRESS_TYPE_ISA, + VIR_DOMAIN_DEVICE_ADDRESS_TYPE_DIMM, VIR_DOMAIN_DEVICE_ADDRESS_TYPE_LAST } virDomainDeviceAddressType; @@ -310,6 +311,13 @@ struct _virDomainDeviceISAAddress { unsigned int irq; }; +typedef struct _virDomainDeviceDimmAddress virDomainDeviceDimmAddress; +typedef virDomainDeviceDimmAddress *virDomainDeviceDimmAddressPtr; +struct _virDomainDeviceDimmAddress { + unsigned int slot; + unsigned long long base; +}; + typedef struct _virDomainDeviceInfo virDomainDeviceInfo; typedef virDomainDeviceInfo *virDomainDeviceInfoPtr; struct _virDomainDeviceInfo { @@ -328,6 +336,7 @@ struct _virDomainDeviceInfo { virDomainDeviceSpaprVioAddress spaprvio; virDomainDeviceCCWAddress ccw; virDomainDeviceISAAddress isa; + virDomainDeviceDimmAddress dimm; } addr; int mastertype; union { -- 2.2.2

Peter Krempa

10:19 a.m.

New subject: [libvirt] [PATCHv3 06/12] conf: Add interface to parse and format memory device information

This patch adds code that parses and formats configuration for memory devices. A simple configuration would be: <memory model='dimm'> <target> <size unit='KiB'>524287</size> <node>0</node> </target> </memory> A complete configuration of a memory device: <memory model='dimm'> <source> <pagesize unit='KiB'>4096</pagesize> <nodemask>1-3</nodemask> </source> <target> <size unit='KiB'>524287</size> <node>1</node> </target> </memory> This patch preemptively forbids use of the <memory> device in individual drivers so the users are warned right away that the device is not supported. --- Notes: Version 3: - target node is stored now as unsigned and all uses are tweaked - fixed documentation wording as suggested - added documentation for behavior of <memory> element once memory devices are used Version 2: - dropped the ACPI prefix - fixed memory leak in the free function - tweaked docs Version 3: - target node is stored now as unsigned and all uses are tweaked - fixed documentation wording as suggested Version 2: - dropped the ACPI prefix - fixed memory leak in the free function - tweaked docs Version 3: - target node is stored now as unsigned and all uses are tweaked - fixed documentation wording as suggested Version 2: - dropped the ACPI prefix - fixed memory leak in the free function - tweaked docs Version 2: - dropped the ACPI prefix - fixed memory leak in the free function - tweaked docs docs/formatdomain.html.in | 92 +++++- docs/schemas/domaincommon.rng | 50 +++ src/bhyve/bhyve_domain.c | 5 +- src/conf/domain_conf.c | 336 ++++++++++++++++++++- src/conf/domain_conf.h | 33 ++ src/libvirt_private.syms | 2 + src/libxl/libxl_domain.c | 3 + src/lxc/lxc_domain.c | 4 + src/openvz/openvz_driver.c | 3 + src/qemu/qemu_domain.c | 3 + src/qemu/qemu_driver.c | 13 + src/qemu/qemu_hotplug.c | 3 + src/uml/uml_driver.c | 3 + src/xen/xen_driver.c | 3 + src/xenapi/xenapi_driver.c | 3 + .../qemuxml2argv-memory-hotplug-dimm.xml | 50 +++ 16 files changed, 601 insertions(+), 5 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-dimm.xml diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index c167e1c..e1aa79c 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -684,7 +684,9 @@ <dl> <dt><code>memory</code></dt> - <dd>The maximum allocation of memory for the guest at boot time. + <dd>The maximum allocation of memory for the guest at boot time. The + memory allocation includes possible additional memory devices specified + at start or hotplugged later. The units for this value are determined by the optional attribute <code>unit</code>, which defaults to "KiB" (kibibytes, 210 or blocks of 1024 bytes). Valid @@ -702,6 +704,9 @@ supported by the hypervisor. Some hypervisors also enforce a minimum, such as 4000KiB. + In case <a href="#elementsCPU">NUMA</a> is configured for the guest the + <code>memory</code> element can be omitted. + In the case of crash, optional attribute <code>dumpCore</code> can be used to control whether the guest memory should be included in the generated coredump or not (values "on", "off"). @@ -5909,6 +5914,91 @@ qemu-kvm -net nic,model=? /dev/null </dd> </dl> + <h4><a name="elementsMemory">Memory devices</a></h4> + + + In addition to the initial memory assigned to the guest, memory devices + allow additional memory to be assigned to the guest in the form of + memory modules. + + A memory device can be hot-plugged or hot-unplugged depending on the + guests' memory resource needs. + + Some hypervisors may require NUMA configured for the guest. + Since 1.2.14 + + + + Example: usage of the memory devices + +<pre> + ... + <devices> + <memory model='dimm'> + <target> + <size unit='KiB'>524287</size> + <node>0</node> + </target> + </memory> + <memory model='dimm'> + <source> + <pagesize unit='KiB'>4096</pagesize> + <nodemask>1-3</nodemask> + </source> + <target> + <size unit='KiB'>524287</size> + <node>1</node> + </target> + </memory> + </devices> + ... +</pre> + <dl> + <dt><code>model</code></dt> + <dd> + + Currently only the <code>dimm</code> model is supported in order to + add a virtual DIMM module to the guest. + + </dd> + + <dt><code>source</code></dt> + <dd> + + The optional source element allows to fine tune the source of the + memory used for the given memory device. If the element is not + provided defaults configured via <code>numatune</code> are used. + + + <code>pagesize</code> can optionally be used to override the default + host page size used for backing the memory device. + + The configured value must correspond to a page size supported by the + host. + + + <code>nodemask</code> can optionally be used to override the default + set of NUMA nodes where the memory would be allocated. + + </dd> + + <dt><code>target</code></dt> + <dd> + + The mandatory <code>target</code> element configures the placement and + sizing of the added memory from the perspective of the guest. + + + The mandatory <code>size</code> subelement configures the size of the + added memory as a scaled integer. + + + The mandatory <code>node</code> subelement configures the guest NUMA + node to attach the memory to. + + </dd> + </dl> + <h3><a name="seclabel">Security label</a></h3> diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index 1f4df8e..e66b467 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -4071,6 +4071,7 @@ <ref name="rng"/> <ref name="tpm"/> <ref name="shmem"/> + <ref name="memorydev"/> </choice> </zeroOrMore> <optional> @@ -4487,6 +4488,55 @@ </element> </define> + <define name="memorydev"> + <element name="memory"> + <attribute name="model"> + <choice> + <value>dimm</value> + </choice> + </attribute> + <interleave> + <optional> + <ref name="memorydev-source"/> + </optional> + <ref name="memorydev-target"/> + <optional> + <ref name="address"/> + </optional> + </interleave> + </element> + </define> + + <define name="memorydev-source"> + <element name="source"> + <interleave> + <optional> + <element name="pagesize"> + <ref name="scaledInteger"/> + </element> + </optional> + <optional> + <element name="nodemask"> + <ref name="cpuset"/> + </element> + </optional> + </interleave> + </element> + </define> + + <define name="memorydev-target"> + <element name="target"> + <interleave> + <element name="size"> + <ref name="scaledInteger"/> + </element> + <element name="node"> + <ref name="unsignedInt"/> + </element> + </interleave> + </element> + </define> + <define name="rng"> <element name="rng"> <attribute name="model"> diff --git a/src/bhyve/bhyve_domain.c b/src/bhyve/bhyve_domain.c index 25ef852..890963e 100644 --- a/src/bhyve/bhyve_domain.c +++ b/src/bhyve/bhyve_domain.c @@ -75,11 +75,14 @@ bhyveDomainDefPostParse(virDomainDefPtr def, } static int -bhyveDomainDeviceDefPostParse(virDomainDeviceDefPtr dev ATTRIBUTE_UNUSED, +bhyveDomainDeviceDefPostParse(virDomainDeviceDefPtr dev, const virDomainDef *def ATTRIBUTE_UNUSED, virCapsPtr caps ATTRIBUTE_UNUSED, void *opaque ATTRIBUTE_UNUSED) { + if (virDomainDeviceDefCheckUnsupportedMemoryDevice(dev) < 0) + return -1; + return 0; } diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 2d765ba..8c2234f 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -236,7 +236,8 @@ VIR_ENUM_IMPL(virDomainDevice, VIR_DOMAIN_DEVICE_LAST, "rng", "shmem", "tpm", - "panic") + "panic", + "memory") VIR_ENUM_IMPL(virDomainDeviceAddress, VIR_DOMAIN_DEVICE_ADDRESS_TYPE_LAST, "none", @@ -779,6 +780,9 @@ VIR_ENUM_DECL(virDomainBlockJob) VIR_ENUM_IMPL(virDomainBlockJob, VIR_DOMAIN_BLOCK_JOB_TYPE_LAST, "", "", "copy", "", "active-commit") +VIR_ENUM_IMPL(virDomainMemoryModel, VIR_DOMAIN_MEMORY_MODEL_LAST, + "", "dimm") + static virClassPtr virDomainObjClass; static virClassPtr virDomainObjListClass; static virClassPtr virDomainXMLOptionClass; @@ -1003,6 +1007,27 @@ virDomainDefCheckUnsupportedMemoryHotplug(virDomainDefPtr def) } +/** + * virDomainDeviceDefCheckUnsupportedMemoryDevice: + * @dev: device definition + * + * Returns -1 if the device definition describes a memory device and reports an + * error. Otherwise returns 0. + */ +int +virDomainDeviceDefCheckUnsupportedMemoryDevice(virDomainDeviceDefPtr dev) +{ + /* This driver doesn't yet know how to handle memory devices */ + if (dev->type == VIR_DOMAIN_DEVICE_MEMORY) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("memory devices are not supported by this driver")); + return -1; + } + + return 0; +} + + static void virDomainObjListDataFree(void *payload, const void *name ATTRIBUTE_UNUSED) { @@ -1936,6 +1961,16 @@ void virDomainRedirFilterDefFree(virDomainRedirFilterDefPtr def) VIR_FREE(def); } +void virDomainMemoryDefFree(virDomainMemoryDefPtr def) +{ + if (!def) + return; + + virBitmapFree(def->sourceNodes); + virDomainDeviceInfoClear(&def->info); + VIR_FREE(def); +} + void virDomainDeviceDefFree(virDomainDeviceDefPtr def) { if (!def) @@ -2005,6 +2040,9 @@ void virDomainDeviceDefFree(virDomainDeviceDefPtr def) case VIR_DOMAIN_DEVICE_PANIC: virDomainPanicDefFree(def->data.panic); break; + case VIR_DOMAIN_DEVICE_MEMORY: + virDomainMemoryDefFree(def->data.memory); + break; case VIR_DOMAIN_DEVICE_LAST: case VIR_DOMAIN_DEVICE_NONE: break; @@ -2202,6 +2240,10 @@ void virDomainDefFree(virDomainDefPtr def) virDomainRNGDefFree(def->rngs[i]); VIR_FREE(def->rngs); + for (i = 0; i < def->nmems; i++) + virDomainMemoryDefFree(def->mems[i]); + VIR_FREE(def->mems); + virDomainTPMDefFree(def->tpm); virDomainPanicDefFree(def->panic); @@ -2743,6 +2785,8 @@ virDomainDeviceGetInfo(virDomainDeviceDefPtr device) return &device->data.tpm->info; case VIR_DOMAIN_DEVICE_PANIC: return &device->data.panic->info; + case VIR_DOMAIN_DEVICE_MEMORY: + return &device->data.memory->info; /* The following devices do not contain virDomainDeviceInfo */ case VIR_DOMAIN_DEVICE_LEASE: @@ -3063,6 +3107,13 @@ virDomainDeviceInfoIterateInternal(virDomainDefPtr def, return -1; } + device.type = VIR_DOMAIN_DEVICE_MEMORY; + for (i = 0; i < def->nmems; i++) { + device.data.memory = def->mems[i]; + if (cb(def, &device, &def->mems[i]->info, opaque) < 0) + return -1; + } + /* Coverity is not very happy with this - all dead_error_condition */ #if !STATIC_ANALYSIS /* This switch statement is here to trigger compiler warning when adding @@ -3095,6 +3146,7 @@ virDomainDeviceInfoIterateInternal(virDomainDefPtr def, case VIR_DOMAIN_DEVICE_PANIC: case VIR_DOMAIN_DEVICE_LAST: case VIR_DOMAIN_DEVICE_RNG: + case VIR_DOMAIN_DEVICE_MEMORY: break; } #endif @@ -7082,10 +7134,16 @@ unsigned long long virDomainDefGetMemoryInitial(virDomainDefPtr def) { unsigned long long ret; + size_t i; /* return NUMA memory size total in case numa is enabled */ - if ((ret = virDomainNumaGetMemorySize(def->numa)) > 0) + if ((ret = virDomainNumaGetMemorySize(def->numa)) > 0) { return ret; + } else { + ret = def->mem.max_balloon; + for (i = 0; i < def->nmems; i++) + ret -= def->mems[i]->size; + } return def->mem.max_balloon; } @@ -7117,7 +7175,17 @@ virDomainDefSetMemoryInitial(virDomainDefPtr def, unsigned long long virDomainDefGetMemoryActual(virDomainDefPtr def) { - return virDomainDefGetMemoryInitial(def); + unsigned long long ret; + size_t i; + + if ((ret = virDomainNumaGetMemorySize(def->numa)) > 0) { + for (i = 0; i < def->nmems; i++) + ret += def->mems[i]->size; + } else { + ret = def->mem.max_balloon; + } + + return ret; } @@ -11476,6 +11544,119 @@ virDomainPMStateParseXML(xmlXPathContextPtr ctxt, return ret; } + +static int +virDomainMemorySourceDefParseXML(xmlNodePtr node, + xmlXPathContextPtr ctxt, + virDomainMemoryDefPtr def) +{ + int ret = -1; + char *nodemask = NULL; + xmlNodePtr save = ctxt->node; + ctxt->node = node; + + if (virDomainParseMemory("./pagesize", "./pagesize/@unit", ctxt, + &def->pagesize, false, false) < 0) + goto cleanup; + + if ((nodemask = virXPathString("string(./nodemask)", ctxt))) { + if (virBitmapParse(nodemask, 0, &def->sourceNodes, + VIR_DOMAIN_CPUMASK_LEN) < 0) + goto cleanup; + } + + ret = 0; + + cleanup: + VIR_FREE(nodemask); + ctxt->node = save; + return ret; +} + + +static int +virDomainMemoryTargetDefParseXML(xmlNodePtr node, + xmlXPathContextPtr ctxt, + virDomainMemoryDefPtr def) +{ + int ret = -1; + xmlNodePtr save = ctxt->node; + ctxt->node = node; + + if (virXPathUInt("string(./node)", ctxt, &def->targetNode) < 0) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("invalid or missing value of memory device node")); + goto cleanup; + } + + if (virDomainParseMemory("./size", "./size/@unit", ctxt, + &def->size, true, false) < 0) + goto cleanup; + + ret = 0; + + cleanup: + ctxt->node = save; + return ret; +} + + +static virDomainMemoryDefPtr +virDomainMemoryDefParseXML(xmlNodePtr memdevNode, + xmlXPathContextPtr ctxt, + unsigned int flags) +{ + char *tmp = NULL; + xmlNodePtr save = ctxt->node; + xmlNodePtr node; + virDomainMemoryDefPtr def; + + ctxt->node = memdevNode; + + if (VIR_ALLOC(def) < 0) + return NULL; + + if (!(tmp = virXMLPropString(memdevNode, "model"))) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("missing memory model")); + goto error; + } + + if ((def->model = virDomainMemoryModelTypeFromString(tmp)) < 0) { + virReportError(VIR_ERR_XML_ERROR, + _("invalid memory model '%s'"), tmp); + goto error; + } + VIR_FREE(tmp); + + /* source */ + if ((node = virXPathNode("./source", ctxt)) && + virDomainMemorySourceDefParseXML(node, ctxt, def) < 0) + goto error; + + /* target */ + if (!(node = virXPathNode("./target", ctxt))) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("missing <target> element for <memory> device")); + goto error; + } + + if (virDomainMemoryTargetDefParseXML(node, ctxt, def) < 0) + goto error; + + if (virDomainDeviceInfoParseXML(memdevNode, NULL, &def->info, flags) < 0) + goto error; + + return def; + + error: + VIR_FREE(tmp); + virDomainMemoryDefFree(def); + ctxt->node = save; + return NULL; +} + + virDomainDeviceDefPtr virDomainDeviceDefParse(const char *xmlStr, const virDomainDef *def, @@ -11610,6 +11791,10 @@ virDomainDeviceDefParse(const char *xmlStr, if (!(dev->data.panic = virDomainPanicDefParseXML(node))) goto error; break; + case VIR_DOMAIN_DEVICE_MEMORY: + if (!(dev->data.memory = virDomainMemoryDefParseXML(node, ctxt, flags))) + goto error; + break; case VIR_DOMAIN_DEVICE_NONE: case VIR_DOMAIN_DEVICE_LAST: break; @@ -15009,6 +15194,23 @@ virDomainDefParseXML(xmlDocPtr xml, ctxt->node = node; VIR_FREE(nodes); + /* analysis of memory devices */ + if ((n = virXPathNodeSet("./devices/memory", ctxt, &nodes)) < 0) + goto error; + if (n && VIR_ALLOC_N(def->mems, n) < 0) + goto error; + + for (i = 0; i < n; i++) { + virDomainMemoryDefPtr mem = virDomainMemoryDefParseXML(nodes[i], + ctxt, + flags); + if (!mem) + goto error; + + def->mems[def->nmems++] = mem; + } + VIR_FREE(nodes); + /* analysis of the user namespace mapping */ if ((n = virXPathNodeSet("./idmap/uid", ctxt, &nodes)) < 0) goto error; @@ -16248,6 +16450,39 @@ virDomainPanicDefCheckABIStability(virDomainPanicDefPtr src, return virDomainDeviceInfoCheckABIStability(&src->info, &dst->info); } +static bool +virDomainMemoryDefCheckABIStability(virDomainMemoryDefPtr src, + virDomainMemoryDefPtr dst) +{ + if (src->model != dst->model) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target memory device model '%s' " + "doesn't match source model '%s'"), + virDomainMemoryModelTypeToString(dst->model), + virDomainMemoryModelTypeToString(src->model)); + return false; + } + + if (src->targetNode != dst->targetNode) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target memory device targetNode '%u' " + "doesn't match source targetNode '%u'"), + dst->targetNode, src->targetNode); + return false; + } + + if (src->size != dst->size) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target memory device size '%llu' doesn't match " + "source memory device size '%llu'"), + dst->size, src->size); + return false; + } + + return virDomainDeviceInfoCheckABIStability(&src->info, &dst->info); +} + + /* This compares two configurations and looks for any differences * which will affect the guest ABI. This is primarily to allow * validation of custom XML config passed in during migration @@ -16666,6 +16901,18 @@ virDomainDefCheckABIStability(virDomainDefPtr src, goto error; } + if (src->nmems != dst->nmems) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target domain memory device count %zu " + "does not match source %zu"), dst->nmems, src->nmems); + goto error; + } + + for (i = 0; i < src->nmems; i++) { + if (!virDomainMemoryDefCheckABIStability(src->mems[i], dst->mems[i])) + goto error; + } + /* Coverity is not very happy with this - all dead_error_condition */ #if !STATIC_ANALYSIS /* This switch statement is here to trigger compiler warning when adding @@ -16697,6 +16944,7 @@ virDomainDefCheckABIStability(virDomainDefPtr src, case VIR_DOMAIN_DEVICE_TPM: case VIR_DOMAIN_DEVICE_PANIC: case VIR_DOMAIN_DEVICE_SHMEM: + case VIR_DOMAIN_DEVICE_MEMORY: break; } #endif @@ -19184,6 +19432,81 @@ virDomainRNGDefFree(virDomainRNGDefPtr def) VIR_FREE(def); } + +static int +virDomainMemorySourceDefFormat(virBufferPtr buf, + virDomainMemoryDefPtr def) +{ + char *bitmap = NULL; + int ret = -1; + + if (!def->pagesize && !def->sourceNodes) + return 0; + + virBufferAddLit(buf, "<source>\n"); + virBufferAdjustIndent(buf, 2); + + if (def->sourceNodes) { + if (!(bitmap = virBitmapFormat(def->sourceNodes))) + goto cleanup; + + virBufferAsprintf(buf, "<nodemask>%s</nodemask>\n", bitmap); + } + + if (def->pagesize) + virBufferAsprintf(buf, "<pagesize unit='KiB'>%llu</pagesize>\n", + def->pagesize); + + virBufferAdjustIndent(buf, -2); + virBufferAddLit(buf, "</source>\n"); + + ret = 0; + + cleanup: + VIR_FREE(bitmap); + return ret; +} + + +static void +virDomainMemoryTargetDefFormat(virBufferPtr buf, + virDomainMemoryDefPtr def) +{ + virBufferAddLit(buf, "<target>\n"); + virBufferAdjustIndent(buf, 2); + + virBufferAsprintf(buf, "<size unit='KiB'>%llu</size>\n", def->size); + virBufferAsprintf(buf, "<node>%u</node>\n", def->targetNode); + + virBufferAdjustIndent(buf, -2); + virBufferAddLit(buf, "</target>\n"); +} + +static int +virDomainMemoryDefFormat(virBufferPtr buf, + virDomainMemoryDefPtr def, + unsigned int flags) +{ + const char *model = virDomainMemoryModelTypeToString(def->model); + + virBufferAsprintf(buf, "<memory model='%s'>\n", model); + virBufferAdjustIndent(buf, 2); + + if (virDomainMemorySourceDefFormat(buf, def) < 0) + return -1; + + virDomainMemoryTargetDefFormat(buf, def); + + if (virDomainDeviceInfoNeedsFormat(&def->info, flags)) { + if (virDomainDeviceInfoFormat(buf, &def->info, flags) < 0) + return -1; + } + + virBufferAdjustIndent(buf, -2); + virBufferAddLit(buf, "</memory>\n"); + return 0; +} + static void virDomainVideoAccelDefFormat(virBufferPtr buf, virDomainVideoAccelDefPtr def) @@ -20770,6 +21093,10 @@ virDomainDefFormatInternal(virDomainDefPtr def, if (virDomainShmemDefFormat(buf, def->shmems[n], flags) < 0) goto error; + for (n = 0; n < def->nmems; n++) + if (virDomainMemoryDefFormat(buf, def->mems[n], flags) < 0) + goto error; + virBufferAdjustIndent(buf, -2); virBufferAddLit(buf, "</devices>\n"); @@ -22174,6 +22501,9 @@ virDomainDeviceDefCopy(virDomainDeviceDefPtr src, case VIR_DOMAIN_DEVICE_PANIC: rc = virDomainPanicDefFormat(&buf, src->data.panic); break; + case VIR_DOMAIN_DEVICE_MEMORY: + rc = virDomainMemoryDefFormat(&buf, src->data.memory, flags); + break; case VIR_DOMAIN_DEVICE_NONE: case VIR_DOMAIN_DEVICE_SMARTCARD: case VIR_DOMAIN_DEVICE_MEMBALLOON: diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index e7376ed..475a174 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -133,6 +133,9 @@ typedef virDomainIdMapDef *virDomainIdMapDefPtr; typedef struct _virDomainPanicDef virDomainPanicDef; typedef virDomainPanicDef *virDomainPanicDefPtr; +typedef struct _virDomainMemoryDef virDomainMemoryDef; +typedef virDomainMemoryDef *virDomainMemoryDefPtr; + /* forward declarations virDomainChrSourceDef, required by * virDomainNetDef */ @@ -169,6 +172,7 @@ typedef enum { VIR_DOMAIN_DEVICE_SHMEM, VIR_DOMAIN_DEVICE_TPM, VIR_DOMAIN_DEVICE_PANIC, + VIR_DOMAIN_DEVICE_MEMORY, VIR_DOMAIN_DEVICE_LAST } virDomainDeviceType; @@ -199,6 +203,7 @@ struct _virDomainDeviceDef { virDomainShmemDefPtr shmem; virDomainTPMDefPtr tpm; virDomainPanicDefPtr panic; + virDomainMemoryDefPtr memory; } data; }; @@ -1970,6 +1975,28 @@ struct _virDomainRNGDef { virDomainDeviceInfo info; }; +typedef enum { + VIR_DOMAIN_MEMORY_MODEL_NONE, + VIR_DOMAIN_MEMORY_MODEL_DIMM, /* dimm hotpluggable memory device */ + + VIR_DOMAIN_MEMORY_MODEL_LAST +} virDomainMemoryModel; + +struct _virDomainMemoryDef { + /* source */ + virBitmapPtr sourceNodes; + unsigned long long pagesize; /* kibibytes */ + + /* target */ + int model; /* virDomainMemoryModel */ + unsigned int targetNode; + unsigned long long size; /* kibibytes */ + + virDomainDeviceInfo info; +}; + +void virDomainMemoryDefFree(virDomainMemoryDefPtr def); + struct _virDomainIdMapEntry { unsigned int start; unsigned int target; @@ -2190,6 +2217,9 @@ struct _virDomainDef { size_t nshmems; virDomainShmemDefPtr *shmems; + size_t nmems; + virDomainMemoryDefPtr *mems; + /* Only 1 */ virDomainWatchdogDefPtr watchdog; virDomainMemballoonDefPtr memballoon; @@ -2352,6 +2382,7 @@ bool virDomainObjTaint(virDomainObjPtr obj, int virDomainDefCheckUnsupportedMemoryHotplug(virDomainDefPtr def); +int virDomainDeviceDefCheckUnsupportedMemoryDevice(virDomainDeviceDefPtr dev); void virDomainPanicDefFree(virDomainPanicDefPtr panic); void virDomainResourceDefFree(virDomainResourceDefPtr resource); @@ -2900,6 +2931,8 @@ VIR_ENUM_DECL(virDomainRNGModel) VIR_ENUM_DECL(virDomainRNGBackend) VIR_ENUM_DECL(virDomainTPMModel) VIR_ENUM_DECL(virDomainTPMBackend) +VIR_ENUM_DECL(virDomainMemoryModel) +VIR_ENUM_DECL(virDomainMemoryBackingModel) /* from libvirt.h */ VIR_ENUM_DECL(virDomainState) VIR_ENUM_DECL(virDomainNostateReason) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 0988bc3..f24b449 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -216,6 +216,7 @@ virDomainDefSetMemoryInitial; virDomainDeleteConfig; virDomainDeviceAddressIsValid; virDomainDeviceAddressTypeToString; +virDomainDeviceDefCheckUnsupportedMemoryDevice; virDomainDeviceDefCopy; virDomainDeviceDefFree; virDomainDeviceDefParse; @@ -333,6 +334,7 @@ virDomainLockFailureTypeFromString; virDomainLockFailureTypeToString; virDomainMemballoonModelTypeFromString; virDomainMemballoonModelTypeToString; +virDomainMemoryDefFree; virDomainNetAppendIpAddress; virDomainNetDefFormat; virDomainNetDefFree; diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c index 3843ae0..ff1e4de 100644 --- a/src/libxl/libxl_domain.c +++ b/src/libxl/libxl_domain.c @@ -538,6 +538,9 @@ libxlDomainDeviceDefPostParse(virDomainDeviceDefPtr dev, } } + if (virDomainDeviceDefCheckUnsupportedMemoryDevice(dev) < 0) + return -1; + return 0; } diff --git a/src/lxc/lxc_domain.c b/src/lxc/lxc_domain.c index 1367b0c..c2180cb 100644 --- a/src/lxc/lxc_domain.c +++ b/src/lxc/lxc_domain.c @@ -113,6 +113,10 @@ virLXCDomainDeviceDefPostParse(virDomainDeviceDefPtr dev, dev->data.chr->targetType == VIR_DOMAIN_CHR_CONSOLE_TARGET_TYPE_NONE) dev->data.chr->targetType = VIR_DOMAIN_CHR_CONSOLE_TARGET_TYPE_LXC; + + if (virDomainDeviceDefCheckUnsupportedMemoryDevice(dev) < 0) + return -1; + return 0; } diff --git a/src/openvz/openvz_driver.c b/src/openvz/openvz_driver.c index 055670a..d29e35b 100644 --- a/src/openvz/openvz_driver.c +++ b/src/openvz/openvz_driver.c @@ -130,6 +130,9 @@ openvzDomainDeviceDefPostParse(virDomainDeviceDefPtr dev, return -1; } + if (virDomainDeviceDefCheckUnsupportedMemoryDevice(dev) < 0) + return -1; + return 0; } diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 3223752..6f328e4 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -1193,6 +1193,9 @@ qemuDomainDeviceDefPostParse(virDomainDeviceDefPtr dev, } } + if (virDomainDeviceDefCheckUnsupportedMemoryDevice(dev) < 0) + goto cleanup; + ret = 0; cleanup: diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index f4b8dab..300bce4 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -7649,6 +7649,9 @@ qemuDomainAttachDeviceLive(virDomainObjPtr vm, dev->data.rng = NULL; break; + /*TODO: implement later */ + case VIR_DOMAIN_DEVICE_MEMORY: + case VIR_DOMAIN_DEVICE_NONE: case VIR_DOMAIN_DEVICE_FS: case VIR_DOMAIN_DEVICE_INPUT: @@ -7726,6 +7729,8 @@ qemuDomainDetachDeviceLive(virDomainObjPtr vm, case VIR_DOMAIN_DEVICE_RNG: ret = qemuDomainDetachRNGDevice(driver, vm, dev->data.rng); break; + case VIR_DOMAIN_DEVICE_MEMORY: + /* TODO: Implement later */ case VIR_DOMAIN_DEVICE_FS: case VIR_DOMAIN_DEVICE_INPUT: case VIR_DOMAIN_DEVICE_SOUND: @@ -7843,6 +7848,7 @@ qemuDomainUpdateDeviceLive(virConnectPtr conn, case VIR_DOMAIN_DEVICE_HOSTDEV: case VIR_DOMAIN_DEVICE_CONTROLLER: case VIR_DOMAIN_DEVICE_REDIRDEV: + case VIR_DOMAIN_DEVICE_MEMORY: case VIR_DOMAIN_DEVICE_CHR: case VIR_DOMAIN_DEVICE_NONE: case VIR_DOMAIN_DEVICE_TPM: @@ -7985,6 +7991,9 @@ qemuDomainAttachDeviceConfig(virQEMUCapsPtr qemuCaps, dev->data.rng = NULL; break; + case VIR_DOMAIN_DEVICE_MEMORY: + /* TODO: implement later */ + case VIR_DOMAIN_DEVICE_INPUT: case VIR_DOMAIN_DEVICE_SOUND: case VIR_DOMAIN_DEVICE_VIDEO: @@ -8110,6 +8119,9 @@ qemuDomainDetachDeviceConfig(virDomainDefPtr vmdef, virDomainRNGDefFree(virDomainRNGRemove(vmdef, idx)); break; + case VIR_DOMAIN_DEVICE_MEMORY: + /* TODO: implement later */ + case VIR_DOMAIN_DEVICE_INPUT: case VIR_DOMAIN_DEVICE_SOUND: case VIR_DOMAIN_DEVICE_VIDEO: @@ -8224,6 +8236,7 @@ qemuDomainUpdateDeviceConfig(virQEMUCapsPtr qemuCaps, case VIR_DOMAIN_DEVICE_CONTROLLER: case VIR_DOMAIN_DEVICE_REDIRDEV: case VIR_DOMAIN_DEVICE_CHR: + case VIR_DOMAIN_DEVICE_MEMORY: case VIR_DOMAIN_DEVICE_NONE: case VIR_DOMAIN_DEVICE_TPM: case VIR_DOMAIN_DEVICE_PANIC: diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 7845fd1..5c5ad0e 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -3070,6 +3070,9 @@ qemuDomainRemoveDevice(virQEMUDriverPtr driver, qemuDomainRemoveRNGDevice(driver, vm, dev->data.rng); break; + /* TODO: implement later */ + case VIR_DOMAIN_DEVICE_MEMORY: + case VIR_DOMAIN_DEVICE_NONE: case VIR_DOMAIN_DEVICE_LEASE: case VIR_DOMAIN_DEVICE_FS: diff --git a/src/uml/uml_driver.c b/src/uml/uml_driver.c index bdfc12e..2d59126 100644 --- a/src/uml/uml_driver.c +++ b/src/uml/uml_driver.c @@ -439,6 +439,9 @@ umlDomainDeviceDefPostParse(virDomainDeviceDefPtr dev, return -1; } + if (virDomainDeviceDefCheckUnsupportedMemoryDevice(dev) < 0) + return -1; + return 0; } diff --git a/src/xen/xen_driver.c b/src/xen/xen_driver.c index b104243..3b11e9a 100644 --- a/src/xen/xen_driver.c +++ b/src/xen/xen_driver.c @@ -370,6 +370,9 @@ xenDomainDeviceDefPostParse(virDomainDeviceDefPtr dev, } } + if (virDomainDeviceDefCheckUnsupportedMemoryDevice(dev) < 0) + return -1; + return 0; } diff --git a/src/xenapi/xenapi_driver.c b/src/xenapi/xenapi_driver.c index 3134441..d495f21 100644 --- a/src/xenapi/xenapi_driver.c +++ b/src/xenapi/xenapi_driver.c @@ -65,6 +65,9 @@ xenapiDomainDeviceDefPostParse(virDomainDeviceDefPtr dev, return -1; } + if (virDomainDeviceDefCheckUnsupportedMemoryDevice(dev) < 0) + return -1; + return 0; } diff --git a/tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-dimm.xml b/tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-dimm.xml new file mode 100644 index 0000000..78088e2 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-dimm.xml @@ -0,0 +1,50 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <maxMemory slots='16' unit='KiB'>1099511627776</maxMemory> + <memory unit='KiB'>1267710</memory> + <currentMemory unit='KiB'>1267710</currentMemory> + <vcpu placement='static' cpuset='0-1'>2</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <cpu> + <topology sockets='2' cores='1' threads='1'/> + <numa> + <cell id='0' cpus='0-1' memory='219136' unit='KiB'/> + </numa> + </cpu> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='hda' bus='ide'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='ide' index='0'/> + <controller type='usb' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <memballoon model='virtio'/> + <memory model='dimm'> + <target> + <size unit='KiB'>524287</size> + <node>0</node> + </target> + </memory> + <memory model='dimm'> + <source> + <nodemask>1-3</nodemask> + <pagesize unit='KiB'>4096</pagesize> + </source> + <target> + <size unit='KiB'>524287</size> + <node>0</node> + </target> + </memory> + </devices> +</domain> -- 2.2.2

Peter Krempa

10:19 a.m.

New subject: [libvirt] [PATCHv3 07/12] qemu: memdev: Add infrastructure to load memory device information

When using 'dimm' memory devices with qemu, some of the information like the slot number and base address need to be reloaded from qemu after process start so that it reflects the actual state. The state then allows to use memory devices across migrations. --- Notes: Version 3: - Don't try to extract data if the qemu command was not successful Version 2: - rebase to the drop of ACPI prefix Version 2: - rebase to the drop of ACPI prefix src/qemu/qemu_domain.c | 49 +++++++++++++++++ src/qemu/qemu_domain.h | 4 ++ src/qemu/qemu_monitor.c | 42 +++++++++++++++ src/qemu/qemu_monitor.h | 14 +++++ src/qemu/qemu_monitor_json.c | 122 +++++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_monitor_json.h | 5 ++ src/qemu/qemu_process.c | 4 ++ 7 files changed, 240 insertions(+) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 6f328e4..825c02e 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -2807,6 +2807,55 @@ qemuDomainUpdateDeviceList(virQEMUDriverPtr driver, return 0; } + +int +qemuDomainUpdateMemoryDeviceInfo(virQEMUDriverPtr driver, + virDomainObjPtr vm, + int asyncJob) +{ + qemuDomainObjPrivatePtr priv = vm->privateData; + virHashTablePtr meminfo = NULL; + int rc; + size_t i; + + if (vm->def->nmems == 0) + return 0; + + if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) < 0) + return -1; + + rc = qemuMonitorGetMemoryDeviceInfo(priv->mon, &meminfo); + + if (qemuDomainObjExitMonitor(driver, vm) < 0) + return -1; + + /* if qemu doesn't support the info request, just carry on */ + if (rc == -2) + return 0; + + if (rc < 0) + return -1; + + for (i = 0; i < vm->def->nmems; i++) { + virDomainMemoryDefPtr mem = vm->def->mems[i]; + qemuMonitorMemoryDeviceInfoPtr dimm; + + if (!mem->info.alias) + continue; + + if (!(dimm = virHashLookup(meminfo, mem->info.alias))) + continue; + + mem->info.type = VIR_DOMAIN_DEVICE_ADDRESS_TYPE_DIMM; + mem->info.addr.dimm.slot = dimm->slot; + mem->info.addr.dimm.base = dimm->address; + } + + virHashFree(meminfo); + return 0; +} + + bool qemuDomainDefCheckABIStability(virQEMUDriverPtr driver, virDomainDefPtr src, diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index ba8d398..75e82f3 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -393,6 +393,10 @@ extern virDomainDefParserConfig virQEMUDriverDomainDefParserConfig; int qemuDomainUpdateDeviceList(virQEMUDriverPtr driver, virDomainObjPtr vm, int asyncJob); +int qemuDomainUpdateMemoryDeviceInfo(virQEMUDriverPtr driver, + virDomainObjPtr vm, + int asyncJob); + bool qemuDomainDefCheckABIStability(virQEMUDriverPtr driver, virDomainDefPtr src, virDomainDefPtr dst); diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index 9b86695..bd4d137 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -4344,3 +4344,45 @@ void qemuMonitorIOThreadsInfoFree(qemuMonitorIOThreadsInfoPtr iothread) VIR_FREE(iothread->name); VIR_FREE(iothread); } + + +/** + * qemuMonitorGetMemoryDeviceInfo: + * @mon: pointer to the monitor + * @info: Location to return the hash of qemuMonitorMemoryDeviceInfo + * + * Retrieve state and addresses of frontend memory devices present in + * the guest. + * + * Returns 0 on success and fills @info with a newly allocated struct; if the + * data can't be retrieved due to lack of support in qemu, returns -2. On + * other errors returns -1. + */ +int +qemuMonitorGetMemoryDeviceInfo(qemuMonitorPtr mon, + virHashTablePtr *info) +{ + VIR_DEBUG("mon=%p info=%p", mon, info); + int ret; + + *info = NULL; + + if (!mon) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("monitor must not be NULL")); + return -1; + } + + if (!mon->json) + return -2; + + if (!(*info = virHashCreate(10, virHashValueFree))) + return -1; + + if ((ret = qemuMonitorJSONGetMemoryDeviceInfo(mon, *info)) < 0) { + virHashFree(*info); + *info = NULL; + } + + return ret; +} diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h index b30da34..8dc4a1c 100644 --- a/src/qemu/qemu_monitor.h +++ b/src/qemu/qemu_monitor.h @@ -877,6 +877,20 @@ int qemuMonitorGetIOThreads(qemuMonitorPtr mon, void qemuMonitorIOThreadsInfoFree(qemuMonitorIOThreadsInfoPtr iothread); +typedef struct _qemuMonitorMemoryDeviceInfo qemuMonitorMemoryDeviceInfo; +typedef qemuMonitorMemoryDeviceInfo *qemuMonitorMemoryDeviceInfoPtr; + +struct _qemuMonitorMemoryDeviceInfo { + unsigned long long address; + unsigned int slot; + bool hotplugged; + bool hotpluggable; +}; + +int qemuMonitorGetMemoryDeviceInfo(qemuMonitorPtr mon, + virHashTablePtr *info) + ATTRIBUTE_NONNULL(2); + /** * When running two dd process and using <> redirection, we need a * shell that will not truncate files. These two strings serve that diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c index c16f3ca..b120b76 100644 --- a/src/qemu/qemu_monitor_json.c +++ b/src/qemu/qemu_monitor_json.c @@ -6392,3 +6392,125 @@ qemuMonitorJSONGetIOThreads(qemuMonitorPtr mon, virJSONValueFree(reply); return ret; } + + +int +qemuMonitorJSONGetMemoryDeviceInfo(qemuMonitorPtr mon, + virHashTablePtr info) +{ + int ret = -1; + virJSONValuePtr cmd; + virJSONValuePtr reply = NULL; + virJSONValuePtr data = NULL; + qemuMonitorMemoryDeviceInfoPtr meminfo = NULL; + ssize_t n; + size_t i; + + if (!(cmd = qemuMonitorJSONMakeCommand("query-memory-devices", NULL))) + return -1; + + ret = qemuMonitorJSONCommand(mon, cmd, &reply); + + if (ret == 0) { + if (qemuMonitorJSONHasError(reply, "CommandNotFound")) { + ret = -2; + goto cleanup; + } + + ret = qemuMonitorJSONCheckError(cmd, reply); + } + + if (ret < 0) + goto cleanup; + + ret = -1; + + if (!(data = virJSONValueObjectGet(reply, "return"))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("query-memory-devices reply was missing return data")); + goto cleanup; + } + + if ((n = virJSONValueArraySize(data)) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("query-memory-devices reply data was not an array")); + goto cleanup; + } + + for (i = 0; i < n; i++) { + virJSONValuePtr elem = virJSONValueArrayGet(data, i); + const char *type; + + if (!(type = virJSONValueObjectGetString(elem, "type"))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("query-memory-devices reply data doesn't contain " + "enum type discriminator")); + goto cleanup; + } + + /* dimm memory devices */ + if (STREQ(type, "dimm")) { + virJSONValuePtr dimminfo; + const char *devalias; + + if (!(dimminfo = virJSONValueObjectGet(elem, "data"))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("query-memory-devices reply data doesn't " + "contain enum data")); + goto cleanup; + } + + if (!(devalias = virJSONValueObjectGetString(dimminfo, "id"))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("dimm memory info data is missing 'id'")); + goto cleanup; + } + + if (VIR_ALLOC(meminfo) < 0) + goto cleanup; + + if (virJSONValueObjectGetNumberUlong(dimminfo, "addr", + &meminfo->address) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("malformed/missing addr in dimm memory info")); + goto cleanup; + } + + if (virJSONValueObjectGetNumberUint(dimminfo, "slot", + &meminfo->slot) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("malformed/missing slot in dimm memory info")); + goto cleanup; + } + + if (virJSONValueObjectGetBoolean(dimminfo, "hotplugged", + &meminfo->hotplugged) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("malformed/missing hotplugged in dimm memory info")); + goto cleanup; + + } + + if (virJSONValueObjectGetBoolean(dimminfo, "hotpluggable", + &meminfo->hotpluggable) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("malformed/missing hotpluggable in dimm memory info")); + goto cleanup; + + } + + if (virHashAddEntry(info, devalias, meminfo) < 0) + goto cleanup; + + meminfo = NULL; + } + } + + ret = 0; + + cleanup: + VIR_FREE(meminfo); + virJSONValueFree(cmd); + virJSONValueFree(reply); + return ret; +} diff --git a/src/qemu/qemu_monitor_json.h b/src/qemu/qemu_monitor_json.h index 8ceea8a..455e96f 100644 --- a/src/qemu/qemu_monitor_json.h +++ b/src/qemu/qemu_monitor_json.h @@ -462,4 +462,9 @@ int qemuMonitorJSONRTCResetReinjection(qemuMonitorPtr mon); int qemuMonitorJSONGetIOThreads(qemuMonitorPtr mon, qemuMonitorIOThreadsInfoPtr **iothreads) ATTRIBUTE_NONNULL(2); + +int qemuMonitorJSONGetMemoryDeviceInfo(qemuMonitorPtr mon, + virHashTablePtr info) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); + #endif /* QEMU_MONITOR_JSON_H */ diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index ae315df..fefb498 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -4908,6 +4908,10 @@ int qemuProcessStart(virConnectPtr conn, if (qemuDomainUpdateDeviceList(driver, vm, asyncJob) < 0) goto cleanup; + VIR_DEBUG("Updating info of memory devices"); + if (qemuDomainUpdateMemoryDeviceInfo(driver, vm, asyncJob) < 0) + goto cleanup; + /* Technically, qemuProcessStart can be called from inside * QEMU_ASYNC_JOB_MIGRATION_IN, but we are okay treating this like * a sync job since no other job can call into the domain until -- 2.2.2

Peter Krempa

10:19 a.m.

New subject: [libvirt] [PATCHv3 08/12] qemu: migration: Forbid migration with memory modules lacking info

Make sure that libvirt has all vital information needed to reliably represent configuration of guest's memory devices in case of a migration. This patch forbids migration in case the required slot number and module base address are not present (failed to be loaded from qemu via monitor). --- Notes: Version 2: - rebased to the drop of ACPI src/qemu/qemu_migration.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 83be435..f36a36c 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -2016,6 +2016,20 @@ qemuMigrationIsAllowed(virQEMUDriverPtr driver, virDomainObjPtr vm, } } + /* Verify that memory device config can be transferred reliably */ + for (i = 0; i < def->nmems; i++) { + virDomainMemoryDefPtr mem = def->mems[i]; + + if (mem->model == VIR_DOMAIN_MEMORY_MODEL_DIMM && + mem->info.type != VIR_DOMAIN_DEVICE_ADDRESS_TYPE_DIMM) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("domain's dimm info lacks slot ID " + "or base address")); + + return false; + } + } + return true; } -- 2.2.2

Peter Krempa

10:19 a.m.

New subject: [libvirt] [PATCHv3 09/12] qemu: add support for memory devices

Add support to start qemu instance with 'pc-dimm' device. Thanks to the refactors we are able to reuse the existing function to determine the parameters. --- Notes: Version 2: - dropped the ACPI naming src/qemu/qemu_command.c | 130 ++++++++++++++++++++- src/qemu/qemu_domain.c | 26 ++++- src/qemu/qemu_domain.h | 1 + .../qemuxml2argv-memory-hotplug-dimm.args | 11 ++ tests/qemuxml2argvtest.c | 2 + tests/qemuxml2xmltest.c | 1 + 6 files changed, 167 insertions(+), 4 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-dimm.args diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index ef8feeb..2d85567 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -1222,6 +1222,10 @@ qemuAssignDeviceAliases(virDomainDefPtr def, virQEMUCapsPtr qemuCaps) if (virAsprintf(&def->tpm->info.alias, "tpm%d", 0) < 0) return -1; } + for (i = 0; i < def->nmems; i++) { + if (virAsprintf(&def->mems[i]->info.alias, "dimm%zu", i) < 0) + return -1; + } return 0; } @@ -4612,8 +4616,7 @@ qemuBuildMemoryBackendStr(unsigned long long size, virDomainHugePagePtr hugepage = NULL; virDomainNumatuneMemMode mode; const long system_page_size = virGetSystemPageSizeKB(); - virNumaMemAccess memAccess = virDomainNumaGetNodeMemoryAccessMode(def->numa, guestNode); - + virNumaMemAccess memAccess = VIR_NUMA_MEM_ACCESS_DEFAULT; size_t i; char *mem_path = NULL; virBitmapPtr nodemask = NULL; @@ -4626,6 +4629,16 @@ qemuBuildMemoryBackendStr(unsigned long long size, if (!(props = virJSONValueNewObject())) return -1; + /* memory devices could provide a invalid guest node */ + if (guestNode >= virDomainNumaGetNodeCount(def->numa)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("can't add memory backend for guest node '%d' as " + "the guest has only '%zu' NUMA nodes configured"), + guestNode, virDomainNumaGetNodeCount(def->numa)); + return -1; + } + + memAccess = virDomainNumaGetNodeMemoryAccessMode(def->numa, guestNode); mode = virDomainNumatuneGetMode(def->numa, guestNode); if (pagesize == 0 || pagesize != system_page_size) { @@ -4823,6 +4836,95 @@ qemuBuildMemoryCellBackendStr(virDomainDefPtr def, } +static char * +qemuBuildMemoryDimmBackendStr(virDomainMemoryDefPtr mem, + virDomainDefPtr def, + virQEMUCapsPtr qemuCaps, + virQEMUDriverConfigPtr cfg) +{ + virJSONValuePtr props = NULL; + char *alias = NULL; + const char *backendType; + char *ret = NULL; + + if (!mem->info.alias) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("memory device alias is not assigned")); + return NULL; + } + + if (virAsprintf(&alias, "mem%s", mem->info.alias) < 0) + goto cleanup; + + if (qemuBuildMemoryBackendStr(mem->size, mem->pagesize, + mem->targetNode, mem->sourceNodes, NULL, + def, qemuCaps, cfg, + &backendType, &props, true) < 0) + goto cleanup; + + ret = qemuBuildObjectCommandlineFromJSON(backendType, alias, props); + + cleanup: + VIR_FREE(alias); + virJSONValueFree(props); + + return ret; +} + + +static char * +qemuBuildMemoryDeviceStr(virDomainMemoryDefPtr mem, + virQEMUCapsPtr qemuCaps) +{ + virBuffer buf = VIR_BUFFER_INITIALIZER; + + if (!mem->info.alias) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("missing alias for memory device")); + return NULL; + } + + switch ((virDomainMemoryModel) mem->model) { + case VIR_DOMAIN_MEMORY_MODEL_DIMM: + if (!virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_PC_DIMM)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("this qemu doesn't support the pc-dimm device")); + return NULL; + } + + if (mem->info.type != VIR_DOMAIN_DEVICE_ADDRESS_TYPE_DIMM && + mem->info.type != VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("only 'dimm' addresses are supported for the " + "pc-dimm device")); + return NULL; + } + + virBufferAsprintf(&buf, "pc-dimm,node=%d,memdev=mem%s,id=%s", + mem->targetNode, mem->info.alias, mem->info.alias); + + if (mem->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_DIMM) { + virBufferAsprintf(&buf, ",slot=%d", mem->info.addr.dimm.slot); + virBufferAsprintf(&buf, ",base=%llu", mem->info.addr.dimm.base); + } + + break; + + case VIR_DOMAIN_MEMORY_MODEL_NONE: + case VIR_DOMAIN_MEMORY_MODEL_LAST: + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("invalid memory device type")); + break; + + } + + if (virBufferCheckError(&buf) < 0) + return NULL; + + return virBufferContentAndReset(&buf); +} + + char * qemuBuildNicStr(virDomainNetDefPtr net, const char *prefix, @@ -8599,10 +8701,32 @@ qemuBuildCommandLine(virConnectPtr conn, } } - if (virDomainNumaGetNodeCount(def->numa)) + if (virDomainNumaGetNodeCount(def->numa)) { if (qemuBuildNumaArgStr(cfg, def, cmd, qemuCaps, nodeset) < 0) goto error; + /* memory hotplug requires NUMA to be enabled - we already checked + * that memory devices are present only when NUMA is */ + for (i = 0; i < def->nmems; i++) { + char *backStr; + char *dimmStr; + + if (!(backStr = qemuBuildMemoryDimmBackendStr(def->mems[i], def, + qemuCaps, cfg))) + goto error; + + if (!(dimmStr = qemuBuildMemoryDeviceStr(def->mems[i], qemuCaps))) { + VIR_FREE(backStr); + goto error; + } + + virCommandAddArgList(cmd, "-object", backStr, "-device", dimmStr, NULL); + + VIR_FREE(backStr); + VIR_FREE(dimmStr); + } + } + if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_UUID)) virCommandAddArgList(cmd, "-uuid", uuid, NULL); if (def->virtType == VIR_DOMAIN_VIRT_XEN || diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 825c02e..b46222d 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -1193,8 +1193,13 @@ qemuDomainDeviceDefPostParse(virDomainDeviceDefPtr dev, } } - if (virDomainDeviceDefCheckUnsupportedMemoryDevice(dev) < 0) + if (dev->type == VIR_DOMAIN_DEVICE_MEMORY && + def->mem.max_memory == 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("maxMemory has to be specified when using memory " + "devices ")); goto cleanup; + } ret = 0; @@ -2950,5 +2955,24 @@ qemuDomainAlignMemorySizes(virDomainDefPtr def) * We'll take the "traditional" path and round it to 1MiB*/ def->mem.max_memory = VIR_ROUND_UP(def->mem.max_memory, 1024); + /* Align memory module sizes */ + for (i = 0; i < def->nmems; i++) + qemuDomainMemoryDeviceAlignSize(def->mems[i]); + return 0; } + + +/** + * qemuDomainMemoryDeviceAlignSize: + * @mem: memory device definition object + * + * Aligns the size of the memory module as qemu enforces it. The size is updated + * inplace. Default rounding is now to 1 MiB (qemu requires rouding to page, + * size so this should be safe). + */ +void +qemuDomainMemoryDeviceAlignSize(virDomainMemoryDefPtr mem) +{ + mem->size = VIR_ROUND_UP(mem->size, 1024); +} diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 75e82f3..28eefac 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -423,5 +423,6 @@ bool qemuDomainDiskBlockJobIsActive(virDomainDiskDefPtr disk); void qemuDomObjEndAPI(virDomainObjPtr *vm); int qemuDomainAlignMemorySizes(virDomainDefPtr def); +void qemuDomainMemoryDeviceAlignSize(virDomainMemoryDefPtr mem); #endif /* __QEMU_DOMAIN_H__ */ diff --git a/tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-dimm.args b/tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-dimm.args new file mode 100644 index 0000000..7fbde33 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-dimm.args @@ -0,0 +1,11 @@ +LC_ALL=C PATH=/bin HOME=/home/test USER=test LOGNAME=test QEMU_AUDIO_DRV=none \ +/usr/bin/qemu -S -M pc -m size=219136k,slots=16,maxmem=1099511627776k -smp 2 \ +-numa node,nodeid=0,cpus=0-1,mem=214 \ +-object memory-backend-ram,id=memdimm0,size=536870912 \ +-device pc-dimm,node=0,memdev=memdimm0,id=dimm0 \ +-object memory-backend-ram,id=memdimm1,size=536870912,host-nodes=1-3,\ +policy=bind \ +-device pc-dimm,node=0,memdev=memdimm1,id=dimm1 \ +-nographic -nodefaults -monitor unix:/tmp/test-monitor,server,nowait \ +-no-acpi -boot c -usb -hda /dev/HostVG/QEMUGuest1 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 387b349..08f374e 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1546,6 +1546,8 @@ mymain(void) DO_TEST_FAILURE("memory-hotplug-nonuma", QEMU_CAPS_DEVICE_PC_DIMM); DO_TEST_FAILURE("memory-hotplug", NONE); DO_TEST("memory-hotplug", QEMU_CAPS_DEVICE_PC_DIMM, QEMU_CAPS_NUMA); + DO_TEST("memory-hotplug-dimm", QEMU_CAPS_DEVICE_PC_DIMM, QEMU_CAPS_NUMA, + QEMU_CAPS_DEVICE, QEMU_CAPS_OBJECT_MEMORY_RAM); virObjectUnref(driver.config); virObjectUnref(driver.caps); diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index ed462b8..ecd77c0 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -429,6 +429,7 @@ mymain(void) DO_TEST("memory-hotplug"); DO_TEST("memory-hotplug-nonuma"); + DO_TEST("memory-hotplug-dimm"); virObjectUnref(driver.caps); virObjectUnref(driver.xmlopt); -- 2.2.2

John Ferlan

19 Mar 19 Mar

6:38 p.m.

New subject: [libvirt] [PATCHv3 09/12] qemu: add support for memory devices

On 03/17/2015 10:19 AM, Peter Krempa wrote:

...

Add support to start qemu instance with 'pc-dimm' device. Thanks to the refactors we are able to reuse the existing function to determine the parameters. ---

Notes: Version 2: - dropped the ACPI naming

src/qemu/qemu_command.c | 130 ++++++++++++++++++++- src/qemu/qemu_domain.c | 26 ++++- src/qemu/qemu_domain.h | 1 + .../qemuxml2argv-memory-hotplug-dimm.args | 11 ++ tests/qemuxml2argvtest.c | 2 + tests/qemuxml2xmltest.c | 1 + 6 files changed, 167 insertions(+), 4 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-dimm.args

diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index ef8feeb..2d85567 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -1222,6 +1222,10 @@ qemuAssignDeviceAliases(virDomainDefPtr def, virQEMUCapsPtr qemuCaps) if (virAsprintf(&def->tpm->info.alias, "tpm%d", 0) < 0) return -1; } + for (i = 0; i < def->nmems; i++) { + if (virAsprintf(&def->mems[i]->info.alias, "dimm%zu", i) < 0) + return -1; + }

return 0; } @@ -4612,8 +4616,7 @@ qemuBuildMemoryBackendStr(unsigned long long size, virDomainHugePagePtr hugepage = NULL; virDomainNumatuneMemMode mode; const long system_page_size = virGetSystemPageSizeKB(); - virNumaMemAccess memAccess = virDomainNumaGetNodeMemoryAccessMode(def->numa, guestNode); - + virNumaMemAccess memAccess = VIR_NUMA_MEM_ACCESS_DEFAULT; size_t i; char *mem_path = NULL; virBitmapPtr nodemask = NULL; @@ -4626,6 +4629,16 @@ qemuBuildMemoryBackendStr(unsigned long long size, if (!(props = virJSONValueNewObject())) return -1;

+ /* memory devices could provide a invalid guest node */ + if (guestNode >= virDomainNumaGetNodeCount(def->numa)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("can't add memory backend for guest node '%d' as " + "the guest has only '%zu' NUMA nodes configured"), + guestNode, virDomainNumaGetNodeCount(def->numa));

Coverity points out that 'props' is being leaked here. So probably should go to cleanup or move this entire hunk above the props = line. In my test I just moved these lines above the props fetch and Coverity was happy John

...

+ return -1; + } + + memAccess = virDomainNumaGetNodeMemoryAccessMode(def->numa, guestNode); mode = virDomainNumatuneGetMode(def->numa, guestNode);

if (pagesize == 0 || pagesize != system_page_size) { @@ -4823,6 +4836,95 @@ qemuBuildMemoryCellBackendStr(virDomainDefPtr def, }

Peter Krempa

17 Mar 17 Mar

10:19 a.m.

New subject: [libvirt] [PATCHv3 10/12] qemu: conf: Add support for memory device cold(un)plug

Add a few helpers that allow to operate with memory device definitions on the domain config and use them to implement memory device coldplug in the qemu driver. --- Notes: Version 2: - no changes src/conf/domain_conf.c | 100 +++++++++++++++++++++++++++++++++++++++++++++++ src/conf/domain_conf.h | 10 +++++ src/libvirt_private.syms | 4 ++ src/qemu/qemu_driver.c | 15 ++++++- 4 files changed, 127 insertions(+), 2 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 8c2234f..1a02e46 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -12811,6 +12811,106 @@ virDomainRNGRemove(virDomainDefPtr def, } +static int +virDomainMemoryFindByDefInternal(virDomainDefPtr def, + virDomainMemoryDefPtr mem, + bool allowAddressFallback) +{ + size_t i; + + for (i = 0; i < def->nmems; i++) { + virDomainMemoryDefPtr tmp = def->mems[i]; + + /* address, if present */ + if (allowAddressFallback) { + if (tmp->info.type != VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE) + continue; + } else { + if (mem->info.type != VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE && + !virDomainDeviceInfoAddressIsEqual(&tmp->info, &mem->info)) + continue; + } + + /* alias, if present */ + if (mem->info.alias && + STRNEQ_NULLABLE(tmp->info.alias, mem->info.alias)) + continue; + + /* target info -> always present */ + if (tmp->model != mem->model || + tmp->targetNode != mem->targetNode || + tmp->size != mem->size) + continue; + + /* source stuff -> match with device */ + if (tmp->pagesize != mem->pagesize) + continue; + + if (!virBitmapEqual(tmp->sourceNodes, mem->sourceNodes)) + continue; + + break; + } + + if (i == def->nmems) + return -1; + + return i; +} + + +int +virDomainMemoryFindByDef(virDomainDefPtr def, + virDomainMemoryDefPtr mem) +{ + return virDomainMemoryFindByDefInternal(def, mem, false); +} + + +int +virDomainMemoryFindInactiveByDef(virDomainDefPtr def, + virDomainMemoryDefPtr mem) +{ + int ret; + + if ((ret = virDomainMemoryFindByDefInternal(def, mem, false)) < 0) + ret = virDomainMemoryFindByDefInternal(def, mem, true); + + return ret; +} + + +int +virDomainMemoryInsert(virDomainDefPtr def, + virDomainMemoryDefPtr mem) +{ + int id = def->nmems; + + if (mem->info.type != VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE && + virDomainDefHasDeviceAddress(def, &mem->info)) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("Domain already contains a device with the same " + "address")); + return -1; + } + + if (VIR_APPEND_ELEMENT(def->mems, def->nmems, mem) < 0) + return -1; + + return id; +} + + +virDomainMemoryDefPtr +virDomainMemoryRemove(virDomainDefPtr def, + int idx) +{ + virDomainMemoryDefPtr ret = def->mems[idx]; + VIR_DELETE_ELEMENT(def->mems, idx, def->nmems); + return ret; +} + + char * virDomainDefGetDefaultEmulator(virDomainDefPtr def, virCapsPtr caps) diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 475a174..eb61aff 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -2860,6 +2860,16 @@ virDomainChrDefGetSecurityLabelDef(virDomainChrDefPtr def, const char *model); typedef const char* (*virEventActionToStringFunc)(int type); typedef int (*virEventActionFromStringFunc)(const char *type); +int virDomainMemoryInsert(virDomainDefPtr def, virDomainMemoryDefPtr mem) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_RETURN_CHECK; +virDomainMemoryDefPtr virDomainMemoryRemove(virDomainDefPtr def, int idx) + ATTRIBUTE_NONNULL(1); +int virDomainMemoryFindByDef(virDomainDefPtr def, virDomainMemoryDefPtr mem) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_RETURN_CHECK; +int virDomainMemoryFindInactiveByDef(virDomainDefPtr def, + virDomainMemoryDefPtr mem) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_RETURN_CHECK; + VIR_ENUM_DECL(virDomainTaint) VIR_ENUM_DECL(virDomainVirt) VIR_ENUM_DECL(virDomainBoot) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index f24b449..239bef4 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -335,6 +335,10 @@ virDomainLockFailureTypeToString; virDomainMemballoonModelTypeFromString; virDomainMemballoonModelTypeToString; virDomainMemoryDefFree; +virDomainMemoryFindByDef; +virDomainMemoryFindInactiveByDef; +virDomainMemoryInsert; +virDomainMemoryRemove; virDomainNetAppendIpAddress; virDomainNetDefFormat; virDomainNetDefFree; diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 300bce4..e948cca 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -7992,7 +7992,10 @@ qemuDomainAttachDeviceConfig(virQEMUCapsPtr qemuCaps, break; case VIR_DOMAIN_DEVICE_MEMORY: - /* TODO: implement later */ + if (virDomainMemoryInsert(vmdef, dev->data.memory) < 0) + return -1; + dev->data.memory = NULL; + break; case VIR_DOMAIN_DEVICE_INPUT: case VIR_DOMAIN_DEVICE_SOUND: @@ -8120,7 +8123,15 @@ qemuDomainDetachDeviceConfig(virDomainDefPtr vmdef, break; case VIR_DOMAIN_DEVICE_MEMORY: - /* TODO: implement later */ + if ((idx = virDomainMemoryFindInactiveByDef(vmdef, + dev->data.memory)) < 0) { + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("matching memory device was not found")); + return -1; + } + + virDomainMemoryDefFree(virDomainMemoryRemove(vmdef, idx)); + break; case VIR_DOMAIN_DEVICE_INPUT: case VIR_DOMAIN_DEVICE_SOUND: -- 2.2.2

Zhu Guihua

18 Mar 18 Mar

1:51 a.m.

New subject: [libvirt] [PATCHv3 10/12] qemu: conf: Add support for memory device cold(un)plug

On 03/17/2015 10:19 PM, Peter Krempa wrote:

...

Add a few helpers that allow to operate with memory device definitions on the domain config and use them to implement memory device coldplug in the qemu driver. ---

Notes: Version 2: - no changes

src/conf/domain_conf.c | 100 +++++++++++++++++++++++++++++++++++++++++++++++ src/conf/domain_conf.h | 10 +++++ src/libvirt_private.syms | 4 ++ src/qemu/qemu_driver.c | 15 ++++++- 4 files changed, 127 insertions(+), 2 deletions(-)

diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 8c2234f..1a02e46 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -12811,6 +12811,106 @@ virDomainRNGRemove(virDomainDefPtr def, }

+static int +virDomainMemoryFindByDefInternal(virDomainDefPtr def, + virDomainMemoryDefPtr mem, + bool allowAddressFallback) +{ + size_t i; + + for (i = 0; i < def->nmems; i++) { + virDomainMemoryDefPtr tmp = def->mems[i]; + + /* address, if present */ + if (allowAddressFallback) { + if (tmp->info.type != VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE) + continue; + } else { + if (mem->info.type != VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE && + !virDomainDeviceInfoAddressIsEqual(&tmp->info, &mem->info)) + continue; + } + + /* alias, if present */ + if (mem->info.alias && + STRNEQ_NULLABLE(tmp->info.alias, mem->info.alias)) + continue; + + /* target info -> always present */ + if (tmp->model != mem->model || + tmp->targetNode != mem->targetNode || + tmp->size != mem->size)

I have tested your series with our qemu memory hot remove patch series, here would be a possible error. When hotplug a memory device, its size has been aligned. So the compare for size here would fail possiblely. Thanks, Zhu

...

+ continue; + + /* source stuff -> match with device */ + if (tmp->pagesize != mem->pagesize) + continue; + + if (!virBitmapEqual(tmp->sourceNodes, mem->sourceNodes)) + continue; + + break; + } + + if (i == def->nmems) + return -1; + + return i; +} + + +int +virDomainMemoryFindByDef(virDomainDefPtr def, + virDomainMemoryDefPtr mem) +{ + return virDomainMemoryFindByDefInternal(def, mem, false); +} + [...]

John Ferlan

19 Mar 19 Mar

6:39 p.m.

New subject: [libvirt] [PATCHv3 10/12] qemu: conf: Add support for memory device cold(un)plug

On 03/18/2015 01:51 AM, Zhu Guihua wrote:

...

On 03/17/2015 10:19 PM, Peter Krempa wrote:

...
Add a few helpers that allow to operate with memory device definitions on the domain config and use them to implement memory device coldplug in the qemu driver. ---

Notes: Version 2: - no changes

src/conf/domain_conf.c | 100 +++++++++++++++++++++++++++++++++++++++++++++++ src/conf/domain_conf.h | 10 +++++ src/libvirt_private.syms | 4 ++ src/qemu/qemu_driver.c | 15 ++++++- 4 files changed, 127 insertions(+), 2 deletions(-)

diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 8c2234f..1a02e46 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -12811,6 +12811,106 @@ virDomainRNGRemove(virDomainDefPtr def, }

+static int +virDomainMemoryFindByDefInternal(virDomainDefPtr def, + virDomainMemoryDefPtr mem, + bool allowAddressFallback) +{ + size_t i; + + for (i = 0; i < def->nmems; i++) { + virDomainMemoryDefPtr tmp = def->mems[i]; + + /* address, if present */ + if (allowAddressFallback) { + if (tmp->info.type != VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE) + continue; + } else { + if (mem->info.type != VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE && + !virDomainDeviceInfoAddressIsEqual(&tmp->info, &mem->info)) + continue; + } + + /* alias, if present */ + if (mem->info.alias && + STRNEQ_NULLABLE(tmp->info.alias, mem->info.alias)) + continue; + + /* target info -> always present */ + if (tmp->model != mem->model || + tmp->targetNode != mem->targetNode || + tmp->size != mem->size)

I have tested your series with our qemu memory hot remove patch series, here would be a possible error.

When hotplug a memory device, its size has been aligned. So the compare for size here would fail possiblely.

hmm.. Not sure that's necessary - although Peter can make the final determination... Commit id '57b215a' doesn't modify each def->mems[i] entry in qemuDomainAlignMemorySizes, rather it gets a value from virDomainDefSetMemoryInitial and then does the rounding. If the stored def->mems[i]->size value is/was modified, then I'd agree, but it doesn't appear to be that way. If there is a rounding of the value - then please just point it out John

...

Thanks, Zhu

...
+ continue; + + /* source stuff -> match with device */ + if (tmp->pagesize != mem->pagesize) + continue; + + if (!virBitmapEqual(tmp->sourceNodes, mem->sourceNodes)) + continue; + + break; + } + + if (i == def->nmems) + return -1; + + return i; +} + + +int +virDomainMemoryFindByDef(virDomainDefPtr def, + virDomainMemoryDefPtr mem) +{ + return virDomainMemoryFindByDefInternal(def, mem, false); +} + [...]

-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list

Zhu Guihua

9:03 p.m.

New subject: [libvirt] [PATCHv3 10/12] qemu: conf: Add support for memory device cold(un)plug

On 03/20/2015 06:39 AM, John Ferlan wrote:

...

On 03/18/2015 01:51 AM, Zhu Guihua wrote:

...
On 03/17/2015 10:19 PM, Peter Krempa wrote:

...
Add a few helpers that allow to operate with memory device definitions on the domain config and use them to implement memory device coldplug in the qemu driver. ---

Notes: Version 2: - no changes

src/conf/domain_conf.c | 100 +++++++++++++++++++++++++++++++++++++++++++++++ src/conf/domain_conf.h | 10 +++++ src/libvirt_private.syms | 4 ++ src/qemu/qemu_driver.c | 15 ++++++- 4 files changed, 127 insertions(+), 2 deletions(-)

diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 8c2234f..1a02e46 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -12811,6 +12811,106 @@ virDomainRNGRemove(virDomainDefPtr def, }

+static int +virDomainMemoryFindByDefInternal(virDomainDefPtr def, + virDomainMemoryDefPtr mem, + bool allowAddressFallback) +{ + size_t i; + + for (i = 0; i < def->nmems; i++) { + virDomainMemoryDefPtr tmp = def->mems[i]; + + /* address, if present */ + if (allowAddressFallback) { + if (tmp->info.type != VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE) + continue; + } else { + if (mem->info.type != VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE && + !virDomainDeviceInfoAddressIsEqual(&tmp->info, &mem->info)) + continue; + } + + /* alias, if present */ + if (mem->info.alias && + STRNEQ_NULLABLE(tmp->info.alias, mem->info.alias)) + continue; + + /* target info -> always present */ + if (tmp->model != mem->model || + tmp->targetNode != mem->targetNode || + tmp->size != mem->size) I have tested your series with our qemu memory hot remove patch series, here would be a possible error.

When hotplug a memory device, its size has been aligned. So the compare for size here would fail possiblely.

hmm.. Not sure that's necessary - although Peter can make the final determination... Commit id '57b215a' doesn't modify each def->mems[i] entry in qemuDomainAlignMemorySizes, rather it gets a value from virDomainDefSetMemoryInitial and then does the rounding.

If the stored def->mems[i]->size value is/was modified, then I'd agree, but it doesn't appear to be that way.

If there is a rounding of the value - then please just point it out

Yes, the stored def->mems[i]->size value was modified. If you assign the size 524287 KiB, the stored value will be 524288. Thanks, Zhu

...

John

...
Thanks, Zhu

...
+ continue; + + /* source stuff -> match with device */ + if (tmp->pagesize != mem->pagesize) + continue; + + if (!virBitmapEqual(tmp->sourceNodes, mem->sourceNodes)) + continue; + + break; + } + + if (i == def->nmems) + return -1; + + return i; +} + + +int +virDomainMemoryFindByDef(virDomainDefPtr def, + virDomainMemoryDefPtr mem) +{ + return virDomainMemoryFindByDefInternal(def, mem, false); +} + [...]

-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list .

John Ferlan

20 Mar 20 Mar

7:40 a.m.

New subject: [libvirt] [PATCHv3 10/12] qemu: conf: Add support for memory device cold(un)plug

...

...
...
I have tested your series with our qemu memory hot remove patch series, here would be a possible error.

When hotplug a memory device, its size has been aligned. So the compare for size here would fail possiblely.

hmm.. Not sure that's necessary - although Peter can make the final determination... Commit id '57b215a' doesn't modify each def->mems[i] entry in qemuDomainAlignMemorySizes, rather it gets a value from virDomainDefSetMemoryInitial and then does the rounding.

If the stored def->mems[i]->size value is/was modified, then I'd agree, but it doesn't appear to be that way.

If there is a rounding of the value - then please just point it out

Yes, the stored def->mems[i]->size value was modified. If you assign the size 524287 KiB, the stored value will be 524288.

Thanks, Zhu

Ah - found it - patch 9 has: + /* Align memory module sizes */ + for (i = 0; i < def->nmems; i++) + qemuDomainMemoryDeviceAlignSize(def->mems[i]); + Which I missed on my first foray through this. Once I cscope'd on VIR_ROUND_UP() instead of ->size, it became apparent So yes, it seems the to be compared size needs a likewise adjustment. John

Peter Krempa

23 Mar 23 Mar

9:41 a.m.

New subject: [libvirt] [PATCHv3 10/12] qemu: conf: Add support for memory device cold(un)plug

On Fri, Mar 20, 2015 at 07:40:21 -0400, John Ferlan wrote:

...

...
...
...
I have tested your series with our qemu memory hot remove patch series, here would be a possible error.

When hotplug a memory device, its size has been aligned. So the compare for size here would fail possiblely.

hmm.. Not sure that's necessary - although Peter can make the final determination... Commit id '57b215a' doesn't modify each def->mems[i] entry in qemuDomainAlignMemorySizes, rather it gets a value from virDomainDefSetMemoryInitial and then does the rounding.

If the stored def->mems[i]->size value is/was modified, then I'd agree, but it doesn't appear to be that way.

If there is a rounding of the value - then please just point it out

Yes, the stored def->mems[i]->size value was modified. If you assign the size 524287 KiB, the stored value will be 524288.

Thanks, Zhu

Ah - found it - patch 9 has:

+ /* Align memory module sizes */ + for (i = 0; i < def->nmems; i++) + qemuDomainMemoryDeviceAlignSize(def->mems[i]); +

Which I missed on my first foray through this. Once I cscope'd on VIR_ROUND_UP() instead of ->size, it became apparent

So yes, it seems the to be compared size needs a likewise adjustment.

Indeed, but the size needs to be aligned only for the active definition as we only align that one, thus it belongs to patch 12/12. I'll be adding the following diff to 12/12: diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 40041d5..9b8d11b 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -4189,6 +4189,8 @@ qemuDomainDetachMemoryDevice(virQEMUDriverPtr driver, return -1; } + qemuDomainMemoryDeviceAlignSize(memdef); + if ((idx = virDomainMemoryFindByDef(vm->def, memdef)) < 0) { virReportError(VIR_ERR_OPERATION_INVALID, "%s", _("device not present in domain configuration")); Peter

Peter Krempa

17 Mar 17 Mar

10:20 a.m.

New subject: [libvirt] [PATCHv3 11/12] qemu: Implement memory device hotplug

Add code to hot-add memory devices to running qemu instances. --- Notes: Version 3: - added comment to clarify that @mem is always consumed by qemuDomainAttachMemory Version 2: - no change Version 2: - no change src/qemu/qemu_command.c | 4 +-- src/qemu/qemu_command.h | 15 ++++++++ src/qemu/qemu_driver.c | 5 ++- src/qemu/qemu_hotplug.c | 95 +++++++++++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_hotplug.h | 3 ++ 5 files changed, 119 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 2d85567..1f72437 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -4599,7 +4599,7 @@ qemuBuildControllerDevStr(virDomainDefPtr domainDef, * other configuration was used (to detect legacy configurations). Returns * -1 in case of an error. */ -static int +int qemuBuildMemoryBackendStr(unsigned long long size, unsigned long long pagesize, int guestNode, @@ -4872,7 +4872,7 @@ qemuBuildMemoryDimmBackendStr(virDomainMemoryDefPtr mem, } -static char * +char * qemuBuildMemoryDeviceStr(virDomainMemoryDefPtr mem, virQEMUCapsPtr qemuCaps) { diff --git a/src/qemu/qemu_command.h b/src/qemu/qemu_command.h index ee81f92..a29db41 100644 --- a/src/qemu/qemu_command.h +++ b/src/qemu/qemu_command.h @@ -162,6 +162,21 @@ char *qemuBuildSoundDevStr(virDomainDefPtr domainDef, virDomainSoundDefPtr sound, virQEMUCapsPtr qemuCaps); +int qemuBuildMemoryBackendStr(unsigned long long size, + unsigned long long pagesize, + int guestNode, + virBitmapPtr userNodeset, + virBitmapPtr autoNodeset, + virDomainDefPtr def, + virQEMUCapsPtr qemuCaps, + virQEMUDriverConfigPtr cfg, + const char **backendType, + virJSONValuePtr *backendProps, + bool force); + +char *qemuBuildMemoryDeviceStr(virDomainMemoryDefPtr mem, + virQEMUCapsPtr qemuCaps); + /* Legacy, pre device support */ char *qemuBuildPCIHostdevPCIDevStr(virDomainHostdevDefPtr dev, virQEMUCapsPtr qemuCaps); diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index e948cca..cbdf279 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -7649,8 +7649,11 @@ qemuDomainAttachDeviceLive(virDomainObjPtr vm, dev->data.rng = NULL; break; - /*TODO: implement later */ case VIR_DOMAIN_DEVICE_MEMORY: + ret = qemuDomainAttachMemory(driver, vm, + dev->data.memory); + dev->data.memory = NULL; + break; case VIR_DOMAIN_DEVICE_NONE: case VIR_DOMAIN_DEVICE_FS: diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 5c5ad0e..88c5e3c 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -1672,6 +1672,101 @@ qemuDomainAttachRNGDevice(virQEMUDriverPtr driver, } +/** + * qemuDomainAttachMemory: + * @driver: qemu driver data + * @vm: VM object + * @mem: Definition of the memory device to be attached. @mem is always consumed + * + * Attaches memory device described by @mem to domain @vm. + * + * Returns 0 on success -1 on error. + */ +int +qemuDomainAttachMemory(virQEMUDriverPtr driver, + virDomainObjPtr vm, + virDomainMemoryDefPtr mem) +{ + qemuDomainObjPrivatePtr priv = vm->privateData; + virQEMUDriverConfigPtr cfg = virQEMUDriverGetConfig(driver); + char *devstr = NULL; + char *objalias = NULL; + const char *backendType; + virJSONValuePtr props = NULL; + int id; + int ret = -1; + + if (virAsprintf(&mem->info.alias, "dimm%zu", vm->def->nmems) < 0) + goto cleanup; + + if (virAsprintf(&objalias, "mem%s", mem->info.alias) < 0) + goto cleanup; + + if (!(devstr = qemuBuildMemoryDeviceStr(mem, priv->qemuCaps))) + goto cleanup; + + qemuDomainMemoryDeviceAlignSize(mem); + + if (qemuBuildMemoryBackendStr(mem->size, mem->pagesize, + mem->targetNode, mem->sourceNodes, NULL, + vm->def, priv->qemuCaps, cfg, + &backendType, &props, true) < 0) + goto cleanup; + + if (virDomainMemoryInsert(vm->def, mem) < 0) { + virJSONValueFree(props); + goto cleanup; + } + + qemuDomainObjEnterMonitor(driver, vm); + if (qemuMonitorAddObject(priv->mon, backendType, objalias, props) < 0) + goto removedef; + + if (qemuMonitorAddDevice(priv->mon, devstr) < 0) { + virErrorPtr err = virSaveLastError(); + ignore_value(qemuMonitorDelObject(priv->mon, objalias)); + virSetError(err); + virFreeError(err); + goto removedef; + } + + if (qemuDomainObjExitMonitor(driver, vm) < 0) { + /* we shouldn't touch mem now, as the def might be freed */ + mem = NULL; + goto cleanup; + } + + /* mem is consumed by vm->def */ + mem = NULL; + + /* this step is best effort, removing the device would be so much trouble */ + ignore_value(qemuDomainUpdateMemoryDeviceInfo(driver, vm, + QEMU_ASYNC_JOB_NONE)); + + ret = 0; + + cleanup: + virObjectUnref(cfg); + VIR_FREE(devstr); + VIR_FREE(objalias); + virDomainMemoryDefFree(mem); + return ret; + + removedef: + if (qemuDomainObjExitMonitor(driver, vm) < 0) { + mem = NULL; + goto cleanup; + } + + if ((id = virDomainMemoryFindByDef(vm->def, mem)) >= 0) + mem = virDomainMemoryRemove(vm->def, id); + else + mem = NULL; + + goto cleanup; +} + + static int qemuDomainAttachHostUSBDevice(virQEMUDriverPtr driver, virDomainObjPtr vm, diff --git a/src/qemu/qemu_hotplug.h b/src/qemu/qemu_hotplug.h index 8a0b313..ad4ff38 100644 --- a/src/qemu/qemu_hotplug.h +++ b/src/qemu/qemu_hotplug.h @@ -57,6 +57,9 @@ int qemuDomainAttachHostDevice(virConnectPtr conn, virDomainHostdevDefPtr hostdev); int qemuDomainFindGraphicsIndex(virDomainDefPtr def, virDomainGraphicsDefPtr dev); +int qemuDomainAttachMemory(virQEMUDriverPtr driver, + virDomainObjPtr vm, + virDomainMemoryDefPtr mem); int qemuDomainChangeGraphics(virQEMUDriverPtr driver, virDomainObjPtr vm, virDomainGraphicsDefPtr dev); -- 2.2.2

John Ferlan

19 Mar 19 Mar

6:40 p.m.

New subject: [libvirt] [PATCHv3 11/12] qemu: Implement memory device hotplug

On 03/17/2015 10:20 AM, Peter Krempa wrote:

...

Add code to hot-add memory devices to running qemu instances. ---

Notes: Version 3: - added comment to clarify that @mem is always consumed by qemuDomainAttachMemory Version 2: - no change

Version 2: - no change

src/qemu/qemu_command.c | 4 +-- src/qemu/qemu_command.h | 15 ++++++++ src/qemu/qemu_driver.c | 5 ++- src/qemu/qemu_hotplug.c | 95 +++++++++++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_hotplug.h | 3 ++ 5 files changed, 119 insertions(+), 3 deletions(-)

diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 2d85567..1f72437 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -4599,7 +4599,7 @@ qemuBuildControllerDevStr(virDomainDefPtr domainDef, * other configuration was used (to detect legacy configurations). Returns * -1 in case of an error. */ -static int +int qemuBuildMemoryBackendStr(unsigned long long size, unsigned long long pagesize, int guestNode, @@ -4872,7 +4872,7 @@ qemuBuildMemoryDimmBackendStr(virDomainMemoryDefPtr mem, }

-static char * +char * qemuBuildMemoryDeviceStr(virDomainMemoryDefPtr mem, virQEMUCapsPtr qemuCaps) { diff --git a/src/qemu/qemu_command.h b/src/qemu/qemu_command.h index ee81f92..a29db41 100644 --- a/src/qemu/qemu_command.h +++ b/src/qemu/qemu_command.h @@ -162,6 +162,21 @@ char *qemuBuildSoundDevStr(virDomainDefPtr domainDef, virDomainSoundDefPtr sound, virQEMUCapsPtr qemuCaps);

+int qemuBuildMemoryBackendStr(unsigned long long size, + unsigned long long pagesize, + int guestNode, + virBitmapPtr userNodeset, + virBitmapPtr autoNodeset, + virDomainDefPtr def, + virQEMUCapsPtr qemuCaps, + virQEMUDriverConfigPtr cfg, + const char **backendType, + virJSONValuePtr *backendProps, + bool force); + +char *qemuBuildMemoryDeviceStr(virDomainMemoryDefPtr mem, + virQEMUCapsPtr qemuCaps); + /* Legacy, pre device support */ char *qemuBuildPCIHostdevPCIDevStr(virDomainHostdevDefPtr dev, virQEMUCapsPtr qemuCaps); diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index e948cca..cbdf279 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -7649,8 +7649,11 @@ qemuDomainAttachDeviceLive(virDomainObjPtr vm, dev->data.rng = NULL; break;

- /*TODO: implement later */ case VIR_DOMAIN_DEVICE_MEMORY: + ret = qemuDomainAttachMemory(driver, vm, + dev->data.memory); + dev->data.memory = NULL; + break;

FWIW: Although there is a note in the AttachMemory code about consumption it may be worth noting here too just so no one in the future "sees" it missing and adds it thinking it's necessary

...

case VIR_DOMAIN_DEVICE_NONE: case VIR_DOMAIN_DEVICE_FS: diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 5c5ad0e..88c5e3c 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -1672,6 +1672,101 @@ qemuDomainAttachRNGDevice(virQEMUDriverPtr driver, }

+/** + * qemuDomainAttachMemory: + * @driver: qemu driver data + * @vm: VM object + * @mem: Definition of the memory device to be attached. @mem is always consumed + * + * Attaches memory device described by @mem to domain @vm. + * + * Returns 0 on success -1 on error. + */ +int +qemuDomainAttachMemory(virQEMUDriverPtr driver, + virDomainObjPtr vm, + virDomainMemoryDefPtr mem) +{ + qemuDomainObjPrivatePtr priv = vm->privateData; + virQEMUDriverConfigPtr cfg = virQEMUDriverGetConfig(driver); + char *devstr = NULL; + char *objalias = NULL; + const char *backendType; + virJSONValuePtr props = NULL; + int id; + int ret = -1; + + if (virAsprintf(&mem->info.alias, "dimm%zu", vm->def->nmems) < 0) + goto cleanup; + + if (virAsprintf(&objalias, "mem%s", mem->info.alias) < 0) + goto cleanup; + + if (!(devstr = qemuBuildMemoryDeviceStr(mem, priv->qemuCaps))) + goto cleanup; + + qemuDomainMemoryDeviceAlignSize(mem); + + if (qemuBuildMemoryBackendStr(mem->size, mem->pagesize, + mem->targetNode, mem->sourceNodes, NULL, + vm->def, priv->qemuCaps, cfg, + &backendType, &props, true) < 0)

Coverity determines that qemuBuildMemoryBackendStr can return props here with a -1 return and thus leak props That's because qemuBuildMemoryBackendStr sets the returned *backendProps and sets the local props to NULL before the (!hugepages) code which if it fails won't cause 'props' to be free'd properly Adding the virJSONValueFree(props); makes Coverity happy again. John

...

+ goto cleanup; + + if (virDomainMemoryInsert(vm->def, mem) < 0) { + virJSONValueFree(props); + goto cleanup; + } + + qemuDomainObjEnterMonitor(driver, vm); + if (qemuMonitorAddObject(priv->mon, backendType, objalias, props) < 0) + goto removedef; + + if (qemuMonitorAddDevice(priv->mon, devstr) < 0) { + virErrorPtr err = virSaveLastError(); + ignore_value(qemuMonitorDelObject(priv->mon, objalias)); + virSetError(err); + virFreeError(err); + goto removedef; + } + + if (qemuDomainObjExitMonitor(driver, vm) < 0) { + /* we shouldn't touch mem now, as the def might be freed */ + mem = NULL; + goto cleanup; + } + + /* mem is consumed by vm->def */ + mem = NULL; + + /* this step is best effort, removing the device would be so much trouble */ + ignore_value(qemuDomainUpdateMemoryDeviceInfo(driver, vm, + QEMU_ASYNC_JOB_NONE)); + + ret = 0; + + cleanup: + virObjectUnref(cfg); + VIR_FREE(devstr); + VIR_FREE(objalias); + virDomainMemoryDefFree(mem); + return ret; + + removedef: + if (qemuDomainObjExitMonitor(driver, vm) < 0) { + mem = NULL; + goto cleanup; + } + + if ((id = virDomainMemoryFindByDef(vm->def, mem)) >= 0) + mem = virDomainMemoryRemove(vm->def, id); + else + mem = NULL; + + goto cleanup; +} + + static int qemuDomainAttachHostUSBDevice(virQEMUDriverPtr driver, virDomainObjPtr vm, diff --git a/src/qemu/qemu_hotplug.h b/src/qemu/qemu_hotplug.h index 8a0b313..ad4ff38 100644 --- a/src/qemu/qemu_hotplug.h +++ b/src/qemu/qemu_hotplug.h @@ -57,6 +57,9 @@ int qemuDomainAttachHostDevice(virConnectPtr conn, virDomainHostdevDefPtr hostdev); int qemuDomainFindGraphicsIndex(virDomainDefPtr def, virDomainGraphicsDefPtr dev); +int qemuDomainAttachMemory(virQEMUDriverPtr driver, + virDomainObjPtr vm, + virDomainMemoryDefPtr mem); int qemuDomainChangeGraphics(virQEMUDriverPtr driver, virDomainObjPtr vm, virDomainGraphicsDefPtr dev);

Peter Krempa

23 Mar 23 Mar

9:17 a.m.

New subject: [libvirt] [PATCHv3 11/12] qemu: Implement memory device hotplug

On Thu, Mar 19, 2015 at 18:40:36 -0400, John Ferlan wrote:

...

On 03/17/2015 10:20 AM, Peter Krempa wrote:

...
Add code to hot-add memory devices to running qemu instances. ---

Notes: Version 3: - added comment to clarify that @mem is always consumed by qemuDomainAttachMemory Version 2: - no change

Version 2: - no change

src/qemu/qemu_command.c | 4 +-- src/qemu/qemu_command.h | 15 ++++++++ src/qemu/qemu_driver.c | 5 ++- src/qemu/qemu_hotplug.c | 95 +++++++++++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_hotplug.h | 3 ++ 5 files changed, 119 insertions(+), 3 deletions(-)

@@ -1672,6 +1672,101 @@ qemuDomainAttachRNGDevice(virQEMUDriverPtr driver,

...

...
}

+/** + * qemuDomainAttachMemory: + * @driver: qemu driver data + * @vm: VM object + * @mem: Definition of the memory device to be attached. @mem is always consumed + * + * Attaches memory device described by @mem to domain @vm. + * + * Returns 0 on success -1 on error. + */ +int +qemuDomainAttachMemory(virQEMUDriverPtr driver, + virDomainObjPtr vm, + virDomainMemoryDefPtr mem) +{ + qemuDomainObjPrivatePtr priv = vm->privateData; + virQEMUDriverConfigPtr cfg = virQEMUDriverGetConfig(driver); + char *devstr = NULL; + char *objalias = NULL; + const char *backendType; + virJSONValuePtr props = NULL; + int id; + int ret = -1; + + if (virAsprintf(&mem->info.alias, "dimm%zu", vm->def->nmems) < 0) + goto cleanup; + + if (virAsprintf(&objalias, "mem%s", mem->info.alias) < 0) + goto cleanup; + + if (!(devstr = qemuBuildMemoryDeviceStr(mem, priv->qemuCaps))) + goto cleanup; + + qemuDomainMemoryDeviceAlignSize(mem); + + if (qemuBuildMemoryBackendStr(mem->size, mem->pagesize, + mem->targetNode, mem->sourceNodes, NULL, + vm->def, priv->qemuCaps, cfg, + &backendType, &props, true) < 0)

Coverity determines that qemuBuildMemoryBackendStr can return props here with a -1 return and thus leak props

That's because qemuBuildMemoryBackendStr sets the returned *backendProps and sets the local props to NULL before the (!hugepages) code which if it fails won't cause 'props' to be free'd properly

Adding the virJSONValueFree(props); makes Coverity happy again.

I'll fix qemuBuildMemoryBackendStr separately rather than adding a pseudo-hack that would violate the style we are using for functions that return via argument. Peter

Peter Krempa

17 Mar 17 Mar

10:20 a.m.

New subject: [libvirt] [PATCHv3 12/12] qemu: Implement memory device hotunplug

Add code to hot-remove memory devices from qemu. Unfortunately QEMU doesn't support this right now, so this is just for completenes. --- Notes: Version 3: - moved check for the "-device" capability before any other code Version 2: - no change src/qemu/qemu_driver.c | 4 ++- src/qemu/qemu_hotplug.c | 91 ++++++++++++++++++++++++++++++++++++++++++++++++- src/qemu/qemu_hotplug.h | 3 ++ 3 files changed, 96 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index cbdf279..9731b5f 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -7733,7 +7733,9 @@ qemuDomainDetachDeviceLive(virDomainObjPtr vm, ret = qemuDomainDetachRNGDevice(driver, vm, dev->data.rng); break; case VIR_DOMAIN_DEVICE_MEMORY: - /* TODO: Implement later */ + ret = qemuDomainDetachMemoryDevice(driver, vm, dev->data.memory); + break; + case VIR_DOMAIN_DEVICE_FS: case VIR_DOMAIN_DEVICE_INPUT: case VIR_DOMAIN_DEVICE_SOUND: diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 88c5e3c..3365982 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -2828,6 +2828,44 @@ qemuDomainRemoveControllerDevice(virQEMUDriverPtr driver, } +static int +qemuDomainRemoveMemoryDevice(virQEMUDriverPtr driver, + virDomainObjPtr vm, + virDomainMemoryDefPtr mem) +{ + qemuDomainObjPrivatePtr priv = vm->privateData; + virObjectEventPtr event; + char *backendAlias = NULL; + int rc; + int idx; + + VIR_DEBUG("Removing memory device %s from domain %p %s", + mem->info.alias, vm, vm->def->name); + + if ((event = virDomainEventDeviceRemovedNewFromObj(vm, mem->info.alias))) + qemuDomainEventQueue(driver, event); + + if (virAsprintf(&backendAlias, "mem%s", mem->info.alias) < 0) + goto error; + + qemuDomainObjEnterMonitor(driver, vm); + rc = qemuMonitorDelObject(priv->mon, backendAlias); + if (qemuDomainObjExitMonitor(driver, vm) < 0 || rc < 0) + goto error; + + if ((idx = virDomainMemoryFindByDef(vm->def, mem)) >= 0) + virDomainMemoryRemove(vm->def, idx); + + virDomainMemoryDefFree(mem); + VIR_FREE(backendAlias); + return 0; + + error: + VIR_FREE(backendAlias); + return -1; +} + + static void qemuDomainRemovePCIHostDevice(virQEMUDriverPtr driver, virDomainObjPtr vm, @@ -3165,8 +3203,9 @@ qemuDomainRemoveDevice(virQEMUDriverPtr driver, qemuDomainRemoveRNGDevice(driver, vm, dev->data.rng); break; - /* TODO: implement later */ case VIR_DOMAIN_DEVICE_MEMORY: + ret = qemuDomainRemoveMemoryDevice(driver, vm, dev->data.memory); + break; case VIR_DOMAIN_DEVICE_NONE: case VIR_DOMAIN_DEVICE_LEASE: @@ -4118,3 +4157,53 @@ qemuDomainDetachRNGDevice(virQEMUDriverPtr driver, qemuDomainResetDeviceRemoval(vm); return ret; } + + +int +qemuDomainDetachMemoryDevice(virQEMUDriverPtr driver, + virDomainObjPtr vm, + virDomainMemoryDefPtr memdef) +{ + qemuDomainObjPrivatePtr priv = vm->privateData; + virDomainMemoryDefPtr mem; + int idx; + int rc; + int ret = -1; + + if (!virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_DEVICE)) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("qemu does not support -device")); + return -1; + } + + if ((idx = virDomainMemoryFindByDef(vm->def, memdef)) < 0) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("device not present in domain configuration")); + return -1; + } + + mem = vm->def->mems[idx]; + + if (!mem->info.alias) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("alias for the memory device was not found")); + return -1; + } + + qemuDomainMarkDeviceForRemoval(vm, &mem->info); + + qemuDomainObjEnterMonitor(driver, vm); + rc = qemuMonitorDelDevice(priv->mon, mem->info.alias); + if (qemuDomainObjExitMonitor(driver, vm) < 0 || rc < 0) + goto cleanup; + + rc = qemuDomainWaitForDeviceRemoval(vm); + if (rc == 0 || rc == 1) + ret = qemuDomainRemoveMemoryDevice(driver, vm, mem); + else + ret = 0; + + cleanup: + qemuDomainResetDeviceRemoval(vm); + return ret; +} diff --git a/src/qemu/qemu_hotplug.h b/src/qemu/qemu_hotplug.h index ad4ff38..4140da3 100644 --- a/src/qemu/qemu_hotplug.h +++ b/src/qemu/qemu_hotplug.h @@ -60,6 +60,9 @@ int qemuDomainFindGraphicsIndex(virDomainDefPtr def, int qemuDomainAttachMemory(virQEMUDriverPtr driver, virDomainObjPtr vm, virDomainMemoryDefPtr mem); +int qemuDomainDetachMemoryDevice(virQEMUDriverPtr driver, + virDomainObjPtr vm, + virDomainMemoryDefPtr memdef); int qemuDomainChangeGraphics(virQEMUDriverPtr driver, virDomainObjPtr vm, virDomainGraphicsDefPtr dev); -- 2.2.2

John Ferlan

19 Mar 19 Mar

6:41 p.m.

On 03/17/2015 10:19 AM, Peter Krempa wrote:

...

This version includes review feedback changes from John and also fixes the <memory> element documentation and code that calculates it to support possible non-NUMA configs with memory hotplug if any hypervisor would support that in the future.

Peter Krempa (12): qemu: monitor: Don't leak @props with non-JSON in qemuMonitorAddObject libxl: Refactor logic in domain post parse callback conf: Add support for parsing and formatting max memory and slot count qemu: Implement setup of memory hotplug parameters conf: Add device address type for dimm devices conf: Add interface to parse and format memory device information qemu: memdev: Add infrastructure to load memory device information qemu: migration: Forbid migration with memory modules lacking info qemu: add support for memory devices qemu: conf: Add support for memory device cold(un)plug qemu: Implement memory device hotplug qemu: Implement memory device hotunplug

docs/formatdomain.html.in | 112 +++- docs/schemas/domaincommon.rng | 76 +++ src/bhyve/bhyve_domain.c | 9 +- src/conf/domain_conf.c | 576 ++++++++++++++++++++- src/conf/domain_conf.h | 59 +++ src/libvirt_private.syms | 7 + src/libxl/libxl_domain.c | 15 +- src/lxc/lxc_domain.c | 8 + src/openvz/openvz_driver.c | 14 +- src/parallels/parallels_driver.c | 6 +- src/phyp/phyp_driver.c | 6 +- src/qemu/qemu_command.c | 166 +++++- src/qemu/qemu_command.h | 15 + src/qemu/qemu_domain.c | 80 +++ src/qemu/qemu_domain.h | 5 + src/qemu/qemu_driver.c | 29 ++ src/qemu/qemu_hotplug.c | 187 +++++++ src/qemu/qemu_hotplug.h | 6 + src/qemu/qemu_migration.c | 14 + src/qemu/qemu_monitor.c | 48 +- src/qemu/qemu_monitor.h | 14 + src/qemu/qemu_monitor_json.c | 122 +++++ src/qemu/qemu_monitor_json.h | 5 + src/qemu/qemu_process.c | 4 + src/uml/uml_driver.c | 9 +- src/vbox/vbox_common.c | 6 +- src/vmware/vmware_driver.c | 6 +- src/vmx/vmx.c | 6 +- src/xen/xen_driver.c | 7 + src/xenapi/xenapi_driver.c | 9 +- tests/domainschemadata/maxMemory.xml | 19 + .../qemuxml2argv-memory-hotplug-dimm.args | 11 + .../qemuxml2argv-memory-hotplug-dimm.xml | 50 ++ .../qemuxml2argv-memory-hotplug-nonuma.xml | 22 + .../qemuxml2argv-memory-hotplug.args | 6 + .../qemuxml2argv-memory-hotplug.xml | 34 ++ tests/qemuxml2argvtest.c | 6 + tests/qemuxml2xmltest.c | 4 + 38 files changed, 1747 insertions(+), 31 deletions(-) create mode 100644 tests/domainschemadata/maxMemory.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-dimm.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-dimm.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-nonuma.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug.xml

ACK series with adjustments noted in patches 9 & 11 and of course the assurance about patch 10. John

Peter Krempa

23 Mar 23 Mar

9:58 a.m.

On Thu, Mar 19, 2015 at 18:41:31 -0400, John Ferlan wrote:

...

On 03/17/2015 10:19 AM, Peter Krempa wrote:

...
This version includes review feedback changes from John and also fixes the <memory> element documentation and code that calculates it to support possible non-NUMA configs with memory hotplug if any hypervisor would support that in the future.

Peter Krempa (12): qemu: monitor: Don't leak @props with non-JSON in qemuMonitorAddObject libxl: Refactor logic in domain post parse callback conf: Add support for parsing and formatting max memory and slot count qemu: Implement setup of memory hotplug parameters conf: Add device address type for dimm devices conf: Add interface to parse and format memory device information qemu: memdev: Add infrastructure to load memory device information qemu: migration: Forbid migration with memory modules lacking info qemu: add support for memory devices qemu: conf: Add support for memory device cold(un)plug qemu: Implement memory device hotplug qemu: Implement memory device hotunplug

...

...
38 files changed, 1747 insertions(+), 31 deletions(-) create mode 100644 tests/domainschemadata/maxMemory.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-dimm.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-dimm.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-nonuma.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug.xml

ACK series with adjustments noted in patches 9 & 11 and of course the assurance about patch 10.

I've fixed all the points and pushed the series. Thanks for reviewing these unpleasant amounts of code. Peter

3822

Age (days ago)

3828

Last active (days ago)

List overview

Download

22 comments

3 participants

participants (3)

John Ferlan
Peter Krempa
Zhu Guihua