7:38 a.m.
On 01/30/2015 08:20 AM, Peter Krempa wrote:
!! this series applies on top of the cleanup series posted earlier
!!
Hi,
this is my try to implement memory hotplug in libvirt (for the qemu) driver.
This series is almost code-complete but is lacking tests and docs as I wanted
to agree on design first before attempting to write the documentation.
Additionally this series is also lacking code that would fix memballoon
handling but I'm waiting on a clousure how it's supposed to work from qemu's
side as it appears to be broken there too.
The basic XML used to add a memory device:
<memory model='acpi-dimm'>
<target>
<size unit='KiB'>524287</size>
<node>0</node>
</target>
</memory>
The <target> subelement is mandatory, whereas the <source> subelement (that
contains source numa nodes, source page size ) is optional and is inferred
from the NUMA tuning for given target numa node.
Please note that at least one guest numa node has to be configured for the
guest for this to work (limitation of qemu).
Been kind of back-burner thinking about this series overall for the last
couple of days. After reading Daniel's review - I think he touched on a
few things I was pondering. Since I haven't followed the upstream QEMU
work on this - I assume it's following that work very closely. I also
assume other hypervisor's would/could be interested in providing similar
functionality (hot plug memory) - I'm left wondering - is the model
chosen flexible enough for that? You certainly will need a "healthy
dose" of documentation to describe how to configure values properly.
Perhaps even an extra utility to output what's available to be provided
for the XML.
The one area that seems to be very QEMU implementation specific is the
"<address type='acpi-dimm' slot='#'
base='0xHEX'/>" XML. Maybe that
becomes a "<driver name='qemu' slot='#'
base='0xHEX'/>" instead leaving
"<address type='acpi-dimm'/>" or even empty? Allowing users to
provide
"base" implies either extra validation to ensure it's 'correct'
(format
& usage/duplication) or documentation to describe what/how to provide
the data. Perhaps even an extra utility to provide "valid values". If
QEMU is expecting specific values - maybe it'd be in our best interests
not to expect the values. I suppose I'm having a difficult time
visualizing the values and what they express. Also what if someone
attempts to "poor man's clone" a live guest (dumpxml > file, edit file
to change name, remove/change uuid, change disk source data, etc., then
start). If they don't change those addresses - what will happen? I
would think it would be "expected of" libvirt to ensure what's passed to
QEMU is "uniquely expressed" anyway, right? What's still not clear to me
is the relationship between 'acpi-dimm' and NUMA slots/nodes.
FWIW: As an aside, after patch8, it would be beneficial to update
qemuxml2argv-memory-hotplug.args in order to include the expected output
of the "pc-dimm,node=%d,memdev=mem%s,id=%s" and "-object xxx -device
xxxx"...
A tangent... When I see 'acpi-' in the XML - I think firmware or some
external memory device which firmware needs to find/recognize... Long
ago I worked on a project that used a memory device as a cluster
interconnect (memory channel). A "practical use" in a virtual world
could be as an interconnect for migrations. This is what I thought of
when I saw a <memory> "device" with 'acpi-...' as opposed to a
memory
device that's providing host NUMA nodes/slots to the guest as extra
available hot-plug memory.
John
What's missing in this series:
- tests
- docs
- commit message touch-up
- code to audit the memory size changes
- code to make memory balloning working with correct size
Peter Krempa (12):
qemu: caps: Add capability bit for the "pc-dimm" device
conf: Add support for parsing and formatting max memory and slot count
qemu: Implement setup of memory hotplug parameters
conf: Add device address type for dimm devices
conf: Add interface to parse and format memory device information
qemu: memdev: Add infrastructure to load memory device information
qemu: migration: Forbid migration with memory modules lacking info
qemu: add support for memory devices
conf: Introduce helper to find duplicate device address
qemu: conf: Add support for memory device cold(un)plug
qemu: Implement memory device hotplug
qemu: Implement memory device hotunplug
docs/formatdomain.html.in | 19 +
docs/schemas/domaincommon.rng | 76 +++
src/bhyve/bhyve_domain.c | 9 +-
src/conf/domain_conf.c | 655 ++++++++++++++++++++-
src/conf/domain_conf.h | 63 ++
src/libvirt_private.syms | 7 +
src/libxl/libxl_domain.c | 8 +
src/lxc/lxc_domain.c | 8 +
src/openvz/openvz_driver.c | 14 +-
src/parallels/parallels_driver.c | 6 +-
src/phyp/phyp_driver.c | 6 +-
src/qemu/qemu_capabilities.c | 2 +
src/qemu/qemu_capabilities.h | 1 +
src/qemu/qemu_command.c | 157 ++++-
src/qemu/qemu_command.h | 15 +
src/qemu/qemu_domain.c | 58 ++
src/qemu/qemu_domain.h | 6 +
src/qemu/qemu_driver.c | 29 +
src/qemu/qemu_hotplug.c | 178 ++++++
src/qemu/qemu_hotplug.h | 6 +
src/qemu/qemu_migration.c | 14 +
src/qemu/qemu_monitor.c | 45 ++
src/qemu/qemu_monitor.h | 14 +
src/qemu/qemu_monitor_json.c | 116 ++++
src/qemu/qemu_monitor_json.h | 5 +
src/qemu/qemu_process.c | 4 +
src/uml/uml_driver.c | 9 +-
src/vbox/vbox_common.c | 6 +-
src/vmware/vmware_driver.c | 6 +-
src/vmx/vmx.c | 6 +-
src/xen/xen_driver.c | 7 +
src/xenapi/xenapi_driver.c | 9 +-
tests/domainschemadata/maxMemory.xml | 19 +
tests/qemucapabilitiesdata/caps_2.1.1-1.caps | 1 +
.../qemuxml2argv-memory-hotplug-dimm.xml | 47 ++
.../qemuxml2argv-memory-hotplug-nonuma.xml | 22 +
.../qemuxml2argv-memory-hotplug.args | 6 +
.../qemuxml2argv-memory-hotplug.xml | 34 ++
tests/qemuxml2argvtest.c | 4 +
tests/qemuxml2xmltest.c | 4 +
40 files changed, 1682 insertions(+), 19 deletions(-)
create mode 100644 tests/domainschemadata/maxMemory.xml
create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-dimm.xml
create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug-nonuma.xml
create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug.args
create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-memory-hotplug.xml