----- Original Message -----
From: "Michal Privoznik" <mprivozn(a)redhat.com>
To: "Zack Cornelius" <zack.cornelius(a)kove.net>
Cc: "libvir-list" <libvir-list(a)redhat.com>
Sent: Friday, September 29, 2017 2:44:13 AM
Subject: Re: [libvirt] Exposing mem-path in domain XML
For this configuration, two memory-backend-file objects are created.
The
first one is for the guest RAM (for the NUMA node), second is for the
DIMM module. While the alias for the DIMM module is exposed in the XML,
the alias for the NUMA node is missing:
-object
memory-backend-file,id=ram-node0,mem-path=/var/lib/libvirt/qemu/ram,share=yes,size=4294967296
-numa node,nodeid=0,cpus=0-3,memdev=ram-node0
-object
memory-backend-file,id=memdimm0,mem-path=/var/lib/libvirt/qemu/ram,share=yes,size=536870912
-device pc-dimm,node=0,memdev=memdimm0,id=dimm0,slot=0
This complicates things IMO. I guess what I'm saying is that we can
generate full paths including the device alias as filename, but I'm not
sure it is going to help, is it?
Looking at the examples, it seems like these automatically added memory-backend-file
objects would always be generated in a consistent manner. As long as this is true, and
these objects always end up with a consistent filenames, our integration components can
assume the filenames in a consistent way.
For instance, as long as every numa <cell> element which has memory defined
generates a file named ram-node<numa_node_number>, we can handle that mapping within
our integration components by parsing the XML and identifying the NUMA nodes with memory,
and allocating an appropriate amount, with the correct filename.
Alternatively, and I admit I don't know much about Kove, if
libvirt
would put all the files into per-domain directories, would that be
enough for you? In the example above all the files are put into generic
path. However, if the path looked like this:
$memoryBackingDir/$domain/
You could differentiate which files belong to which domain.
For our integration to work well for users, we need to allocate the external memory prior
to qemu starting, so we can give descriptive error messages as part of our integration
components.
The directory based approach would require us to allocate external memory when the
individual files are created or mmap'd in qemu. This may lead to a situation where
there is insufficient external memory, leading to the failure happening within qemu, and a
much more generic qemu unable to allocate memory error.
With the external memory allocations and files being created and sized prior to qemu
startup via our integration components called via the libvirt prepare hook, we can detect
this situation during the libvirt hook, and return a more appropriate error message.
--Zack