[PATCH v2] qemu: Don't prealloc mem for real NVDIMMs

Currently, we configure QEMU to prealloc memory almost by default. Well, by default for NVDIMMs, hugepages and if user asked us to (via memoryBacking <allocation mode="immediate"/>). However, when guest's NVDIMM is backed by real life NVDIMM this approach is not the best. In this case users should put <pmem/> into the <memory/> device <source/>, like this: <memory model='nvdimm' access='shared'> <source> <path>/dev/pmem0</path> <pmem/> </source> </memory> Instructing QEMU to do prealloc in this case means that each page of the NVDIMM is "touched" (the first byte is read and written back - see QEMU commit v2.9.0-rc1~26^2) which cripples device wear. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1894053 Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- v2 of: https://www.redhat.com/archives/libvir-list/2020-November/msg01568.html diff to v1: - I've dropped the first hunk of v1 which forbade prealloc even if user requested it explicitly. src/qemu/qemu_command.c | 5 ++++- .../memory-hotplug-nvdimm-pmem.x86_64-latest.args | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index b06a086e18..f572ed64c1 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -3085,7 +3085,10 @@ qemuBuildMemoryBackendProps(virJSONValuePtr *backendProps, if (mem->nvdimmPath) { memPath = g_strdup(mem->nvdimmPath); - prealloc = true; + /* If the NVDIMM is a real device then there's nothing to prealloc. + * If anyhing, we would be only wearing off the device. */ + if (!mem->nvdimmPmem) + prealloc = true; } else if (useHugepage) { if (qemuGetDomainHupageMemPath(priv->driver, def, pagesize, &memPath) < 0) return -1; diff --git a/tests/qemuxml2argvdata/memory-hotplug-nvdimm-pmem.x86_64-latest.args b/tests/qemuxml2argvdata/memory-hotplug-nvdimm-pmem.x86_64-latest.args index cac02a6f6d..fb4ae4b518 100644 --- a/tests/qemuxml2argvdata/memory-hotplug-nvdimm-pmem.x86_64-latest.args +++ b/tests/qemuxml2argvdata/memory-hotplug-nvdimm-pmem.x86_64-latest.args @@ -20,7 +20,7 @@ file=/tmp/lib/domain--1-QEMUGuest1/master-key.aes \ -object memory-backend-ram,id=ram-node0,size=224395264 \ -numa node,nodeid=0,cpus=0-1,memdev=ram-node0 \ -object memory-backend-file,id=memnvdimm0,mem-path=/tmp/nvdimm,share=no,\ -prealloc=yes,size=536870912,pmem=yes \ +size=536870912,pmem=yes \ -device nvdimm,node=0,memdev=memnvdimm0,id=nvdimm0,slot=0 \ -uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ -display none \ -- 2.26.2

On 1/5/21 8:39 AM, Michal Privoznik wrote:
Currently, we configure QEMU to prealloc memory almost by default. Well, by default for NVDIMMs, hugepages and if user asked us to (via memoryBacking <allocation mode="immediate"/>).
However, when guest's NVDIMM is backed by real life NVDIMM this approach is not the best. In this case users should put <pmem/> into the <memory/> device <source/>, like this:
<memory model='nvdimm' access='shared'> <source> <path>/dev/pmem0</path> <pmem/> </source> </memory>
Instructing QEMU to do prealloc in this case means that each page of the NVDIMM is "touched" (the first byte is read and written back - see QEMU commit v2.9.0-rc1~26^2) which cripples device wear.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1894053 Signed-off-by: Michal Privoznik <mprivozn@redhat.com> ---
v2 of:
https://www.redhat.com/archives/libvir-list/2020-November/msg01568.html
diff to v1: - I've dropped the first hunk of v1 which forbade prealloc even if user requested it explicitly.
src/qemu/qemu_command.c | 5 ++++- .../memory-hotplug-nvdimm-pmem.x86_64-latest.args | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index b06a086e18..f572ed64c1 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -3085,7 +3085,10 @@ qemuBuildMemoryBackendProps(virJSONValuePtr *backendProps,
if (mem->nvdimmPath) { memPath = g_strdup(mem->nvdimmPath); - prealloc = true; + /* If the NVDIMM is a real device then there's nothing to prealloc. + * If anyhing, we would be only wearing off the device. */
s/anyhing/anything Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
+ if (!mem->nvdimmPmem) + prealloc = true; } else if (useHugepage) { if (qemuGetDomainHupageMemPath(priv->driver, def, pagesize, &memPath) < 0) return -1; diff --git a/tests/qemuxml2argvdata/memory-hotplug-nvdimm-pmem.x86_64-latest.args b/tests/qemuxml2argvdata/memory-hotplug-nvdimm-pmem.x86_64-latest.args index cac02a6f6d..fb4ae4b518 100644 --- a/tests/qemuxml2argvdata/memory-hotplug-nvdimm-pmem.x86_64-latest.args +++ b/tests/qemuxml2argvdata/memory-hotplug-nvdimm-pmem.x86_64-latest.args @@ -20,7 +20,7 @@ file=/tmp/lib/domain--1-QEMUGuest1/master-key.aes \ -object memory-backend-ram,id=ram-node0,size=224395264 \ -numa node,nodeid=0,cpus=0-1,memdev=ram-node0 \ -object memory-backend-file,id=memnvdimm0,mem-path=/tmp/nvdimm,share=no,\ -prealloc=yes,size=536870912,pmem=yes \ +size=536870912,pmem=yes \ -device nvdimm,node=0,memdev=memnvdimm0,id=nvdimm0,slot=0 \ -uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ -display none \

On Tue, Jan 05, 2021 at 12:39:00PM +0100, Michal Privoznik wrote:
Currently, we configure QEMU to prealloc memory almost by default. Well, by default for NVDIMMs, hugepages and if user asked us to (via memoryBacking <allocation mode="immediate"/>).
However, when guest's NVDIMM is backed by real life NVDIMM this approach is not the best. In this case users should put <pmem/> into the <memory/> device <source/>, like this:
<memory model='nvdimm' access='shared'> <source> <path>/dev/pmem0</path> <pmem/> </source> </memory>
Instructing QEMU to do prealloc in this case means that each page of the NVDIMM is "touched" (the first byte is read and written back - see QEMU commit v2.9.0-rc1~26^2) which cripples device wear.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1894053 Signed-off-by: Michal Privoznik <mprivozn@redhat.com> ---
v2 of:
https://www.redhat.com/archives/libvir-list/2020-November/msg01568.html
diff to v1: - I've dropped the first hunk of v1 which forbade prealloc even if user requested it explicitly.
src/qemu/qemu_command.c | 5 ++++- .../memory-hotplug-nvdimm-pmem.x86_64-latest.args | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
participants (3)
-
Daniel Henrique Barboza
-
Daniel P. Berrangé
-
Michal Privoznik