[PATCH] lib: Document VIR_DUMP_LIVE flag quirk

From: Michal Privoznik <mprivozn@redhat.com> The virDomainCoreDump() API has VIR_DUMP_LIVE flag which is documented to leave vCPUs running throughout making of the dump of guest memory. Well, this is not the case for QEMU which pauses vCPUs unconditionally (it calls vm_stop() in dump_init()). Document this quirk. And also mention it in 'virsh dump --live' manapage. Resolves: https://gitlab.com/libvirt/libvirt/-/issues/646 Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- docs/manpages/virsh.rst | 3 ++- src/libvirt-domain.c | 2 ++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index 3a00778467..14f22b32c3 100644 --- a/docs/manpages/virsh.rst +++ b/docs/manpages/virsh.rst @@ -2927,7 +2927,8 @@ dump Dumps the core of a domain to a file for analysis. If *--live* is specified, the domain continues to run until the core -dump is complete, rather than pausing up front. +dump is complete, rather than pausing up front. Although, the hypervisor might +still decide to pause the guest's vCPUs. If *--crash* is specified, the domain is halted with a crashed status, rather than merely left in a paused state. If *--reset* is specified, the domain is reset after successful dump. diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index 93e8f5b853..6d1cd2dba1 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -1445,6 +1445,8 @@ virDomainSaveImageDefineXML(virConnectPtr conn, const char *file, * the guest to run; otherwise, the guest is suspended during the dump. * VIR_DUMP_RESET flag forces reset of the guest after dump. * The above three flags are mutually exclusive. + * However, note that even if VIR_DUMP_LIVE flag is specified, the hypervisor + * might temporarily suspend the guest vCPUs anyway. * * Additionally, if @flags includes VIR_DUMP_BYPASS_CACHE, then libvirt * will attempt to bypass the file system cache while creating the file, -- 2.49.0

On Mon, Jun 02, 2025 at 03:27:32PM +0200, Michal Privoznik via Devel wrote:
From: Michal Privoznik <mprivozn@redhat.com>
The virDomainCoreDump() API has VIR_DUMP_LIVE flag which is documented to leave vCPUs running throughout making of the dump of guest memory. Well, this is not the case for QEMU which pauses vCPUs unconditionally (it calls vm_stop() in dump_init()). Document this quirk. And also mention it in 'virsh dump --live' manapage.
How long does QEMU keep the vCPUs paused for ? If it is paused for the entire duration of the dump, then IMHO it is a libvirt bug to allow the VIR_DUMP_LIVE flag to have been set to begin with, rather than a behavioural quirk. Essentially I'd consider VIR_DUMP_LIVE to mean that the vCPUs are running for the time window when the guest memory is being captured. If there are small synchronizaiton steps a hypervisor does which result in temporarily pausing CPUs for only a fraction of a second that is acceptable wrt VIR_DUMP_LIVE semantics.
Resolves: https://gitlab.com/libvirt/libvirt/-/issues/646 Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- docs/manpages/virsh.rst | 3 ++- src/libvirt-domain.c | 2 ++ 2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index 3a00778467..14f22b32c3 100644 --- a/docs/manpages/virsh.rst +++ b/docs/manpages/virsh.rst @@ -2927,7 +2927,8 @@ dump
Dumps the core of a domain to a file for analysis. If *--live* is specified, the domain continues to run until the core -dump is complete, rather than pausing up front. +dump is complete, rather than pausing up front. Although, the hypervisor might +still decide to pause the guest's vCPUs. If *--crash* is specified, the domain is halted with a crashed status, rather than merely left in a paused state. If *--reset* is specified, the domain is reset after successful dump. diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index 93e8f5b853..6d1cd2dba1 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -1445,6 +1445,8 @@ virDomainSaveImageDefineXML(virConnectPtr conn, const char *file, * the guest to run; otherwise, the guest is suspended during the dump. * VIR_DUMP_RESET flag forces reset of the guest after dump. * The above three flags are mutually exclusive. + * However, note that even if VIR_DUMP_LIVE flag is specified, the hypervisor + * might temporarily suspend the guest vCPUs anyway. * * Additionally, if @flags includes VIR_DUMP_BYPASS_CACHE, then libvirt * will attempt to bypass the file system cache while creating the file, -- 2.49.0
With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On 6/2/25 16:18, Daniel P. Berrangé wrote:
On Mon, Jun 02, 2025 at 03:27:32PM +0200, Michal Privoznik via Devel wrote:
From: Michal Privoznik <mprivozn@redhat.com>
The virDomainCoreDump() API has VIR_DUMP_LIVE flag which is documented to leave vCPUs running throughout making of the dump of guest memory. Well, this is not the case for QEMU which pauses vCPUs unconditionally (it calls vm_stop() in dump_init()). Document this quirk. And also mention it in 'virsh dump --live' manapage.
How long does QEMU keep the vCPUs paused for ? If it is paused for the entire duration of the dump, then IMHO it is a libvirt bug to allow the VIR_DUMP_LIVE flag to have been set to begin with, rather than a behavioural quirk.
Essentially I'd consider VIR_DUMP_LIVE to mean that the vCPUs are running for the time window when the guest memory is being captured.
If there are small synchronizaiton steps a hypervisor does which result in temporarily pausing CPUs for only a fraction of a second that is acceptable wrt VIR_DUMP_LIVE semantics.
Unfortunately, it's the former: 2025-06-02 08:44:54.531+0000: 343: info : qemuMonitorIOWrite:339 : QEMU_MONITOR_IO_WRITE: mon=0x7f9a4c04bcb0 buf={"execute":"dump-guest-memory","arguments":{"paging":false,"protocol":"fd:dump","detach":true},"id":"libvirt-502"} len=116 ret=116 errno=0 2025-06-02 08:44:54.532+0000: 343: debug : qemuMonitorJSONIOProcessLine:193 : Line [{"timestamp": {"seconds": 1748853894, "microseconds": 532565}, "event": "STOP"}] ... 2025-06-02 08:44:55.720+0000: 343: debug : qemuMonitorJSONIOProcessLine:193 : Line [{"timestamp": {"seconds": 1748853895, "microseconds": 720431}, "event": "DUMP_COMPLETED", "data": {"result": {"total": 4303486976, "status": "completed", "completed": 4303486976}}}] 2025-06-02 08:44:55.720+0000: 343: info : .... 2025-06-02 08:44:55.720+0000: 343: debug : qemuProcessHandleDumpCompleted:1645 : Dump completed for domain 0x7f9a401c4bf0 fedora with stats=0x7f9a50ffd760 error='<null>' 2025-06-02 08:44:55.720+0000: 343: debug : qemuMonitorJSONIOProcessLine:193 : Line [{"timestamp": {"seconds": 1748853895, "microseconds": 720492}, "event": "RESUME"}] 2025-06-02 08:44:55.720+0000: 343: info : Okay, I'll just make qemu driver refuse the flag. Thanks! Michal
participants (3)
-
Daniel P. Berrangé
-
Michal Privoznik
-
Michal Prívozník