[PATCH 0/3] qemu: Introduce 'manual' snapshot mode for storage providers not managed by libvirt
Peter Krempa (3): conf: snapshot: Introduce 'manual' mode for snapshot of a disk qemuSnapshotCreateActiveExternal: Implement manual snapshot mode kbase: Introduce 'snapshots' page and describe the new 'manual' snapshot docs/formatdomain.rst | 15 +++++----- docs/formatsnapshot.rst | 9 ++++++ docs/kbase/index.rst | 3 ++ docs/kbase/meson.build | 1 + docs/kbase/snapshots.rst | 53 +++++++++++++++++++++++++++++++++ docs/schemas/domainsnapshot.rng | 3 ++ src/conf/domain_conf.c | 1 + src/conf/domain_conf.h | 1 + src/conf/snapshot_conf.c | 6 ++++ src/qemu/qemu_snapshot.c | 24 +++++++++++---- src/test/test_driver.c | 17 +++++++++++ 11 files changed, 121 insertions(+), 12 deletions(-) create mode 100644 docs/kbase/snapshots.rst -- 2.35.1
The idea of the manual mode is to allow a synchronized snapshot in cases when the storage is outsourced to an unmanaged storage provider which requires cooperation with snapshotting. The mode will instruct the hypervisor to pause along when the other components are snapshotted and the 'manual' disk can be snapshotted along. This increases latency of the snapshot but allows them in otherwise impossible situations. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- docs/formatdomain.rst | 15 ++++++++------- docs/formatsnapshot.rst | 9 +++++++++ docs/schemas/domainsnapshot.rng | 3 +++ src/conf/domain_conf.c | 1 + src/conf/domain_conf.h | 1 + src/conf/snapshot_conf.c | 6 ++++++ src/qemu/qemu_snapshot.c | 5 +++++ src/test/test_driver.c | 17 +++++++++++++++++ 8 files changed, 50 insertions(+), 7 deletions(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index 9b1b69bb4d..d188de4858 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -2620,13 +2620,14 @@ paravirtualized driver is specified via the ``disk`` element. Indicates the default behavior of the disk during disk snapshots: "``internal``" requires a file format such as qcow2 that can store both the snapshot and the data changes since the snapshot; "``external``" will - separate the snapshot from the live data; and "``no``" means the disk will - not participate in snapshots. Read-only disks default to "``no``", while - the default for other disks depends on the hypervisor's capabilities. Some - hypervisors allow a per-snapshot choice as well, during `domain snapshot - creation <formatsnapshot.html>`__. Not all snapshot modes are supported; - for example, enabling snapshots with a transient disk generally does not - make sense. :since:`Since 0.9.5` + separate the snapshot from the live data; "``no``" means the disk will + not participate in snapshots; and ``manual`` allows snapshotting done via + an unmanaged storage provider. Read-only disks default to "``no``", while + the default for other disks depends on the hypervisor's capabilities. + Some hypervisors allow a per-snapshot choice as well, during `domain + snapshot creation <formatsnapshot.html>`__. Not all snapshot modes are + supported; for example, enabling snapshots with a transient disk + generally does not make sense. :since:`Since 0.9.5` ``source`` Representation of the disk ``source`` depends on the disk ``type`` attribute diff --git a/docs/formatsnapshot.rst b/docs/formatsnapshot.rst index 0fee35d89c..4635df89cb 100644 --- a/docs/formatsnapshot.rst +++ b/docs/formatsnapshot.rst @@ -124,6 +124,15 @@ The top-level ``domainsnapshot`` element may contain the following elements: corresponding domain disk, while others like qemu allow this field to override the domain default. + :since:`Since 8.2.0` the ``snapshot`` attribute supports the ``manual`` + value which instructs the hypervisor to create the snapshot and keep a + synchronized state by pausing the VM which allows to snapshot disk + storage from outside of the hypervisor if the storage provider supports + it. The caller is responsible for resuming a VM paused by requesting a + ``manual`` snapshot When reverting such snapshot, the expectation is that + the storage is configured in a way where the hypervisor will see the + correct image state. + :since:`Since 1.2.2` the ``disk`` element supports an optional attribute ``type`` if the ``snapshot`` attribute is set to ``external``. This attribute specifies the snapshot target storage type and allows to diff --git a/docs/schemas/domainsnapshot.rng b/docs/schemas/domainsnapshot.rng index 58c370878d..a5d1a40493 100644 --- a/docs/schemas/domainsnapshot.rng +++ b/docs/schemas/domainsnapshot.rng @@ -200,6 +200,9 @@ <ref name="diskSourceNetwork"/> </choice> </group> + <attribute name="snapshot"> + <value>manual</value> + </attribute> </choice> </element> </define> diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index e0dfc9e45f..153954a0b0 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -1408,6 +1408,7 @@ VIR_ENUM_IMPL(virDomainSnapshotLocation, "no", "internal", "external", + "manual", ); /* Internal mapping: subset of block job types that can be present in diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index a4de46773c..b69abfa270 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -542,6 +542,7 @@ typedef enum { VIR_DOMAIN_SNAPSHOT_LOCATION_NO, VIR_DOMAIN_SNAPSHOT_LOCATION_INTERNAL, VIR_DOMAIN_SNAPSHOT_LOCATION_EXTERNAL, + VIR_DOMAIN_SNAPSHOT_LOCATION_MANUAL, VIR_DOMAIN_SNAPSHOT_LOCATION_LAST } virDomainSnapshotLocation; diff --git a/src/conf/snapshot_conf.c b/src/conf/snapshot_conf.c index e2442441d0..80946beba9 100644 --- a/src/conf/snapshot_conf.c +++ b/src/conf/snapshot_conf.c @@ -312,6 +312,12 @@ virDomainSnapshotDefParse(xmlXPathContextPtr ctxt, &def->memory, VIR_DOMAIN_SNAPSHOT_LOCATION_DEFAULT) < 0) return NULL; + + if (def->memory == VIR_DOMAIN_SNAPSHOT_LOCATION_MANUAL) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("'manual' memory snapshot mode not supported")); + return NULL; + } } if (def->memory == VIR_DOMAIN_SNAPSHOT_LOCATION_DEFAULT) { diff --git a/src/qemu/qemu_snapshot.c b/src/qemu/qemu_snapshot.c index a7901779fc..3fa6caaaec 100644 --- a/src/qemu/qemu_snapshot.c +++ b/src/qemu/qemu_snapshot.c @@ -757,6 +757,11 @@ qemuSnapshotPrepare(virDomainObj *vm, external++; break; + case VIR_DOMAIN_SNAPSHOT_LOCATION_MANUAL: + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("'manual' disk snapshot mode not yet implemented")); + return -1; + case VIR_DOMAIN_SNAPSHOT_LOCATION_NO: /* Remember seeing a disk that has snapshot disabled */ if (!virStorageSourceIsEmpty(dom_disk->src) && diff --git a/src/test/test_driver.c b/src/test/test_driver.c index 9ceb0b45c8..4020662353 100644 --- a/src/test/test_driver.c +++ b/src/test/test_driver.c @@ -8715,6 +8715,23 @@ testDomainSnapshotAlignDisks(virDomainObj *vm, unsigned int flags) { virDomainSnapshotLocation align_location = VIR_DOMAIN_SNAPSHOT_LOCATION_INTERNAL; + size_t i; + + for (i = 0; i < def->ndisks; i++) { + switch (def->disks[i].snapshot) { + case VIR_DOMAIN_SNAPSHOT_LOCATION_DEFAULT: + case VIR_DOMAIN_SNAPSHOT_LOCATION_NO: + case VIR_DOMAIN_SNAPSHOT_LOCATION_INTERNAL: + case VIR_DOMAIN_SNAPSHOT_LOCATION_EXTERNAL: + case VIR_DOMAIN_SNAPSHOT_LOCATION_LAST: + break; + + case VIR_DOMAIN_SNAPSHOT_LOCATION_MANUAL: + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", + _("'manual' snapshot mode is not supported by the test driver")); + return -1; + } + } if (flags & VIR_DOMAIN_SNAPSHOT_CREATE_DISK_ONLY) { align_location = VIR_DOMAIN_SNAPSHOT_LOCATION_EXTERNAL; -- 2.35.1
On a Friday in 2022, Peter Krempa wrote:
The idea of the manual mode is to allow a synchronized snapshot in cases when the storage is outsourced to an unmanaged storage provider which requires cooperation with snapshotting.
The mode will instruct the hypervisor to pause along when the other components are snapshotted and the 'manual' disk can be snapshotted along. This increases latency of the snapshot but allows them in otherwise impossible situations.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- docs/formatdomain.rst | 15 ++++++++------- docs/formatsnapshot.rst | 9 +++++++++ docs/schemas/domainsnapshot.rng | 3 +++ src/conf/domain_conf.c | 1 + src/conf/domain_conf.h | 1 + src/conf/snapshot_conf.c | 6 ++++++ src/qemu/qemu_snapshot.c | 5 +++++ src/test/test_driver.c | 17 +++++++++++++++++ 8 files changed, 50 insertions(+), 7 deletions(-)
diff --git a/docs/formatsnapshot.rst b/docs/formatsnapshot.rst index 0fee35d89c..4635df89cb 100644 --- a/docs/formatsnapshot.rst +++ b/docs/formatsnapshot.rst @@ -124,6 +124,15 @@ The top-level ``domainsnapshot`` element may contain the following elements: corresponding domain disk, while others like qemu allow this field to override the domain default.
+ :since:`Since 8.2.0` the ``snapshot`` attribute supports the ``manual`` + value which instructs the hypervisor to create the snapshot and keep a + synchronized state by pausing the VM which allows to snapshot disk + storage from outside of the hypervisor if the storage provider supports + it. The caller is responsible for resuming a VM paused by requesting a + ``manual`` snapshot When reverting such snapshot, the expectation is that
Missing period.
+ the storage is configured in a way where the hypervisor will see the + correct image state. + :since:`Since 1.2.2` the ``disk`` element supports an optional attribute ``type`` if the ``snapshot`` attribute is set to ``external``. This attribute specifies the snapshot target storage type and allows to
Reviewed-by: Ján Tomko <jtomko@redhat.com> Jano
In case we are snapshotting at least one 'manual' disk we will pause the VM and keep it paused. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1866400 Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_snapshot.c | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/src/qemu/qemu_snapshot.c b/src/qemu/qemu_snapshot.c index 3fa6caaaec..03c070b911 100644 --- a/src/qemu/qemu_snapshot.c +++ b/src/qemu/qemu_snapshot.c @@ -458,6 +458,9 @@ qemuSnapshotPrepareDiskExternalActive(virDomainObj *vm, { int actualType = virStorageSourceGetActualType(snapdisk->src); + if (snapdisk->snapshot == VIR_DOMAIN_SNAPSHOT_LOCATION_MANUAL) + return 0; + if (domdisk->device == VIR_DOMAIN_DISK_DEVICE_LUN) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("external active snapshots are not supported on scsi " @@ -677,6 +680,7 @@ qemuSnapshotPrepareDiskInternal(virDomainDiskDef *disk, static int qemuSnapshotPrepare(virDomainObj *vm, virDomainSnapshotDef *def, + bool *has_manual, unsigned int *flags) { qemuDomainObjPrivate *priv = vm->privateData; @@ -758,9 +762,9 @@ qemuSnapshotPrepare(virDomainObj *vm, break; case VIR_DOMAIN_SNAPSHOT_LOCATION_MANUAL: - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("'manual' disk snapshot mode not yet implemented")); - return -1; + *has_manual = true; + forbid_internal = true; + break; case VIR_DOMAIN_SNAPSHOT_LOCATION_NO: /* Remember seeing a disk that has snapshot disabled */ @@ -1341,6 +1345,7 @@ qemuSnapshotCreateActiveExternal(virQEMUDriver *driver, virDomainObj *vm, virDomainMomentObj *snap, virQEMUDriverConfig *cfg, + bool has_manual, unsigned int flags) { virObjectEvent *event; @@ -1391,11 +1396,14 @@ qemuSnapshotCreateActiveExternal(virQEMUDriver *driver, } else if (virDomainObjGetState(vm, NULL) == VIR_DOMAIN_RUNNING) { /* For full system external snapshots (those with memory), the guest * must pause (either by libvirt up front, or by qemu after - * _LIVE converges). */ - if (memory) + * _LIVE converges). We don't want to unpause it though if user has + * elected to manually snapshot some disks */ + if (memory && !has_manual) resume = true; - if (memory && !(flags & VIR_DOMAIN_SNAPSHOT_CREATE_LIVE)) { + /* we need to pause the VM even when we aren't taking a memory snapshot + * when the user wants to manually snapshot some disks */ + if (((memory || has_manual) && !(flags & VIR_DOMAIN_SNAPSHOT_CREATE_LIVE))) { if (qemuProcessStopCPUs(driver, vm, VIR_DOMAIN_PAUSED_SNAPSHOT, QEMU_ASYNC_JOB_SNAPSHOT) < 0) goto cleanup; @@ -1765,11 +1773,12 @@ qemuSnapshotCreate(virDomainObj *vm, virDomainMomentObj *snap = NULL; virDomainMomentObj *current = NULL; virDomainSnapshotPtr ret = NULL; + bool has_manual = false; /* user wants to manually snapshot some disks */ if (qemuSnapshotCreateAlignDisks(vm, snapdef, driver, flags) < 0) return NULL; - if (qemuSnapshotPrepare(vm, snapdef, &flags) < 0) + if (qemuSnapshotPrepare(vm, snapdef, &has_manual, &flags) < 0) return NULL; if (flags & VIR_DOMAIN_SNAPSHOT_CREATE_NO_METADATA) { @@ -1790,7 +1799,7 @@ qemuSnapshotCreate(virDomainObj *vm, if (flags & VIR_DOMAIN_SNAPSHOT_CREATE_DISK_ONLY || virDomainSnapshotObjGetDef(snap)->memory == VIR_DOMAIN_SNAPSHOT_LOCATION_EXTERNAL) { /* external full system or disk snapshot */ - if (qemuSnapshotCreateActiveExternal(driver, vm, snap, cfg, flags) < 0) + if (qemuSnapshotCreateActiveExternal(driver, vm, snap, cfg, has_manual, flags) < 0) goto error; } else { /* internal full system */ -- 2.35.1
Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- docs/kbase/index.rst | 3 +++ docs/kbase/meson.build | 1 + docs/kbase/snapshots.rst | 53 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 57 insertions(+) create mode 100644 docs/kbase/snapshots.rst diff --git a/docs/kbase/index.rst b/docs/kbase/index.rst index 09b19ed1af..77b23e96a8 100644 --- a/docs/kbase/index.rst +++ b/docs/kbase/index.rst @@ -55,6 +55,9 @@ Usage `Memory devices <memorydevices.html>`__ Memory devices and their use +`Snapshots <snapshots.html>` + Details about snapshotting a VM + Internals / Debugging --------------------- diff --git a/docs/kbase/meson.build b/docs/kbase/meson.build index af067db453..d632f9bda1 100644 --- a/docs/kbase/meson.build +++ b/docs/kbase/meson.build @@ -18,6 +18,7 @@ docs_kbase_files = [ 'rpm-deployment', 's390_protected_virt', 'secureusage', + 'snapshots', 'systemtap', 'virtiofs', ] diff --git a/docs/kbase/snapshots.rst b/docs/kbase/snapshots.rst new file mode 100644 index 0000000000..bc457197a0 --- /dev/null +++ b/docs/kbase/snapshots.rst @@ -0,0 +1,53 @@ +========= +Snapshots +========= + +.. contents:: + +Manual storage snapshotting +=========================== + +Certain use cases such as block storage on LVM or disks backed via storage +exported through the ``vhost-user-blk`` protocol may require that snapshots are +done in conjunction with the storage provider which is not managed by **libvirt**. + +To achieve this such disks can use ``snapshot`` mode ``manual``. When a snapshot +has a disk in manual mode the following happens: + + #. ``libvirt`` takes snapshot of the VM memory if requested + + #. If a live snapshot is requested (``VIR_DOMAIN_SNAPSHOT_CREATE_LIVE``) the + VM runs until the memory snapshot phase completes and is then paused. + #. Otherwise the VM is paused right away. + + #. Snapshot of disks which are marked for external snapsot is executed + #. The API return success, the VM is paused. + #. The user snapshots the externally managed storage + #. The user resumes the execution of the VM (``virsh resume $VM``) + +*Note:* It's not required to wait until the API finishes. The snapshotting of +``manual`` disks can be done right after a event of +``VIR_DOMAIN_EVENT_ID_LIFECYCLE`` type with value of +``VIR_DOMAIN_EVENT_SUSPENDED`` is emitted by libvirtd. + +Overview of manual snapshots +---------------------------- + +Manual snapshot of a disk is requested by setting the ``snapshot`` property to +``manual`` in the snapshot XML :: + + <domainsnapshot> + <memory file='/path/to/memory/img'/> + <disks> + <disk name='vda' snapshot='manual'/> + <disk name='vdb' snapshot='external'/> + <disk name='vdc' snapshot='no'/> + </disks> + </domainsnapshot> + +or ``--diskspec vda,snapshot=manual`` when using ``virsh snapshot-create-as``:: + + $ virsh snapshot-create-as --diskspec vda,snapshot=manual \ + --diskspec vdb,snapshot=external \ + --diskspec vdc,snapshot=no $VM \ + --memspec file=/path/to/memory/img -- 2.35.1
On a Friday in 2022, Peter Krempa wrote:
Peter Krempa (3): conf: snapshot: Introduce 'manual' mode for snapshot of a disk qemuSnapshotCreateActiveExternal: Implement manual snapshot mode kbase: Introduce 'snapshots' page and describe the new 'manual' snapshot
docs/formatdomain.rst | 15 +++++----- docs/formatsnapshot.rst | 9 ++++++ docs/kbase/index.rst | 3 ++ docs/kbase/meson.build | 1 + docs/kbase/snapshots.rst | 53 +++++++++++++++++++++++++++++++++ docs/schemas/domainsnapshot.rng | 3 ++ src/conf/domain_conf.c | 1 + src/conf/domain_conf.h | 1 + src/conf/snapshot_conf.c | 6 ++++ src/qemu/qemu_snapshot.c | 24 +++++++++++---- src/test/test_driver.c | 17 +++++++++++ 11 files changed, 121 insertions(+), 12 deletions(-) create mode 100644 docs/kbase/snapshots.rst
Reviewed-by: Ján Tomko <jtomko@redhat.com> Jano
participants (2)
-
Ján Tomko -
Peter Krempa