[libvirt] [PATCH v6 0/9] Incremental backups, focus on API

I'm fairly confident that these API are ready to go (that is, I've got qemu code in the wings to implement these API for the qemu driver, as demonstrated at last KVM forum, and it shouldn't be too hard to add support in the test driver to get some 'make check' coverage similar to what I recently added for snapshots). I'm hoping the APIs make it in for 5.2, even if I'm still dealing with review churn on the later parts of my v5 series (there has been a lot of rebasing from earlier review comments, so v5 is currently still the most recent version that I was able to run demos with, although I hope to post the rest of v6 soon). The biggest diffs compared to v4/v5 are addressing many of John Ferlan's grammar suggestions, removing some TODO items from the public API (although implementing those changes, such as the new VIR_DOMAIN_SNAPSHOT_CREATE_QUIESCE, is still on my todo list), splitting some large patches (checkpoints and backups are now separated), moving the backup APIs into libvirt-domain.c instead of piggybacking on libvirt-domain-checkpoint.c, and adding 2 more API (virDomainSnapshotCreateXML2 to create a checkpoint at the same time as a snapshot; virDomainListJobIds to learn the job ids of all currently running backup jobs [the initial implementation will only ever have job id 1 due to qemu 4.0 limitations, but the API was designed with future expansion in mind]). I've also got some preliminary patches to snapshot in this posting based on things noticed while working checkpoints. 001/9:[down] 'Revert "snapshot: Add virDomainSnapshotObjListParse"' 002/9:[down] 'Revert "snapshot: Add virDomainSnapshotObjListFormat"' 003/9:[down] 'snapshot: Various doc tweaks' 004/9:[0005] [FC] 'backup: Introduce virDomainCheckpointPtr' 005/9:[down] 'backup: Document new XML for checkpoints' 006/9:[0562] [FC] 'backup: Document new XML for backups' 007/9:[0173] [FC] 'backup: Introduce virDomainCheckpoint APIs' 008/9:[0296] [FC] 'backup: Introduce virDomainBackup APIs' 009/9:[0039] [FC] 'backup: Document nuances between different state capture APIs' Eric Blake (9): Revert "snapshot: Add virDomainSnapshotObjListParse" Revert "snapshot: Add virDomainSnapshotObjListFormat" snapshot: Various doc tweaks backup: Introduce virDomainCheckpointPtr backup: Document new XML for checkpoints backup: Document new XML for backups backup: Introduce virDomainCheckpoint APIs backup: Introduce virDomainBackup APIs backup: Document nuances between different state capture APIs include/libvirt/virterror.h | 7 +- src/util/virerror.c | 12 +- include/libvirt/libvirt-domain-checkpoint.h | 161 ++++ include/libvirt/libvirt-domain-snapshot.h | 7 +- include/libvirt/libvirt-domain.h | 59 +- include/libvirt/libvirt.h | 3 +- src/conf/snapshot_conf.h | 7 - src/conf/virdomainmomentobjlist.h | 5 +- src/conf/virdomainsnapshotobjlist.h | 13 - src/datatypes.h | 33 +- src/driver-hypervisor.h | 87 ++- src/qemu/qemu_blockjob.h | 1 + docs/Makefile.am | 3 + docs/apibuild.py | 2 + docs/docs.html.in | 10 +- docs/domainstatecapture.html.in | 313 ++++++++ docs/format.html.in | 2 + docs/formatbackup.html.in | 181 +++++ docs/formatcheckpoint.html.in | 204 +++++ docs/formatsnapshot.html.in | 82 +- docs/index.html.in | 4 +- docs/schemas/domainbackup.rng | 219 ++++++ docs/schemas/domaincheckpoint.rng | 94 +++ examples/object-events/event-test.c | 3 + libvirt.spec.in | 3 + mingw-libvirt.spec.in | 6 + po/POTFILES | 1 + src/Makefile.am | 2 + src/conf/domain_conf.c | 2 +- src/conf/snapshot_conf.c | 2 +- src/conf/virdomainsnapshotobjlist.c | 166 ---- src/datatypes.c | 63 +- src/libvirt-domain-checkpoint.c | 749 +++++++++++++++++++ src/libvirt-domain-snapshot.c | 311 ++++---- src/libvirt-domain.c | 289 ++++++- src/libvirt_private.syms | 4 +- src/libvirt_public.syms | 21 + tests/Makefile.am | 4 + tests/domainbackupxml2xmlin/backup-pull.xml | 9 + tests/domainbackupxml2xmlin/backup-push.xml | 9 + tests/domainbackupxml2xmlin/empty.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 9 + tests/domainbackupxml2xmlout/backup-push.xml | 9 + tests/domainbackupxml2xmlout/empty.xml | 7 + tests/domaincheckpointxml2xmlin/empty.xml | 1 + tests/domaincheckpointxml2xmlin/sample.xml | 7 + tests/domaincheckpointxml2xmlout/empty.xml | 10 + tests/domaincheckpointxml2xmlout/sample.xml | 16 + tests/virschematest.c | 4 + tools/virsh-domain.c | 8 +- 50 files changed, 2794 insertions(+), 431 deletions(-) create mode 100644 include/libvirt/libvirt-domain-checkpoint.h create mode 100644 docs/domainstatecapture.html.in create mode 100644 docs/formatbackup.html.in create mode 100644 docs/formatcheckpoint.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 docs/schemas/domaincheckpoint.rng create mode 100644 src/libvirt-domain-checkpoint.c create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml create mode 100644 tests/domainbackupxml2xmlin/empty.xml create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml create mode 100644 tests/domainbackupxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml -- 2.20.1

This reverts commit 1b57269cbcfcfe998a065c0c9f0f8db408710d87, and subsequent refactorings of the function into new files. There are no callers of this function - I had originally proposed it for implementing a new bulk snapshot API, but that proved to be too invasive given RPC limits. I also tried using it for streamlining how the qemu driver stores snapshot state across libvirtd restarts internally, but in the end, the risks of a new internal format outweighed the benefits of one file per snapshot. Signed-off-by: Eric Blake <eblake@redhat.com> --- src/conf/virdomainsnapshotobjlist.h | 6 -- src/conf/virdomainsnapshotobjlist.c | 105 ---------------------------- src/libvirt_private.syms | 1 - 3 files changed, 112 deletions(-) diff --git a/src/conf/virdomainsnapshotobjlist.h b/src/conf/virdomainsnapshotobjlist.h index 3691662b4c..626cdcafa7 100644 --- a/src/conf/virdomainsnapshotobjlist.h +++ b/src/conf/virdomainsnapshotobjlist.h @@ -30,12 +30,6 @@ virDomainSnapshotObjListPtr virDomainSnapshotObjListNew(void); void virDomainSnapshotObjListFree(virDomainSnapshotObjListPtr snapshots); -int virDomainSnapshotObjListParse(const char *xmlStr, - const unsigned char *domain_uuid, - virDomainSnapshotObjListPtr snapshots, - virCapsPtr caps, - virDomainXMLOptionPtr xmlopt, - unsigned int flags); int virDomainSnapshotObjListFormat(virBufferPtr buf, const char *uuidstr, virDomainSnapshotObjListPtr snapshots, diff --git a/src/conf/virdomainsnapshotobjlist.c b/src/conf/virdomainsnapshotobjlist.c index 9057ecf587..20f4ff22f2 100644 --- a/src/conf/virdomainsnapshotobjlist.c +++ b/src/conf/virdomainsnapshotobjlist.c @@ -39,111 +39,6 @@ struct _virDomainSnapshotObjList { }; -/* Parse a <snapshots> XML entry into snapshots, which must start - * empty. Any <domain> sub-elements of a <domainsnapshot> must match - * domain_uuid. @flags is virDomainSnapshotParseFlags. Return the - * number of snapshots parsed, or -1 on error. - */ -int -virDomainSnapshotObjListParse(const char *xmlStr, - const unsigned char *domain_uuid, - virDomainSnapshotObjListPtr snapshots, - virCapsPtr caps, - virDomainXMLOptionPtr xmlopt, - unsigned int flags) -{ - int ret = -1; - xmlDocPtr xml; - xmlNodePtr root; - xmlXPathContextPtr ctxt = NULL; - int n; - size_t i; - int keepBlanksDefault = xmlKeepBlanksDefault(0); - virDomainMomentObjPtr snap; - VIR_AUTOFREE(xmlNodePtr *) nodes = NULL; - VIR_AUTOFREE(char *) current = NULL; - - if (!(flags & VIR_DOMAIN_SNAPSHOT_PARSE_REDEFINE) || - (flags & VIR_DOMAIN_SNAPSHOT_PARSE_INTERNAL)) { - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("incorrect flags for bulk parse")); - return -1; - } - if (virDomainMomentObjListSize(snapshots->base) != 0) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("bulk define of snapshots only possible with " - "no existing snapshot")); - return -1; - } - - if (!(xml = virXMLParse(NULL, xmlStr, _("(domain_snapshot)")))) - return -1; - - root = xmlDocGetRootElement(xml); - if (!virXMLNodeNameEqual(root, "snapshots")) { - virReportError(VIR_ERR_XML_ERROR, - _("unexpected root element <%s>, " - "expecting <snapshots>"), root->name); - goto cleanup; - } - ctxt = xmlXPathNewContext(xml); - if (ctxt == NULL) { - virReportOOMError(); - goto cleanup; - } - ctxt->node = root; - current = virXMLPropString(root, "current"); - - if ((n = virXPathNodeSet("./domainsnapshot", ctxt, &nodes)) < 0) - goto cleanup; - - for (i = 0; i < n; i++) { - virDomainSnapshotDefPtr def; - - def = virDomainSnapshotDefParseNode(xml, nodes[i], caps, xmlopt, NULL, - flags); - if (!def) - goto cleanup; - if (!(snap = virDomainSnapshotAssignDef(snapshots, def))) { - virDomainSnapshotDefFree(def); - goto cleanup; - } - if (virDomainSnapshotRedefineValidate(def, domain_uuid, NULL, NULL, - flags) < 0) - goto cleanup; - } - - if (virDomainSnapshotUpdateRelations(snapshots) < 0) { - virReportError(VIR_ERR_XML_ERROR, "%s", - _("<snapshots> contains inconsistent parent-child " - "relationships")); - goto cleanup; - } - - if (current) { - snap = virDomainSnapshotFindByName(snapshots, current); - if (!snap) { - virReportError(VIR_ERR_NO_DOMAIN_SNAPSHOT, - _("no snapshot matching current='%s'"), current); - goto cleanup; - } - virDomainSnapshotSetCurrent(snapshots, snap); - } - - ret = n; - cleanup: - if (ret < 0) { - /* There were no snapshots before this call; so on error, just - * blindly delete anything created before the failure. */ - virDomainMomentObjListRemoveAll(snapshots->base); - } - xmlXPathFreeContext(ctxt); - xmlFreeDoc(xml); - xmlKeepBlanksDefault(keepBlanksDefault); - return ret; -} - - /* Struct and callback function used as a hash table callback; each call * appends another snapshot XML to buf, with the caller clearing the * buffer if any callback fails. */ diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index bf5625fbf4..03b4db98fe 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1001,7 +1001,6 @@ virDomainSnapshotObjListFree; virDomainSnapshotObjListGetNames; virDomainSnapshotObjListNew; virDomainSnapshotObjListNum; -virDomainSnapshotObjListParse; virDomainSnapshotObjListRemove; virDomainSnapshotObjListRemoveAll; virDomainSnapshotSetCurrent; -- 2.20.1

On Tue, Mar 26, 2019 at 01:13:45AM -0500, Eric Blake wrote:
This reverts commit 1b57269cbcfcfe998a065c0c9f0f8db408710d87, and subsequent refactorings of the function into new files. There are no callers of this function - I had originally proposed it for implementing a new bulk snapshot API, but that proved to be too invasive given RPC limits. I also tried using it for streamlining how the qemu driver stores snapshot state across libvirtd restarts internally, but in the end, the risks of a new internal format outweighed the benefits of one file per snapshot.
Signed-off-by: Eric Blake <eblake@redhat.com> --- src/conf/virdomainsnapshotobjlist.h | 6 -- src/conf/virdomainsnapshotobjlist.c | 105 ---------------------------- src/libvirt_private.syms | 1 - 3 files changed, 112 deletions(-)
Reviewed-by: Ján Tomko <jtomko@redhat.com> Jano

On Tue, Mar 26, 2019 at 01:13:45AM -0500, Eric Blake wrote:
This reverts commit 1b57269cbcfcfe998a065c0c9f0f8db408710d87, and subsequent refactorings of the function into new files. There are no callers of this function - I had originally proposed it for implementing a new bulk snapshot API, but that proved to be too invasive given RPC limits. I also tried using it for streamlining how the qemu driver stores snapshot state across libvirtd restarts internally, but in the end, the risks of a new internal format outweighed the benefits of one file per snapshot.
Signed-off-by: Eric Blake <eblake@redhat.com> --- src/conf/virdomainsnapshotobjlist.h | 6 -- src/conf/virdomainsnapshotobjlist.c | 105 ---------------------------- src/libvirt_private.syms | 1 - 3 files changed, 112 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

This reverts commit 86c0ed6f70268dfa7c3bba95a0ba96fcfe2ab039, and subsequent refactorings of the function into new files. There are no callers of this function - I had originally proposed it for implementing a new bulk snapshot API, but that proved to be too invasive given RPC limits. I also tried using it for streamlining how the qemu driver stores snapshot state across libvirtd restarts internally, but in the end, the risks of a new internal format outweighed the benefits of one file per snapshot. Signed-off-by: Eric Blake <eblake@redhat.com> --- src/conf/snapshot_conf.h | 7 ---- src/conf/virdomainsnapshotobjlist.h | 7 ---- src/conf/snapshot_conf.c | 2 +- src/conf/virdomainsnapshotobjlist.c | 61 ----------------------------- src/libvirt_private.syms | 1 - 5 files changed, 1 insertion(+), 77 deletions(-) diff --git a/src/conf/snapshot_conf.h b/src/conf/snapshot_conf.h index d082ffbad0..cc8f384bf7 100644 --- a/src/conf/snapshot_conf.h +++ b/src/conf/snapshot_conf.h @@ -119,13 +119,6 @@ char *virDomainSnapshotDefFormat(const char *uuidstr, virCapsPtr caps, virDomainXMLOptionPtr xmlopt, unsigned int flags); -int virDomainSnapshotDefFormatInternal(virBufferPtr buf, - const char *uuidstr, - virDomainSnapshotDefPtr def, - virCapsPtr caps, - virDomainXMLOptionPtr xmlopt, - unsigned int flags); - int virDomainSnapshotAlignDisks(virDomainSnapshotDefPtr snapshot, int default_snapshot, bool require_match); diff --git a/src/conf/virdomainsnapshotobjlist.h b/src/conf/virdomainsnapshotobjlist.h index 626cdcafa7..38d34ea010 100644 --- a/src/conf/virdomainsnapshotobjlist.h +++ b/src/conf/virdomainsnapshotobjlist.h @@ -30,13 +30,6 @@ virDomainSnapshotObjListPtr virDomainSnapshotObjListNew(void); void virDomainSnapshotObjListFree(virDomainSnapshotObjListPtr snapshots); -int virDomainSnapshotObjListFormat(virBufferPtr buf, - const char *uuidstr, - virDomainSnapshotObjListPtr snapshots, - virCapsPtr caps, - virDomainXMLOptionPtr xmlopt, - unsigned int flags); - virDomainMomentObjPtr virDomainSnapshotAssignDef(virDomainSnapshotObjListPtr snapshots, virDomainSnapshotDefPtr def); diff --git a/src/conf/snapshot_conf.c b/src/conf/snapshot_conf.c index cc3f71ab6f..4ce120451e 100644 --- a/src/conf/snapshot_conf.c +++ b/src/conf/snapshot_conf.c @@ -787,7 +787,7 @@ virDomainSnapshotDiskDefFormat(virBufferPtr buf, /* Append XML describing def into buf. Return 0 on success, or -1 on * failure with buf cleared. */ -int +static int virDomainSnapshotDefFormatInternal(virBufferPtr buf, const char *uuidstr, virDomainSnapshotDefPtr def, diff --git a/src/conf/virdomainsnapshotobjlist.c b/src/conf/virdomainsnapshotobjlist.c index 20f4ff22f2..04221134da 100644 --- a/src/conf/virdomainsnapshotobjlist.c +++ b/src/conf/virdomainsnapshotobjlist.c @@ -39,67 +39,6 @@ struct _virDomainSnapshotObjList { }; -/* Struct and callback function used as a hash table callback; each call - * appends another snapshot XML to buf, with the caller clearing the - * buffer if any callback fails. */ -struct virDomainSnapshotFormatData { - virBufferPtr buf; - const char *uuidstr; - virCapsPtr caps; - virDomainXMLOptionPtr xmlopt; - unsigned int flags; -}; - -static int -virDomainSnapshotFormatOne(void *payload, - const void *name ATTRIBUTE_UNUSED, - void *opaque) -{ - virDomainMomentObjPtr snap = payload; - struct virDomainSnapshotFormatData *data = opaque; - return virDomainSnapshotDefFormatInternal(data->buf, data->uuidstr, - virDomainSnapshotObjGetDef(snap), - data->caps, data->xmlopt, - data->flags); -} - - -/* Format the XML for all snapshots in the list into buf. @flags is - * virDomainSnapshotFormatFlags. On error, clear the buffer and return - * -1. */ -int -virDomainSnapshotObjListFormat(virBufferPtr buf, - const char *uuidstr, - virDomainSnapshotObjListPtr snapshots, - virCapsPtr caps, - virDomainXMLOptionPtr xmlopt, - unsigned int flags) -{ - struct virDomainSnapshotFormatData data = { - .buf = buf, - .uuidstr = uuidstr, - .caps = caps, - .xmlopt = xmlopt, - .flags = flags, - }; - - virCheckFlags(VIR_DOMAIN_SNAPSHOT_FORMAT_SECURE, -1); - virBufferAddLit(buf, "<snapshots"); - virBufferEscapeString(buf, " current='%s'", - virDomainSnapshotGetCurrentName(snapshots)); - virBufferAddLit(buf, ">\n"); - virBufferAdjustIndent(buf, 2); - if (virDomainSnapshotForEach(snapshots, virDomainSnapshotFormatOne, - &data) < 0) { - virBufferFreeAndReset(buf); - return -1; - } - virBufferAdjustIndent(buf, -2); - virBufferAddLit(buf, "</snapshots>\n"); - return 0; -} - - virDomainMomentObjPtr virDomainSnapshotAssignDef(virDomainSnapshotObjListPtr snapshots, virDomainSnapshotDefPtr def) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 03b4db98fe..80abbfd0ce 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -996,7 +996,6 @@ virDomainSnapshotForEach; virDomainSnapshotGetCurrent; virDomainSnapshotGetCurrentName; virDomainSnapshotIsCurrentName; -virDomainSnapshotObjListFormat; virDomainSnapshotObjListFree; virDomainSnapshotObjListGetNames; virDomainSnapshotObjListNew; -- 2.20.1

On Tue, Mar 26, 2019 at 01:13:46AM -0500, Eric Blake wrote:
This reverts commit 86c0ed6f70268dfa7c3bba95a0ba96fcfe2ab039, and subsequent refactorings of the function into new files. There are no callers of this function - I had originally proposed it for implementing a new bulk snapshot API, but that proved to be too invasive given RPC limits. I also tried using it for streamlining how the qemu driver stores snapshot state across libvirtd restarts internally, but in the end, the risks of a new internal format outweighed the benefits of one file per snapshot.
Signed-off-by: Eric Blake <eblake@redhat.com> --- src/conf/snapshot_conf.h | 7 ---- src/conf/virdomainsnapshotobjlist.h | 7 ---- src/conf/snapshot_conf.c | 2 +- src/conf/virdomainsnapshotobjlist.c | 61 ----------------------------- src/libvirt_private.syms | 1 - 5 files changed, 1 insertion(+), 77 deletions(-)
Reviewed-by: Ján Tomko <jtomko@redhat.com> Jano

On Tue, Mar 26, 2019 at 01:13:46AM -0500, Eric Blake wrote:
This reverts commit 86c0ed6f70268dfa7c3bba95a0ba96fcfe2ab039, and subsequent refactorings of the function into new files. There are no callers of this function - I had originally proposed it for implementing a new bulk snapshot API, but that proved to be too invasive given RPC limits. I also tried using it for streamlining how the qemu driver stores snapshot state across libvirtd restarts internally, but in the end, the risks of a new internal format outweighed the benefits of one file per snapshot.
Signed-off-by: Eric Blake <eblake@redhat.com> --- src/conf/snapshot_conf.h | 7 ---- src/conf/virdomainsnapshotobjlist.h | 7 ---- src/conf/snapshot_conf.c | 2 +- src/conf/virdomainsnapshotobjlist.c | 61 ----------------------------- src/libvirt_private.syms | 1 - 5 files changed, 1 insertion(+), 77 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

Since I was copying this text to form checkpoint XML and API documentation, I might as well make improvements along the way. Most of these changes are based on reviews of the checkpoint docs. Among other things: grammar tweaks, point to a single source of documentation rather than repeating verbosity, reword things for easier legibility. Signed-off-by: Eric Blake <eblake@redhat.com> --- docs/formatsnapshot.html.in | 80 ++++++------ src/libvirt-domain-snapshot.c | 222 +++++++--------------------------- 2 files changed, 89 insertions(+), 213 deletions(-) diff --git a/docs/formatsnapshot.html.in b/docs/formatsnapshot.html.in index 146908ce76..1bbbf06205 100644 --- a/docs/formatsnapshot.html.in +++ b/docs/formatsnapshot.html.in @@ -99,16 +99,14 @@ </p> <dl> <dt><code>name</code></dt> - <dd>The name for this snapshot. If the name is specified when - initially creating the snapshot, then the snapshot will have - that particular name. If the name is omitted when initially - creating the snapshot, then libvirt will make up a name for - the snapshot, based on the time when it was created. + <dd>The optional name for this snapshot. If the name is + omitted, libvirt will create a name based on the time of the + creation. </dd> <dt><code>description</code></dt> - <dd>A human-readable description of the snapshot. If the - description is omitted when initially creating the snapshot, - then this field will be empty. + <dd>An optional human-readable description of the snapshot. If + the description is omitted when initially creating the + snapshot, then this field will be empty. </dd> <dt><code>memory</code></dt> <dd>On input, this is an optional request for how to handle VM @@ -200,46 +198,52 @@ </dl> </dd> <dt><code>creationTime</code></dt> - <dd>The time this snapshot was created. The time is specified - in seconds since the Epoch, UTC (i.e. Unix time). Readonly. + <dd>A readonly representation of the time this snapshot was + created. The time is specified in seconds since the Epoch, + UTC (i.e. Unix time). </dd> <dt><code>state</code></dt> - <dd>The state of the domain at the time this snapshot was taken. - If a full system snapshot was created, then this - is the state of the domain at that time. When the domain is - reverted to this snapshot, the domain's state will default to - this state, unless overridden + <dd>A readonly representation of the state of the domain at the + time this snapshot was taken. If a full system snapshot was + created, then this is the state of the domain at that + time. When the domain is reverted to this snapshot, the + domain's state will default to this state, unless overridden by <code>virDomainRevertToSnapshot()</code> flags to revert to - a running or paused state. Additionally, - this field can be the value "disk-snapshot" - (<span class="since">since 0.9.5</span>) when it represents - only a disk snapshot (no VM memory state), and reverting to this - snapshot will default to an inactive guest. Readonly. + a running or paused state. Additionally, this field can be the + value "disk-snapshot" (<span class="since">since 0.9.5</span>) + when it represents only a disk snapshot (no VM memory state), + and reverting to this snapshot will default to an inactive + guest. </dd> <dt><code>parent</code></dt> - <dd>The parent of this snapshot. If present, this element - contains exactly one child element, name. This specifies the - name of the parent snapshot of this snapshot, and is used to - represent trees of snapshots. Readonly. + <dd>An optional readonly representation of the parent of this + snapshot. If present, this element contains exactly one child + element, <code>name</code>. This specifies the name of the + parent snapshot of this snapshot, and is used to represent + trees of snapshots. </dd> <dt><code>domain</code></dt> - <dd>The domain that this snapshot was taken against. Older - versions of libvirt stored only a single child element, uuid; - reverting to a snapshot like this is risky if the current - state of the domain differs from the state that the domain was - created in, and requires the use of the - <code>VIR_DOMAIN_SNAPSHOT_REVERT_FORCE</code> flag + <dd>A readonly representation of the domain that this snapshot + was taken against. Older versions of libvirt stored only a + single child element, uuid; reverting to a snapshot like this + is risky if the current state of the domain differs from the + state that the domain was created in, and requires the use of + the <code>VIR_DOMAIN_SNAPSHOT_REVERT_FORCE</code> flag in <code>virDomainRevertToSnapshot()</code>. Newer versions - of libvirt (<span class="since">since 0.9.5</span>) store the entire - inactive <a href="formatdomain.html">domain configuration</a> - at the time of the snapshot (<span class="since">since - 0.9.5</span>). Readonly. + of libvirt (<span class="since">since 0.9.5</span>) store the + entire inactive <a href="formatdomain.html">domain + configuration</a> at the time of the snapshot + (<span class="since">since 0.9.5</span>). The domain will have + security-sensitive information omitted + unless the flag <code>VIR_DOMAIN_SNAPSHOT_XML_SECURE</code> is + provided on a read-write connection. </dd> <dt><code>cookie</code></dt> - <dd>Save image cookie containing additional data libvirt may need to - properly restore a domain from an active snapshot when such data - cannot be stored directly in the <code>domain</code> to maintain - compatibility with older libvirt or hypervisor. Readonly. + <dd>An optional readonly representation of a save image cookie + containing additional data libvirt may need to properly + restore a domain from an active snapshot when such data cannot + be stored directly in the <code>domain</code> to maintain + compatibility with older libvirt or hypervisor. </dd> </dl> diff --git a/src/libvirt-domain-snapshot.c b/src/libvirt-domain-snapshot.c index be9bf71af9..0c8023d9f6 100644 --- a/src/libvirt-domain-snapshot.c +++ b/src/libvirt-domain-snapshot.c @@ -142,8 +142,10 @@ virDomainSnapshotGetConnect(virDomainSnapshotPtr snapshot) * support these flags. * * If @flags includes VIR_DOMAIN_SNAPSHOT_CREATE_NO_METADATA, then the - * domain's disk images are modified according to @xmlDesc, but then - * the just-created snapshot has its metadata deleted. This flag is + * domain's disk images are modified according to @xmlDesc, but + * libvirt does not track any metadata (similar to immediately calling + * virDomainSnapshotDelete() with + * VIR_DOMAIN_SNAPSHOT_DELETE_METADATA_ONLY). This flag is * incompatible with VIR_DOMAIN_SNAPSHOT_CREATE_REDEFINE. * * If @flags includes VIR_DOMAIN_SNAPSHOT_CREATE_HALT, then the domain @@ -203,7 +205,8 @@ virDomainSnapshotGetConnect(virDomainSnapshotPtr snapshot) * virDomainSnapshotFree should be used to free the resources after the * snapshot object is no longer needed. * - * Returns an (opaque) virDomainSnapshotPtr on success, NULL on failure. + * Returns an (opaque) new virDomainSnapshotPtr on success or NULL on + * failure. */ virDomainSnapshotPtr virDomainSnapshotCreateXML(virDomainPtr domain, @@ -260,7 +263,7 @@ virDomainSnapshotCreateXML(virDomainPtr domain, * VIR_DOMAIN_SNAPSHOT_XML_SECURE; this flag is rejected on read-only * connections. * - * Returns a 0 terminated UTF-8 encoded XML instance, or NULL in case + * Returns a 0 terminated UTF-8 encoded XML instance or NULL in case * of error. The caller must free() the returned value. */ char * @@ -308,37 +311,11 @@ virDomainSnapshotGetXMLDesc(virDomainSnapshotPtr snapshot, * @flags only if virDomainSnapshotListNames() can honor it, although * the flag has no other effect here. * - * By default, this command covers all snapshots; it is also possible to - * limit things to just snapshots with no parents, when @flags includes - * VIR_DOMAIN_SNAPSHOT_LIST_ROOTS. Additional filters are provided in - * groups, where each group contains bits that describe mutually exclusive - * attributes of a snapshot, and where all bits within a group describe - * all possible snapshots. Some hypervisors might reject explicit bits - * from a group where the hypervisor cannot make a distinction. For a - * group supported by a given hypervisor, the behavior when no bits of a - * group are set is identical to the behavior when all bits in that group - * are set. When setting bits from more than one group, it is possible to - * select an impossible combination, in that case a hypervisor may return - * either 0 or an error. - * - * The first group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_LEAVES and - * VIR_DOMAIN_SNAPSHOT_LIST_NO_LEAVES, to filter based on snapshots that - * have no further children (a leaf snapshot). - * - * The next group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_METADATA and - * VIR_DOMAIN_SNAPSHOT_LIST_NO_METADATA, for filtering snapshots based on - * whether they have metadata that would prevent the removal of the last - * reference to a domain. - * - * The next group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_INACTIVE, - * VIR_DOMAIN_SNAPSHOT_LIST_ACTIVE, and VIR_DOMAIN_SNAPSHOT_LIST_DISK_ONLY, - * for filtering snapshots based on what domain state is tracked by the - * snapshot. - * - * The next group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_INTERNAL and - * VIR_DOMAIN_SNAPSHOT_LIST_EXTERNAL, for filtering snapshots based on - * whether the snapshot is stored inside the disk images or as - * additional files. + * By default, this command covers all snapshots. It is also possible + * to limit things to just snapshots with no parents, when @flags + * includes VIR_DOMAIN_SNAPSHOT_LIST_ROOTS. Additional filters are + * provided via the same @flags values as documented in + * virDomainListAllSnapshots(). * * Returns the number of domain snapshots found or -1 in case of error. */ @@ -386,37 +363,11 @@ virDomainSnapshotNum(virDomainPtr domain, unsigned int flags) * starting from that earlier snapshot; otherwise, the order of * snapshots in the resulting list is unspecified. * - * By default, this command covers all snapshots; it is also possible to - * limit things to just snapshots with no parents, when @flags includes - * VIR_DOMAIN_SNAPSHOT_LIST_ROOTS. Additional filters are provided in - * groups, where each group contains bits that describe mutually exclusive - * attributes of a snapshot, and where all bits within a group describe - * all possible snapshots. Some hypervisors might reject explicit bits - * from a group where the hypervisor cannot make a distinction. For a - * group supported by a given hypervisor, the behavior when no bits of a - * group are set is identical to the behavior when all bits in that group - * are set. When setting bits from more than one group, it is possible to - * select an impossible combination, in that case a hypervisor may return - * either 0 or an error. - * - * The first group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_LEAVES and - * VIR_DOMAIN_SNAPSHOT_LIST_NO_LEAVES, to filter based on snapshots that - * have no further children (a leaf snapshot). - * - * The next group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_METADATA and - * VIR_DOMAIN_SNAPSHOT_LIST_NO_METADATA, for filtering snapshots based on - * whether they have metadata that would prevent the removal of the last - * reference to a domain. - * - * The next group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_INACTIVE, - * VIR_DOMAIN_SNAPSHOT_LIST_ACTIVE, and VIR_DOMAIN_SNAPSHOT_LIST_DISK_ONLY, - * for filtering snapshots based on what domain state is tracked by the - * snapshot. - * - * The next group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_INTERNAL and - * VIR_DOMAIN_SNAPSHOT_LIST_EXTERNAL, for filtering snapshots based on - * whether the snapshot is stored inside the disk images or as - * additional files. + * By default, this command covers all snapshots. It is also possible + * to limit things to just snapshots with no parents, when @flags + * includes VIR_DOMAIN_SNAPSHOT_LIST_ROOTS. Additional filters are + * provided via the same @flags values as documented in + * virDomainListAllSnapshots(). * * Note that this command is inherently racy: another connection can * define a new snapshot between a call to virDomainSnapshotNum() and @@ -465,12 +416,12 @@ virDomainSnapshotListNames(virDomainPtr domain, char **names, int nameslen, /** * virDomainListAllSnapshots: * @domain: a domain object - * @snaps: pointer to variable to store the array containing snapshot objects, + * @snaps: pointer to variable to store the array containing snapshot objects * or NULL if the list is not required (just returns number of * snapshots) * @flags: bitwise-OR of supported virDomainSnapshotListFlags * - * Collect the list of domain snapshots for the given domain, and allocate + * Collect the list of domain snapshots for the given domain and allocate * an array to store those objects. This API solves the race inherent in * virDomainSnapshotListNames(). * @@ -482,18 +433,15 @@ virDomainSnapshotListNames(virDomainPtr domain, char **names, int nameslen, * otherwise, the order of snapshots in the resulting list is * unspecified. * - * By default, this command covers all snapshots; it is also possible to - * limit things to just snapshots with no parents, when @flags includes - * VIR_DOMAIN_SNAPSHOT_LIST_ROOTS. Additional filters are provided in - * groups, where each group contains bits that describe mutually exclusive - * attributes of a snapshot, and where all bits within a group describe - * all possible snapshots. Some hypervisors might reject explicit bits - * from a group where the hypervisor cannot make a distinction. For a - * group supported by a given hypervisor, the behavior when no bits of a - * group are set is identical to the behavior when all bits in that group - * are set. When setting bits from more than one group, it is possible to - * select an impossible combination, in that case a hypervisor may return - * either 0 or an error. + * By default, this command covers all snapshots. It is also possible + * to limit things to just snapshots with no parents, when @flags + * includes VIR_DOMAIN_SNAPSHOT_LIST_ROOTS. Additional filters are + * provided in groups listed below. Within a group, bits are mutually + * exclusive, where all possible snapshots are described by exactly + * one bit from the group. Some hypervisors might reject particular + * flags where it cannot make a distinction for filtering. If the set + * of filter flags selected forms an impossible combination, the + * hypervisor may return either 0 or an error. * * The first group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_LEAVES and * VIR_DOMAIN_SNAPSHOT_LIST_NO_LEAVES, to filter based on snapshots that @@ -562,37 +510,11 @@ virDomainListAllSnapshots(virDomainPtr domain, virDomainSnapshotPtr **snaps, * @flags only if virDomainSnapshotListChildrenNames() can honor it, * although the flag has no other effect here. * - * By default, this command covers only direct children; it is also possible - * to expand things to cover all descendants, when @flags includes - * VIR_DOMAIN_SNAPSHOT_LIST_DESCENDANTS. Also, some filters are provided in - * groups, where each group contains bits that describe mutually exclusive - * attributes of a snapshot, and where all bits within a group describe - * all possible snapshots. Some hypervisors might reject explicit bits - * from a group where the hypervisor cannot make a distinction. For a - * group supported by a given hypervisor, the behavior when no bits of a - * group are set is identical to the behavior when all bits in that group - * are set. When setting bits from more than one group, it is possible to - * select an impossible combination, in that case a hypervisor may return - * either 0 or an error. - * - * The first group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_LEAVES and - * VIR_DOMAIN_SNAPSHOT_LIST_NO_LEAVES, to filter based on snapshots that - * have no further children (a leaf snapshot). - * - * The next group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_METADATA and - * VIR_DOMAIN_SNAPSHOT_LIST_NO_METADATA, for filtering snapshots based on - * whether they have metadata that would prevent the removal of the last - * reference to a domain. - * - * The next group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_INACTIVE, - * VIR_DOMAIN_SNAPSHOT_LIST_ACTIVE, and VIR_DOMAIN_SNAPSHOT_LIST_DISK_ONLY, - * for filtering snapshots based on what domain state is tracked by the - * snapshot. - * - * The next group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_INTERNAL and - * VIR_DOMAIN_SNAPSHOT_LIST_EXTERNAL, for filtering snapshots based on - * whether the snapshot is stored inside the disk images or as - * additional files. + * By default, this command covers only direct children. It is also + * possible to expand things to cover all descendants, when @flags + * includes VIR_DOMAIN_SNAPSHOT_LIST_DESCENDANTS. Additional filters + * are provided via the same @flags values as documented in + * virDomainSnapshotListAllChildren(). * * Returns the number of domain snapshots found or -1 in case of error. */ @@ -642,39 +564,12 @@ virDomainSnapshotNumChildren(virDomainSnapshotPtr snapshot, unsigned int flags) * earlier snapshot; otherwise, the order of snapshots in the * resulting list is unspecified. * - * By default, this command covers only direct children; it is also possible - * to expand things to cover all descendants, when @flags includes - * VIR_DOMAIN_SNAPSHOT_LIST_DESCENDANTS. Also, some filters are provided in - * groups, where each group contains bits that describe mutually exclusive - * attributes of a snapshot, and where all bits within a group describe - * all possible snapshots. Some hypervisors might reject explicit bits - * from a group where the hypervisor cannot make a distinction. For a - * group supported by a given hypervisor, the behavior when no bits of a - * group are set is identical to the behavior when all bits in that group - * are set. When setting bits from more than one group, it is possible to - * select an impossible combination, in that case a hypervisor may return - * either 0 or an error. + * By default, this command covers only direct children. It is also + * possible to expand things to cover all descendants, when @flags + * includes VIR_DOMAIN_SNAPSHOT_LIST_DESCENDANTS. Additional filters + * are provided via the same @flags values as documented in + * virDomainSnapshotListAllChildren(). * - * The first group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_LEAVES and - * VIR_DOMAIN_SNAPSHOT_LIST_NO_LEAVES, to filter based on snapshots that - * have no further children (a leaf snapshot). - * - * The next group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_METADATA and - * VIR_DOMAIN_SNAPSHOT_LIST_NO_METADATA, for filtering snapshots based on - * whether they have metadata that would prevent the removal of the last - * reference to a domain. - * - * The next group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_INACTIVE, - * VIR_DOMAIN_SNAPSHOT_LIST_ACTIVE, and VIR_DOMAIN_SNAPSHOT_LIST_DISK_ONLY, - * for filtering snapshots based on what domain state is tracked by the - * snapshot. - * - * The next group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_INTERNAL and - * VIR_DOMAIN_SNAPSHOT_LIST_EXTERNAL, for filtering snapshots based on - * whether the snapshot is stored inside the disk images or as - * additional files. - * - * Returns the number of domain snapshots found or -1 in case of error. * Note that this command is inherently racy: another connection can * define a new snapshot between a call to virDomainSnapshotNumChildren() * and this call. You are only guaranteed that all currently defined @@ -725,7 +620,7 @@ virDomainSnapshotListChildrenNames(virDomainSnapshotPtr snapshot, /** * virDomainSnapshotListAllChildren: * @snapshot: a domain snapshot object - * @snaps: pointer to variable to store the array containing snapshot objects, + * @snaps: pointer to variable to store the array containing snapshot objects * or NULL if the list is not required (just returns number of * snapshots) * @flags: bitwise-OR of supported virDomainSnapshotListFlags @@ -742,37 +637,14 @@ virDomainSnapshotListChildrenNames(virDomainSnapshotPtr snapshot, * starting from that earlier snapshot; otherwise, the order of * snapshots in the resulting list is unspecified. * - * By default, this command covers only direct children; it is also possible - * to expand things to cover all descendants, when @flags includes - * VIR_DOMAIN_SNAPSHOT_LIST_DESCENDANTS. Also, some filters are provided in - * groups, where each group contains bits that describe mutually exclusive - * attributes of a snapshot, and where all bits within a group describe - * all possible snapshots. Some hypervisors might reject explicit bits - * from a group where the hypervisor cannot make a distinction. For a - * group supported by a given hypervisor, the behavior when no bits of a - * group are set is identical to the behavior when all bits in that group - * are set. When setting bits from more than one group, it is possible to - * select an impossible combination, in that case a hypervisor may return - * either 0 or an error. - * - * The first group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_LEAVES and - * VIR_DOMAIN_SNAPSHOT_LIST_NO_LEAVES, to filter based on snapshots that - * have no further children (a leaf snapshot). - * - * The next group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_METADATA and - * VIR_DOMAIN_SNAPSHOT_LIST_NO_METADATA, for filtering snapshots based on - * whether they have metadata that would prevent the removal of the last - * reference to a domain. - * - * The next group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_INACTIVE, - * VIR_DOMAIN_SNAPSHOT_LIST_ACTIVE, and VIR_DOMAIN_SNAPSHOT_LIST_DISK_ONLY, - * for filtering snapshots based on what domain state is tracked by the - * snapshot. - * - * The next group of @flags is VIR_DOMAIN_SNAPSHOT_LIST_INTERNAL and - * VIR_DOMAIN_SNAPSHOT_LIST_EXTERNAL, for filtering snapshots based on - * whether the snapshot is stored inside the disk images or as - * additional files. + * By default, this command covers only direct children. It is also + * possible to expand things to cover all descendants, when @flags + * includes VIR_DOMAIN_SNAPSHOT_LIST_DESCENDANTS. Additional filters + * are provided via the remaining @flags values as documented in + * virDomainListAllSnapshots(), with the exception that + * VIR_DOMAIN_SNAPSHOT_LIST_ROOTS is not supported (in fact, + * VIR_DOMAIN_SNAPSHOT_LIST_DESCENDANTS has the same bit value but + * opposite semantics of widening rather than narrowing the listing). * * Returns the number of domain snapshots found or -1 and sets @snaps to * NULL in case of error. On success, the array stored into @snaps is -- 2.20.1

On Tue, Mar 26, 2019 at 01:13:47AM -0500, Eric Blake wrote:
Since I was copying this text to form checkpoint XML and API documentation, I might as well make improvements along the way. Most of these changes are based on reviews of the checkpoint docs.
Among other things: grammar tweaks, point to a single source of documentation rather than repeating verbosity, reword things for easier legibility.
Signed-off-by: Eric Blake <eblake@redhat.com> --- docs/formatsnapshot.html.in | 80 ++++++------ src/libvirt-domain-snapshot.c | 222 +++++++--------------------------- 2 files changed, 89 insertions(+), 213 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

Prepare for introducing a bunch of new public APIs related to backup checkpoints by first introducing a new internal type and errors associated with that type. Checkpoints are modeled heavily after virDomainSnapshotPtr (both represent a point in time of the guest), although a snapshot exists with the intent of rolling back to that state, while a checkpoint exists to make it possible to create an incremental backup at a later time. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com> --- include/libvirt/virterror.h | 7 +++-- src/util/virerror.c | 12 ++++++- include/libvirt/libvirt.h | 6 +++- src/datatypes.h | 33 ++++++++++++++++++- src/datatypes.c | 63 ++++++++++++++++++++++++++++++++++++- src/libvirt_private.syms | 2 ++ 6 files changed, 117 insertions(+), 6 deletions(-) diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h index 3c19ff5e2e..6dc83a17cc 100644 --- a/include/libvirt/virterror.h +++ b/include/libvirt/virterror.h @@ -4,7 +4,7 @@ * Description: Provides the interfaces of the libvirt library to handle * errors raised while using the library. * - * Copyright (C) 2006-2016 Red Hat, Inc. + * Copyright (C) 2006-2019 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -132,6 +132,7 @@ typedef enum { VIR_FROM_LIBSSH = 66, /* Error from libssh connection transport */ VIR_FROM_RESCTRL = 67, /* Error from resource control */ VIR_FROM_FIREWALLD = 68, /* Error from firewalld */ + VIR_FROM_DOMAIN_CHECKPOINT = 69, /* Error from domain checkpoint */ # ifdef VIR_ENUM_SENTINELS VIR_ERR_DOMAIN_LAST @@ -322,11 +323,13 @@ typedef enum { VIR_ERR_DEVICE_MISSING = 99, /* fail to find the desired device */ VIR_ERR_INVALID_NWFILTER_BINDING = 100, /* invalid nwfilter binding */ VIR_ERR_NO_NWFILTER_BINDING = 101, /* no nwfilter binding */ + VIR_ERR_INVALID_DOMAIN_CHECKPOINT = 102, /* invalid domain checkpoint */ + VIR_ERR_NO_DOMAIN_CHECKPOINT = 103, /* domain checkpoint not found */ + VIR_ERR_NO_DOMAIN_BACKUP = 104, /* domain backup job id not found */ # ifdef VIR_ENUM_SENTINELS VIR_ERR_NUMBER_LAST # endif - } virErrorNumber; /** diff --git a/src/util/virerror.c b/src/util/virerror.c index 91a513160f..05e535d859 100644 --- a/src/util/virerror.c +++ b/src/util/virerror.c @@ -1,7 +1,7 @@ /* * virerror.c: error handling and reporting code for libvirt * - * Copyright (C) 2006, 2008-2016 Red Hat, Inc. + * Copyright (C) 2006-2019 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -139,6 +139,7 @@ VIR_ENUM_IMPL(virErrorDomain, VIR_ERR_DOMAIN_LAST, "Libssh transport layer", "Resource control", "FirewallD", + "Domain Checkpoint", ); @@ -1214,6 +1215,15 @@ const virErrorMsgTuple virErrorMsgStrings[VIR_ERR_NUMBER_LAST] = { [VIR_ERR_NO_NWFILTER_BINDING] = { N_("Network filter binding not found"), N_("Network filter binding not found: %s") }, + [VIR_ERR_INVALID_DOMAIN_CHECKPOINT] = { + N_("Invalid domain checkpoint"), + N_("Invalid domain checkpoint: %s") }, + [VIR_ERR_NO_DOMAIN_CHECKPOINT] = { + N_("Domain checkpoint not found"), + N_("Domain checkpoint not found: %s") }, + [VIR_ERR_NO_DOMAIN_BACKUP] = { + N_("Domain backup job id not found"), + N_("Domain backup job id not found: %s") }, }; diff --git a/include/libvirt/libvirt.h b/include/libvirt/libvirt.h index 20e5d276a7..13de151cb6 100644 --- a/include/libvirt/libvirt.h +++ b/include/libvirt/libvirt.h @@ -4,7 +4,7 @@ * Description: Provides the interfaces of the libvirt library to handle * virtualized domains * - * Copyright (C) 2005-2006, 2010-2014 Red Hat, Inc. + * Copyright (C) 2005-2019 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -34,6 +34,10 @@ extern "C" { # include <libvirt/libvirt-common.h> # include <libvirt/libvirt-host.h> # include <libvirt/libvirt-domain.h> +/* FIXME: Temporary hack until later patch creates new + * libvirt-domain-checkpoint.h file */ +typedef struct _virDomainCheckpoint virDomainCheckpoint; +typedef virDomainCheckpoint *virDomainCheckpointPtr; # include <libvirt/libvirt-domain-snapshot.h> # include <libvirt/libvirt-event.h> # include <libvirt/libvirt-interface.h> diff --git a/src/datatypes.h b/src/datatypes.h index 12015679f3..87f3d5e095 100644 --- a/src/datatypes.h +++ b/src/datatypes.h @@ -1,7 +1,7 @@ /* * datatypes.h: management of structs for public data types * - * Copyright (C) 2006-2015 Red Hat, Inc. + * Copyright (C) 2006-2019 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -31,6 +31,7 @@ extern virClassPtr virConnectClass; extern virClassPtr virDomainClass; +extern virClassPtr virDomainCheckpointClass; extern virClassPtr virDomainSnapshotClass; extern virClassPtr virInterfaceClass; extern virClassPtr virNetworkClass; @@ -292,6 +293,21 @@ extern virClassPtr virAdmClientClass; } \ } while (0) +# define virCheckDomainCheckpointReturn(obj, retval) \ + do { \ + virDomainCheckpointPtr _check = (obj); \ + if (!virObjectIsClass(_check, virDomainCheckpointClass) || \ + !virObjectIsClass(_check->domain, virDomainClass) || \ + !virObjectIsClass(_check->domain->conn, virConnectClass)) { \ + virReportErrorHelper(VIR_FROM_DOMAIN_CHECKPOINT, \ + VIR_ERR_INVALID_DOMAIN_CHECKPOINT, \ + __FILE__, __FUNCTION__, __LINE__, \ + __FUNCTION__); \ + virDispatchError(NULL); \ + return retval; \ + } \ + } while (0) + # define virCheckDomainSnapshotReturn(obj, retval) \ do { \ virDomainSnapshotPtr _snap = (obj); \ @@ -668,6 +684,19 @@ struct _virStream { virFreeCallback ff; }; + +/** + * _virDomainCheckpoint + * + * Internal structure associated with a domain checkpoint + */ +struct _virDomainCheckpoint { + virObject parent; + char *name; + virDomainPtr domain; +}; + + /** * _virDomainSnapshot * @@ -744,6 +773,8 @@ virNWFilterPtr virGetNWFilter(virConnectPtr conn, virNWFilterBindingPtr virGetNWFilterBinding(virConnectPtr conn, const char *portdev, const char *filtername); +virDomainCheckpointPtr virGetDomainCheckpoint(virDomainPtr domain, + const char *name); virDomainSnapshotPtr virGetDomainSnapshot(virDomainPtr domain, const char *name); diff --git a/src/datatypes.c b/src/datatypes.c index 9b92d892d5..14ad5442fd 100644 --- a/src/datatypes.c +++ b/src/datatypes.c @@ -1,7 +1,7 @@ /* * datatypes.c: management of structs for public data types * - * Copyright (C) 2006-2015 Red Hat, Inc. + * Copyright (C) 2006-2019 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -36,6 +36,7 @@ VIR_LOG_INIT("datatypes"); virClassPtr virConnectClass; virClassPtr virConnectCloseCallbackDataClass; virClassPtr virDomainClass; +virClassPtr virDomainCheckpointClass; virClassPtr virDomainSnapshotClass; virClassPtr virInterfaceClass; virClassPtr virNetworkClass; @@ -50,6 +51,7 @@ virClassPtr virStoragePoolClass; static void virConnectDispose(void *obj); static void virConnectCloseCallbackDataDispose(void *obj); static void virDomainDispose(void *obj); +static void virDomainCheckpointDispose(void *obj); static void virDomainSnapshotDispose(void *obj); static void virInterfaceDispose(void *obj); static void virNetworkDispose(void *obj); @@ -86,6 +88,7 @@ virDataTypesOnceInit(void) DECLARE_CLASS_LOCKABLE(virConnect); DECLARE_CLASS_LOCKABLE(virConnectCloseCallbackData); DECLARE_CLASS(virDomain); + DECLARE_CLASS(virDomainCheckpoint); DECLARE_CLASS(virDomainSnapshot); DECLARE_CLASS(virInterface); DECLARE_CLASS(virNetwork); @@ -900,6 +903,64 @@ virNWFilterBindingDispose(void *obj) } +/** + * virGetDomainCheckpoint: + * @domain: the domain to checkpoint + * @name: pointer to the domain checkpoint name + * + * Allocates a new domain checkpoint object. When the object is no longer needed, + * virObjectUnref() must be called in order to not leak data. + * + * Returns a pointer to the domain checkpoint object, or NULL on error. + */ +virDomainCheckpointPtr +virGetDomainCheckpoint(virDomainPtr domain, + const char *name) +{ + virDomainCheckpointPtr ret = NULL; + + if (virDataTypesInitialize() < 0) + return NULL; + + virCheckDomainGoto(domain, error); + virCheckNonNullArgGoto(name, error); + + if (!(ret = virObjectNew(virDomainCheckpointClass))) + goto error; + if (VIR_STRDUP(ret->name, name) < 0) + goto error; + + ret->domain = virObjectRef(domain); + + return ret; + + error: + virObjectUnref(ret); + return NULL; +} + + +/** + * virDomainCheckpointDispose: + * @obj: the domain checkpoint to release + * + * Unconditionally release all memory associated with a checkpoint. + * The checkpoint object must not be used once this method returns. + * + * It will also unreference the associated connection object, + * which may also be released if its ref count hits zero. + */ +static void +virDomainCheckpointDispose(void *obj) +{ + virDomainCheckpointPtr checkpoint = obj; + VIR_DEBUG("release checkpoint %p %s", checkpoint, checkpoint->name); + + VIR_FREE(checkpoint->name); + virObjectUnref(checkpoint->domain); +} + + /** * virGetDomainSnapshot: * @domain: the domain to snapshot diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 80abbfd0ce..9c2d5e75fa 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1252,10 +1252,12 @@ virConnectCloseCallbackDataClass; virConnectCloseCallbackDataGetCallback; virConnectCloseCallbackDataRegister; virConnectCloseCallbackDataUnregister; +virDomainCheckpointClass; virDomainClass; virDomainSnapshotClass; virGetConnect; virGetDomain; +virGetDomainCheckpoint; virGetDomainSnapshot; virGetInterface; virGetNetwork; -- 2.20.1

On Tue, Mar 26, 2019 at 01:13:48AM -0500, Eric Blake wrote:
Prepare for introducing a bunch of new public APIs related to backup checkpoints by first introducing a new internal type and errors associated with that type. Checkpoints are modeled heavily after virDomainSnapshotPtr (both represent a point in time of the guest), although a snapshot exists with the intent of rolling back to that state, while a checkpoint exists to make it possible to create an incremental backup at a later time.
Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com> --- include/libvirt/virterror.h | 7 +++-- src/util/virerror.c | 12 ++++++- include/libvirt/libvirt.h | 6 +++- src/datatypes.h | 33 ++++++++++++++++++- src/datatypes.c | 63 ++++++++++++++++++++++++++++++++++++- src/libvirt_private.syms | 2 ++ 6 files changed, 117 insertions(+), 6 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

Prepare for new checkpoint APIs by describing the XML that will represent a checkpoint. The checkpoint XML is modeled heavily after virDomainSnapshotPtr. (See the docs for more details). Add testsuite coverage for some minimal uses of the XML. Signed-off-by: Eric Blake <eblake@redhat.com> --- docs/docs.html.in | 3 +- docs/format.html.in | 1 + docs/formatcheckpoint.html.in | 202 ++++++++++++++++++++ docs/index.html.in | 3 +- docs/schemas/domaincheckpoint.rng | 94 +++++++++ libvirt.spec.in | 1 + mingw-libvirt.spec.in | 2 + tests/Makefile.am | 2 + tests/domaincheckpointxml2xmlin/empty.xml | 1 + tests/domaincheckpointxml2xmlin/sample.xml | 7 + tests/domaincheckpointxml2xmlout/empty.xml | 10 + tests/domaincheckpointxml2xmlout/sample.xml | 16 ++ tests/virschematest.c | 2 + 13 files changed, 342 insertions(+), 2 deletions(-) create mode 100644 docs/formatcheckpoint.html.in create mode 100644 docs/schemas/domaincheckpoint.rng create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml diff --git a/docs/docs.html.in b/docs/docs.html.in index d0ff844d0c..f6e599b77b 100644 --- a/docs/docs.html.in +++ b/docs/docs.html.in @@ -80,7 +80,8 @@ <a href="formatstoragecaps.html">storage pool capabilities</a>, <a href="formatnode.html">node devices</a>, <a href="formatsecret.html">secrets</a>, - <a href="formatsnapshot.html">snapshots</a></dd> + <a href="formatsnapshot.html">snapshots</a>, + <a href="formatcheckpoint.html">checkpoints</a></dd> <dt><a href="uri.html">URI format</a></dt> <dd>The URI formats used for connecting to libvirt</dd> diff --git a/docs/format.html.in b/docs/format.html.in index 640a9957ee..20f9ef3348 100644 --- a/docs/format.html.in +++ b/docs/format.html.in @@ -25,6 +25,7 @@ <li><a href="formatnode.html">Node devices</a></li> <li><a href="formatsecret.html">Secrets</a></li> <li><a href="formatsnapshot.html">Snapshots</a></li> + <li><a href="formatcheckpoint.html">Checkpoints</a></li> </ul> <h2>Command line validation</h2> diff --git a/docs/formatcheckpoint.html.in b/docs/formatcheckpoint.html.in new file mode 100644 index 0000000000..5c844f1f27 --- /dev/null +++ b/docs/formatcheckpoint.html.in @@ -0,0 +1,202 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE html> +<html xmlns="http://www.w3.org/1999/xhtml"> + <body> + <h1>Checkpoint XML format</h1> + + <ul id="toc"></ul> + + <h2><a id="CheckpointAttributes">Checkpoint XML</a></h2> + + <p> + One method of capturing domain disk backups is via the use of + incremental backups. Right now, incremental backups are only + supported for the QEMU hypervisor when using qcow2 disks at the + active layer; if other disk formats are in use, capturing disk + backups requires different libvirt APIs. + </p> + <p> + Libvirt is able to facilitate incremental backups by tracking + disk checkpoints, which are points in time against which it is + easy to compute which portion of the disk has changed. Given a + full backup (a backup created from the creation of the disk to a + given point in time), coupled with the creation of a disk + checkpoint at that time, and an incremental backup (a backup + created from just the dirty portion of the disk between the + first checkpoint and the second backup operation), it is + possible to do an offline reconstruction of the state of the + disk at the time of the second backup without having to copy as + much data as a second full backup would require. Most disk + checkpoints are created in conjunction with a backup + via <code>virDomainBackupBegin()</code> or with a snapshot + via <code>virDomainSnapshotCreateXML2</code>; however, libvirt + also exposes enough support to create disk checkpoints + independently from a backup operation + via <code>virDomainCheckpointCreateXML()</code>. + </p> + <p> + Attributes of libvirt checkpoints are stored as child elements + of the <code>domaincheckpoint</code> element. At checkpoint + creation time, normally only + the <code>name</code>, <code>description</code>, + and <code>disks</code> elements are settable. The rest of the + fields are ignored on creation and will be filled in by libvirt + in for informational purposes + by <code>virDomainCheckpointGetXMLDesc()</code>. However, when + redefining a checkpoint, with + the <code>VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE</code> flag + of <code>virDomainCheckpointCreateXML()</code>, all of the XML + fields described here are relevant on input, even the fields + that are normally described as readonly for output. + </p> + <p> + Checkpoints are maintained in a hierarchy. A domain can have a + current checkpoint, which is the most recent checkpoint compared + to the current state of the domain (although a domain might have + checkpoints without a current checkpoint, if checkpoints have + been deleted in the meantime). Creating a checkpoint sets that + checkpoint as current and the prior current checkpoint is the + parent of the new checkpoint. In the future, reverting to a + domain snapshot may also affect the current checkpoint. + </p> + <p> + The top-level <code>domaincheckpoint</code> element may contain + the following elements: + </p> + <dl> + <dt><code>name</code></dt> + <dd>The optional name for this checkpoint. If the name is + omitted, libvirt will create a name based on the time of the + creation. + </dd> + <dt><code>description</code></dt> + <dd>An optional human-readable description of the checkpoint. + If the description is omitted when initially creating the + checkpoint, then this field will be empty. + </dd> + <dt><code>disks</code></dt> + <dd>On input, this is an optional listing of specific + instructions for disk checkpoints; it is needed when making a + checkpoint on only a subset of the disks associated with a + domain. In particular, since QEMU checkpoints require qcow2 + disks, this element may be needed on input for excluding guest + disks that are not in qcow2 format. If the entire element was + omitted on input, then all disks participate in the + checkpoint, otherwise, only the disks explicitly listed which + do not also use <code>checkpoint='no'</code> will + participate. On output, this is the checkpoint state of each + of the domain's disks. + <dl> + <dt><code>disk</code></dt> + <dd>This sub-element describes the checkpoint properties of + a specific disk with the following attributes: + <dl> + <dt><code>name</code></dt> + <dd>A mandatory attribute which must match either + the <code><target dev='name'/></code> or an + unambiguous <code><source file='name'/></code> + of one of + the <a href="formatdomain.html#elementsDisks">disk + devices</a> specified for the domain at the time of + the checkpoint.</dd> + <dt><code>checkpoint</code></dt> + <dd>An optional attribute; possible values + are <code>no</code> when the disk does not participate + in this checkpoint; or <code>bitmap</code> if the disk + will track all changes since the creation of this + checkpoint via a bitmap.</dd> + <dt><code>bitmap</code></dt> + <dd>The attribute <code>bitmap</code> is only valid + if <code>checkpoint='bitmap'</code>; it describes the + name of the tracking bitmap (defaulting to the + checkpoint name).</dd> + <dt><code>size</code></dt> + <dd>The attribute <code>size</code> is ignored on input; + on output, it is only present if + the <code>VIR_DOMAIN_CHECKPOINT_XML_SIZE</code> flag + was used to perform a dynamic query of the estimated + size in bytes of the changes made since the checkpoint + was created.</dd> + </dl> + </dd> + </dl> + </dd> + <dt><code>creationTime</code></dt> + <dd>A readonly representation of the time this checkpoint was + created. The time is specified in seconds since the Epoch, + UTC (i.e. Unix time). + </dd> + <dt><code>parent</code></dt> + <dd>An optional readonly representation of the parent of this + checkpoint. If present, this element contains exactly one + child element, <code>name</code>. + </dd> + <dt><code>domain</code></dt> + <dd>A readonly representation of the + inactive <a href="formatdomain.html">domain configuration</a> + at the time the checkpoint was created. This element may be + omitted for output brevity by supplying + the <code>VIR_DOMAIN_CHECKPOINT_XML_NO_DOMAIN</code> flag, but + the resulting XML is no longer viable for use with + the <code>VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE</code> flag + of <code>virDomainCheckpointCreateXML()</code>. The domain + will have security-sensitive information omitted unless the + flag <code>VIR_DOMAIN_SNAPSHOT_XML_SECURE</code> is provided + on a read-write connection. + </dd> + </dl> + + <h2><a id="example">Examples</a></h2> + + <p>Using this XML to create a checkpoint of just vda on a qemu + domain with two disks and a prior checkpoint:</p> + <pre> +<domaincheckpoint> + <description>Completion of updates after OS install</description> + <disks> + <disk name='vda' checkpoint='bitmap'/> + <disk name='vdb' checkpoint='no'/> + </disks> +</domaincheckpoint></pre> + + <p>will result in XML similar to this from + <code>virDomainCheckpointGetXMLDesc()</code>:</p> + <pre> +<domaincheckpoint> + <name>1525889631</name> + <description>Completion of updates after OS install</description> + <creationTime>1525889631</creationTime> + <parent> + <name>1525111885</name> + </parent> + <disks> + <disk name='vda' checkpoint='bitmap' bitmap='1525889631'/> + <disk name='vdb' checkpoint='no'/> + </disks> + <domain type='qemu'> + <name>fedora</name> + <uuid>93a5c045-6457-2c09-e56c-927cdf34e178</uuid> + <memory>1048576</memory> + ... + <devices> + <disk type='file' device='disk'> + <driver name='qemu' type='qcow2'/> + <source file='/path/to/file1'/> + <target dev='vda' bus='virtio'/> + </disk> + <disk type='file' device='disk' snapshot='external'> + <driver name='qemu' type='raw'/> + <source file='/path/to/file2'/> + <target dev='vdb' bus='virtio'/> + </disk> + ... + </devices> + </domain> +</domaincheckpoint></pre> + + <p>With that checkpoint created, the qcow2 image is now tracking + all changes that occur in the image since the checkpoint via + the persistent bitmap named <code>1525889631</code>. + </p> + </body> +</html> diff --git a/docs/index.html.in b/docs/index.html.in index f593445d06..0315915402 100644 --- a/docs/index.html.in +++ b/docs/index.html.in @@ -69,7 +69,8 @@ <a href="formatstoragecaps.html">storage pool capabilities</a>, <a href="formatnode.html">node devices</a>, <a href="formatsecret.html">secrets</a>, - <a href="formatsnapshot.html">snapshots</a></dd> + <a href="formatsnapshot.html">snapshots</a>, + <a href="formatcheckpoint.html">checkpoints</a></dd> <dt><a href="http://wiki.libvirt.org">Wiki</a></dt> <dd>Read further community contributed content</dd> </dl> diff --git a/docs/schemas/domaincheckpoint.rng b/docs/schemas/domaincheckpoint.rng new file mode 100644 index 0000000000..d8dfda9f1c --- /dev/null +++ b/docs/schemas/domaincheckpoint.rng @@ -0,0 +1,94 @@ +<?xml version="1.0"?> +<!-- A Relax NG schema for the libvirt domain checkpoint properties XML format --> +<grammar xmlns="http://relaxng.org/ns/structure/1.0"> + <start> + <ref name='domaincheckpoint'/> + </start> + + <include href='domaincommon.rng'/> + + <define name='domaincheckpoint'> + <element name='domaincheckpoint'> + <interleave> + <optional> + <element name='name'> + <text/> + </element> + </optional> + <optional> + <element name='description'> + <text/> + </element> + </optional> + <optional> + <element name='creationTime'> + <text/> + </element> + </optional> + <optional> + <element name='disks'> + <oneOrMore> + <ref name='diskcheckpoint'/> + </oneOrMore> + </element> + </optional> + <optional> + <choice> + <element name='domain'> + <element name='uuid'> + <ref name="UUID"/> + </element> + </element> + <!-- Nested grammar ensures that any of our overrides of + storagecommon/domaincommon defines do not conflict + with any domain.rng overrides. --> + <grammar> + <include href='domain.rng'/> + </grammar> + </choice> + </optional> + <optional> + <element name='parent'> + <element name='name'> + <text/> + </element> + </element> + </optional> + </interleave> + </element> + </define> + + <define name='diskcheckpoint'> + <element name='disk'> + <attribute name='name'> + <choice> + <ref name='diskTarget'/> + <ref name='absFilePath'/> + </choice> + </attribute> + <choice> + <attribute name='checkpoint'> + <value>no</value> + </attribute> + <group> + <optional> + <attribute name='checkpoint'> + <value>bitmap</value> + </attribute> + </optional> + <optional> + <attribute name='bitmap'> + <text/> + </attribute> + </optional> + <optional> + <attribute name='size'> + <ref name="unsignedLong"/> + </attribute> + </optional> + </group> + </choice> + </element> + </define> + +</grammar> diff --git a/libvirt.spec.in b/libvirt.spec.in index b7a35a0fb1..6a730befdd 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -1800,6 +1800,7 @@ exit 0 %{_datadir}/libvirt/schemas/cputypes.rng %{_datadir}/libvirt/schemas/domain.rng %{_datadir}/libvirt/schemas/domaincaps.rng +%{_datadir}/libvirt/schemas/domaincheckpoint.rng %{_datadir}/libvirt/schemas/domaincommon.rng %{_datadir}/libvirt/schemas/domainsnapshot.rng %{_datadir}/libvirt/schemas/interface.rng diff --git a/mingw-libvirt.spec.in b/mingw-libvirt.spec.in index bea822fd73..553533c88f 100644 --- a/mingw-libvirt.spec.in +++ b/mingw-libvirt.spec.in @@ -240,6 +240,7 @@ rm -rf $RPM_BUILD_ROOT%{mingw64_libexecdir}/libvirt-guests.sh %{mingw32_datadir}/libvirt/schemas/cputypes.rng %{mingw32_datadir}/libvirt/schemas/domain.rng %{mingw32_datadir}/libvirt/schemas/domaincaps.rng +%{mingw32_datadir}/libvirt/schemas/domaincheckpoint.rng %{mingw32_datadir}/libvirt/schemas/domaincommon.rng %{mingw32_datadir}/libvirt/schemas/domainsnapshot.rng %{mingw32_datadir}/libvirt/schemas/interface.rng @@ -328,6 +329,7 @@ rm -rf $RPM_BUILD_ROOT%{mingw64_libexecdir}/libvirt-guests.sh %{mingw64_datadir}/libvirt/schemas/cputypes.rng %{mingw64_datadir}/libvirt/schemas/domain.rng %{mingw64_datadir}/libvirt/schemas/domaincaps.rng +%{mingw64_datadir}/libvirt/schemas/domaincheckpoint.rng %{mingw64_datadir}/libvirt/schemas/domaincommon.rng %{mingw64_datadir}/libvirt/schemas/domainsnapshot.rng %{mingw64_datadir}/libvirt/schemas/interface.rng diff --git a/tests/Makefile.am b/tests/Makefile.am index d3cdbff8bb..c552d08e12 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -83,6 +83,8 @@ EXTRA_DIST = \ commanddata \ cputestdata \ domaincapsschemadata \ + domaincheckpointxml2xmlin \ + domaincheckpointxml2xmlout \ domainconfdata \ domainschemadata \ domainsnapshotxml2xmlin \ diff --git a/tests/domaincheckpointxml2xmlin/empty.xml b/tests/domaincheckpointxml2xmlin/empty.xml new file mode 100644 index 0000000000..dc36449142 --- /dev/null +++ b/tests/domaincheckpointxml2xmlin/empty.xml @@ -0,0 +1 @@ +<domaincheckpoint/> diff --git a/tests/domaincheckpointxml2xmlin/sample.xml b/tests/domaincheckpointxml2xmlin/sample.xml new file mode 100644 index 0000000000..70ed964e1e --- /dev/null +++ b/tests/domaincheckpointxml2xmlin/sample.xml @@ -0,0 +1,7 @@ +<domaincheckpoint> + <description>Completion of updates after OS install</description> + <disks> + <disk name='vda' checkpoint='bitmap'/> + <disk name='vdb' checkpoint='no'/> + </disks> +</domaincheckpoint> diff --git a/tests/domaincheckpointxml2xmlout/empty.xml b/tests/domaincheckpointxml2xmlout/empty.xml new file mode 100644 index 0000000000..a26c7caab0 --- /dev/null +++ b/tests/domaincheckpointxml2xmlout/empty.xml @@ -0,0 +1,10 @@ +<domaincheckpoint> + <name>1525889631</name> + <creationTime>1525889631</creationTime> + <disks> + <disk name='vda' checkpoint='bitmap' bitmap='1525889631'/> + </disks> + <domain> + <uuid>9d37b878-a7cc-9f9a-b78f-49b3abad25a8</uuid> + </domain> +</domaincheckpoint> diff --git a/tests/domaincheckpointxml2xmlout/sample.xml b/tests/domaincheckpointxml2xmlout/sample.xml new file mode 100644 index 0000000000..559b29c8c1 --- /dev/null +++ b/tests/domaincheckpointxml2xmlout/sample.xml @@ -0,0 +1,16 @@ +<domaincheckpoint> + <name>1525889631</name> + <description>Completion of updates after OS install</description> + <creationTime>1525889631</creationTime> + <parent> + <name>1525111885</name> + </parent> + <disks> + <disk name='vda' checkpoint='bitmap' bitmap='1525889631'/> + <disk name='vdb' checkpoint='no'/> + </disks> + <domain type='qemu'> + <name>fedora</name> + <uuid>93a5c045-6457-2c09-e56c-927cdf34e178</uuid> + </domain> +</domaincheckpoint> diff --git a/tests/virschematest.c b/tests/virschematest.c index 56bdcb2f88..fa99e67a57 100644 --- a/tests/virschematest.c +++ b/tests/virschematest.c @@ -222,6 +222,8 @@ mymain(void) "genericxml2xmloutdata", "xlconfigdata", "libxlxml2domconfigdata", "qemuhotplugtestdomains"); DO_TEST_DIR("domaincaps.rng", "domaincapsschemadata"); + DO_TEST_DIR("domaincheckpoint.rng", "domaincheckpointxml2xmlin", + "domaincheckpointxml2xmlout"); DO_TEST_DIR("domainsnapshot.rng", "domainsnapshotxml2xmlin", "domainsnapshotxml2xmlout"); DO_TEST_DIR("interface.rng", "interfaceschemadata"); -- 2.20.1

On Tue, Mar 26, 2019 at 01:13:49AM -0500, Eric Blake wrote:
Prepare for new checkpoint APIs by describing the XML that will represent a checkpoint. The checkpoint XML is modeled heavily after virDomainSnapshotPtr. (See the docs for more details).
Add testsuite coverage for some minimal uses of the XML.
Signed-off-by: Eric Blake <eblake@redhat.com> --- docs/docs.html.in | 3 +- docs/format.html.in | 1 + docs/formatcheckpoint.html.in | 202 ++++++++++++++++++++ docs/index.html.in | 3 +- docs/schemas/domaincheckpoint.rng | 94 +++++++++ libvirt.spec.in | 1 + mingw-libvirt.spec.in | 2 + tests/Makefile.am | 2 + tests/domaincheckpointxml2xmlin/empty.xml | 1 + tests/domaincheckpointxml2xmlin/sample.xml | 7 + tests/domaincheckpointxml2xmlout/empty.xml | 10 + tests/domaincheckpointxml2xmlout/sample.xml | 16 ++ tests/virschematest.c | 2 + 13 files changed, 342 insertions(+), 2 deletions(-) create mode 100644 docs/formatcheckpoint.html.in create mode 100644 docs/schemas/domaincheckpoint.rng create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

Prepare for new backup APIs by describing the XML that will represent a backup. The XML resembles snapshots and checkpoints in being able to select actions for a set of disks, but has other differences. It can support both push model (the hypervisor does the backup directly into the destination file) and pull model (the hypervisor exposes an access port for a third party to grab what is necessary). Add testsuite coverage for some minimal uses of the XML. The <disk> element within <domainbackup> tries to model the same elements as a <disk> under <domain>, but sharing the RNG grammar proved to be hairy. That is in part because while <domain> use <source> to describe a host resource in use by the guest, a backup job is using a host resource that is not visible to the guest: a push backup action is instead describing a <target> (which ultimately could be a remote network resource, but for simplicity the RNG just validates a local file for now), and a pull backup action is instead describing a temporary local file <scratch> (which probably should not be a remote resource). A future refactoring may thus introduce some way to parameterize RNG to accept <disk type='FOO'>...</disk> so that the name of the subelement can be <source> for domain, or <target> or <scratch> as needed for backups. Future patches may improve this area of code. Signed-off-by: Eric Blake <eblake@redhat.com> --- docs/docs.html.in | 3 +- docs/format.html.in | 1 + docs/formatbackup.html.in | 179 +++++++++++++++ docs/index.html.in | 3 +- docs/schemas/domainbackup.rng | 219 +++++++++++++++++++ libvirt.spec.in | 1 + mingw-libvirt.spec.in | 2 + tests/Makefile.am | 2 + tests/domainbackupxml2xmlin/backup-pull.xml | 9 + tests/domainbackupxml2xmlin/backup-push.xml | 9 + tests/domainbackupxml2xmlin/empty.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 9 + tests/domainbackupxml2xmlout/backup-push.xml | 9 + tests/domainbackupxml2xmlout/empty.xml | 7 + tests/virschematest.c | 2 + 15 files changed, 454 insertions(+), 2 deletions(-) create mode 100644 docs/formatbackup.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml create mode 100644 tests/domainbackupxml2xmlin/empty.xml create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml create mode 100644 tests/domainbackupxml2xmlout/empty.xml diff --git a/docs/docs.html.in b/docs/docs.html.in index f6e599b77b..b6b03313fa 100644 --- a/docs/docs.html.in +++ b/docs/docs.html.in @@ -81,7 +81,8 @@ <a href="formatnode.html">node devices</a>, <a href="formatsecret.html">secrets</a>, <a href="formatsnapshot.html">snapshots</a>, - <a href="formatcheckpoint.html">checkpoints</a></dd> + <a href="formatcheckpoint.html">checkpoints</a>, + <a href="formatbackup.html">backup jobs</a></dd> <dt><a href="uri.html">URI format</a></dt> <dd>The URI formats used for connecting to libvirt</dd> diff --git a/docs/format.html.in b/docs/format.html.in index 20f9ef3348..e287b430bc 100644 --- a/docs/format.html.in +++ b/docs/format.html.in @@ -26,6 +26,7 @@ <li><a href="formatsecret.html">Secrets</a></li> <li><a href="formatsnapshot.html">Snapshots</a></li> <li><a href="formatcheckpoint.html">Checkpoints</a></li> + <li><a href="formatbackup.html">Backup jobs</a></li> </ul> <h2>Command line validation</h2> diff --git a/docs/formatbackup.html.in b/docs/formatbackup.html.in new file mode 100644 index 0000000000..b590bc6a12 --- /dev/null +++ b/docs/formatbackup.html.in @@ -0,0 +1,179 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE html> +<html xmlns="http://www.w3.org/1999/xhtml"> + <body> + <h1>Backup XML format</h1> + + <ul id="toc"></ul> + + <h2><a id="BackupAttributes">Backup XML</a></h2> + + <p> + Creating a backup, whether full or incremental, is done + via <code>virDomainBackupBegin()</code>, which takes an XML + description of the actions to perform, as well as an optional + second XML document <a href="formatcheckpoint.html">describing a + checkpoint</a> to create at the same point in time. + </p> + <p> + There are two general modes for backups: a push mode (where the + hypervisor writes out the data to the destination file, which + may be local or remote), and a pull mode (where the hypervisor + creates an NBD server that a third-party client can then read as + needed, and which requires the use of temporary storage, + typically local, until the backup is complete). + </p> + <p> + The instructions for beginning a backup job are provided as + attributes and elements of the + top-level <code>domainbackup</code> element. This element + includes an optional attribute <code>mode</code> which can be + either "push" or "pull" (default + push). <code>virDomainBackupGetXMLDesc()</code> can be used to + see the actual values selected for elements omitted during + creation (for example, learning which port the NBD server is + using in the pull model or what file names libvirt generated + when none were supplied). The following child elements are + supported: + </p> + <dl> + <dt><code>id</code></dt> + <dd>Ignored on input, this element is the job id of the backup + operation returned on success + from <code>virDomainBackupBegin()</code>; it can also be + obtained from <code>virDomainListJobIds()</code>, and is used + for selecting which backup operation to target + during <code>virDomainBackupGetXMLDesc()</code> + and <code>virDomainBackupEnd()</code>.</dd> + <dt><code>incremental</code></dt> + <dd>An optional element giving the name of an existing + checkpoint of the domain, which will be used to make this + backup an incremental one. In the push model, only changes + since the named checkpoint are written to the destination. In + the pull model, the NBD server uses the + NBD_OPT_SET_META_CONTEXT extension to advertise to the client + which portions of the export contain changes since the named + checkpoint. If omitted, a full backup is performed. + </dd> + <dt><code>server</code></dt> + <dd>Present only for a pull mode backup. Contains the same + attributes as + the <a href="formatdomain.html#elementsDisks"><code>protocol</code> + element of a disk</a> attached via NBD in the domain (such as + transport, socket, name, port, or tls), necessary to set up an + NBD server that exposes the content of each disk at the time + the backup is started. + </dd> + <dt><code>disks</code></dt> + <dd>An optional listing of instructions for disks participating + in the backup (if omitted, all disks participate and libvirt + attempts to generate filenames by appending the current + timestamp as a suffix). If the entire element was omitted on + input, then all disks participate in the backup, otherwise, + only the disks explicitly listed which do not also + use <code>backup='no'</code> will participate. On output, this + is the state of each of the domain's disk in relation to the + backup operation. + <dl> + <dt><code>disk</code></dt> + <dd>This sub-element describes the backup properties of a + specific disk, with the following attributes and child + elements: + <dl> + <dt><code>name</code></dt> + <dd>A mandatory attribute which must match either + the <code><target dev='name'/></code> or an + unambiguous <code><source file='name'/></code> + of one of + the <a href="formatdomain.html#elementsDisks">disk + devices</a> specified for the domain at the time of + the checkpoint.</dd> + <dt><code>backup</code></dt> + <dd>An optional attribute to describe the state of the + backup for the given disk. On input, the + value <code>no</code> skips a backup of the disk, and + any other value is ignored; on output, other values + such as <code>begin</code>, <code>inprogress</code>, + or <code>ready</code> track the backup progress that + libvirt has observed for that disk.</dd> + <dt><code>type</code></dt> + <dd>An optional attribute to describe the type of the + disk, except when <code>backup='no'</code> is + used. Valid values include <code>file</code> + or <code>block</code> for both push and pull model + backups; in the future, <code>network</code> may be + added for push model backups. Similar to a disk + declaration for a domain, the choice of type controls + what additional sub-elements are needed to describe + the destination (such as <code>protocol</code> for a + network destination).</dd> + <dt><code>target</code></dt> + <dd>Valid only for push mode backups, this is the + primary sub-element that describes the file name of + the backup destination, similar to + the <code>source</code> sub-element of a domain + disk. An optional sub-element <code>driver</code> can + also be used, with an attribute <code>type</code> to + specify a destination format different from + qcow2. Additionally, if a push backup is not + incremental, <code>target</code> may contain an + optional attribute <code>shallow="on"</code> so that + the destination file copies only the top-most source + file in a backing chain, rather than collapsing the + entire chain into the copy.</dd> + <dt><code>scratch</code></dt> + <dd>Valid only for pull mode backups, this is the + primary sub-element that describes the file name of + the local scratch file to be used in facilitating the + backup, and is similar to the <code>source</code> + sub-element of a domain disk.</dd> + </dl> + </dd> + </dl> + </dd> + </dl> + + <h2><a id="example">Examples</a></h2> + + <p>Use <code>virDomainBackupBegin()</code> to perform a full + backup using push mode. The example lets libvirt pick the + destination and format for 'vda', fully specifies that we want a + raw backup of 'vdb', and omits 'vdc' from the operation. + </p> + <pre> +<domainbackup> + <disks/> + <disk name='vda'/> + <disk name='vdb' type='file'> + <target file='/path/to/vdb.backup'/> + <driver type='raw'/> + </disk> + <disk name='vdc' backup='no'/> + </disks/> +</domainbackup> + </pre> + + <p>If the previous full backup also passed a parameter describing + <a href="formatcheckpoint.html">checkpoint XML</a> that resulted + in a checkpoint named <code>1525889631</code>, we can make + another call to <code>virDomainBackupBegin()</code> to perform + an incremental backup of just the data changed since that + checkpoint, this time using the following XML to start a pull + model export of the 'vda' and 'vdb' disks, where a third-party + NBD client connecting to '/path/to/server' completes the backup + (omitting 'vdc' from the explicit list has the same effect as + the backup='no' from the previous example): + </p> + <pre> +<domainbackup mode="pull"> + <incremental>1525889631</incremental> + <server transport="unix" socket="/path/to/server"/> + <disks/> + <disk name='vda' type='file'> + <scratch file='/path/to/file1.scratch'/> + </disk> + </disks/> +</domainbackup> + </pre> + </body> +</html> diff --git a/docs/index.html.in b/docs/index.html.in index 0315915402..167a58da58 100644 --- a/docs/index.html.in +++ b/docs/index.html.in @@ -70,7 +70,8 @@ <a href="formatnode.html">node devices</a>, <a href="formatsecret.html">secrets</a>, <a href="formatsnapshot.html">snapshots</a>, - <a href="formatcheckpoint.html">checkpoints</a></dd> + <a href="formatcheckpoint.html">checkpoints</a>, + <a href="formatbackup.html">backup jobs</a></dd> <dt><a href="http://wiki.libvirt.org">Wiki</a></dt> <dd>Read further community contributed content</dd> </dl> diff --git a/docs/schemas/domainbackup.rng b/docs/schemas/domainbackup.rng new file mode 100644 index 0000000000..92327e7077 --- /dev/null +++ b/docs/schemas/domainbackup.rng @@ -0,0 +1,219 @@ +<?xml version="1.0"?> +<!-- A Relax NG schema for the libvirt domain backup properties XML format --> +<grammar xmlns="http://relaxng.org/ns/structure/1.0"> + <start> + <ref name='domainbackup'/> + </start> + + <include href='domaincommon.rng'/> + + <define name='domainbackup'> + <element name='domainbackup'> + <optional> + <attribute name='id'> + <ref name="unsignedInt"/> + </attribute> + </optional> + <interleave> + <optional> + <element name='incremental'> + <text/> + </element> + </optional> + <choice> + <group> + <optional> + <attribute name='mode'> + <value>push</value> + </attribute> + </optional> + <ref name='backupDisksPush'/> + </group> + <group> + <attribute name='mode'> + <value>pull</value> + </attribute> + <interleave> + <element name='server'> + <choice> + <group> + <optional> + <attribute name='transport'> + <value>tcp</value> + </attribute> + </optional> + <attribute name='name'> + <choice> + <ref name='dnsName'/> + <ref name='ipAddr'/> + </choice> + </attribute> + <optional> + <attribute name='port'> + <ref name='unsignedInt'/> + </attribute> + </optional> + <!-- add tls? --> + </group> + <group> + <attribute name='transport'> + <value>unix</value> + </attribute> + <attribute name='socket'> + <ref name='absFilePath'/> + </attribute> + </group> + </choice> + </element> + <ref name='backupDisksPull'/> + </interleave> + </group> + </choice> + </interleave> + </element> + </define> + + <define name='backupPushDriver'> + <optional> + <element name='driver'> + <attribute name='type'> + <ref name='storageFormat'/> + </attribute> + </element> + </optional> + </define> + + <define name='backupAttr'> + <!-- valid values of <disk backup='XXX'> other than 'no' --> + <optional> + <attribute name='backup'> + <choice> + <value>begin</value> + <value>inprogress</value> + <value>ready</value> + </choice> + </attribute> + </optional> + </define> + + <define name='backupDisksPush'> + <optional> + <element name='disks'> + <oneOrMore> + <element name='disk'> + <attribute name='name'> + <choice> + <ref name='diskTarget'/> + <ref name='absFilePath'/> + </choice> + </attribute> + <optional> + <attribute name='shallow'> + <value>on</value> + </attribute> + </optional> + <choice> + <group> + <attribute name='backup'> + <value>no</value> + </attribute> + </group> + <!-- FIXME allow push to a network location, perhaps by + refactoring 'diskSource' to take element name by a + per-grammar ref --> + <group> + <ref name='backupAttr'/> + <optional> + <attribute name='type'> + <value>file</value> + </attribute> + </optional> + <interleave> + <optional> + <element name='target'> + <attribute name='file'> + <ref name='absFilePath'/> + </attribute> + </element> + </optional> + <ref name='backupPushDriver'/> + </interleave> + </group> + <group> + <ref name='backupAttr'/> + <attribute name='type'> + <value>block</value> + </attribute> + <interleave> + <optional> + <element name='target'> + <attribute name='dev'> + <ref name='absFilePath'/> + </attribute> + </element> + </optional> + <ref name='backupPushDriver'/> + </interleave> + </group> + <!-- + <group> + <ref name='backupAttr'/> + <attribute name='type'> + <value>network</value> + </attribute> + ... + </group> + --> + </choice> + </element> + </oneOrMore> + </element> + </optional> + </define> + + <define name='backupDisksPull'> + <optional> + <element name='disks'> + <oneOrMore> + <element name='disk'> + <attribute name='name'> + <choice> + <ref name='diskTarget'/> + <ref name='absFilePath'/> + </choice> + </attribute> + <choice> + <group> + <optional> + <attribute name='type'> + <value>file</value> + </attribute> + </optional> + <optional> + <element name='scratch'> + <attribute name='file'> + <ref name='absFilePath'/> + </attribute> + </element> + </optional> + </group> + <group> + <attribute name='type'> + <value>block</value> + </attribute> + <optional> + <element name='scratch'> + <attribute name='dev'> + <ref name='absFilePath'/> + </attribute> + </element> + </optional> + </group> + </choice> + </element> + </oneOrMore> + </element> + </optional> + </define> + +</grammar> diff --git a/libvirt.spec.in b/libvirt.spec.in index 6a730befdd..2235404075 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -1799,6 +1799,7 @@ exit 0 %{_datadir}/libvirt/schemas/capability.rng %{_datadir}/libvirt/schemas/cputypes.rng %{_datadir}/libvirt/schemas/domain.rng +%{_datadir}/libvirt/schemas/domainbackup.rng %{_datadir}/libvirt/schemas/domaincaps.rng %{_datadir}/libvirt/schemas/domaincheckpoint.rng %{_datadir}/libvirt/schemas/domaincommon.rng diff --git a/mingw-libvirt.spec.in b/mingw-libvirt.spec.in index 553533c88f..f407526e1a 100644 --- a/mingw-libvirt.spec.in +++ b/mingw-libvirt.spec.in @@ -239,6 +239,7 @@ rm -rf $RPM_BUILD_ROOT%{mingw64_libexecdir}/libvirt-guests.sh %{mingw32_datadir}/libvirt/schemas/capability.rng %{mingw32_datadir}/libvirt/schemas/cputypes.rng %{mingw32_datadir}/libvirt/schemas/domain.rng +%{mingw32_datadir}/libvirt/schemas/domainbackup.rng %{mingw32_datadir}/libvirt/schemas/domaincaps.rng %{mingw32_datadir}/libvirt/schemas/domaincheckpoint.rng %{mingw32_datadir}/libvirt/schemas/domaincommon.rng @@ -328,6 +329,7 @@ rm -rf $RPM_BUILD_ROOT%{mingw64_libexecdir}/libvirt-guests.sh %{mingw64_datadir}/libvirt/schemas/capability.rng %{mingw64_datadir}/libvirt/schemas/cputypes.rng %{mingw64_datadir}/libvirt/schemas/domain.rng +%{mingw64_datadir}/libvirt/schemas/domainbackup.rng %{mingw64_datadir}/libvirt/schemas/domaincaps.rng %{mingw64_datadir}/libvirt/schemas/domaincheckpoint.rng %{mingw64_datadir}/libvirt/schemas/domaincommon.rng diff --git a/tests/Makefile.am b/tests/Makefile.am index c552d08e12..666b3945c6 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -82,6 +82,8 @@ EXTRA_DIST = \ capabilityschemadata \ commanddata \ cputestdata \ + domainbackupxml2xmlin \ + domainbackupxml2xmlout \ domaincapsschemadata \ domaincheckpointxml2xmlin \ domaincheckpointxml2xmlout \ diff --git a/tests/domainbackupxml2xmlin/backup-pull.xml b/tests/domainbackupxml2xmlin/backup-pull.xml new file mode 100644 index 0000000000..2ce5cd6711 --- /dev/null +++ b/tests/domainbackupxml2xmlin/backup-pull.xml @@ -0,0 +1,9 @@ +<domainbackup mode="pull"> + <incremental>1525889631</incremental> + <server transport='tcp' name='localhost' port='10809'/> + <disks> + <disk name='vda' type='file'> + <scratch file='/path/to/file'/> + </disk> + </disks> +</domainbackup> diff --git a/tests/domainbackupxml2xmlin/backup-push.xml b/tests/domainbackupxml2xmlin/backup-push.xml new file mode 100644 index 0000000000..1b7d3061fd --- /dev/null +++ b/tests/domainbackupxml2xmlin/backup-push.xml @@ -0,0 +1,9 @@ +<domainbackup mode="push"> + <incremental>1525889631</incremental> + <disks> + <disk name='vda' type='file'> + <driver type='raw'/> + <target file='/path/to/file'/> + </disk> + </disks> +</domainbackup> diff --git a/tests/domainbackupxml2xmlin/empty.xml b/tests/domainbackupxml2xmlin/empty.xml new file mode 100644 index 0000000000..7ed511f97b --- /dev/null +++ b/tests/domainbackupxml2xmlin/empty.xml @@ -0,0 +1 @@ +<domainbackup/> diff --git a/tests/domainbackupxml2xmlout/backup-pull.xml b/tests/domainbackupxml2xmlout/backup-pull.xml new file mode 100644 index 0000000000..2ce5cd6711 --- /dev/null +++ b/tests/domainbackupxml2xmlout/backup-pull.xml @@ -0,0 +1,9 @@ +<domainbackup mode="pull"> + <incremental>1525889631</incremental> + <server transport='tcp' name='localhost' port='10809'/> + <disks> + <disk name='vda' type='file'> + <scratch file='/path/to/file'/> + </disk> + </disks> +</domainbackup> diff --git a/tests/domainbackupxml2xmlout/backup-push.xml b/tests/domainbackupxml2xmlout/backup-push.xml new file mode 100644 index 0000000000..1b7d3061fd --- /dev/null +++ b/tests/domainbackupxml2xmlout/backup-push.xml @@ -0,0 +1,9 @@ +<domainbackup mode="push"> + <incremental>1525889631</incremental> + <disks> + <disk name='vda' type='file'> + <driver type='raw'/> + <target file='/path/to/file'/> + </disk> + </disks> +</domainbackup> diff --git a/tests/domainbackupxml2xmlout/empty.xml b/tests/domainbackupxml2xmlout/empty.xml new file mode 100644 index 0000000000..13600fbb1c --- /dev/null +++ b/tests/domainbackupxml2xmlout/empty.xml @@ -0,0 +1,7 @@ +<domainbackup mode="push"> + <disks> + <disk name="vda" type="file"> + <target file="/path/to/file1.copy"/> + </disk> + </disks> +</domainbackup> diff --git a/tests/virschematest.c b/tests/virschematest.c index fa99e67a57..82690db8ab 100644 --- a/tests/virschematest.c +++ b/tests/virschematest.c @@ -221,6 +221,8 @@ mymain(void) "lxcxml2xmloutdata", "bhyvexml2argvdata", "genericxml2xmlindata", "genericxml2xmloutdata", "xlconfigdata", "libxlxml2domconfigdata", "qemuhotplugtestdomains"); + DO_TEST_DIR("domainbackup.rng", "domainbackupxml2xmlin", + "domainbackupxml2xmlout"); DO_TEST_DIR("domaincaps.rng", "domaincapsschemadata"); DO_TEST_DIR("domaincheckpoint.rng", "domaincheckpointxml2xmlin", "domaincheckpointxml2xmlout"); -- 2.20.1

On Tue, Mar 26, 2019 at 01:13:50AM -0500, Eric Blake wrote:
Prepare for new backup APIs by describing the XML that will represent a backup. The XML resembles snapshots and checkpoints in being able to select actions for a set of disks, but has other differences. It can support both push model (the hypervisor does the backup directly into the destination file) and pull model (the hypervisor exposes an access port for a third party to grab what is necessary). Add testsuite coverage for some minimal uses of the XML.
The <disk> element within <domainbackup> tries to model the same elements as a <disk> under <domain>, but sharing the RNG grammar proved to be hairy. That is in part because while <domain> use <source> to describe a host resource in use by the guest, a backup job is using a host resource that is not visible to the guest: a push backup action is instead describing a <target> (which ultimately could be a remote network resource, but for simplicity the RNG just validates a local file for now), and a pull backup action is instead describing a temporary local file <scratch> (which probably should not be a remote resource). A future refactoring may thus introduce some way to parameterize RNG to accept <disk type='FOO'>...</disk> so that the name of the subelement can be <source> for domain, or <target> or <scratch> as needed for backups. Future patches may improve this area of code.
Signed-off-by: Eric Blake <eblake@redhat.com> --- docs/docs.html.in | 3 +- docs/format.html.in | 1 + docs/formatbackup.html.in | 179 +++++++++++++++ docs/index.html.in | 3 +- docs/schemas/domainbackup.rng | 219 +++++++++++++++++++ libvirt.spec.in | 1 + mingw-libvirt.spec.in | 2 + tests/Makefile.am | 2 + tests/domainbackupxml2xmlin/backup-pull.xml | 9 + tests/domainbackupxml2xmlin/backup-push.xml | 9 + tests/domainbackupxml2xmlin/empty.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 9 + tests/domainbackupxml2xmlout/backup-push.xml | 9 + tests/domainbackupxml2xmlout/empty.xml | 7 + tests/virschematest.c | 2 + 15 files changed, 454 insertions(+), 2 deletions(-) create mode 100644 docs/formatbackup.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml create mode 100644 tests/domainbackupxml2xmlin/empty.xml create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml create mode 100644 tests/domainbackupxml2xmlout/empty.xml
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Tue, Mar 26, 2019 at 8:14 AM Eric Blake <eblake@redhat.com> wrote: ...
+ <value>pull</value> + </attribute> + <interleave> + <element name='server'> + <choice>
...
+ <group> + <attribute name='transport'> + <value>unix</value> + </attribute> + <attribute name='socket'> + <ref name='absFilePath'/> + </attribute> + </group> + </choice> + </element> + <ref name='backupDisksPull'/> + </interleave> + </group> + </choice>
We are trying to use backup_v5a from: https://repo.or.cz/libvirt/ericb.git And when we start a backup with this xml: <domainbackup mode="pull"> <server socket="/var/run/vdsm/nbd/4f594cec-1a27-43fd-acf8-3283cacd6347.sock" transport="unix" /> <disks> <disk name="sda" type="file" /> </disks> </domainbackup> Libvirt fail with: 'Operation not supported: <domainbackup> must specify TCP server for now' Which comes from: if (!def->server || def->server->transport != VIR_STORAGE_NET_HOST_TRANS_TCP) { virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", _("<domainbackup> must specify TCP server for now")); goto cleanup; } Unix socket support is very important for us, for several reasons: - always local, no way to configure it in the wrong way - having permissions on the socket - easier to manage using file system (persistence is free) - usually perforamnce better then tcp I have tests running our nbd client with qemu, starting a backup with unix socket using qmp, so there is no reason that libvirt would not support it. Can we expect this to be fixed soon? If we must use tcp for now, can we use port=0 to get a random port from the kernel, and find the port by getting the backup xml? I hope that we don't need to managed pull of ports for backups per host. Nir

On 3/27/19 5:16 AM, Nir Soffer wrote:
On Tue, Mar 26, 2019 at 8:14 AM Eric Blake <eblake@redhat.com> wrote:
We are trying to use backup_v5a from: https://repo.or.cz/libvirt/ericb.git And when we start a backup with this xml:
<domainbackup mode="pull"> <server socket="/var/run/vdsm/nbd/4f594cec-1a27-43fd-acf8-3283cacd6347.sock" transport="unix" /> <disks> <disk name="sda" type="file" /> </disks> </domainbackup>
Libvirt fail with:
'Operation not supported: <domainbackup> must specify TCP server for now'
Which comes from:
if (!def->server || def->server->transport != VIR_STORAGE_NET_HOST_TRANS_TCP) { virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", _("<domainbackup> must specify TCP server for now")); goto cleanup; }
Still broken that way in v7.
Unix socket support is very important for us, for several reasons: - always local, no way to configure it in the wrong way - having permissions on the socket - easier to manage using file system (persistence is free) - usually perforamnce better then tcp
I have tests running our nbd client with qemu, starting a backup with unix socket using qmp, so there is no reason that libvirt would not support it.
Other than the time it takes someone (me probably) to wire it in.
Can we expect this to be fixed soon?
Yes, particularly now that you've called it out. If my v7 makes it into 5.2, I'll also make sure Unix sockets work.
If we must use tcp for now, can we use port=0 to get a random port from the kernel, and find the port by getting the backup xml?
I hope that we don't need to managed pull of ports for backups per host.
Nir
-- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org

Introduce a bunch of new public APIs related to backup checkpoints. Checkpoints are modeled heavily after virDomainSnapshotPtr (both represent a point in time of the guest), although a snapshot exists with the intent of rolling back to that state, while a checkpoint exists to make it possible to create an incremental backup at a later time. The following map shows the API relations to snapshots, with new APIs on the right: Operate on a domain object to create/redefine a child: virDomainSnapshotCreateXML virDomainCheckpointCreateXML Operate on a child object for lifetime management: virDomainSnapshotDelete virDomainCheckpointDelete virDomainSnapshotFree virDomainCheckpointFree virDomainSnapshotRef virDomainCheckpointRef Operate on a child object to learn more about it: virDomainSnapshotGetXMLDesc virDomainCheckpointGetXMLDesc virDomainSnapshotGetConnect virDomainCheckpointGetConnect virDomainSnapshotGetDomain virDomainCheckpointGetDomain virDomainSnapshotGetName virDomainCheckpiontGetName virDomainSnapshotGetParent virDomainCheckpiontGetParent virDomainSnapshotHasMetadata virDomainCheckpointHasMetadata virDomainSnapshotIsCurrent virDomainCheckpointIsCurrent Operate on a domain object to list all children: virDomainSnapshotNum (no counterpart, this is the old virDomainSnapshotListNames racy interface) virDomainSnapshotListAllSnapshots virDomainListCheckpoints Operate on a child object to list descendents: virDomainSnapshotNumChildren (no counterpart, this is the old virDomainSnapshotListChildrenNames racy interface) virDomainSnapshotListAllChildren virDomainCheckpointListChildren Operate on a domain to locate a particular child: virDomainSnapshotLookupByName virDomainCheckpointLookupByName virDomainHasCurrentSnapshot virDomainHasCurrentSnapshot virDomainSnapshotCurrent virDomainCheckpointCurrent Operate on a snapshot to roll back to earlier state: virDomainSnapshotRevert (no counterpart, instead checkpoints are used in incremental backups via XML to virDomainBackupBegin) Signed-off-by: Eric Blake <eblake@redhat.com> --- include/libvirt/libvirt-domain-checkpoint.h | 161 +++++ include/libvirt/libvirt-domain.h | 6 + include/libvirt/libvirt.h | 5 +- src/conf/virdomainmomentobjlist.h | 5 +- src/driver-hypervisor.h | 60 +- docs/Makefile.am | 3 + docs/apibuild.py | 2 + docs/docs.html.in | 1 + libvirt.spec.in | 1 + mingw-libvirt.spec.in | 2 + po/POTFILES | 1 + src/Makefile.am | 2 + src/libvirt-domain-checkpoint.c | 749 ++++++++++++++++++++ src/libvirt-domain.c | 4 +- src/libvirt_public.syms | 16 + 15 files changed, 1010 insertions(+), 8 deletions(-) create mode 100644 include/libvirt/libvirt-domain-checkpoint.h create mode 100644 src/libvirt-domain-checkpoint.c diff --git a/include/libvirt/libvirt-domain-checkpoint.h b/include/libvirt/libvirt-domain-checkpoint.h new file mode 100644 index 0000000000..332df93755 --- /dev/null +++ b/include/libvirt/libvirt-domain-checkpoint.h @@ -0,0 +1,161 @@ +/* + * libvirt-domain-checkpoint.h + * Summary: APIs for management of domain checkpoints + * Description: Provides APIs for the management of domain checkpoints + * + * Copyright (C) 2006-2019 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + */ + +#ifndef LIBVIRT_DOMAIN_CHECKPOINT_H +# define LIBVIRT_DOMAIN_CHECKPOINT_H + +# ifndef __VIR_LIBVIRT_H_INCLUDES__ +# error "Don't include this file directly, only use libvirt/libvirt.h" +# endif + +/** + * virDomainCheckpoint: + * + * A virDomainCheckpoint is a private structure representing a checkpoint of + * a domain. A checkpoint is useful for tracking which portions of the + * domain disks have been altered since a point in time, but by itself does + * not allow reverting back to that point in time. + */ +typedef struct _virDomainCheckpoint virDomainCheckpoint; + +/** + * virDomainCheckpointPtr: + * + * A virDomainCheckpointPtr is pointer to a virDomainCheckpoint + * private structure, and is the type used to reference a domain + * checkpoint in the API. + */ +typedef virDomainCheckpoint *virDomainCheckpointPtr; + +const char *virDomainCheckpointGetName(virDomainCheckpointPtr checkpoint); +virDomainPtr virDomainCheckpointGetDomain(virDomainCheckpointPtr checkpoint); +virConnectPtr virDomainCheckpointGetConnect(virDomainCheckpointPtr checkpoint); + +typedef enum { + VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE = (1 << 0), /* Restore or alter + metadata */ + VIR_DOMAIN_CHECKPOINT_CREATE_CURRENT = (1 << 1), /* With redefine, make + checkpoint current */ + VIR_DOMAIN_CHECKPOINT_CREATE_NO_METADATA = (1 << 2), /* Make checkpoint without + remembering it */ + VIR_DOMAIN_CHECKPOINT_CREATE_QUIESCE = (1 << 3), /* use guest agent to + quiesce all mounted + file systems within + the domain */ +} virDomainCheckpointCreateFlags; + +/* Create a checkpoint using the current VM state. */ +virDomainCheckpointPtr virDomainCheckpointCreateXML(virDomainPtr domain, + const char *xmlDesc, + unsigned int flags); + +typedef enum { + VIR_DOMAIN_CHECKPOINT_XML_SECURE = (1 << 0), /* Include sensitive data */ + VIR_DOMAIN_CHECKPOINT_XML_NO_DOMAIN = (1 << 1), /* Suppress <domain> + subelement */ + VIR_DOMAIN_CHECKPOINT_XML_SIZE = (1 << 2), /* Include dynamic + per-<disk> size */ +} virDomainCheckpointXMLFlags; + +/* Dump the XML of a checkpoint */ +char *virDomainCheckpointGetXMLDesc(virDomainCheckpointPtr checkpoint, + unsigned int flags); + +/** + * virDomainCheckpointListFlags: + * + * Flags valid for virDomainListCheckpoints() and + * virDomainCheckpointListChildren(). Note that the interpretation of + * flag (1<<0) depends on which function it is passed to; but serves + * to toggle the per-call default of whether the listing is shallow or + * recursive. Remaining bits come in groups; if all bits from a group + * are 0, then that group is not used to filter results. */ +typedef enum { + VIR_DOMAIN_CHECKPOINT_LIST_ROOTS = (1 << 0), /* Filter by checkpoints + with no parents, when + listing a domain */ + VIR_DOMAIN_CHECKPOINT_LIST_DESCENDANTS = (1 << 0), /* List all descendants, + not just children, when + listing a checkpoint */ + VIR_DOMAIN_CHECKPOINT_LIST_TOPOLOGICAL = (1 << 1), /* Ensure parents occur + before children in + the resulting list */ + + VIR_DOMAIN_CHECKPOINT_LIST_LEAVES = (1 << 2), /* Filter by checkpoints + with no children */ + VIR_DOMAIN_CHECKPOINT_LIST_NO_LEAVES = (1 << 3), /* Filter by checkpoints + that have children */ + + VIR_DOMAIN_CHECKPOINT_LIST_METADATA = (1 << 4), /* Filter by checkpoints + which have metadata */ + VIR_DOMAIN_CHECKPOINT_LIST_NO_METADATA = (1 << 5), /* Filter by checkpoints + with no metadata */ +} virDomainCheckpointListFlags; + +/* Get all checkpoint objects for this domain */ +int virDomainListCheckpoints(virDomainPtr domain, + virDomainCheckpointPtr **checkpoints, + unsigned int flags); + +/* Get all checkpoint object children for this checkpoint */ +int virDomainCheckpointListChildren(virDomainCheckpointPtr checkpoint, + virDomainCheckpointPtr **children, + unsigned int flags); + +/* Get a handle to a named checkpoint */ +virDomainCheckpointPtr virDomainCheckpointLookupByName(virDomainPtr domain, + const char *name, + unsigned int flags); + +/* Check whether a domain has a checkpoint which is currently used */ +int virDomainHasCurrentCheckpoint(virDomainPtr domain, unsigned int flags); + +/* Get a handle to the current checkpoint */ +virDomainCheckpointPtr virDomainCheckpointCurrent(virDomainPtr domain, + unsigned int flags); + +/* Get a handle to the parent checkpoint, if one exists */ +virDomainCheckpointPtr virDomainCheckpointGetParent(virDomainCheckpointPtr checkpoint, + unsigned int flags); + +/* Determine if a checkpoint is the current checkpoint of its domain. */ +int virDomainCheckpointIsCurrent(virDomainCheckpointPtr checkpoint, + unsigned int flags); + +/* Determine if checkpoint has metadata that would prevent domain deletion. */ +int virDomainCheckpointHasMetadata(virDomainCheckpointPtr checkpoint, + unsigned int flags); + +/* Delete a checkpoint */ +typedef enum { + VIR_DOMAIN_CHECKPOINT_DELETE_CHILDREN = (1 << 0), /* Also delete children */ + VIR_DOMAIN_CHECKPOINT_DELETE_METADATA_ONLY = (1 << 1), /* Delete just metadata */ + VIR_DOMAIN_CHECKPOINT_DELETE_CHILDREN_ONLY = (1 << 2), /* Delete just children */ +} virDomainCheckpointDeleteFlags; + +int virDomainCheckpointDelete(virDomainCheckpointPtr checkpoint, + unsigned int flags); + +int virDomainCheckpointRef(virDomainCheckpointPtr checkpoint); +int virDomainCheckpointFree(virDomainCheckpointPtr checkpoint); + +#endif /* LIBVIRT_DOMAIN_CHECKPOINT_H */ diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 1d5bdb545d..94504e70a6 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1774,6 +1774,9 @@ typedef enum { VIR_DOMAIN_UNDEFINE_NVRAM = (1 << 2), /* Also remove any nvram file */ VIR_DOMAIN_UNDEFINE_KEEP_NVRAM = (1 << 3), /* Keep nvram file */ + VIR_DOMAIN_UNDEFINE_CHECKPOINTS_METADATA= (1 << 4),/* If last use of domain, + then also remove any + checkpoint metadata */ /* Future undefine control flags should come here. */ } virDomainUndefineFlagsValues; @@ -1812,6 +1815,9 @@ typedef enum { VIR_CONNECT_LIST_DOMAINS_HAS_SNAPSHOT = 1 << 12, VIR_CONNECT_LIST_DOMAINS_NO_SNAPSHOT = 1 << 13, + + VIR_CONNECT_LIST_DOMAINS_HAS_CHECKPOINT = 1 << 14, + VIR_CONNECT_LIST_DOMAINS_NO_CHECKPOINT = 1 << 15, } virConnectListAllDomainsFlags; int virConnectListAllDomains (virConnectPtr conn, diff --git a/include/libvirt/libvirt.h b/include/libvirt/libvirt.h index 13de151cb6..f93e3bfa85 100644 --- a/include/libvirt/libvirt.h +++ b/include/libvirt/libvirt.h @@ -34,10 +34,7 @@ extern "C" { # include <libvirt/libvirt-common.h> # include <libvirt/libvirt-host.h> # include <libvirt/libvirt-domain.h> -/* FIXME: Temporary hack until later patch creates new - * libvirt-domain-checkpoint.h file */ -typedef struct _virDomainCheckpoint virDomainCheckpoint; -typedef virDomainCheckpoint *virDomainCheckpointPtr; +# include <libvirt/libvirt-domain-checkpoint.h> # include <libvirt/libvirt-domain-snapshot.h> # include <libvirt/libvirt-event.h> # include <libvirt/libvirt-interface.h> diff --git a/src/conf/virdomainmomentobjlist.h b/src/conf/virdomainmomentobjlist.h index b67af24bba..a113a9f872 100644 --- a/src/conf/virdomainmomentobjlist.h +++ b/src/conf/virdomainmomentobjlist.h @@ -71,8 +71,9 @@ virDomainMomentObjPtr virDomainMomentAssignDef(virDomainMomentObjListPtr moments virDomainMomentDefPtr def); /* Various enum bits that map to public API filters. Note that the - * values of the internal bits are not necessarily the same as the - * public ones. */ + * values of the internal bits are not the same as the public ones for + * snapshot, however, this list should be kept in sync with the public + * ones for checkpoint. */ typedef enum { VIR_DOMAIN_MOMENT_LIST_ROOTS = (1 << 0), VIR_DOMAIN_MOMENT_LIST_DESCENDANTS = (1 << 0), diff --git a/src/driver-hypervisor.h b/src/driver-hypervisor.h index 5315e33dde..e6efce1bfe 100644 --- a/src/driver-hypervisor.h +++ b/src/driver-hypervisor.h @@ -1,7 +1,7 @@ /* * driver-hypervisor.h: entry points for hypervisor drivers * - * Copyright (C) 2006-2015 Red Hat, Inc. + * Copyright (C) 2006-2019 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -1328,6 +1328,53 @@ typedef int int *nparams, unsigned int flags); +typedef virDomainCheckpointPtr +(*virDrvDomainCheckpointCreateXML)(virDomainPtr domain, + const char *xmlDesc, + unsigned int flags); + +typedef char * +(*virDrvDomainCheckpointGetXMLDesc)(virDomainCheckpointPtr checkpoint, + unsigned int flags); + +typedef int +(*virDrvDomainListCheckpoints)(virDomainPtr domain, + virDomainCheckpointPtr **checkpoints, + unsigned int flags); + +typedef int +(*virDrvDomainCheckpointListChildren)(virDomainCheckpointPtr checkpoint, + virDomainCheckpointPtr **children, + unsigned int flags); + +typedef virDomainCheckpointPtr +(*virDrvDomainCheckpointLookupByName)(virDomainPtr domain, + const char *name, + unsigned int flags); + +typedef int +(*virDrvDomainHasCurrentCheckpoint)(virDomainPtr domain, + unsigned int flags); + +typedef virDomainCheckpointPtr +(*virDrvDomainCheckpointGetParent)(virDomainCheckpointPtr checkpoint, + unsigned int flags); + +typedef virDomainCheckpointPtr +(*virDrvDomainCheckpointCurrent)(virDomainPtr domain, + unsigned int flags); + +typedef int +(*virDrvDomainCheckpointIsCurrent)(virDomainCheckpointPtr checkpoint, + unsigned int flags); + +typedef int +(*virDrvDomainCheckpointHasMetadata)(virDomainCheckpointPtr checkpoint, + unsigned int flags); + +typedef int +(*virDrvDomainCheckpointDelete)(virDomainCheckpointPtr checkpoint, + unsigned int flags); typedef struct _virHypervisorDriver virHypervisorDriver; typedef virHypervisorDriver *virHypervisorDriverPtr; @@ -1580,6 +1627,17 @@ struct _virHypervisorDriver { virDrvConnectBaselineHypervisorCPU connectBaselineHypervisorCPU; virDrvNodeGetSEVInfo nodeGetSEVInfo; virDrvDomainGetLaunchSecurityInfo domainGetLaunchSecurityInfo; + virDrvDomainCheckpointCreateXML domainCheckpointCreateXML; + virDrvDomainCheckpointGetXMLDesc domainCheckpointGetXMLDesc; + virDrvDomainListCheckpoints domainListCheckpoints; + virDrvDomainCheckpointListChildren domainCheckpointListChildren; + virDrvDomainCheckpointLookupByName domainCheckpointLookupByName; + virDrvDomainHasCurrentCheckpoint domainHasCurrentCheckpoint; + virDrvDomainCheckpointGetParent domainCheckpointGetParent; + virDrvDomainCheckpointCurrent domainCheckpointCurrent; + virDrvDomainCheckpointIsCurrent domainCheckpointIsCurrent; + virDrvDomainCheckpointHasMetadata domainCheckpointHasMetadata; + virDrvDomainCheckpointDelete domainCheckpointDelete; }; diff --git a/docs/Makefile.am b/docs/Makefile.am index bd7bc1a431..88c1e83275 100644 --- a/docs/Makefile.am +++ b/docs/Makefile.am @@ -25,6 +25,7 @@ apihtml = \ apihtml_generated = \ html/libvirt-libvirt-common.html \ html/libvirt-libvirt-domain.html \ + html/libvirt-libvirt-domain-checkpoint.html \ html/libvirt-libvirt-domain-snapshot.html \ html/libvirt-libvirt-event.html \ html/libvirt-libvirt-host.html \ @@ -316,6 +317,7 @@ $(python_generated_files): $(APIBUILD_STAMP) $(APIBUILD_STAMP): $(srcdir)/apibuild.py \ $(top_srcdir)/include/libvirt/libvirt.h \ $(top_srcdir)/include/libvirt/libvirt-common.h.in \ + $(top_srcdir)/include/libvirt/libvirt-domain-checkpoint.h \ $(top_srcdir)/include/libvirt/libvirt-domain-snapshot.h \ $(top_srcdir)/include/libvirt/libvirt-domain.h \ $(top_srcdir)/include/libvirt/libvirt-event.h \ @@ -332,6 +334,7 @@ $(APIBUILD_STAMP): $(srcdir)/apibuild.py \ $(top_srcdir)/include/libvirt/libvirt-admin.h \ $(top_srcdir)/include/libvirt/virterror.h \ $(top_srcdir)/src/libvirt.c \ + $(top_srcdir)/src/libvirt-domain-checkpoint.c \ $(top_srcdir)/src/libvirt-domain-snapshot.c \ $(top_srcdir)/src/libvirt-domain.c \ $(top_srcdir)/src/libvirt-host.c \ diff --git a/docs/apibuild.py b/docs/apibuild.py index 9e04871220..dbdc1c95af 100755 --- a/docs/apibuild.py +++ b/docs/apibuild.py @@ -26,6 +26,7 @@ debugsym = None included_files = { "libvirt-common.h": "header with general libvirt API definitions", "libvirt-domain.h": "header with general libvirt API definitions", + "libvirt-domain-checkpoint.h": "header with general libvirt API definitions", "libvirt-domain-snapshot.h": "header with general libvirt API definitions", "libvirt-event.h": "header with general libvirt API definitions", "libvirt-host.h": "header with general libvirt API definitions", @@ -39,6 +40,7 @@ included_files = { "virterror.h": "header with error specific API definitions", "libvirt.c": "Main interfaces for the libvirt library", "libvirt-domain.c": "Domain interfaces for the libvirt library", + "libvirt-domain-checkpoint.c": "Domain checkpoint interfaces for the libvirt library", "libvirt-domain-snapshot.c": "Domain snapshot interfaces for the libvirt library", "libvirt-host.c": "Host interfaces for the libvirt library", "libvirt-interface.c": "Interface interfaces for the libvirt library", diff --git a/docs/docs.html.in b/docs/docs.html.in index b6b03313fa..400b149791 100644 --- a/docs/docs.html.in +++ b/docs/docs.html.in @@ -99,6 +99,7 @@ <dd>Reference manual for the C public API, split in <a href="html/libvirt-libvirt-common.html">common</a>, <a href="html/libvirt-libvirt-domain.html">domain</a>, + <a href="html/libvirt-libvirt-domain-checkpoint.html">domain checkpoint</a>, <a href="html/libvirt-libvirt-domain-snapshot.html">domain snapshot</a>, <a href="html/libvirt-virterror.html">error</a>, <a href="html/libvirt-libvirt-event.html">event</a>, diff --git a/libvirt.spec.in b/libvirt.spec.in index 2235404075..c4de6bc2ac 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -1860,6 +1860,7 @@ exit 0 %{_includedir}/libvirt/libvirt-admin.h %{_includedir}/libvirt/libvirt-common.h %{_includedir}/libvirt/libvirt-domain.h +%{_includedir}/libvirt/libvirt-domain-checkpoint.h %{_includedir}/libvirt/libvirt-domain-snapshot.h %{_includedir}/libvirt/libvirt-event.h %{_includedir}/libvirt/libvirt-host.h diff --git a/mingw-libvirt.spec.in b/mingw-libvirt.spec.in index f407526e1a..924725a5b6 100644 --- a/mingw-libvirt.spec.in +++ b/mingw-libvirt.spec.in @@ -272,6 +272,7 @@ rm -rf $RPM_BUILD_ROOT%{mingw64_libexecdir}/libvirt-guests.sh %{mingw32_includedir}/libvirt/libvirt.h %{mingw32_includedir}/libvirt/libvirt-common.h %{mingw32_includedir}/libvirt/libvirt-domain.h +%{mingw32_includedir}/libvirt/libvirt-domain-checkpoint.h %{mingw32_includedir}/libvirt/libvirt-domain-snapshot.h %{mingw32_includedir}/libvirt/libvirt-event.h %{mingw32_includedir}/libvirt/libvirt-host.h @@ -362,6 +363,7 @@ rm -rf $RPM_BUILD_ROOT%{mingw64_libexecdir}/libvirt-guests.sh %{mingw64_includedir}/libvirt/libvirt.h %{mingw64_includedir}/libvirt/libvirt-common.h %{mingw64_includedir}/libvirt/libvirt-domain.h +%{mingw64_includedir}/libvirt/libvirt-domain-checkpoint.h %{mingw64_includedir}/libvirt/libvirt-domain-snapshot.h %{mingw64_includedir}/libvirt/libvirt-event.h %{mingw64_includedir}/libvirt/libvirt-host.h diff --git a/po/POTFILES b/po/POTFILES index 9dd4ee7d99..88af551664 100644 --- a/po/POTFILES +++ b/po/POTFILES @@ -69,6 +69,7 @@ src/interface/interface_backend_netcf.c src/interface/interface_backend_udev.c src/internal.h src/libvirt-admin.c +src/libvirt-domain-checkpoint.c src/libvirt-domain-snapshot.c src/libvirt-domain.c src/libvirt-host.c diff --git a/src/Makefile.am b/src/Makefile.am index a73f43c483..8a08213880 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -167,6 +167,7 @@ DRIVER_SOURCES += \ $(DATATYPES_SOURCES) \ libvirt.c libvirt_internal.h \ libvirt-domain.c \ + libvirt-domain-checkpoint.c \ libvirt-domain-snapshot.c \ libvirt-host.c \ libvirt-interface.c \ @@ -718,6 +719,7 @@ libvirt_setuid_rpc_client_la_SOURCES = \ datatypes.c \ libvirt.c \ libvirt-domain.c \ + libvirt-domain-checkpoint.c \ libvirt-domain-snapshot.c \ libvirt-host.c \ libvirt-interface.c \ diff --git a/src/libvirt-domain-checkpoint.c b/src/libvirt-domain-checkpoint.c new file mode 100644 index 0000000000..43701ff91d --- /dev/null +++ b/src/libvirt-domain-checkpoint.c @@ -0,0 +1,749 @@ +/* + * libvirt-domain-checkpoint.c: entry points for virDomainCheckpointPtr APIs + * + * Copyright (C) 2006-2019 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + */ + +#include <config.h> + +#include "datatypes.h" +#include "virlog.h" + +VIR_LOG_INIT("libvirt.domain-checkpoint"); + +#define VIR_FROM_THIS VIR_FROM_DOMAIN_CHECKPOINT + +/** + * virDomainCheckpointGetName: + * @checkpoint: a checkpoint object + * + * Get the public name for that checkpoint + * + * Returns a pointer to the name or NULL, the string need not be deallocated + * as its lifetime will be the same as the checkpoint object. + */ +const char * +virDomainCheckpointGetName(virDomainCheckpointPtr checkpoint) +{ + VIR_DEBUG("checkpoint=%p", checkpoint); + + virResetLastError(); + + virCheckDomainCheckpointReturn(checkpoint, NULL); + + return checkpoint->name; +} + + +/** + * virDomainCheckpointGetDomain: + * @checkpoint: a checkpoint object + * + * Provides the domain pointer associated with a checkpoint. The + * reference counter on the domain is not increased by this + * call. + * + * Returns the domain or NULL. + */ +virDomainPtr +virDomainCheckpointGetDomain(virDomainCheckpointPtr checkpoint) +{ + VIR_DEBUG("checkpoint=%p", checkpoint); + + virResetLastError(); + + virCheckDomainCheckpointReturn(checkpoint, NULL); + + return checkpoint->domain; +} + + +/** + * virDomainCheckpointGetConnect: + * @checkpoint: a checkpoint object + * + * Provides the connection pointer associated with a checkpoint. The + * reference counter on the connection is not increased by this + * call. + * + * Returns the connection or NULL. + */ +virConnectPtr +virDomainCheckpointGetConnect(virDomainCheckpointPtr checkpoint) +{ + VIR_DEBUG("checkpoint=%p", checkpoint); + + virResetLastError(); + + virCheckDomainCheckpointReturn(checkpoint, NULL); + + return checkpoint->domain->conn; +} + + +/** + * virDomainCheckpointCreateXML: + * @domain: a domain object + * @xmlDesc: description of the checkpoint to create + * @flags: bitwise-OR of supported virDomainCheckpointCreateFlags + * + * Create a new checkpoint using @xmlDesc, with a top-level + * <domaincheckpoint> element, on a running @domain. Typically, it is + * more common to create a new checkpoint as part of kicking off a + * backup job with virDomainBackupBegin() or when creating a snapshot + * with virDomainSnapshotCreateXML2(); however, it is also possible to + * start a checkpoint without a backup. + * + * See <a href=formatcheckpoint.html#CheckpointAttributes">Checkpoint XML</a> + * for more details on @xmlDesc. In particular, some hypervisors may require + * particular disk formats, such as qcow2, in order to support this + * command; where @xmlDesc can be used to limit the checkpoint to a working + * subset of the domain's disks. + * + * If @flags includes VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE, then this + * is a request to reinstate checkpoint metadata that was previously + * captured from virDomainCheckpointGetXMLDesc() before removing that + * metadata, rather than creating a new checkpoint. Note that while + * original creation can omit a number of elements from @xmlDesc (and + * libvirt will supply sane defaults based on the domain state at that + * point in time), a redefinition must supply more elements (as the + * domain may have changed in the meantime, so that libvirt no longer + * has a way to resupply correct defaults). When redefining + * checkpoint metadata, the domain's current checkpoint will not be + * altered unless the VIR_DOMAIN_CHECKPOINT_CREATE_CURRENT flag is + * also present. It is an error to request the + * VIR_DOMAIN_CHECKPOINT_CREATE_CURRENT flag without + * VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE. Not all hypervisors support + * these flags. + * + * If @flags includes VIR_DOMAIN_CHECKPOINT_CREATE_NO_METADATA, then + * the domain's disk images are modified according to @xmlDesc, but + * libvirt does not track any metadata (similar to immediately calling + * virDomainCheckpointDelete() with + * VIR_DOMAIN_CHECKPOINT_DELETE_METADATA_ONLY). This flag is + * incompatible with VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE. + * + * If @flags includes VIR_DOMAIN_CHECKPOINT_CREATE_QUIESCE, then the + * libvirt will attempt to use guest agent to freeze and thaw all file + * systems in use within domain OS. However, if the guest agent is not + * present, an error is thrown. This flag is incompatible with + * VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE. + * + * Returns an (opaque) new virDomainCheckpointPtr on success or NULL + * on failure. + */ +virDomainCheckpointPtr +virDomainCheckpointCreateXML(virDomainPtr domain, + const char *xmlDesc, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "xmlDesc=%s, flags=0x%x", xmlDesc, flags); + + virResetLastError(); + + virCheckDomainReturn(domain, NULL); + conn = domain->conn; + + virCheckNonNullArgGoto(xmlDesc, error); + virCheckReadOnlyGoto(conn->flags, error); + + VIR_REQUIRE_FLAG_GOTO(VIR_DOMAIN_CHECKPOINT_CREATE_CURRENT, + VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE, + error); + + VIR_EXCLUSIVE_FLAGS_GOTO(VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE, + VIR_DOMAIN_CHECKPOINT_CREATE_NO_METADATA, + error); + VIR_EXCLUSIVE_FLAGS_GOTO(VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE, + VIR_DOMAIN_CHECKPOINT_CREATE_QUIESCE, + error); + + if (conn->driver->domainCheckpointCreateXML) { + virDomainCheckpointPtr ret; + ret = conn->driver->domainCheckpointCreateXML(domain, xmlDesc, flags); + if (!ret) + goto error; + return ret; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return NULL; +} + + +/** + * virDomainCheckpointGetXMLDesc: + * @checkpoint: a domain checkpoint object + * @flags: bitwise-OR of supported virDomainCheckpointXMLFlags + * + * Provide an XML description of the domain checkpoint. + * + * No security-sensitive data will be included unless @flags contains + * VIR_DOMAIN_CHECKPOINT_XML_SECURE; this flag is rejected on read-only + * connections. + * + * Normally, the XML description includes an element giving a full + * description of the domain at the time the snapshot was created; to + * reduce parsing time, it will be suppressed when @flags contains + * VIR_DOMAIN_CHECKPOINT_XML_NO_DOMAIN. + * + * By default, the XML description contains only static information that + * does not change over time. However, when @flags contains + * VIR_DOMAIN_CHECKPOINT_XML_SIZE, each <disk> listing adds an additional + * attribute that shows an estimate of the current size in bytes that + * have been dirtied between the time the checkpoint was created and the + * current point in time. + * + * Returns a 0 terminated UTF-8 encoded XML instance or NULL in case + * of error. The caller must free() the returned value. + */ +char * +virDomainCheckpointGetXMLDesc(virDomainCheckpointPtr checkpoint, + unsigned int flags) +{ + virConnectPtr conn; + VIR_DEBUG("checkpoint=%p, flags=0x%x", checkpoint, flags); + + virResetLastError(); + + virCheckDomainCheckpointReturn(checkpoint, NULL); + conn = checkpoint->domain->conn; + + if ((conn->flags & VIR_CONNECT_RO) && + (flags & VIR_DOMAIN_CHECKPOINT_XML_SECURE)) { + virReportError(VIR_ERR_OPERATION_DENIED, "%s", + _("virDomainCheckpointGetXMLDesc with secure flag")); + goto error; + } + + if (conn->driver->domainCheckpointGetXMLDesc) { + char *ret; + ret = conn->driver->domainCheckpointGetXMLDesc(checkpoint, flags); + if (!ret) + goto error; + return ret; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return NULL; +} + + +/** + * virDomainListCheckpoints: + * @domain: a domain object + * @checkpoints: pointer to variable to store the array containing checkpoint + * object, or NULL if the list is not required (just returns + * number of checkpoints) + * @flags: bitwise-OR of supported virDomainCheckpoinListFlags + * + * Collect the list of domain checkpoints for the given domain and allocate + * an array to store those objects. + * + * If @flags contains VIR_DOMAIN_CHECKPOINT_LIST_TOPOLOGICAL, + * @checkpoints is non-NULL, and no other connection is modifying + * checkpoints, then it is guaranteed that for any checkpoint in the + * resulting list, no checkpoints later in the list can be reached by + * a sequence of virDomainCheckpointGetParent() starting from that + * earlier checkpoint; otherwise, the order of checkpoints in the + * resulting list is unspecified. + * + * By default, this command covers all checkpoints. It is also + * possible to limit things to just checkpoints with no parents, when + * @flags includes VIR_DOMAIN_CHECKPOINT_LIST_ROOTS. Additional + * filters are provided in groups listed below. Within a group, bits + * are mutually exclusive, where all possible checkpoints are + * described by exactly one bit from the group. Some hypervisors might + * reject particular flags where it cannot make a distinction for + * filtering. If the set of filter flags selected forms an impossible + * combination, the hypervisor may return either 0 or an error. + * + * The first group of @flags is VIR_DOMAIN_CHECKPOINT_LIST_LEAVES and + * VIR_DOMAIN_CHECKPOINT_LIST_NO_LEAVES, to filter based on checkpoints that + * have no further children (a leaf checkpoint). + * + * The next group of @flags is VIR_DOMAIN_CHECKPOINT_LIST_METADATA and + * VIR_DOMAIN_CHECKPOINT_LIST_NO_METADATA, for filtering checkpoints based on + * whether they have metadata that would prevent the removal of the last + * reference to a domain. + * + * Returns the number of domain checkpoints found or -1 and sets @checkpoints + * to NULL in case of error. On success, the array stored into @checkpoints + * is guaranteed to have an extra allocated element set to NULL but not + * included in the return count, to make iteration easier. The caller is + * responsible for calling virDomainCheckpointFree() on each array element, + * then calling free() on @checkpoints. + */ +int +virDomainListCheckpoints(virDomainPtr domain, + virDomainCheckpointPtr **checkpoints, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "checkpoints=%p, flags=0x%x", checkpoints, flags); + + virResetLastError(); + + if (checkpoints) + *checkpoints = NULL; + + virCheckDomainReturn(domain, -1); + conn = domain->conn; + + if (conn->driver->domainListCheckpoints) { + int ret = conn->driver->domainListCheckpoints(domain, checkpoints, + flags); + if (ret < 0) + goto error; + return ret; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return -1; +} + + +/** + * virDomainCheckpointListChildren: + * @checkpoint: a domain checkpoint object + * @children: pointer to variable to store the array containing checkpoint + * objects or NULL if the list is not required (just returns + * number of checkpoints) + * @flags: bitwise-OR of supported virDomainCheckpointListFlags + * + * Collect the list of domain checkpoints that are children of the given + * checkpoint, and allocate an array to store those objects. + * + * If @flags contains VIR_DOMAIN_CHECKPOINT_LIST_TOPOLOGICAL, + * @checkpoints is non-NULL, and no other connection is modifying + * checkpoints, then it is guaranteed that for any checkpoint in the + * resulting list, no checkpoints later in the list can be reached by + * a sequence of virDomainCheckpointGetParent() starting from that + * earlier checkpoint; otherwise, the order of checkpoints in the + * resulting list is unspecified. + * + * By default, this command covers only direct children. It is also + * possible to expand things to cover all descendants, when @flags + * includes VIR_DOMAIN_CHECKPOINT_LIST_DESCENDANTS. Additional + * are provided via the remaining @flags values as documented in + * virDomainListCheckpoints(), with the exception that + * VIR_DOMAIN_CHECKPOINT_LIST_ROOTS is not supported (in fact, + * VIR_DOMAIN_CHECKPOINT_LIST_DESCENDANTS has the same bit value but + * opposite semantics of widening rather than narrowing the listing). + * + * Returns the number of domain checkpoints found or -1 and sets @children to + * NULL in case of error. On success, the array stored into @children is + * guaranteed to have an extra allocated element set to NULL but not included + * in the return count, to make iteration easier. The caller is responsible + * for calling virDomainCheckpointFree() on each array element, then calling + * free() on @children. + */ +int +virDomainCheckpointListChildren(virDomainCheckpointPtr checkpoint, + virDomainCheckpointPtr **children, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DEBUG("checkpoint=%p, children=%p, flags=0x%x", + checkpoint, children, flags); + + virResetLastError(); + + if (children) + *children = NULL; + + virCheckDomainCheckpointReturn(checkpoint, -1); + conn = checkpoint->domain->conn; + + if (conn->driver->domainCheckpointListChildren) { + int ret = conn->driver->domainCheckpointListChildren(checkpoint, + children, flags); + if (ret < 0) + goto error; + return ret; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return -1; +} + + +/** + * virDomainCheckpointLookupByName: + * @domain: a domain object + * @name: name for the domain checkpoint + * @flags: extra flags; not used yet, so callers should always pass 0 + * + * Try to lookup a domain checkpoint based on its name. + * + * Returns a domain checkpoint object or NULL in case of failure. If the + * domain checkpoint cannot be found, then the VIR_ERR_NO_DOMAIN_CHECKPOINT + * error is raised. + */ +virDomainCheckpointPtr +virDomainCheckpointLookupByName(virDomainPtr domain, + const char *name, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "name=%s, flags=0x%x", name, flags); + + virResetLastError(); + + virCheckDomainReturn(domain, NULL); + conn = domain->conn; + + virCheckNonNullArgGoto(name, error); + + if (conn->driver->domainCheckpointLookupByName) { + virDomainCheckpointPtr dom; + dom = conn->driver->domainCheckpointLookupByName(domain, name, flags); + if (!dom) + goto error; + return dom; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return NULL; +} + + +/** + * virDomainHasCurrentCheckpoint: + * @domain: pointer to the domain object + * @flags: extra flags; not used yet, so callers should always pass 0 + * + * Determine if the domain has a current checkpoint. + * + * Returns 1 if such checkpoint exists, 0 if it doesn't, -1 on error. + */ +int +virDomainHasCurrentCheckpoint(virDomainPtr domain, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "flags=0x%x", flags); + + virResetLastError(); + + virCheckDomainReturn(domain, -1); + conn = domain->conn; + + if (conn->driver->domainHasCurrentCheckpoint) { + int ret = conn->driver->domainHasCurrentCheckpoint(domain, flags); + if (ret < 0) + goto error; + return ret; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return -1; +} + + +/** + * virDomainCheckpointCurrent: + * @domain: a domain object + * @flags: extra flags; not used yet, so callers should always pass 0 + * + * Get the current checkpoint for a domain, if any. + * + * virDomainCheckpointFree should be used to free the resources after the + * checkpoint object is no longer needed. + * + * Returns a domain checkpoint object or NULL in case of failure. If the + * current domain checkpoint cannot be found, then the + * VIR_ERR_NO_DOMAIN_CHECKPOINT error is raised. + */ +virDomainCheckpointPtr +virDomainCheckpointCurrent(virDomainPtr domain, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "flags=0x%x", flags); + + virResetLastError(); + + virCheckDomainReturn(domain, NULL); + conn = domain->conn; + + if (conn->driver->domainCheckpointCurrent) { + virDomainCheckpointPtr snap; + snap = conn->driver->domainCheckpointCurrent(domain, flags); + if (!snap) + goto error; + return snap; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return NULL; +} + + +/** + * virDomainCheckpointGetParent: + * @checkpoint: a checkpoint object + * @flags: extra flags; not used yet, so callers should always pass 0 + * + * Get the parent checkpoint for @checkpoint, if any. + * + * virDomainCheckpointFree should be used to free the resources after the + * checkpoint object is no longer needed. + * + * Returns a domain checkpoint object or NULL in case of failure. If the + * given checkpoint is a root (no parent), then the VIR_ERR_NO_DOMAIN_CHECKPOINT + * error is raised. + */ +virDomainCheckpointPtr +virDomainCheckpointGetParent(virDomainCheckpointPtr checkpoint, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DEBUG("checkpoint=%p, flags=0x%x", checkpoint, flags); + + virResetLastError(); + + virCheckDomainCheckpointReturn(checkpoint, NULL); + conn = checkpoint->domain->conn; + + if (conn->driver->domainCheckpointGetParent) { + virDomainCheckpointPtr snap; + snap = conn->driver->domainCheckpointGetParent(checkpoint, flags); + if (!snap) + goto error; + return snap; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return NULL; +} + + +/** + * virDomainCheckpointIsCurrent: + * @checkpoint: a checkpoint object + * @flags: extra flags; not used yet, so callers should always pass 0 + * + * Determine if the given checkpoint is the domain's current checkpoint. See + * also virDomainHasCurrentCheckpoint(). + * + * Returns 1 if current, 0 if not current, or -1 on error. + */ +int +virDomainCheckpointIsCurrent(virDomainCheckpointPtr checkpoint, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DEBUG("checkpoint=%p, flags=0x%x", checkpoint, flags); + + virResetLastError(); + + virCheckDomainCheckpointReturn(checkpoint, -1); + conn = checkpoint->domain->conn; + + if (conn->driver->domainCheckpointIsCurrent) { + int ret; + ret = conn->driver->domainCheckpointIsCurrent(checkpoint, flags); + if (ret < 0) + goto error; + return ret; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return -1; +} + + +/** + * virDomainCheckpointHasMetadata: + * @checkpoint: a checkpoint object + * @flags: extra flags; not used yet, so callers should always pass 0 + * + * Determine if the given checkpoint is associated with libvirt metadata + * that would prevent the deletion of the domain. + * + * Returns 1 if the checkpoint has metadata, 0 if the checkpoint exists without + * help from libvirt, or -1 on error. + */ +int +virDomainCheckpointHasMetadata(virDomainCheckpointPtr checkpoint, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DEBUG("checkpoint=%p, flags=0x%x", checkpoint, flags); + + virResetLastError(); + + virCheckDomainCheckpointReturn(checkpoint, -1); + conn = checkpoint->domain->conn; + + if (conn->driver->domainCheckpointHasMetadata) { + int ret; + ret = conn->driver->domainCheckpointHasMetadata(checkpoint, flags); + if (ret < 0) + goto error; + return ret; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return -1; +} + + +/** + * virDomainCheckpointDelete: + * @checkpoint: the checkpoint to remove + * @flags: bitwise-OR of supported virDomainCheckpointDeleteFlags + * + * Removes a checkpoint from the domain. + * + * When removing a checkpoint, the record of which portions of the + * disk were dirtied after the checkpoint will be merged into the + * record tracked by the parent checkpoint, if any. Likewise, if the + * checkpoint being deleted was the current checkpoint, the parent + * checkpoint becomes the new current checkpoint. + * + * If @flags includes VIR_DOMAIN_CHECKPOINT_DELETE_CHILDREN, then any + * descendant checkpoints are also deleted. If @flags includes + * VIR_DOMAIN_CHECKPOINT_DELETE_CHILDREN_ONLY, then any descendant + * checkepoints are deleted, but this checkpoint remains. These two + * flags are mutually exclusive. + * + * If @flags includes VIR_DOMAIN_CHECKPOINT_DELETE_METADATA_ONLY, then + * any checkpoint metadata tracked by libvirt is removed while keeping + * the checkpoint contents intact; if a hypervisor does not require + * any libvirt metadata to track checkpoints, then this flag is + * silently ignored. + * + * Returns 0 on success, -1 on error. + */ +int +virDomainCheckpointDelete(virDomainCheckpointPtr checkpoint, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DEBUG("checkpoint=%p, flags=0x%x", checkpoint, flags); + + virResetLastError(); + + virCheckDomainCheckpointReturn(checkpoint, -1); + conn = checkpoint->domain->conn; + + virCheckReadOnlyGoto(conn->flags, error); + + VIR_EXCLUSIVE_FLAGS_GOTO(VIR_DOMAIN_CHECKPOINT_DELETE_CHILDREN, + VIR_DOMAIN_CHECKPOINT_DELETE_CHILDREN_ONLY, + error); + + if (conn->driver->domainCheckpointDelete) { + int ret = conn->driver->domainCheckpointDelete(checkpoint, flags); + if (ret < 0) + goto error; + return ret; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return -1; +} + + +/** + * virDomainCheckpointRef: + * @checkpoint: the checkpoint to hold a reference on + * + * Increment the reference count on the checkpoint. For each + * additional call to this method, there shall be a corresponding + * call to virDomainCheckpointFree to release the reference count, once + * the caller no longer needs the reference to this object. + * + * This method is typically useful for applications where multiple + * threads are using a connection, and it is required that the + * connection and domain remain open until all threads have finished + * using the checkpoint. ie, each new thread using a checkpoint would + * increment the reference count. + * + * Returns 0 in case of success and -1 in case of failure. + */ +int +virDomainCheckpointRef(virDomainCheckpointPtr checkpoint) +{ + VIR_DEBUG("checkpoint=%p, refs=%d", checkpoint, + checkpoint ? checkpoint->parent.u.s.refs : 0); + + virResetLastError(); + + virCheckDomainCheckpointReturn(checkpoint, -1); + + virObjectRef(checkpoint); + return 0; +} + + +/** + * virDomainCheckpointFree: + * @checkpoint: a domain checkpoint object + * + * Free the domain checkpoint object. The checkpoint itself is not modified. + * The data structure is freed and should not be used thereafter. + * + * Returns 0 in case of success and -1 in case of failure. + */ +int +virDomainCheckpointFree(virDomainCheckpointPtr checkpoint) +{ + VIR_DEBUG("checkpoint=%p", checkpoint); + + virResetLastError(); + + virCheckDomainCheckpointReturn(checkpoint, -1); + + virObjectUnref(checkpoint); + return 0; +} diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index be5b1f6740..1ff4962fc1 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -6444,7 +6444,9 @@ virConnectListDefinedDomains(virConnectPtr conn, char **const names, * VIR_CONNECT_LIST_DOMAINS_NO_AUTOSTART, for filtering based on autostart; * VIR_CONNECT_LIST_DOMAINS_HAS_SNAPSHOT and * VIR_CONNECT_LIST_DOMAINS_NO_SNAPSHOT, for filtering based on whether - * a domain has snapshots. + * a domain has snapshots; VIR_CONNECT_LIST_DOMAINS_HAS_CHECKPOINT and + * VIR_CONNECT_LIST_DOMAINS_NO_CHECKPOINT, for filtering based on whether + * a domain has checkpoints. * * Example of usage: * diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index dbce3336d5..d026b33f53 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -817,6 +817,22 @@ LIBVIRT_4.10.0 { LIBVIRT_5.2.0 { global: virConnectGetStoragePoolCapabilities; + virDomainCheckpointCreateXML; + virDomainCheckpointCurrent; + virDomainCheckpointDelete; + virDomainCheckpointFree; + virDomainCheckpointGetConnect; + virDomainCheckpointGetDomain; + virDomainCheckpointGetName; + virDomainCheckpointGetParent; + virDomainCheckpointGetXMLDesc; + virDomainCheckpointHasMetadata; + virDomainCheckpointIsCurrent; + virDomainCheckpointListChildren; + virDomainCheckpointLookupByName; + virDomainCheckpointRef; + virDomainHasCurrentCheckpoint; + virDomainListCheckpoints; } LIBVIRT_4.10.0; # .... define new API here using predicted next version number .... -- 2.20.1

On Tue, Mar 26, 2019 at 01:13:51AM -0500, Eric Blake wrote:
Introduce a bunch of new public APIs related to backup checkpoints. Checkpoints are modeled heavily after virDomainSnapshotPtr (both represent a point in time of the guest), although a snapshot exists with the intent of rolling back to that state, while a checkpoint exists to make it possible to create an incremental backup at a later time.
The following map shows the API relations to snapshots, with new APIs on the right:
Operate on a domain object to create/redefine a child: virDomainSnapshotCreateXML virDomainCheckpointCreateXML
Operate on a child object for lifetime management: virDomainSnapshotDelete virDomainCheckpointDelete virDomainSnapshotFree virDomainCheckpointFree virDomainSnapshotRef virDomainCheckpointRef
Operate on a child object to learn more about it: virDomainSnapshotGetXMLDesc virDomainCheckpointGetXMLDesc virDomainSnapshotGetConnect virDomainCheckpointGetConnect virDomainSnapshotGetDomain virDomainCheckpointGetDomain virDomainSnapshotGetName virDomainCheckpiontGetName virDomainSnapshotGetParent virDomainCheckpiontGetParent virDomainSnapshotHasMetadata virDomainCheckpointHasMetadata virDomainSnapshotIsCurrent virDomainCheckpointIsCurrent
Operate on a domain object to list all children: virDomainSnapshotNum (no counterpart, this is the old virDomainSnapshotListNames racy interface) virDomainSnapshotListAllSnapshots virDomainListCheckpoints
The snapshot API was actually virDomainListAllSnapshots. For naming consistency I'd prefer to see "All" in the name of the name API too eg virDomainListAllCheckpoints
Operate on a child object to list descendents: virDomainSnapshotNumChildren (no counterpart, this is the old virDomainSnapshotListChildrenNames racy interface) virDomainSnapshotListAllChildren virDomainCheckpointListChildren
and virDomainCheckpointListAllChildren here too
Operate on a domain to locate a particular child: virDomainSnapshotLookupByName virDomainCheckpointLookupByName virDomainHasCurrentSnapshot virDomainHasCurrentSnapshot
Presumably the second should be virDomainhasCurrentCheckpoint
virDomainSnapshotCurrent virDomainCheckpointCurrent
Operate on a snapshot to roll back to earlier state: virDomainSnapshotRevert (no counterpart, instead checkpoints are used in incremental backups via XML to virDomainBackupBegin)
Signed-off-by: Eric Blake <eblake@redhat.com> --- include/libvirt/libvirt-domain-checkpoint.h | 161 +++++ include/libvirt/libvirt-domain.h | 6 + include/libvirt/libvirt.h | 5 +- src/conf/virdomainmomentobjlist.h | 5 +- src/driver-hypervisor.h | 60 +- docs/Makefile.am | 3 + docs/apibuild.py | 2 + docs/docs.html.in | 1 + libvirt.spec.in | 1 + mingw-libvirt.spec.in | 2 + po/POTFILES | 1 + src/Makefile.am | 2 + src/libvirt-domain-checkpoint.c | 749 ++++++++++++++++++++ src/libvirt-domain.c | 4 +- src/libvirt_public.syms | 16 + 15 files changed, 1010 insertions(+), 8 deletions(-) create mode 100644 include/libvirt/libvirt-domain-checkpoint.h create mode 100644 src/libvirt-domain-checkpoint.c
diff --git a/include/libvirt/libvirt-domain-checkpoint.h b/include/libvirt/libvirt-domain-checkpoint.h new file mode 100644 index 0000000000..332df93755 --- /dev/null +++ b/include/libvirt/libvirt-domain-checkpoint.h @@ -0,0 +1,161 @@ +/* + * libvirt-domain-checkpoint.h + * Summary: APIs for management of domain checkpoints + * Description: Provides APIs for the management of domain checkpoints + * + * Copyright (C) 2006-2019 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + */ + +#ifndef LIBVIRT_DOMAIN_CHECKPOINT_H +# define LIBVIRT_DOMAIN_CHECKPOINT_H + +# ifndef __VIR_LIBVIRT_H_INCLUDES__ +# error "Don't include this file directly, only use libvirt/libvirt.h" +# endif + +/** + * virDomainCheckpoint: + * + * A virDomainCheckpoint is a private structure representing a checkpoint of + * a domain. A checkpoint is useful for tracking which portions of the + * domain disks have been altered since a point in time, but by itself does + * not allow reverting back to that point in time. + */ +typedef struct _virDomainCheckpoint virDomainCheckpoint; + +/** + * virDomainCheckpointPtr: + * + * A virDomainCheckpointPtr is pointer to a virDomainCheckpoint + * private structure, and is the type used to reference a domain + * checkpoint in the API. + */ +typedef virDomainCheckpoint *virDomainCheckpointPtr; + +const char *virDomainCheckpointGetName(virDomainCheckpointPtr checkpoint); +virDomainPtr virDomainCheckpointGetDomain(virDomainCheckpointPtr checkpoint); +virConnectPtr virDomainCheckpointGetConnect(virDomainCheckpointPtr checkpoint); + +typedef enum { + VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE = (1 << 0), /* Restore or alter + metadata */ + VIR_DOMAIN_CHECKPOINT_CREATE_CURRENT = (1 << 1), /* With redefine, make + checkpoint current */ + VIR_DOMAIN_CHECKPOINT_CREATE_NO_METADATA = (1 << 2), /* Make checkpoint without + remembering it */ + VIR_DOMAIN_CHECKPOINT_CREATE_QUIESCE = (1 << 3), /* use guest agent to + quiesce all mounted + file systems within + the domain */ +} virDomainCheckpointCreateFlags; + +/* Create a checkpoint using the current VM state. */ +virDomainCheckpointPtr virDomainCheckpointCreateXML(virDomainPtr domain, + const char *xmlDesc, + unsigned int flags); + +typedef enum { + VIR_DOMAIN_CHECKPOINT_XML_SECURE = (1 << 0), /* Include sensitive data */ + VIR_DOMAIN_CHECKPOINT_XML_NO_DOMAIN = (1 << 1), /* Suppress <domain> + subelement */ + VIR_DOMAIN_CHECKPOINT_XML_SIZE = (1 << 2), /* Include dynamic + per-<disk> size */ +} virDomainCheckpointXMLFlags; + +/* Dump the XML of a checkpoint */ +char *virDomainCheckpointGetXMLDesc(virDomainCheckpointPtr checkpoint, + unsigned int flags); + +/** + * virDomainCheckpointListFlags: + * + * Flags valid for virDomainListCheckpoints() and + * virDomainCheckpointListChildren(). Note that the interpretation of + * flag (1<<0) depends on which function it is passed to; but serves + * to toggle the per-call default of whether the listing is shallow or + * recursive. Remaining bits come in groups; if all bits from a group + * are 0, then that group is not used to filter results. */ +typedef enum { + VIR_DOMAIN_CHECKPOINT_LIST_ROOTS = (1 << 0), /* Filter by checkpoints + with no parents, when + listing a domain */ + VIR_DOMAIN_CHECKPOINT_LIST_DESCENDANTS = (1 << 0), /* List all descendants, + not just children, when + listing a checkpoint */ + VIR_DOMAIN_CHECKPOINT_LIST_TOPOLOGICAL = (1 << 1), /* Ensure parents occur + before children in + the resulting list */ + + VIR_DOMAIN_CHECKPOINT_LIST_LEAVES = (1 << 2), /* Filter by checkpoints + with no children */ + VIR_DOMAIN_CHECKPOINT_LIST_NO_LEAVES = (1 << 3), /* Filter by checkpoints + that have children */ + + VIR_DOMAIN_CHECKPOINT_LIST_METADATA = (1 << 4), /* Filter by checkpoints + which have metadata */ + VIR_DOMAIN_CHECKPOINT_LIST_NO_METADATA = (1 << 5), /* Filter by checkpoints + with no metadata */ +} virDomainCheckpointListFlags; + +/* Get all checkpoint objects for this domain */ +int virDomainListCheckpoints(virDomainPtr domain, + virDomainCheckpointPtr **checkpoints, + unsigned int flags); + +/* Get all checkpoint object children for this checkpoint */ +int virDomainCheckpointListChildren(virDomainCheckpointPtr checkpoint, + virDomainCheckpointPtr **children, + unsigned int flags); + +/* Get a handle to a named checkpoint */ +virDomainCheckpointPtr virDomainCheckpointLookupByName(virDomainPtr domain, + const char *name, + unsigned int flags); + +/* Check whether a domain has a checkpoint which is currently used */ +int virDomainHasCurrentCheckpoint(virDomainPtr domain, unsigned int flags); + +/* Get a handle to the current checkpoint */ +virDomainCheckpointPtr virDomainCheckpointCurrent(virDomainPtr domain, + unsigned int flags); + +/* Get a handle to the parent checkpoint, if one exists */ +virDomainCheckpointPtr virDomainCheckpointGetParent(virDomainCheckpointPtr checkpoint, + unsigned int flags); + +/* Determine if a checkpoint is the current checkpoint of its domain. */ +int virDomainCheckpointIsCurrent(virDomainCheckpointPtr checkpoint, + unsigned int flags); + +/* Determine if checkpoint has metadata that would prevent domain deletion. */ +int virDomainCheckpointHasMetadata(virDomainCheckpointPtr checkpoint, + unsigned int flags); + +/* Delete a checkpoint */ +typedef enum { + VIR_DOMAIN_CHECKPOINT_DELETE_CHILDREN = (1 << 0), /* Also delete children */ + VIR_DOMAIN_CHECKPOINT_DELETE_METADATA_ONLY = (1 << 1), /* Delete just metadata */ + VIR_DOMAIN_CHECKPOINT_DELETE_CHILDREN_ONLY = (1 << 2), /* Delete just children */ +} virDomainCheckpointDeleteFlags; + +int virDomainCheckpointDelete(virDomainCheckpointPtr checkpoint, + unsigned int flags); + +int virDomainCheckpointRef(virDomainCheckpointPtr checkpoint); +int virDomainCheckpointFree(virDomainCheckpointPtr checkpoint); + +#endif /* LIBVIRT_DOMAIN_CHECKPOINT_H */ diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 1d5bdb545d..94504e70a6 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1774,6 +1774,9 @@ typedef enum { VIR_DOMAIN_UNDEFINE_NVRAM = (1 << 2), /* Also remove any nvram file */ VIR_DOMAIN_UNDEFINE_KEEP_NVRAM = (1 << 3), /* Keep nvram file */ + VIR_DOMAIN_UNDEFINE_CHECKPOINTS_METADATA= (1 << 4),/* If last use of domain, + then also remove any + checkpoint metadata */
Whitespace missing before the "=" and between ",/"
/* Future undefine control flags should come here. */ } virDomainUndefineFlagsValues; @@ -1812,6 +1815,9 @@ typedef enum {
VIR_CONNECT_LIST_DOMAINS_HAS_SNAPSHOT = 1 << 12, VIR_CONNECT_LIST_DOMAINS_NO_SNAPSHOT = 1 << 13, + + VIR_CONNECT_LIST_DOMAINS_HAS_CHECKPOINT = 1 << 14, + VIR_CONNECT_LIST_DOMAINS_NO_CHECKPOINT = 1 << 15, } virConnectListAllDomainsFlags;
int virConnectListAllDomains (virConnectPtr conn,
Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On 3/26/19 11:55 AM, Daniel P. Berrangé wrote:
On Tue, Mar 26, 2019 at 01:13:51AM -0500, Eric Blake wrote:
Introduce a bunch of new public APIs related to backup checkpoints. Checkpoints are modeled heavily after virDomainSnapshotPtr (both represent a point in time of the guest), although a snapshot exists with the intent of rolling back to that state, while a checkpoint exists to make it possible to create an incremental backup at a later time.
The following map shows the API relations to snapshots, with new APIs on the right:
Operate on a domain object to create/redefine a child: virDomainSnapshotCreateXML virDomainCheckpointCreateXML
Operate on a child object for lifetime management: virDomainSnapshotDelete virDomainCheckpointDelete virDomainSnapshotFree virDomainCheckpointFree virDomainSnapshotRef virDomainCheckpointRef
Operate on a child object to learn more about it: virDomainSnapshotGetXMLDesc virDomainCheckpointGetXMLDesc virDomainSnapshotGetConnect virDomainCheckpointGetConnect virDomainSnapshotGetDomain virDomainCheckpointGetDomain virDomainSnapshotGetName virDomainCheckpiontGetName virDomainSnapshotGetParent virDomainCheckpiontGetParent virDomainSnapshotHasMetadata virDomainCheckpointHasMetadata virDomainSnapshotIsCurrent virDomainCheckpointIsCurrent
Operate on a domain object to list all children: virDomainSnapshotNum (no counterpart, this is the old virDomainSnapshotListNames racy interface) virDomainSnapshotListAllSnapshots virDomainListCheckpoints
The snapshot API was actually virDomainListAllSnapshots.
For naming consistency I'd prefer to see "All" in the name of the name API too eg virDomainListAllCheckpoints
Can do. That one's quite mechanical to fix.
Operate on a child object to list descendents: virDomainSnapshotNumChildren (no counterpart, this is the old virDomainSnapshotListChildrenNames racy interface) virDomainSnapshotListAllChildren virDomainCheckpointListChildren
and virDomainCheckpointListAllChildren here too
Yep, consistency in both places makes sense.
Operate on a domain to locate a particular child: virDomainSnapshotLookupByName virDomainCheckpointLookupByName virDomainHasCurrentSnapshot virDomainHasCurrentSnapshot
Presumably the second should be virDomainhasCurrentCheckpoint
Correct, will fix my commit message.
virDomainSnapshotCurrent virDomainCheckpointCurrent
Operate on a snapshot to roll back to earlier state: virDomainSnapshotRevert (no counterpart, instead checkpoints are used in incremental backups via XML to virDomainBackupBegin)
Signed-off-by: Eric Blake <eblake@redhat.com> ---
+++ b/include/libvirt/libvirt-domain.h @@ -1774,6 +1774,9 @@ typedef enum { VIR_DOMAIN_UNDEFINE_NVRAM = (1 << 2), /* Also remove any nvram file */ VIR_DOMAIN_UNDEFINE_KEEP_NVRAM = (1 << 3), /* Keep nvram file */ + VIR_DOMAIN_UNDEFINE_CHECKPOINTS_METADATA= (1 << 4),/* If last use of domain, + then also remove any + checkpoint metadata */
Whitespace missing before the "=" and between ",/"
I'm not sure how best to fit things; then again, it looks like we don't always try to hit 80 columns in the public headers, so I don't mind adding the space. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org

Introduce a few more new public APIs related to incremental backups. This builds on the previous notion of a checkpoint (without an existing checkpoint, the new API is a full backup, differing from virDomainBlockCopy in the point of time chosen and in operation on multiple disks at once); and also allows creation of a new checkpoint at the same time as starting the backup (after all, an incremental backup is only useful if it covers the state since the previous backup). Snapshot creation is also a point in time at which creating a checkpoint atomically can be useful. A backup job also affects filtering a listing of domains, as well as adding event reporting for signaling when a push model backup completes (where the hypervisor creates the backup); note that the pull model does not have an event (starting the backup lets a third party access the data, and only the third party knows when it is finished). Since multiple backup jobs can be run in parallel in the future (well, qemu doesn't support it yet, but we don't want to preclude the idea), virDomainBackupBegin() returns a job id, which can also be queried by virDomainListJobIds(), and this job id must be used for virDomainBackupGetXMLDesc() and virDomainBackupEnd(). In the future, we may also extend other jobs (migration as the default global job impacting virDomainJobStats(), and the various block jobs) to all have ids, where the existing APIs act like thin wrappers around more powerful APIs that support a job id everywhere. The full list of new API: virDomainBackupBegin; virDomainBackupEnd; virDomainBackupGetXMLDesc; virDomainListJobIds; virDomainSnapshotCreateXML2; Signed-off-by: Eric Blake <eblake@redhat.com> --- include/libvirt/libvirt-domain-snapshot.h | 7 +- include/libvirt/libvirt-domain.h | 53 +++- src/driver-hypervisor.h | 27 ++ src/qemu/qemu_blockjob.h | 1 + examples/object-events/event-test.c | 3 + src/conf/domain_conf.c | 2 +- src/libvirt-domain-snapshot.c | 89 +++++++ src/libvirt-domain.c | 285 +++++++++++++++++++++- src/libvirt_public.syms | 5 + tools/virsh-domain.c | 8 +- 10 files changed, 463 insertions(+), 17 deletions(-) diff --git a/include/libvirt/libvirt-domain-snapshot.h b/include/libvirt/libvirt-domain-snapshot.h index 602e5def59..632e99c61a 100644 --- a/include/libvirt/libvirt-domain-snapshot.h +++ b/include/libvirt/libvirt-domain-snapshot.h @@ -3,7 +3,7 @@ * Summary: APIs for management of domain snapshots * Description: Provides APIs for the management of domain snapshots * - * Copyright (C) 2006-2014 Red Hat, Inc. + * Copyright (C) 2006-2019 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -78,6 +78,11 @@ virDomainSnapshotPtr virDomainSnapshotCreateXML(virDomainPtr domain, const char *xmlDesc, unsigned int flags); +virDomainSnapshotPtr virDomainSnapshotCreateXML2(virDomainPtr domain, + const char *xmlDesc, + const char *snapshotXml, + unsigned int flags); + typedef enum { VIR_DOMAIN_SNAPSHOT_XML_SECURE = VIR_DOMAIN_XML_SECURE, /* dump security sensitive information too */ } virDomainSnapshotXMLFlags; diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 94504e70a6..59fb645755 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -3,7 +3,7 @@ * Summary: APIs for management of domains * Description: Provides APIs for the management of domains * - * Copyright (C) 2006-2015 Red Hat, Inc. + * Copyright (C) 2006-2019 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -2420,6 +2420,9 @@ typedef enum { * exists as long as sync is active */ VIR_DOMAIN_BLOCK_JOB_TYPE_ACTIVE_COMMIT = 4, + /* Backup (virDomainBackupBegin), job exists until virDomainBackupEnd */ + VIR_DOMAIN_BLOCK_JOB_TYPE_BACKUP = 5, + # ifdef VIR_ENUM_SENTINELS VIR_DOMAIN_BLOCK_JOB_TYPE_LAST # endif @@ -3231,6 +3234,18 @@ int virDomainGetJobStats(virDomainPtr domain, unsigned int flags); int virDomainAbortJob(virDomainPtr dom); +typedef struct _virDomainJobId virDomainJobId; +typedef virDomainJobId *virDomainJobIdPtr; +struct _virDomainJobId { + /* One of virDomainJobType */ + int type; + + /* The job id */ + int id; +}; +int virDomainListJobIds(virDomainPtr dom, virDomainJobId **ids, + unsigned int flags); + typedef enum { VIR_DOMAIN_JOB_OPERATION_UNKNOWN = 0, VIR_DOMAIN_JOB_OPERATION_START = 1, @@ -3241,6 +3256,7 @@ typedef enum { VIR_DOMAIN_JOB_OPERATION_SNAPSHOT = 6, VIR_DOMAIN_JOB_OPERATION_SNAPSHOT_REVERT = 7, VIR_DOMAIN_JOB_OPERATION_DUMP = 8, + VIR_DOMAIN_JOB_OPERATION_BACKUP = 9, # ifdef VIR_ENUM_SENTINELS VIR_DOMAIN_JOB_OPERATION_LAST @@ -3256,6 +3272,14 @@ typedef enum { */ # define VIR_DOMAIN_JOB_OPERATION "operation" +/** + * VIR_DOMAIN_JOB_ID: + * + * virDomainGetJobStats field: the id of the job (so far, only for jobs + * started by virDomainBackupBegin()), as VIR_TYPED_PARAM_INT. + */ +# define VIR_DOMAIN_JOB_ID "id" + /** * VIR_DOMAIN_JOB_TIME_ELAPSED: * @@ -4080,7 +4104,8 @@ typedef void (*virConnectDomainEventMigrationIterationCallback)(virConnectPtr co * @nparams: size of the params array * @opaque: application specific data * - * This callback occurs when a job (such as migration) running on the domain + * This callback occurs when a job (such as migration or push-model + * virDomainBackupBegin()) running on the domain * is completed. The params array will contain statistics of the just completed * job as virDomainGetJobStats would return. The callback must not free @params * (the array will be freed once the callback finishes). @@ -4876,4 +4901,28 @@ int virDomainGetLaunchSecurityInfo(virDomainPtr domain, int *nparams, unsigned int flags); +typedef enum { + VIR_DOMAIN_BACKUP_BEGIN_NO_METADATA = (1 << 0), /* Make checkpoint without + remembering it */ + VIR_DOMAIN_BACKUP_BEGIN_QUIESCE = (1 << 1), /* use guest agent to + quiesce all mounted + file systems within + the domain */ +} virDomainBackupBeginFlags; + +/* Begin an incremental backup job, possibly creating a checkpoint. */ +int virDomainBackupBegin(virDomainPtr domain, const char *diskXml, + const char *checkpointXml, unsigned int flags); + +/* Learn about an ongoing backup job. */ +char *virDomainBackupGetXMLDesc(virDomainPtr domain, int id, + unsigned int flags); + +typedef enum { + VIR_DOMAIN_BACKUP_END_ABORT = (1 << 0), /* Abandon a push model backup */ +} virDomainBackupEndFlags; + +/* Complete (or abort) an incremental backup job. */ +int virDomainBackupEnd(virDomainPtr domain, int id, unsigned int flags); + #endif /* LIBVIRT_DOMAIN_H */ diff --git a/src/driver-hypervisor.h b/src/driver-hypervisor.h index e6efce1bfe..3cdd8a9031 100644 --- a/src/driver-hypervisor.h +++ b/src/driver-hypervisor.h @@ -729,6 +729,11 @@ typedef int typedef int (*virDrvDomainAbortJob)(virDomainPtr domain); +typedef int +(*virDrvDomainListJobIds)(virDomainPtr dom, + virDomainJobId **ids, + unsigned int flags); + typedef int (*virDrvDomainMigrateGetMaxDowntime)(virDomainPtr domain, unsigned long long *downtime, @@ -797,6 +802,12 @@ typedef virDomainSnapshotPtr const char *xmlDesc, unsigned int flags); +typedef virDomainSnapshotPtr +(*virDrvDomainSnapshotCreateXML2)(virDomainPtr domain, + const char *xmlDesc, + const char *checkpointXml, + unsigned int flags); + typedef char * (*virDrvDomainSnapshotGetXMLDesc)(virDomainSnapshotPtr snapshot, unsigned int flags); @@ -1376,6 +1387,17 @@ typedef int (*virDrvDomainCheckpointDelete)(virDomainCheckpointPtr checkpoint, unsigned int flags); +typedef int +(*virDrvDomainBackupBegin)(virDomainPtr domain, const char *diskXml, + const char *checkpointXml, unsigned int flags); + +typedef char * +(*virDrvDomainBackupGetXMLDesc)(virDomainPtr domain, int id, + unsigned int flags); + +typedef int +(*virDrvDomainBackupEnd)(virDomainPtr domain, int id, unsigned int flags); + typedef struct _virHypervisorDriver virHypervisorDriver; typedef virHypervisorDriver *virHypervisorDriverPtr; @@ -1527,6 +1549,7 @@ struct _virHypervisorDriver { virDrvDomainGetJobInfo domainGetJobInfo; virDrvDomainGetJobStats domainGetJobStats; virDrvDomainAbortJob domainAbortJob; + virDrvDomainListJobIds domainListJobIds; virDrvDomainMigrateGetMaxDowntime domainMigrateGetMaxDowntime; virDrvDomainMigrateSetMaxDowntime domainMigrateSetMaxDowntime; virDrvDomainMigrateGetCompressionCache domainMigrateGetCompressionCache; @@ -1541,6 +1564,7 @@ struct _virHypervisorDriver { virDrvDomainManagedSaveGetXMLDesc domainManagedSaveGetXMLDesc; virDrvDomainManagedSaveDefineXML domainManagedSaveDefineXML; virDrvDomainSnapshotCreateXML domainSnapshotCreateXML; + virDrvDomainSnapshotCreateXML2 domainSnapshotCreateXML2; virDrvDomainSnapshotGetXMLDesc domainSnapshotGetXMLDesc; virDrvDomainSnapshotNum domainSnapshotNum; virDrvDomainSnapshotListNames domainSnapshotListNames; @@ -1638,6 +1662,9 @@ struct _virHypervisorDriver { virDrvDomainCheckpointIsCurrent domainCheckpointIsCurrent; virDrvDomainCheckpointHasMetadata domainCheckpointHasMetadata; virDrvDomainCheckpointDelete domainCheckpointDelete; + virDrvDomainBackupBegin domainBackupBegin; + virDrvDomainBackupGetXMLDesc domainBackupGetXMLDesc; + virDrvDomainBackupEnd domainBackupEnd; }; diff --git a/src/qemu/qemu_blockjob.h b/src/qemu/qemu_blockjob.h index c7325c6daf..95f53dde16 100644 --- a/src/qemu/qemu_blockjob.h +++ b/src/qemu/qemu_blockjob.h @@ -55,6 +55,7 @@ typedef enum { QEMU_BLOCKJOB_TYPE_COPY = VIR_DOMAIN_BLOCK_JOB_TYPE_COPY, QEMU_BLOCKJOB_TYPE_COMMIT = VIR_DOMAIN_BLOCK_JOB_TYPE_COMMIT, QEMU_BLOCKJOB_TYPE_ACTIVE_COMMIT = VIR_DOMAIN_BLOCK_JOB_TYPE_ACTIVE_COMMIT, + QEMU_BLOCKJOB_TYPE_BACKUP = VIR_DOMAIN_BLOCK_JOB_TYPE_BACKUP, /* Additional enum values local to qemu */ QEMU_BLOCKJOB_TYPE_INTERNAL, QEMU_BLOCKJOB_TYPE_LAST diff --git a/examples/object-events/event-test.c b/examples/object-events/event-test.c index fcf4492470..98337ad185 100644 --- a/examples/object-events/event-test.c +++ b/examples/object-events/event-test.c @@ -891,6 +891,9 @@ blockJobTypeToStr(int type) case VIR_DOMAIN_BLOCK_JOB_TYPE_ACTIVE_COMMIT: return "active layer block commit"; + + case VIR_DOMAIN_BLOCK_JOB_TYPE_BACKUP: + return "backup"; } return "unknown"; diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 5f2b1f68b5..fc45d6c680 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -1092,7 +1092,7 @@ VIR_ENUM_IMPL(virDomainOsDefFirmware, * <mirror> XML (remaining types are not two-phase). */ VIR_ENUM_DECL(virDomainBlockJob); VIR_ENUM_IMPL(virDomainBlockJob, VIR_DOMAIN_BLOCK_JOB_TYPE_LAST, - "", "", "copy", "", "active-commit", + "", "", "copy", "", "active-commit", "", ); VIR_ENUM_IMPL(virDomainMemoryModel, diff --git a/src/libvirt-domain-snapshot.c b/src/libvirt-domain-snapshot.c index 0c8023d9f6..2a50a84417 100644 --- a/src/libvirt-domain-snapshot.c +++ b/src/libvirt-domain-snapshot.c @@ -202,6 +202,10 @@ virDomainSnapshotGetConnect(virDomainSnapshotPtr snapshot) * block copy operation; in that case, use virDomainBlockJobAbort() * to stop the block copy first. * + * To create a checkpoint object that coincides with the snapshot, in + * order to facilitate later incremental backups from the time of the + * snapshot, use virDomainSnapshotCreateXML2(). + * * virDomainSnapshotFree should be used to free the resources after the * snapshot object is no longer needed. * @@ -251,6 +255,91 @@ virDomainSnapshotCreateXML(virDomainPtr domain, } +/** + * virDomainSnapshotCreateXML2: + * @domain: a domain object + * @xmlDesc: string containing an XML description of the domain snapshot + * @checkpointXml: description of a checkpoint to create or NULL + * @flags: bitwise-OR of virDomainSnapshotCreateFlags + * + * Creates a new snapshot of a domain based on the snapshot xml + * contained in xmlDesc, with a top-level element <domainsnapshot>. + * + * The @checkpointXml parameter is optional; if non-NULL, then libvirt + * behaves as if virDomainCheckpointCreateXML() were called to create + * a checkpoint atomically covering the same point in time as the + * snapshot, using @checkpointXml and forwarding flags + * VIR_DOMAIN_SNAPSHOT_CREATE_QUIESCE and + * VIR_DOMAIN_SNAPSHOT_CREATE_NO_METADATA. The creation of a new + * checkpoint allows for future incremental backups from the time of + * the snapshot. Note that some hypervisors may require a particular + * disk format, such as qcow2, in order to take advantage of + * checkpoints, while allowing arbitrary formats if checkpoints are + * not involved. This parameter is incompatible with the flag + * VIR_DOMAIN_SNAPSHOT_CREATE_REDEFINE. + * + * See virDomainSnapshotCreateXML() for the description of individual + * flags and general behavior. + * + * virDomainSnapshotFree() should be used to free the resources after + * the snapshot object is no longer needed. + * + * Returns an (opaque) new virDomainSnapshotPtr on success or NULL on + * failure. + */ +virDomainSnapshotPtr +virDomainSnapshotCreateXML2(virDomainPtr domain, + const char *xmlDesc, + const char *checkpointXml, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "xmlDesc=%s, checkpointXml=%s, flags=0x%x", + xmlDesc, checkpointXml, flags); + + virResetLastError(); + + virCheckDomainReturn(domain, NULL); + conn = domain->conn; + + virCheckNonNullArgGoto(xmlDesc, error); + virCheckReadOnlyGoto(conn->flags, error); + + VIR_REQUIRE_FLAG_GOTO(VIR_DOMAIN_SNAPSHOT_CREATE_CURRENT, + VIR_DOMAIN_SNAPSHOT_CREATE_REDEFINE, + error); + + VIR_EXCLUSIVE_FLAGS_GOTO(VIR_DOMAIN_SNAPSHOT_CREATE_REDEFINE, + VIR_DOMAIN_SNAPSHOT_CREATE_NO_METADATA, + error); + VIR_EXCLUSIVE_FLAGS_GOTO(VIR_DOMAIN_SNAPSHOT_CREATE_REDEFINE, + VIR_DOMAIN_SNAPSHOT_CREATE_HALT, + error); + + if (checkpointXml && (flags & VIR_DOMAIN_SNAPSHOT_CREATE_REDEFINE)) { + virReportInvalidArg(checkpointXml, "%s", + _("Cannot create checkpoint when redefining " + "snapshot")); + goto error; + } + + if (conn->driver->domainSnapshotCreateXML2) { + virDomainSnapshotPtr ret; + ret = conn->driver->domainSnapshotCreateXML2(domain, xmlDesc, + checkpointXml, flags); + if (!ret) + goto error; + return ret; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return NULL; +} + + /** * virDomainSnapshotGetXMLDesc: * @snapshot: a domain snapshot object diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index 1ff4962fc1..45d187099e 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -1,7 +1,7 @@ /* * libvirt-domain.c: entry points for virDomainPtr APIs * - * Copyright (C) 2006-2015 Red Hat, Inc. + * Copyright (C) 2006-2019 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -8726,8 +8726,9 @@ virDomainIsUpdated(virDomainPtr dom) * @domain: a domain object * @info: pointer to a virDomainJobInfo structure allocated by the user * - * Extract information about progress of a background job on a domain. - * Will return an error if the domain is not active. + * Extract information about progress of the default background job + * (id 0) on a domain. Will return an error if the domain is not + * active. * * This function returns a limited amount of information in comparison * to virDomainGetJobStats(). @@ -8775,13 +8776,13 @@ virDomainGetJobInfo(virDomainPtr domain, virDomainJobInfoPtr info) * @nparams: number of items in @params * @flags: bitwise-OR of virDomainGetJobStatsFlags * - * Extract information about progress of a background job on a domain. - * Will return an error if the domain is not active. The function returns - * a superset of progress information provided by virDomainGetJobInfo. - * Possible fields returned in @params are defined by VIR_DOMAIN_JOB_* - * macros and new fields will likely be introduced in the future so callers - * may receive fields that they do not understand in case they talk to a - * newer server. + * Extract information about progress of the default background job + * (id 0) on a domain. Will return an error if the domain is not + * active. The function returns a superset of progress information + * provided by virDomainGetJobInfo(). Possible fields returned in + * @params are defined by VIR_DOMAIN_JOB_* macros and new fields will + * likely be introduced in the future so callers may receive fields + * that they do not understand in case they talk to a newer server. * * When @flags contains VIR_DOMAIN_JOB_STATS_COMPLETED, the function will * return statistics about a recently completed job. Specifically, this @@ -8839,7 +8840,7 @@ virDomainGetJobStats(virDomainPtr domain, * virDomainAbortJob: * @domain: a domain object * - * Requests that the current background job be aborted at the + * Requests that the current default background job (id 0) be aborted at the * soonest opportunity. In case the job is a migration in a post-copy mode, * virDomainAbortJob will report an error (see virDomainMigrateStartPostCopy * for more details). @@ -8876,6 +8877,57 @@ virDomainAbortJob(virDomainPtr domain) } +/** + * virDomainListJobIds: + * @domain: a domain object + * @ids: Pointer to a variable to store the array containing job ids or NULL + * if the list is not required (just returns number of jobs). + * @flags: extra flags; not used yet, so callers should always pass 0 + * + * Collect a list of all background jobs, and return an allocated + * array of information about the type and id of each. + * + * The default background job (id 0, which is typically migration) + * might not be included in the list; for that, use + * virDomainJobStats(). The job id may be important to other APIs + * related to the job type; for example, a backup job id (created by + * virDomainBackupBegin()) is required for calling + * virDomainBackupEnd(). + * + * Returns the number of jobs found or -1 and sets @ids to NULL in + * case of error. The caller is responsible for calling free() on + * @ids. + */ +int +virDomainListJobIds(virDomainPtr domain, + virDomainJobId **ids, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain); + + virResetLastError(); + + virCheckDomainReturn(domain, -1); + conn = domain->conn; + + if (conn->driver->domainListJobIds) { + int ret; + ret = conn->driver->domainListJobIds(domain, ids, flags); + if (ret < 0) + goto error; + return ret; + } + + virReportUnsupportedError(); + + error: + virDispatchError(conn); + return -1; +} + + /** * virDomainMigrateSetMaxDowntime: * @domain: a domain object @@ -10341,6 +10393,12 @@ virDomainBlockRebase(virDomainPtr dom, const char *disk, * over the destination format, the ability to copy to a destination that * is not a local file, and the possibility of additional tuning parameters. * + * The copy created by this API is not finalized until the job ends, + * and does not lend itself to incremental backups (beyond what + * VIR_DOMAIN_BLOCK_COPY_SHALLOW provides) nor to third-party control + * over the data being copied. For those features, use + * virDomainBackupBegin(). + * * Returns 0 if the operation has started, -1 on failure. */ int @@ -12361,3 +12419,208 @@ int virDomainGetLaunchSecurityInfo(virDomainPtr domain, virDispatchError(domain->conn); return -1; } + + +/** + * virDomainBackupBegin: + * @domain: a domain object + * @diskXml: description of storage to utilize and expose during + * the backup, or NULL + * @checkpointXml: description of a checkpoint to create, or NULL + * @flags: bitwise-OR of supported virDomainBackupBeginFlags + * + * Start a point-in-time backup job for the specified disks of a + * running domain. + * + * A backup job is mutually exclusive with domain migration + * (particularly when the job sets up an NBD export, since it is not + * possible to tell any NBD clients about a server migrating between + * hosts). For now, backup jobs are also mutually exclusive with any + * other block job on the same device, although this restriction may + * be lifted in a future release. Progress of the backup job can be + * tracked via virDomainGetJobStats(). The job remains active until a + * subsequent call to virDomainBackupEnd(), even if it no longer has + * anything to copy. + * + * This API differs from virDomainBlockCopy() because it can grab the + * state of more than one disk in parallel, and because the state is + * captured as of the start of the job, rather than the end. + * + * There are two fundamental backup approaches. The first, called a + * push model, instructs the hypervisor to copy the state of the guest + * disk to the designated storage destination (which may be on the + * local file system or a network device). In this mode, the + * hypervisor writes the content of the guest disk to the destination, + * then emits VIR_DOMAIN_EVENT_ID_JOB_COMPLETED when the backup is + * either complete or failed (the backup image is invalid if the job + * fails or virDomainBackupEnd() is used prior to the event being + * emitted). + * + * The second, called a pull model, instructs the hypervisor to expose + * the state of the guest disk over an NBD export. A third-party + * client can then connect to this export and read whichever portions + * of the disk it desires. In this mode, there is no event; libvirt + * has to be informed via virDomainBackupEnd() when the third-party + * NBD client is done and the backup resources can be released. + * + * The @diskXml parameter is optional but usually provided and + * contains details about the backup in the top-level element + * <domainbackup> , including which backup mode to use, whether the + * backup is incremental from a previous checkpoint, which disks + * participate in the backup, the destination for a push model backup, + * and the temporary storage and NBD server details for a pull model + * backup. If omitted, the backup attempts to default to a push mode + * full backup of all disks, where libvirt generates a filename for + * each disk by appending a suffix of a timestamp in seconds since the + * Epoch. virDomainBackupGetXMLDesc() can be called to learn actual + * values selected. For more information, see + * formatcheckpoint.html#BackupAttributes. + * + * The @checkpointXml parameter is optional; if non-NULL, then libvirt + * behaves as if virDomainCheckpointCreateXML() were called to create + * a checkpoint atomically covering the same point in time as the + * backup, using @checkpointXml and forwarding flags + * VIR_DOMAIN_BACKUP_BEGIN_QUIESCE and + * VIR_DOMAIN_BACKUP_BEGIN_NO_METADATA. The creation of a new + * checkpoint allows for future incremental backups. Note that some + * hypervisors may require a particular disk format, such as qcow2, in + * order to take advantage of checkpoints, while allowing arbitrary + * formats if checkpoints are not involved. + * + * Returns a non-negative job id on success or negative on failure. + * This id is then passed to virDomainBackupGetXMLDesc() and + * virDomainBackupEnd(); it can also be obtained from + * virDomainListJobIds(). This operation returns quickly, such that a + * user can choose to start a backup job between virDomainFSFreeze() + * and virDomainFSThaw() in order to create the backup while guest I/O + * is quiesced. + */ +int +virDomainBackupBegin(virDomainPtr domain, + const char *diskXml, + const char *checkpointXml, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "diskXml=%s, checkpointXml=%s, flags=0x%x", + NULLSTR(diskXml), NULLSTR(checkpointXml), flags); + + virResetLastError(); + + virCheckDomainReturn(domain, -1); + conn = domain->conn; + + virCheckReadOnlyGoto(conn->flags, error); + if (flags & VIR_DOMAIN_BACKUP_BEGIN_NO_METADATA) + virCheckNonNullArgGoto(checkpointXml, error); + + if (conn->driver->domainBackupBegin) { + int ret; + ret = conn->driver->domainBackupBegin(domain, diskXml, checkpointXml, + flags); + if (!ret) + goto error; + return ret; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return -1; +} + + +/** + * virDomainBackupGetXMLDesc: + * @domain: a domain object + * @id: the id of an active backup job previously started with + * virDomainBackupBegin() + * @flags: extra flags; not used yet, so callers should always pass 0 + * + * In some cases, a user can start a backup job without supplying all + * details and rely on libvirt to fill in the rest (for example, + * selecting the port used for an NBD export). This API can then be + * used to learn what default values were chosen. At present, none of + * the information provided is security sensitive. + * + * Returns a NUL-terminated UTF-8 encoded XML instance or NULL in + * case of error. The caller must free() the returned value. + */ +char * +virDomainBackupGetXMLDesc(virDomainPtr domain, int id, unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "id=%d, flags=0x%x", id, flags); + + virResetLastError(); + + virCheckDomainReturn(domain, NULL); + conn = domain->conn; + + virCheckNonNegativeArgGoto(id, error); + + if (conn->driver->domainBackupGetXMLDesc) { + char *ret; + ret = conn->driver->domainBackupGetXMLDesc(domain, id, flags); + if (!ret) + goto error; + return ret; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return NULL; +} + + +/** + * virDomainBackupEnd: + * @domain: a domain object + * @id: the id of an active backup job previously started with + * virDomainBackupBegin() + * @flags: bitwise-OR of supported virDomainBackupEndFlags + * + * Conclude a point-in-time backup job @id on the given domain. + * + * If the backup job uses the push model, but the event marking that + * all data has been copied has not yet been emitted, then the command + * fails unless @flags includes VIR_DOMAIN_BACKUP_END_ABORT. If the + * event has been issued, or if the backup uses the pull model, the + * flag has no effect. + * + * Returns 1 if the backup job completed successfully (the backup + * destination file in a push model is consistent), 0 if the job was + * aborted successfully (only when VIR_DOMAIN_BACKUP_END_ABORT is + * passed; the destination file is unusable), and -1 on failure. + */ +int +virDomainBackupEnd(virDomainPtr domain, int id, unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "id=%d, flags=0x%x", id, flags); + + virResetLastError(); + + virCheckDomainReturn(domain, -1); + conn = domain->conn; + + virCheckReadOnlyGoto(conn->flags, error); + virCheckNonNegativeArgGoto(id, error); + + if (conn->driver->domainBackupEnd) { + int ret; + ret = conn->driver->domainBackupEnd(domain, id, flags); + if (!ret) + goto error; + return ret; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return -1; +} diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index d026b33f53..d13df6bb67 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -817,6 +817,9 @@ LIBVIRT_4.10.0 { LIBVIRT_5.2.0 { global: virConnectGetStoragePoolCapabilities; + virDomainBackupBegin; + virDomainBackupEnd; + virDomainBackupGetXMLDesc; virDomainCheckpointCreateXML; virDomainCheckpointCurrent; virDomainCheckpointDelete; @@ -833,6 +836,8 @@ LIBVIRT_5.2.0 { virDomainCheckpointRef; virDomainHasCurrentCheckpoint; virDomainListCheckpoints; + virDomainListJobIds; + virDomainSnapshotCreateXML2; } LIBVIRT_4.10.0; # .... define new API here using predicted next version number .... diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 01aa1bd017..bb3a6d5b6b 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -2561,7 +2561,9 @@ VIR_ENUM_IMPL(virshDomainBlockJob, N_("Block Pull"), N_("Block Copy"), N_("Block Commit"), - N_("Active Block Commit")); + N_("Active Block Commit"), + N_("Backup"), +); static const char * virshDomainBlockJobToString(int type) @@ -6064,7 +6066,9 @@ VIR_ENUM_IMPL(virshDomainJobOperation, N_("Outgoing migration"), N_("Snapshot"), N_("Snapshot revert"), - N_("Dump")); + N_("Dump"), + N_("Backup"), +); static const char * virshDomainJobOperationToString(int op) -- 2.20.1

On Tue, Mar 26, 2019 at 01:13:52AM -0500, Eric Blake wrote:
Introduce a few more new public APIs related to incremental backups. This builds on the previous notion of a checkpoint (without an existing checkpoint, the new API is a full backup, differing from virDomainBlockCopy in the point of time chosen and in operation on multiple disks at once); and also allows creation of a new checkpoint at the same time as starting the backup (after all, an incremental backup is only useful if it covers the state since the previous backup). Snapshot creation is also a point in time at which creating a checkpoint atomically can be useful. A backup job also affects filtering a listing of domains, as well as adding event reporting for signaling when a push model backup completes (where the hypervisor creates the backup); note that the pull model does not have an event (starting the backup lets a third party access the data, and only the third party knows when it is finished).
Since multiple backup jobs can be run in parallel in the future (well, qemu doesn't support it yet, but we don't want to preclude the idea), virDomainBackupBegin() returns a job id, which can also be queried by virDomainListJobIds(), and this job id must be used for virDomainBackupGetXMLDesc() and virDomainBackupEnd(). In the future, we may also extend other jobs (migration as the default global job impacting virDomainJobStats(), and the various block jobs) to all have ids, where the existing APIs act like thin wrappers around more powerful APIs that support a job id everywhere.
The full list of new API: virDomainBackupBegin; virDomainBackupEnd; virDomainBackupGetXMLDesc; virDomainListJobIds; virDomainSnapshotCreateXML2;
Signed-off-by: Eric Blake <eblake@redhat.com> --- include/libvirt/libvirt-domain-snapshot.h | 7 +- include/libvirt/libvirt-domain.h | 53 +++- src/driver-hypervisor.h | 27 ++ src/qemu/qemu_blockjob.h | 1 + examples/object-events/event-test.c | 3 + src/conf/domain_conf.c | 2 +- src/libvirt-domain-snapshot.c | 89 +++++++ src/libvirt-domain.c | 285 +++++++++++++++++++++- src/libvirt_public.syms | 5 + tools/virsh-domain.c | 8 +- 10 files changed, 463 insertions(+), 17 deletions(-)
+virDomainSnapshotPtr virDomainSnapshotCreateXML2(virDomainPtr domain, + const char *xmlDesc, + const char *snapshotXml, + unsigned int flags);
s/snapshotXml/checkpointXml/ based on later docs. On IRC, you had said an alternative would be to put the checkpointXml as a <domaincheckpoint> child of the main <domainsnapshot> or <domainbackup> XML document. IIUC, the <domaincheckpoint> XML is merely forwarded on the checkpoint APIs. IOW, if you later call virDomainSnapshotGetXMLDesc, you would *not* expect to see the <domaincheckpoint> child again ? If that is correct, then having it via the separate API parameter makes more sense than as a XML child element. I'd only want it as an XML child if that where the canonical representation & storage location. So the separate API looks ok to me.
+ typedef enum { VIR_DOMAIN_SNAPSHOT_XML_SECURE = VIR_DOMAIN_XML_SECURE, /* dump security sensitive information too */ } virDomainSnapshotXMLFlags; diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 94504e70a6..59fb645755 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -3,7 +3,7 @@ * Summary: APIs for management of domains * Description: Provides APIs for the management of domains * - * Copyright (C) 2006-2015 Red Hat, Inc. + * Copyright (C) 2006-2019 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -2420,6 +2420,9 @@ typedef enum { * exists as long as sync is active */ VIR_DOMAIN_BLOCK_JOB_TYPE_ACTIVE_COMMIT = 4,
+ /* Backup (virDomainBackupBegin), job exists until virDomainBackupEnd */ + VIR_DOMAIN_BLOCK_JOB_TYPE_BACKUP = 5, + # ifdef VIR_ENUM_SENTINELS VIR_DOMAIN_BLOCK_JOB_TYPE_LAST # endif @@ -3231,6 +3234,18 @@ int virDomainGetJobStats(virDomainPtr domain, unsigned int flags); int virDomainAbortJob(virDomainPtr dom);
+typedef struct _virDomainJobId virDomainJobId; +typedef virDomainJobId *virDomainJobIdPtr; +struct _virDomainJobId {
Shouldn't this be called just "virDomainJob" ? Id is just one piece of info inside the struct. Should we be making this struct opaque, and adding virDomainJobGetID and virDomainJobGetType accessors, and thne passing a virDomainJobPtr to the other APIs instead of just an id ? It feels safer if virDomainBackupGetXMLDesc were given the full virDomainJobPtr, as then it can validate that the "type" field represents an backup job. This could detect the case where a stale job ID was passed in that now points to a completely different job type.
+ /* One of virDomainJobType */ + int type; + + /* The job id */ + int id; +}; +int virDomainListJobIds(virDomainPtr dom, virDomainJobId **ids, + unsigned int flags); + typedef enum { VIR_DOMAIN_JOB_OPERATION_UNKNOWN = 0, VIR_DOMAIN_JOB_OPERATION_START = 1, @@ -3241,6 +3256,7 @@ typedef enum { VIR_DOMAIN_JOB_OPERATION_SNAPSHOT = 6, VIR_DOMAIN_JOB_OPERATION_SNAPSHOT_REVERT = 7, VIR_DOMAIN_JOB_OPERATION_DUMP = 8, + VIR_DOMAIN_JOB_OPERATION_BACKUP = 9,
# ifdef VIR_ENUM_SENTINELS VIR_DOMAIN_JOB_OPERATION_LAST @@ -3256,6 +3272,14 @@ typedef enum { */ # define VIR_DOMAIN_JOB_OPERATION "operation"
+/** + * VIR_DOMAIN_JOB_ID: + * + * virDomainGetJobStats field: the id of the job (so far, only for jobs + * started by virDomainBackupBegin()), as VIR_TYPED_PARAM_INT. + */ +# define VIR_DOMAIN_JOB_ID "id" + /** * VIR_DOMAIN_JOB_TIME_ELAPSED: * @@ -4080,7 +4104,8 @@ typedef void (*virConnectDomainEventMigrationIterationCallback)(virConnectPtr co * @nparams: size of the params array * @opaque: application specific data * - * This callback occurs when a job (such as migration) running on the domain + * This callback occurs when a job (such as migration or push-model + * virDomainBackupBegin()) running on the domain * is completed. The params array will contain statistics of the just completed * job as virDomainGetJobStats would return. The callback must not free @params * (the array will be freed once the callback finishes). @@ -4876,4 +4901,28 @@ int virDomainGetLaunchSecurityInfo(virDomainPtr domain, int *nparams, unsigned int flags);
+typedef enum { + VIR_DOMAIN_BACKUP_BEGIN_NO_METADATA = (1 << 0), /* Make checkpoint without + remembering it */ + VIR_DOMAIN_BACKUP_BEGIN_QUIESCE = (1 << 1), /* use guest agent to + quiesce all mounted + file systems within + the domain */ +} virDomainBackupBeginFlags; + +/* Begin an incremental backup job, possibly creating a checkpoint. */ +int virDomainBackupBegin(virDomainPtr domain, const char *diskXml, + const char *checkpointXml, unsigned int flags); + +/* Learn about an ongoing backup job. */ +char *virDomainBackupGetXMLDesc(virDomainPtr domain, int id, + unsigned int flags); + +typedef enum { + VIR_DOMAIN_BACKUP_END_ABORT = (1 << 0), /* Abandon a push model backup */ +} virDomainBackupEndFlags; + +/* Complete (or abort) an incremental backup job. */ +int virDomainBackupEnd(virDomainPtr domain, int id, unsigned int flags); + #endif /* LIBVIRT_DOMAIN_H */ diff --git a/src/driver-hypervisor.h b/src/driver-hypervisor.h index e6efce1bfe..3cdd8a9031 100644 --- a/src/driver-hypervisor.h +++ b/src/driver-hypervisor.h @@ -729,6 +729,11 @@ typedef int typedef int (*virDrvDomainAbortJob)(virDomainPtr domain);
+typedef int +(*virDrvDomainListJobIds)(virDomainPtr dom, + virDomainJobId **ids, + unsigned int flags); + typedef int (*virDrvDomainMigrateGetMaxDowntime)(virDomainPtr domain, unsigned long long *downtime, @@ -797,6 +802,12 @@ typedef virDomainSnapshotPtr const char *xmlDesc, unsigned int flags);
+typedef virDomainSnapshotPtr +(*virDrvDomainSnapshotCreateXML2)(virDomainPtr domain, + const char *xmlDesc, + const char *checkpointXml, + unsigned int flags); + typedef char * (*virDrvDomainSnapshotGetXMLDesc)(virDomainSnapshotPtr snapshot, unsigned int flags); @@ -1376,6 +1387,17 @@ typedef int (*virDrvDomainCheckpointDelete)(virDomainCheckpointPtr checkpoint, unsigned int flags);
+typedef int +(*virDrvDomainBackupBegin)(virDomainPtr domain, const char *diskXml, + const char *checkpointXml, unsigned int flags); + +typedef char * +(*virDrvDomainBackupGetXMLDesc)(virDomainPtr domain, int id, + unsigned int flags); + +typedef int +(*virDrvDomainBackupEnd)(virDomainPtr domain, int id, unsigned int flags); + typedef struct _virHypervisorDriver virHypervisorDriver; typedef virHypervisorDriver *virHypervisorDriverPtr;
@@ -1527,6 +1549,7 @@ struct _virHypervisorDriver { virDrvDomainGetJobInfo domainGetJobInfo; virDrvDomainGetJobStats domainGetJobStats; virDrvDomainAbortJob domainAbortJob; + virDrvDomainListJobIds domainListJobIds; virDrvDomainMigrateGetMaxDowntime domainMigrateGetMaxDowntime; virDrvDomainMigrateSetMaxDowntime domainMigrateSetMaxDowntime; virDrvDomainMigrateGetCompressionCache domainMigrateGetCompressionCache; @@ -1541,6 +1564,7 @@ struct _virHypervisorDriver { virDrvDomainManagedSaveGetXMLDesc domainManagedSaveGetXMLDesc; virDrvDomainManagedSaveDefineXML domainManagedSaveDefineXML; virDrvDomainSnapshotCreateXML domainSnapshotCreateXML; + virDrvDomainSnapshotCreateXML2 domainSnapshotCreateXML2; virDrvDomainSnapshotGetXMLDesc domainSnapshotGetXMLDesc; virDrvDomainSnapshotNum domainSnapshotNum; virDrvDomainSnapshotListNames domainSnapshotListNames; @@ -1638,6 +1662,9 @@ struct _virHypervisorDriver { virDrvDomainCheckpointIsCurrent domainCheckpointIsCurrent; virDrvDomainCheckpointHasMetadata domainCheckpointHasMetadata; virDrvDomainCheckpointDelete domainCheckpointDelete; + virDrvDomainBackupBegin domainBackupBegin; + virDrvDomainBackupGetXMLDesc domainBackupGetXMLDesc; + virDrvDomainBackupEnd domainBackupEnd; };
diff --git a/src/qemu/qemu_blockjob.h b/src/qemu/qemu_blockjob.h index c7325c6daf..95f53dde16 100644 --- a/src/qemu/qemu_blockjob.h +++ b/src/qemu/qemu_blockjob.h @@ -55,6 +55,7 @@ typedef enum { QEMU_BLOCKJOB_TYPE_COPY = VIR_DOMAIN_BLOCK_JOB_TYPE_COPY, QEMU_BLOCKJOB_TYPE_COMMIT = VIR_DOMAIN_BLOCK_JOB_TYPE_COMMIT, QEMU_BLOCKJOB_TYPE_ACTIVE_COMMIT = VIR_DOMAIN_BLOCK_JOB_TYPE_ACTIVE_COMMIT, + QEMU_BLOCKJOB_TYPE_BACKUP = VIR_DOMAIN_BLOCK_JOB_TYPE_BACKUP, /* Additional enum values local to qemu */ QEMU_BLOCKJOB_TYPE_INTERNAL, QEMU_BLOCKJOB_TYPE_LAST diff --git a/examples/object-events/event-test.c b/examples/object-events/event-test.c index fcf4492470..98337ad185 100644 --- a/examples/object-events/event-test.c +++ b/examples/object-events/event-test.c @@ -891,6 +891,9 @@ blockJobTypeToStr(int type)
case VIR_DOMAIN_BLOCK_JOB_TYPE_ACTIVE_COMMIT: return "active layer block commit"; + + case VIR_DOMAIN_BLOCK_JOB_TYPE_BACKUP: + return "backup"; }
return "unknown"; diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 5f2b1f68b5..fc45d6c680 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -1092,7 +1092,7 @@ VIR_ENUM_IMPL(virDomainOsDefFirmware, * <mirror> XML (remaining types are not two-phase). */ VIR_ENUM_DECL(virDomainBlockJob); VIR_ENUM_IMPL(virDomainBlockJob, VIR_DOMAIN_BLOCK_JOB_TYPE_LAST, - "", "", "copy", "", "active-commit", + "", "", "copy", "", "active-commit", "", );
VIR_ENUM_IMPL(virDomainMemoryModel, diff --git a/src/libvirt-domain-snapshot.c b/src/libvirt-domain-snapshot.c index 0c8023d9f6..2a50a84417 100644 --- a/src/libvirt-domain-snapshot.c +++ b/src/libvirt-domain-snapshot.c @@ -202,6 +202,10 @@ virDomainSnapshotGetConnect(virDomainSnapshotPtr snapshot) * block copy operation; in that case, use virDomainBlockJobAbort() * to stop the block copy first. * + * To create a checkpoint object that coincides with the snapshot, in + * order to facilitate later incremental backups from the time of the + * snapshot, use virDomainSnapshotCreateXML2(). + * * virDomainSnapshotFree should be used to free the resources after the * snapshot object is no longer needed. * @@ -251,6 +255,91 @@ virDomainSnapshotCreateXML(virDomainPtr domain, }
+/** + * virDomainSnapshotCreateXML2: + * @domain: a domain object + * @xmlDesc: string containing an XML description of the domain snapshot + * @checkpointXml: description of a checkpoint to create or NULL + * @flags: bitwise-OR of virDomainSnapshotCreateFlags + * + * Creates a new snapshot of a domain based on the snapshot xml + * contained in xmlDesc, with a top-level element <domainsnapshot>. + * + * The @checkpointXml parameter is optional; if non-NULL, then libvirt + * behaves as if virDomainCheckpointCreateXML() were called to create + * a checkpoint atomically covering the same point in time as the + * snapshot, using @checkpointXml and forwarding flags + * VIR_DOMAIN_SNAPSHOT_CREATE_QUIESCE and + * VIR_DOMAIN_SNAPSHOT_CREATE_NO_METADATA. The creation of a new + * checkpoint allows for future incremental backups from the time of + * the snapshot. Note that some hypervisors may require a particular + * disk format, such as qcow2, in order to take advantage of + * checkpoints, while allowing arbitrary formats if checkpoints are + * not involved. This parameter is incompatible with the flag + * VIR_DOMAIN_SNAPSHOT_CREATE_REDEFINE. + * + * See virDomainSnapshotCreateXML() for the description of individual + * flags and general behavior. + * + * virDomainSnapshotFree() should be used to free the resources after + * the snapshot object is no longer needed. + * + * Returns an (opaque) new virDomainSnapshotPtr on success or NULL on + * failure. + */ +virDomainSnapshotPtr +virDomainSnapshotCreateXML2(virDomainPtr domain, + const char *xmlDesc, + const char *checkpointXml, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "xmlDesc=%s, checkpointXml=%s, flags=0x%x", + xmlDesc, checkpointXml, flags); + + virResetLastError(); + + virCheckDomainReturn(domain, NULL); + conn = domain->conn; + + virCheckNonNullArgGoto(xmlDesc, error); + virCheckReadOnlyGoto(conn->flags, error); + + VIR_REQUIRE_FLAG_GOTO(VIR_DOMAIN_SNAPSHOT_CREATE_CURRENT, + VIR_DOMAIN_SNAPSHOT_CREATE_REDEFINE, + error); + + VIR_EXCLUSIVE_FLAGS_GOTO(VIR_DOMAIN_SNAPSHOT_CREATE_REDEFINE, + VIR_DOMAIN_SNAPSHOT_CREATE_NO_METADATA, + error); + VIR_EXCLUSIVE_FLAGS_GOTO(VIR_DOMAIN_SNAPSHOT_CREATE_REDEFINE, + VIR_DOMAIN_SNAPSHOT_CREATE_HALT, + error); + + if (checkpointXml && (flags & VIR_DOMAIN_SNAPSHOT_CREATE_REDEFINE)) { + virReportInvalidArg(checkpointXml, "%s", + _("Cannot create checkpoint when redefining " + "snapshot")); + goto error; + } + + if (conn->driver->domainSnapshotCreateXML2) { + virDomainSnapshotPtr ret; + ret = conn->driver->domainSnapshotCreateXML2(domain, xmlDesc, + checkpointXml, flags); + if (!ret) + goto error; + return ret; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return NULL; +} + + /** * virDomainSnapshotGetXMLDesc: * @snapshot: a domain snapshot object diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index 1ff4962fc1..45d187099e 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -1,7 +1,7 @@ /* * libvirt-domain.c: entry points for virDomainPtr APIs * - * Copyright (C) 2006-2015 Red Hat, Inc. + * Copyright (C) 2006-2019 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -8726,8 +8726,9 @@ virDomainIsUpdated(virDomainPtr dom) * @domain: a domain object * @info: pointer to a virDomainJobInfo structure allocated by the user * - * Extract information about progress of a background job on a domain. - * Will return an error if the domain is not active. + * Extract information about progress of the default background job + * (id 0) on a domain. Will return an error if the domain is not + * active. * * This function returns a limited amount of information in comparison * to virDomainGetJobStats(). @@ -8775,13 +8776,13 @@ virDomainGetJobInfo(virDomainPtr domain, virDomainJobInfoPtr info) * @nparams: number of items in @params * @flags: bitwise-OR of virDomainGetJobStatsFlags * - * Extract information about progress of a background job on a domain. - * Will return an error if the domain is not active. The function returns - * a superset of progress information provided by virDomainGetJobInfo. - * Possible fields returned in @params are defined by VIR_DOMAIN_JOB_* - * macros and new fields will likely be introduced in the future so callers - * may receive fields that they do not understand in case they talk to a - * newer server. + * Extract information about progress of the default background job + * (id 0) on a domain. Will return an error if the domain is not + * active. The function returns a superset of progress information + * provided by virDomainGetJobInfo(). Possible fields returned in + * @params are defined by VIR_DOMAIN_JOB_* macros and new fields will + * likely be introduced in the future so callers may receive fields + * that they do not understand in case they talk to a newer server. * * When @flags contains VIR_DOMAIN_JOB_STATS_COMPLETED, the function will * return statistics about a recently completed job. Specifically, this @@ -8839,7 +8840,7 @@ virDomainGetJobStats(virDomainPtr domain, * virDomainAbortJob: * @domain: a domain object * - * Requests that the current background job be aborted at the + * Requests that the current default background job (id 0) be aborted at the * soonest opportunity. In case the job is a migration in a post-copy mode, * virDomainAbortJob will report an error (see virDomainMigrateStartPostCopy * for more details). @@ -8876,6 +8877,57 @@ virDomainAbortJob(virDomainPtr domain) }
+/** + * virDomainListJobIds: + * @domain: a domain object + * @ids: Pointer to a variable to store the array containing job ids or NULL + * if the list is not required (just returns number of jobs). + * @flags: extra flags; not used yet, so callers should always pass 0 + * + * Collect a list of all background jobs, and return an allocated + * array of information about the type and id of each. + * + * The default background job (id 0, which is typically migration) + * might not be included in the list; for that, use
This line makes me a little uncomfortable ? Why would we exclude the default background job (sometimes) ? I feel it is preferrable to always return all jobs.
+ * virDomainJobStats(). The job id may be important to other APIs + * related to the job type; for example, a backup job id (created by + * virDomainBackupBegin()) is required for calling + * virDomainBackupEnd(). + * + * Returns the number of jobs found or -1 and sets @ids to NULL in + * case of error. The caller is responsible for calling free() on + * @ids. + */ +int +virDomainListJobIds(virDomainPtr domain, + virDomainJobId **ids, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain); + + virResetLastError(); + + virCheckDomainReturn(domain, -1); + conn = domain->conn; + + if (conn->driver->domainListJobIds) { + int ret; + ret = conn->driver->domainListJobIds(domain, ids, flags); + if (ret < 0) + goto error; + return ret; + } + + virReportUnsupportedError(); + + error: + virDispatchError(conn); + return -1; +} + + /** * virDomainMigrateSetMaxDowntime: * @domain: a domain object @@ -10341,6 +10393,12 @@ virDomainBlockRebase(virDomainPtr dom, const char *disk, * over the destination format, the ability to copy to a destination that * is not a local file, and the possibility of additional tuning parameters. * + * The copy created by this API is not finalized until the job ends, + * and does not lend itself to incremental backups (beyond what + * VIR_DOMAIN_BLOCK_COPY_SHALLOW provides) nor to third-party control + * over the data being copied. For those features, use + * virDomainBackupBegin(). + * * Returns 0 if the operation has started, -1 on failure. */ int @@ -12361,3 +12419,208 @@ int virDomainGetLaunchSecurityInfo(virDomainPtr domain, virDispatchError(domain->conn); return -1; } + + +/** + * virDomainBackupBegin: + * @domain: a domain object + * @diskXml: description of storage to utilize and expose during + * the backup, or NULL + * @checkpointXml: description of a checkpoint to create, or NULL + * @flags: bitwise-OR of supported virDomainBackupBeginFlags + * + * Start a point-in-time backup job for the specified disks of a + * running domain. + * + * A backup job is mutually exclusive with domain migration + * (particularly when the job sets up an NBD export, since it is not + * possible to tell any NBD clients about a server migrating between + * hosts). For now, backup jobs are also mutually exclusive with any + * other block job on the same device, although this restriction may + * be lifted in a future release. Progress of the backup job can be + * tracked via virDomainGetJobStats(). The job remains active until a + * subsequent call to virDomainBackupEnd(), even if it no longer has + * anything to copy. + * + * This API differs from virDomainBlockCopy() because it can grab the + * state of more than one disk in parallel, and because the state is + * captured as of the start of the job, rather than the end. + * + * There are two fundamental backup approaches. The first, called a + * push model, instructs the hypervisor to copy the state of the guest + * disk to the designated storage destination (which may be on the + * local file system or a network device). In this mode, the + * hypervisor writes the content of the guest disk to the destination, + * then emits VIR_DOMAIN_EVENT_ID_JOB_COMPLETED when the backup is + * either complete or failed (the backup image is invalid if the job + * fails or virDomainBackupEnd() is used prior to the event being + * emitted). + * + * The second, called a pull model, instructs the hypervisor to expose + * the state of the guest disk over an NBD export. A third-party + * client can then connect to this export and read whichever portions + * of the disk it desires. In this mode, there is no event; libvirt + * has to be informed via virDomainBackupEnd() when the third-party + * NBD client is done and the backup resources can be released. + * + * The @diskXml parameter is optional but usually provided and + * contains details about the backup in the top-level element + * <domainbackup> , including which backup mode to use, whether the + * backup is incremental from a previous checkpoint, which disks + * participate in the backup, the destination for a push model backup, + * and the temporary storage and NBD server details for a pull model + * backup. If omitted, the backup attempts to default to a push mode + * full backup of all disks, where libvirt generates a filename for + * each disk by appending a suffix of a timestamp in seconds since the + * Epoch. virDomainBackupGetXMLDesc() can be called to learn actual + * values selected. For more information, see + * formatcheckpoint.html#BackupAttributes. + * + * The @checkpointXml parameter is optional; if non-NULL, then libvirt + * behaves as if virDomainCheckpointCreateXML() were called to create + * a checkpoint atomically covering the same point in time as the + * backup, using @checkpointXml and forwarding flags + * VIR_DOMAIN_BACKUP_BEGIN_QUIESCE and + * VIR_DOMAIN_BACKUP_BEGIN_NO_METADATA. The creation of a new + * checkpoint allows for future incremental backups. Note that some + * hypervisors may require a particular disk format, such as qcow2, in + * order to take advantage of checkpoints, while allowing arbitrary + * formats if checkpoints are not involved. + * + * Returns a non-negative job id on success or negative on failure. + * This id is then passed to virDomainBackupGetXMLDesc() and + * virDomainBackupEnd(); it can also be obtained from + * virDomainListJobIds(). This operation returns quickly, such that a + * user can choose to start a backup job between virDomainFSFreeze() + * and virDomainFSThaw() in order to create the backup while guest I/O + * is quiesced. + */ +int +virDomainBackupBegin(virDomainPtr domain, + const char *diskXml, + const char *checkpointXml, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "diskXml=%s, checkpointXml=%s, flags=0x%x", + NULLSTR(diskXml), NULLSTR(checkpointXml), flags); + + virResetLastError(); + + virCheckDomainReturn(domain, -1); + conn = domain->conn; + + virCheckReadOnlyGoto(conn->flags, error); + if (flags & VIR_DOMAIN_BACKUP_BEGIN_NO_METADATA) + virCheckNonNullArgGoto(checkpointXml, error); + + if (conn->driver->domainBackupBegin) { + int ret; + ret = conn->driver->domainBackupBegin(domain, diskXml, checkpointXml, + flags); + if (!ret) + goto error; + return ret; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return -1; +} + + +/** + * virDomainBackupGetXMLDesc: + * @domain: a domain object + * @id: the id of an active backup job previously started with + * virDomainBackupBegin() + * @flags: extra flags; not used yet, so callers should always pass 0 + * + * In some cases, a user can start a backup job without supplying all + * details and rely on libvirt to fill in the rest (for example, + * selecting the port used for an NBD export). This API can then be + * used to learn what default values were chosen. At present, none of + * the information provided is security sensitive. + * + * Returns a NUL-terminated UTF-8 encoded XML instance or NULL in + * case of error. The caller must free() the returned value. + */ +char * +virDomainBackupGetXMLDesc(virDomainPtr domain, int id, unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "id=%d, flags=0x%x", id, flags); + + virResetLastError(); + + virCheckDomainReturn(domain, NULL); + conn = domain->conn; + + virCheckNonNegativeArgGoto(id, error); + + if (conn->driver->domainBackupGetXMLDesc) { + char *ret; + ret = conn->driver->domainBackupGetXMLDesc(domain, id, flags); + if (!ret) + goto error; + return ret; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return NULL; +} + + +/** + * virDomainBackupEnd: + * @domain: a domain object + * @id: the id of an active backup job previously started with + * virDomainBackupBegin() + * @flags: bitwise-OR of supported virDomainBackupEndFlags + * + * Conclude a point-in-time backup job @id on the given domain. + * + * If the backup job uses the push model, but the event marking that + * all data has been copied has not yet been emitted, then the command + * fails unless @flags includes VIR_DOMAIN_BACKUP_END_ABORT. If the + * event has been issued, or if the backup uses the pull model, the + * flag has no effect. + * + * Returns 1 if the backup job completed successfully (the backup + * destination file in a push model is consistent), 0 if the job was + * aborted successfully (only when VIR_DOMAIN_BACKUP_END_ABORT is + * passed; the destination file is unusable), and -1 on failure. + */ +int +virDomainBackupEnd(virDomainPtr domain, int id, unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "id=%d, flags=0x%x", id, flags); + + virResetLastError(); + + virCheckDomainReturn(domain, -1); + conn = domain->conn; + + virCheckReadOnlyGoto(conn->flags, error); + virCheckNonNegativeArgGoto(id, error); + + if (conn->driver->domainBackupEnd) { + int ret; + ret = conn->driver->domainBackupEnd(domain, id, flags); + if (!ret) + goto error; + return ret; + } + + virReportUnsupportedError(); + error: + virDispatchError(conn); + return -1; +} diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index d026b33f53..d13df6bb67 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -817,6 +817,9 @@ LIBVIRT_4.10.0 { LIBVIRT_5.2.0 { global: virConnectGetStoragePoolCapabilities; + virDomainBackupBegin; + virDomainBackupEnd; + virDomainBackupGetXMLDesc; virDomainCheckpointCreateXML; virDomainCheckpointCurrent; virDomainCheckpointDelete; @@ -833,6 +836,8 @@ LIBVIRT_5.2.0 { virDomainCheckpointRef; virDomainHasCurrentCheckpoint; virDomainListCheckpoints; + virDomainListJobIds; + virDomainSnapshotCreateXML2; } LIBVIRT_4.10.0;
Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On 3/26/19 12:08 PM, Daniel P. Berrangé wrote:
On Tue, Mar 26, 2019 at 01:13:52AM -0500, Eric Blake wrote:
Introduce a few more new public APIs related to incremental backups. This builds on the previous notion of a checkpoint (without an existing checkpoint, the new API is a full backup, differing from virDomainBlockCopy in the point of time chosen and in operation on multiple disks at once); and also allows creation of a new checkpoint at the same time as starting the backup (after all, an incremental backup is only useful if it covers the state since the previous backup). Snapshot creation is also a point in time at which creating a checkpoint atomically can be useful. A backup job also affects filtering a listing of domains, as well as adding event reporting for signaling when a push model backup completes (where the hypervisor creates the backup); note that the pull model does not have an event (starting the backup lets a third party access the data, and only the third party knows when it is finished).
Since multiple backup jobs can be run in parallel in the future (well, qemu doesn't support it yet, but we don't want to preclude the idea), virDomainBackupBegin() returns a job id, which can also be queried by virDomainListJobIds(), and this job id must be used for virDomainBackupGetXMLDesc() and virDomainBackupEnd(). In the future, we may also extend other jobs (migration as the default global job impacting virDomainJobStats(), and the various block jobs) to all have ids, where the existing APIs act like thin wrappers around more powerful APIs that support a job id everywhere.
This is the part that has me most worried about 5.2 - anything that we find a future libvirt supplying about job management APIs will not be backported without a .so bump if it didn't make it into 5.2. So the hope is that whatever lands now is usable (even if only one job at a time) by whatever downstreams pick 5.2 as their starting point, without wishing we had additional Job apis available.
The full list of new API: virDomainBackupBegin; virDomainBackupEnd; virDomainBackupGetXMLDesc; virDomainListJobIds; virDomainSnapshotCreateXML2;
10 files changed, 463 insertions(+), 17 deletions(-)
+virDomainSnapshotPtr virDomainSnapshotCreateXML2(virDomainPtr domain, + const char *xmlDesc, + const char *snapshotXml, + unsigned int flags);
s/snapshotXml/checkpointXml/ based on later docs.
Correct.
On IRC, you had said an alternative would be to put the checkpointXml as a <domaincheckpoint> child of the main <domainsnapshot> or <domainbackup> XML document.
IIUC, the <domaincheckpoint> XML is merely forwarded on the checkpoint APIs. IOW, if you later call virDomainSnapshotGetXMLDesc, you would *not* expect to see the <domaincheckpoint> child again ? If that is correct, then having it via the separate API parameter makes more sense than as a XML child element. I'd only want it as an XML child if that where the canonical representation & storage location. So the separate API looks ok to me.
All right, that answers a question I've had for a while. You are correct that the <domaincheckpoint> is independent from the snapshot, and you would NOT expect to see it under the listing of a <domainsnapshot> after the fact. Embedding things also makes it conceptually harder for <incremental>name</incremental> to refer back to a particular checkpoint if checkpoints are not independent objects, but if they ARE independent objects, then embedding them just to avoid a new API seems fishy. So my only remaining question is if there is any better name than virDomainSnapshotCreateXML2(), but since I modeled it after virDomainMigrate[23](), I think we're okay.
@@ -3231,6 +3234,18 @@ int virDomainGetJobStats(virDomainPtr domain, unsigned int flags); int virDomainAbortJob(virDomainPtr dom);
+typedef struct _virDomainJobId virDomainJobId; +typedef virDomainJobId *virDomainJobIdPtr; +struct _virDomainJobId {
Shouldn't this be called just "virDomainJob" ? Id is just one piece of info inside the struct.
Should we be making this struct opaque, and adding virDomainJobGetID and virDomainJobGetType accessors, and thne passing a virDomainJobPtr to the other APIs instead of just an id ?
It's a late request on a late API addition, but it sounds reasonable. We have the benefit that it is NOT a type that has to be backed by on-disk XML, so while it requires more patching to datatypes.h and figuring out how to express the type over RPC, it should be a lot less grunt-work than what I've been doing with CheckpointXML and to a lesser extent Backup XML.
It feels safer if virDomainBackupGetXMLDesc were given the full virDomainJobPtr, as then it can validate that the "type" field represents an backup job. This could detect the case where a stale job ID was passed in that now points to a completely different job type.
Interesting idea; and not that much harder (s/int/virDomainJobPtr/).
+ /* One of virDomainJobType */ + int type; + + /* The job id */ + int id; +}; +int virDomainListJobIds(virDomainPtr dom, virDomainJobId **ids, + unsigned int flags);
And this would be virDomainJobIdPtr **ids, where the caller then has to free both each id pointer and the overall list (but we have plenty of other opaque types to copy that practice from, including how the generator would work with it). Naming wise, I guess the most consistent name would be virDomainListAllJobs?
+/** + * virDomainListJobIds: + * @domain: a domain object + * @ids: Pointer to a variable to store the array containing job ids or NULL + * if the list is not required (just returns number of jobs). + * @flags: extra flags; not used yet, so callers should always pass 0 + * + * Collect a list of all background jobs, and return an allocated + * array of information about the type and id of each. + * + * The default background job (id 0, which is typically migration) + * might not be included in the list; for that, use
This line makes me a little uncomfortable ? Why would we exclude the default background job (sometimes) ? I feel it is preferrable to always return all jobs.
Okay, I'll drop the line. As I have not yet done a full implementation of job listing, it shouldn't be too hard to get that right (and the fact that right now, we only support 1 [migration], soon to be 2 [backup], orthogonal domain jobs that can't be run simultaneously, plus an additional maximum of 1 block job per domain disk, means it's still not hard to get an initial version correct). I'll go ahead and push the patches for Checkpoint APIs in time for the 5.2 freeze tomorrow, as well as post another round of the Backup APIs later tonight to see how far I can get with the idea of virDomainJobPtr being an opaque type. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org

On 3/26/19 1:13 PM, Eric Blake wrote: TL;DR: After more thought, I'm reluctant to make a domain job an opaque type. I'm also leaning towards withdrawing virDomainListJobIds() from the set of APIs I'm proposing for 5.2, on the grounds that with a documented magic job id of '0' that means "the currently running job" for both virDomainBackupGetXMLDesc() and virDomainBackupEnd(), we don't lose any functionality over what qemu currently provides (of no parallel jobs) while still having a sane API that can be properly used with non-zero job ids in the future when we do add parallel job support.
Since multiple backup jobs can be run in parallel in the future (well, qemu doesn't support it yet, but we don't want to preclude the idea), virDomainBackupBegin() returns a job id, which can also be queried by virDomainListJobIds(), and this job id must be used for virDomainBackupGetXMLDesc() and virDomainBackupEnd(). In the future, we may also extend other jobs (migration as the default global job impacting virDomainJobStats(), and the various block jobs) to all have ids, where the existing APIs act like thin wrappers around more powerful APIs that support a job id everywhere.
This is the part that has me most worried about 5.2 - anything that we find a future libvirt supplying about job management APIs will not be backported without a .so bump if it didn't make it into 5.2. So the hope is that whatever lands now is usable (even if only one job at a time) by whatever downstreams pick 5.2 as their starting point, without wishing we had additional Job apis available.
@@ -3231,6 +3234,18 @@ int virDomainGetJobStats(virDomainPtr domain, unsigned int flags); int virDomainAbortJob(virDomainPtr dom);
+typedef struct _virDomainJobId virDomainJobId; +typedef virDomainJobId *virDomainJobIdPtr; +struct _virDomainJobId {
Shouldn't this be called just "virDomainJob" ? Id is just one piece of info inside the struct.
Should we be making this struct opaque, and adding virDomainJobGetID and virDomainJobGetType accessors, and thne passing a virDomainJobPtr to the other APIs instead of just an id ?
It's a late request on a late API addition, but it sounds reasonable. We have the benefit that it is NOT a type that has to be backed by on-disk XML, so while it requires more patching to datatypes.h and figuring out how to express the type over RPC, it should be a lot less grunt-work than what I've been doing with CheckpointXML and to a lesser extent Backup XML.
It feels safer if virDomainBackupGetXMLDesc were given the full virDomainJobPtr, as then it can validate that the "type" field represents an backup job. This could detect the case where a stale job ID was passed in that now points to a completely different job type.
Interesting idea; and not that much harder (s/int/virDomainJobPtr/).
Actually, it turns out to make life a bit more difficult if the type is opaque. My demo at KVM Forum showed the following: $ virsh backup-begin $dom backup_pull_full.xml Backup id 1 created backup used description from 'backup_pull_full.xml' $ # do thired-party copy $ virsh backup-end $dom 1 Backup id 1 completed Note the time and process separation between creation and completion; if we make virDomainBackupEnd() take an opaque pointer, then the first virsh instance has to call virDomainJobFree(job) prior to exiting (well, that depends a lot on whether we make virDomainJobPtr light-weight scalar-only that just records type/id where a leak is harmless, or heavy-weight where it also records the virConnectionPtr and/or virDomainPtr it was associated for better API safety, but where freeing the pointer is important in a long-running client). Then, the second virsh call would have to call virDomainJobLookupId(domain, 1) to re-obtain the proper virDomainJobPtr object to hand to virDomainBackupEnd(domain, job), and then call virDomainJobFree(ptr) again to conclude things. Thus, making it an opaque type adds a lot of API overhead to using things, compared to the interface that virsh wants to give the user of just printing an integer and letting the user re-input that integer; we also have to worry about whether multiple consecutive API calls can result in TOCTTOU races that could confuse things. You raised the question of whether storing type/id as an opaque type would add any safety to prevent the user from requesting an API on an unrelated job ID. But after thinking about it more, my argument is that it doesn't really help - when calling virDomainBackupEnd(domain, 1), we've already implied that we want to operate on backup job 1 (and throw an error if there is no backup job with id 1, even if there is some other job type that overlaps namespaces and has an id 1). Calling virDomainBackupEnd(domain, virDomainJobCompose(domain, VIR_DOMAIN_JOB_BACKUP, 1)) adds verbosity, but I don't see it adding safety. Along those lines we can look at file descriptors as an example - they are plain integers, but still serve as an opaque type (you use fcntl(F_GETFD/F_GETFL) to learn things about the fd, and have various errors like ESPIPE or EISDIR when operating on the wrong id compared to the function trying to do something on an fd). Then there's the question of what precedence we have for opaque vs. user-visible types in public headers. We already expose virTypedParameter as a user-visible type rather than opaque, and the more time I've spent on the code this afternoon playing with the idea of making jobs full-bodied and opaque, the more I'm finding it to be too much overhead. For example, the python generator can easily support ints, but needs a lot more modifications to support a new opaque type. If we insist on making virDomainBackupBegin() return an opaque type instead of an integer id, I'm very doubtful on whether the changes to libvirt-python would be made in time for making the 5.2 release. Thinking for the future, when we eventually do get around to adding full parallel job ID support, it will touch a lot of APIs: - virDomainGetJobInfo(), virDomainGetJobStats(), and virDomainAbortJob() - currently operate on a default job (in particular, the only job actually affected by these are jobs started by virDomainMigrate() and friends) - virDomainBlockJobAbort(), virDomainGetBlockJobInfo(), virDomainBlockJobSetSpeed() - currently act on a single job per disk, where multiple jobs can be run in parallel if you have multiple disks, but again with no id; affects jobs started by virDomainBlockCopy(), virDomainBlockCommit(), virDomainBlockPull(), and friends - my new virDomainBlockBackup() - which I'm trying to design for the future by returning a job id - we may want to add a virDomainJobGetDefaultID(domain) or even virDomainBlockJobGetID(domain, diskname) to make it easy to reverse-engineer which job id was utilized by the legacy APIs, but at the same time would want new APIs that return the job id directly when creating one of those jobs - we may want to add a virDomainJobSetDefaultID(domain, id) to select which job (out of multiple running in parallel) will be targetted by the existing job functions that don't take a job id - side note: virDomainAbortJob() vs. virDomainBlockJobAbort() is confusing; it's hard to remember which name form to use So, that said, what are the options for 5.2? If we commit virDomainBackupEnd(virDomainPtr, int job, flags) in time for Wednesday's freeze, we can still change the API (go with a different parameter type) and/or revert the addition up until Friday. If the release happens with virDomainBackupEnd() but no virDomainListAllJobs(), we can still run backup jobs (you just can't easily query what job id was chosen, but have to assume a default job). But I could also document that although virDomainBackupBegin() always returns a non-zero job id on success, we guarantee that the reserved job id '0' stands for "the currently-running job". If there is no active backup job, or if there are more than one active backup jobs, then id "0" will fail - but we can also document that if a server does not support the virDomainListAllJobs() API, then that server does not support parallel active backup jobs. Thus, the use of a job id is a formality for 5.2 (we know that qemu won't have parallel jobs, so we can blindly pass 0 for the job id to all calls that document a job id - but still code virsh to allow the user to pass job id '1' or whatever the output from backup-begin said if they don't want to rely on the default of job '0'). At the same time, down the road (in release 6.0 or whenever we finally add parallel job ids) that we DO finally add virDomainListAllJobs() and all the other APIs needed for true job management, we've avoided needing to add wrapper APIs to fix virDomainBackupBegin/End not working with job ids (even though we have to add wrappers for all the other legacy job-related APIs that did not work with ids from the get-go). We could decide that a job id is too much for right now, and state that virDomainBackupBegin() returns 0 on success rather than a job id (for exactly one job possible), and where virDomainBackupEnd() no longer needs a job id parameter (in other words, making it more like existing virDomainMigrate()/virDomainAbortJob()). Then future parallel jobs would have to add a new API for backup as well as everything else (less work now, and at least consistent with other APIs, but more work down the road). I'm less in favor of this option, because it feels like adding technical debt. If the release happens now with both virDomainBackupEnd() taking a job id, AND the addition of a virDomainListAllJobs() API, then we are stuck supporting the virDomainListAllJobs() API even if it turns out we don't like the design it locked us into. But if we think we have a sufficiently extensible design for that one API, even without the rest of a nice parallel job framework, and feel we can reasonably add remaining job API enhancements later, then more power to us for getting it right. So having said all that, I'm leaning towards keeping job ids as an integer, having no virDomainListAllJobs() in this release, and adding sufficient documentation and magic around a reserved job id '0' so that callers can start experimenting with backup jobs in 5.2. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org

Now that various new API have been added, it is worth a landing page that gives an overview of capturing various pieces of guest state, and which APIs are best suited to which tasks. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com> --- docs/docs.html.in | 5 + docs/domainstatecapture.html.in | 313 ++++++++++++++++++++++++++++++++ docs/formatbackup.html.in | 4 +- docs/formatcheckpoint.html.in | 4 +- docs/formatsnapshot.html.in | 2 + 5 files changed, 326 insertions(+), 2 deletions(-) create mode 100644 docs/domainstatecapture.html.in diff --git a/docs/docs.html.in b/docs/docs.html.in index 400b149791..8c08ace402 100644 --- a/docs/docs.html.in +++ b/docs/docs.html.in @@ -124,6 +124,11 @@ <dt><a href="secureusage.html">Secure usage</a></dt> <dd>Secure usage of the libvirt APIs</dd> + + <dt><a href="domainstatecapture.html">Domain state + capture</a></dt> + <dd>Comparison between different methods of capturing domain + state</dd> </dl> </div> diff --git a/docs/domainstatecapture.html.in b/docs/domainstatecapture.html.in new file mode 100644 index 0000000000..27a72a70b5 --- /dev/null +++ b/docs/domainstatecapture.html.in @@ -0,0 +1,313 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE html> +<html xmlns="http://www.w3.org/1999/xhtml"> + <body> + + <h1>Domain state capture using Libvirt</h1> + + <ul id="toc"></ul> + + <p> + In order to aid application developers to choose which + operations best suit their needs, this page compares the + different means for capturing state related to a domain managed + by libvirt. + </p> + + <p> + The information here is primarily geared towards capturing the + state of an active domain. Capturing the state of an inactive + domain essentially amounts to copying the contents of guest + disks, followed by a fresh boot with disks restored to that + state. + </p> + + <h2><a id="definitions">State capture trade-offs</a></h2> + + <p>One of the features made possible with virtual machines is live + migration -- transferring all state related to the guest from + one host to another with minimal interruption to the guest's + activity. In this case, state includes domain memory (including + register and device contents), and domain storage (whether the + guest's view of the disks are backed by local storage on the + host, or by the hypervisor accessing shared storage over a + network). A clever observer will then note that if all state is + available for live migration, then there is nothing stopping a + user from saving some or all of that state at a given point of + time in order to be able to later rewind guest execution back to + the state it previously had. The astute reader will also realize + that state capture at any level requires that the data must be + stored and managed by some mechanism. This processing might fit + in a single file, or more likely require a chain of related + files, and may require synchronization with third-party tools + built around managing the amount of data resulting from + capturing the state of multiple guests that each use multiple + disks. + </p> + + <p> + There are several libvirt APIs associated with capturing the + state of a guest, which can later be used to rewind that guest + to the conditions it was in earlier. The following is a list of + trade-offs and differences between the various facets that + affect capturing domain state for active domains: + </p> + + <dl> + <dt>Duration</dt> + <dd>Capturing state can be a lengthy process, so while the + captured state ideally represents an atomic point in time + corresponding to something the guest was actually executing, + capturing state tends to focus on minimizing guest downtime + while performing the rest of the state capture in parallel + with guest execution. Some interfaces require up-front + preparation (the state captured is not complete until the API + ends, which may be some time after the command was first + started), while other interfaces track the state when the + command was first issued, regardless of the time spent in + capturing the rest of the state. Also, time spent in state + capture may be longer than the time required for live + migration, when state must be duplicated rather than shared. + </dd> + + <dt>Amount of state</dt> + <dd>For an online guest, there is a choice between capturing the + guest's memory (all that is needed during live migration when + the storage is already shared between source and destination), + the guest's disk state (all that is needed if there are no + pending guest I/O transactions that would be lost without the + corresponding memory state), or both together. Reverting to + partial state may still be viable, but typically, booting from + captured disk state without corresponding memory is comparable + to rebooting a machine that had power cut before I/O could be + flushed. Guests may need to use proper journaling methods to + avoid problems when booting from partial state. + </dd> + + <dt>Quiescing of data</dt> + <dd>Even if a guest has no pending I/O, capturing disk state may + catch the guest at a time when the contents of the disk are + inconsistent. Cooperating with the guest to perform data + quiescing is an optional step to ensure that captured disk + state is fully consistent without requiring additional memory + state, rather than just crash-consistent. But guest + cooperation may also have time constraints, where the guest + can rightfully panic if there is too much downtime while I/O + is frozen. + </dd> + + <dt>Quantity of files</dt> + <dd>When capturing state, some approaches store all state within + the same file (internal), while others expand a chain of + related files that must be used together (external), for more + files that a management application must track. + </dd> + + <dt>Impact to guest definition</dt> + <dd>Capturing state may require temporary changes to the guest + definition, such as associating new files into the domain + definition. While state capture should never impact the + running guest, a change to the domain's active XML may have + impact on other host operations being performed on the domain. + </dd> + + <dt>Third-party integration</dt> + <dd>When capturing state, there are tradeoffs to how much of the + process must be done directly by the hypervisor, and how much + can be off-loaded to third-party software. Since capturing + state is not instantaneous, it is essential that any + third-party integration see consistent data even if the + running guest continues to modify that data after the point in + time of the capture.</dd> + + <dt>Full vs. incremental</dt> + <dd>When periodically repeating the action of state capture, it + is useful to minimize the amount of state that must be + captured by exploiting the relation to a previous capture, + such as focusing only on the portions of the disk that the + guest has modified in the meantime. Some approaches are able + to take advantage of checkpoints to provide an incremental + backup, while others are only capable of a full backup even if + that means re-capturing unchanged portions of the disk.</dd> + + <dt>Local vs. remote</dt> + <dd>Domains that completely use remote storage may only need + some mechanism to keep track of guest memory state while using + external means to manage storage. Still, hypervisor and guest + cooperation to ensure points in time when no I/O is in flight + across the network can be important for properly capturing + disk state.</dd> + + <dt>Network latency</dt> + <dd>Whether it's domain storage or saving domain state into + remote storage, network latency has an impact on snapshot + data. Having dedicated network capacity, bandwidth, or quality + of service levels may play a role, as well as planning for how + much of the backup process needs to be local.</dd> + </dl> + + <p> + An example of the various facets in action is migration of a + running guest. In order for the guest to be able to resume on + the destination at the same place it left off at the source, the + hypervisor has to get to a point where execution on the source + is stopped, the last remaining changes occurring since the + migration started are then transferred, and the guest is started + on the target. The management software thus must keep track of + the starting point and any changes since the starting + point. These last changes are often referred to as dirty page + tracking or dirty disk block bitmaps. At some point in time + during the migration, the management software must freeze the + source guest, transfer the dirty data, and then start the guest + on the target. This period of time must be minimal. To minimize + overall migration time, one is advised to use a dedicated + network connection with a high quality of service. Alternatively + saving the current state of the running guest can just be a + point in time type operation which doesn't require updating the + "last vestiges" of state prior to writing out the saved state + file. The state file is the point in time of whatever is current + and may contain incomplete data which if used to restart the + guest could cause confusion or problems because some operation + wasn't completed depending upon where in time the operation was + commenced. + </p> + + <h2><a id="apis">State capture APIs</a></h2> + <p>With those definitions, the following libvirt APIs related to + state capture have these properties:</p> + <dl> + <dt><a href="html/libvirt-libvit-domain.html#virDomainManagedSave"><code>virDomainManagedSave</code></a></dt> + <dd>This API saves guest memory, with libvirt managing all of + the saved state, then stops the guest. While stopped, the + disks can be copied by a third party. However, since any + subsequent restart of the guest by libvirt API will restore + the memory state (which typically only works if the disk state + is unchanged in the meantime), and since it is not possible to + get at the memory state that libvirt is managing, this is not + viable as a means for rolling back to earlier saved states, + but is rather more suited to situations such as suspending a + guest prior to rebooting the host in order to resume the guest + when the host is back up. This API also has a drawback of + potentially long guest downtime, and therefore does not lend + itself well to live backups.</dd> + + <dt><a href="html/libvirt-libvit-domain.html#virDomainSave"><code>virDomainSave</code></a></dt> + <dd>This API is similar to virDomainManagedSave(), but moves the + burden on managing the stored memory state to the user. As + such, the user can now couple saved state with copies of the + disks to perform a revert to an arbitrary earlier saved state. + However, changing who manages the memory state does not change + the drawback of potentially long guest downtime when capturing + state.</dd> + + <dt><a href="html/libvirt-libvit-domain-snapshot.html#virDomainSnapshotCreateXML"><code>virDomainSnapshotCreateXML</code></a></dt> + <dd>This API wraps several approaches for capturing guest state, + with a general premise of creating a snapshot (where the + current guest resources are frozen in time and a new wrapper + layer is opened for tracking subsequent guest changes). It + can operate on both offline and running guests, can choose + whether to capture the state of memory, disk, or both when + used on a running guest, and can choose between internal and + external storage for captured state. However, it is geared + towards post-event captures (when capturing both memory and + disk state, the disk state is not captured until all memory + state has been collected first). Using QEMU as the + hypervisor, internal snapshots currently have lengthy downtime + that is incompatible with freezing guest I/O, but external + snapshots are quick. Since creating an external snapshot + changes which disk image resource is in use by the guest, this + API can be coupled with <a href="html/libvirt-libvirt-domain.html#virDomainBlockCommit"><code>virDomainBlockCommit()</code></a> to + restore things back to the guest using its original disk + image, where a third-party tool can read the backing file + prior to the live commit. See also + the <a href="formatsnapshot.html">XML details</a> used with + this command.</dd> + + <dt><a href="html/libvirt-libvit-domain.html#virDomainFSFreeze"><code>virDomainFSFreeze</code></a>, <a href="html/libvirt-libvit-domain.html#virDomainFSThaw"><code>virDomainFSThaw</code></a></dt> + <dd>This pair of APIs does not directly capture guest state, but + can be used to coordinate with a trusted live guest that state + capture is about to happen, and therefore guest I/O should be + quiesced so that the state capture is fully consistent, rather + than merely crash consistent. Some APIs are able to + automatically perform a freeze and thaw via a flags parameter, + rather than having to make separate calls to these + functions. Also, note that freezing guest I/O is only possible + with trusted guests running a guest agent, and that some + guests place maximum time limits on how long I/O can be + frozen.</dd> + + <dt><a href="html/libvirt-libvit-domain.html#virDomainBlockCopy"><code>virDomainBlockCopy</code></a></dt> + <dd>This API wraps approaches for capturing the disk state (but + not memory) of a running guest, but does not track + accompanying guest memory state, and can only operate on one + block device per job. To get a consistent copy of multiple + disks, multiple jobs must be run in parallel, then the domain + must be paused before ending all of the jobs. The capture is + consistent only at the end of the operation with a choice for + future guest changes to either pivot to the new file or to + resume to just using the original file. The resulting backup + file is thus the other file no longer in use by the + guest.</dd> + + <dt><a href="html/libvirt-libvit-domain-checkpoint.html#virDomainCheckpointCreateXML"><code>virDomainCheckpointCreateXML</code></a></dt> + <dd>This API does not actually capture guest state, rather it + makes it possible to track which portions of guest disks have + changed between a checkpoint and the current live execution of + the guest. However, while it is possible use this API to + create checkpoints in isolation, it is more typical to create + a checkpoint as a side-effect of starting a new incremental + backup with <code>virDomainBackupBegin()</code>, since a + second incremental backup is most useful when using the + checkpoint created during the first. See also + the <a href="formatcheckpoint.html">XML details</a> used with + this command.</dd> + + <dt><a href="html/libvirt-libvit-domain.html#virDomainBackupBegin"><code>virDomainBackupBegin</code></a>, <a href="html/libvirt-libvit-domain.html#virDomainBackupEnd"><code>virDomainBackupEnd</code></a></dt> + <dd>This API wraps approaches for capturing the state of disks + of a running guest, but does not track accompanying guest + memory state. The capture is consistent to the start of the + operation, where the captured state is stored independently + from the disk image in use with the guest and where it can be + easily integrated with a third-party for capturing the disk + state. Since the backup operation is stored externally from + the guest resources, there is no need to commit data back in + at the completion of the operation. When coupled with + checkpoints, this can be used to capture incremental backups + instead of full.</dd> + </dl> + + <h2><a id="examples">Examples</a></h2> + <p>The following two sequences both accomplish the task of + capturing the disk state of a running guest, then wrapping + things up so that the guest is still running with the same file + as its disk image as before the sequence of operations began. + The difference between the two sequences boils down to the + impact of an unexpected interruption made at any point in the + middle of the sequence: with such an interruption, the first + example leaves the guest tied to a temporary wrapper file rather + than the original disk, and requires manual clean up of the + domain definition; while the second example has no impact to the + domain definition.</p> + + <p>1. Backup via temporary snapshot + <pre> +virDomainFSFreeze() +virDomainSnapshotCreateXML(VIR_DOMAIN_SNAPSHOT_CREATE_DISK_ONLY) +virDomainFSThaw() +third-party copy the backing file to backup storage # most time spent here +virDomainBlockCommit(VIR_DOMAIN_BLOCK_COMMIT_ACTIVE) per disk +wait for commit ready event per disk +virDomainBlockJobAbort() per disk + </pre></p> + + <p>2. Direct backup + <pre> +virDomainFSFreeze() +virDomainBackupBegin() +virDomainFSThaw() +wait for push mode event, or pull data over NBD # most time spent here +virDomainBackupEnd() + </pre></p> + + </body> +</html> diff --git a/docs/formatbackup.html.in b/docs/formatbackup.html.in index b590bc6a12..f7dcb2113b 100644 --- a/docs/formatbackup.html.in +++ b/docs/formatbackup.html.in @@ -13,7 +13,9 @@ via <code>virDomainBackupBegin()</code>, which takes an XML description of the actions to perform, as well as an optional second XML document <a href="formatcheckpoint.html">describing a - checkpoint</a> to create at the same point in time. + checkpoint</a> to create at the same point in time. See + also <a href="domainstatecapture.html">a comparison</a> between + the various state capture APIs. </p> <p> There are two general modes for backups: a push mode (where the diff --git a/docs/formatcheckpoint.html.in b/docs/formatcheckpoint.html.in index 5c844f1f27..9662d94186 100644 --- a/docs/formatcheckpoint.html.in +++ b/docs/formatcheckpoint.html.in @@ -13,7 +13,9 @@ incremental backups. Right now, incremental backups are only supported for the QEMU hypervisor when using qcow2 disks at the active layer; if other disk formats are in use, capturing disk - backups requires different libvirt APIs. + backups requires different libvirt APIs + (see <a href="domainstatecapture.html">domain state capture</a> + for a comparison between APIs). </p> <p> Libvirt is able to facilitate incremental backups by tracking diff --git a/docs/formatsnapshot.html.in b/docs/formatsnapshot.html.in index 1bbbf06205..d98b1a5f62 100644 --- a/docs/formatsnapshot.html.in +++ b/docs/formatsnapshot.html.in @@ -9,6 +9,8 @@ <h2><a id="SnapshotAttributes">Snapshot XML</a></h2> <p> + Snapshots are one form + of <a href="domainstatecapture.html">domain state capture</a>. There are several types of snapshots: </p> <dl> -- 2.20.1

On Tue, Mar 26, 2019 at 01:13:53AM -0500, Eric Blake wrote:
Now that various new API have been added, it is worth a landing page that gives an overview of capturing various pieces of guest state, and which APIs are best suited to which tasks.
Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Ferlan <jferlan@redhat.com> --- docs/docs.html.in | 5 + docs/domainstatecapture.html.in | 313 ++++++++++++++++++++++++++++++++ docs/formatbackup.html.in | 4 +- docs/formatcheckpoint.html.in | 4 +- docs/formatsnapshot.html.in | 2 + 5 files changed, 326 insertions(+), 2 deletions(-) create mode 100644 docs/domainstatecapture.html.in
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Tue, Mar 26, 2019 at 01:13:44 -0500, Eric Blake wrote:
I'm fairly confident that these API are ready to go (that is, I've got qemu code in the wings to implement these API for the qemu driver, as demonstrated at last KVM forum, and it shouldn't be too hard to add support in the test driver to get some 'make check' coverage similar to what I recently added for snapshots). I'm hoping the APIs make it in for 5.2, even if I'm still dealing with review churn on the later parts of my v5 series (there has been a lot of rebasing from earlier review comments, so v5 is currently still the most recent version that I was able to run demos with, although I hope to post the rest of v6 soon).
I'm still not persuaded that pushing any API without implementation is a good idea. (No, test driver implementation does not count). If the consensus of others is that it's good to go then go ahead, but I want to voice this concern here.

On 3/26/19 12:41 PM, Peter Krempa wrote:
On Tue, Mar 26, 2019 at 01:13:44 -0500, Eric Blake wrote:
I'm fairly confident that these API are ready to go (that is, I've got qemu code in the wings to implement these API for the qemu driver, as demonstrated at last KVM forum, and it shouldn't be too hard to add support in the test driver to get some 'make check' coverage similar to what I recently added for snapshots). I'm hoping the APIs make it in for 5.2, even if I'm still dealing with review churn on the later parts of my v5 series (there has been a lot of rebasing from earlier review comments, so v5 is currently still the most recent version that I was able to run demos with, although I hope to post the rest of v6 soon).
I'm still not persuaded that pushing any API without implementation is a good idea. (No, test driver implementation does not count). If the consensus of others is that it's good to go then go ahead, but I want to voice this concern here.
I do have the qemu implementation for checkpoints fully tested, as well as working for pull mode backups (the push mode backups is still demo quality, but as of v5, I was able to get a push mode backup started even if the events weren't wired up correctly). -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org

On Tue, Mar 26, 2019 at 13:14:48 -0500, Eric Blake wrote:
On 3/26/19 12:41 PM, Peter Krempa wrote:
On Tue, Mar 26, 2019 at 01:13:44 -0500, Eric Blake wrote:
I'm fairly confident that these API are ready to go (that is, I've got qemu code in the wings to implement these API for the qemu driver, as demonstrated at last KVM forum, and it shouldn't be too hard to add support in the test driver to get some 'make check' coverage similar to what I recently added for snapshots). I'm hoping the APIs make it in for 5.2, even if I'm still dealing with review churn on the later parts of my v5 series (there has been a lot of rebasing from earlier review comments, so v5 is currently still the most recent version that I was able to run demos with, although I hope to post the rest of v6 soon).
I'm still not persuaded that pushing any API without implementation is a good idea. (No, test driver implementation does not count). If the consensus of others is that it's good to go then go ahead, but I want to voice this concern here.
I do have the qemu implementation for checkpoints fully tested, as well as working for pull mode backups (the push mode backups is still demo quality, but as of v5, I was able to get a push mode backup started even if the events weren't wired up correctly).
That is okay. If there is a working subset with a real driver I don't object. I just don't see any value in pushing API without a real implementation from upstream point of view.
participants (5)
-
Daniel P. Berrangé
-
Eric Blake
-
Ján Tomko
-
Nir Soffer
-
Peter Krempa