6:23 a.m.
On Sat, Jul 06, 2019 at 22:56:01 -0500, Eric Blake wrote:
Prepare for new checkpoint APIs by describing the XML that will
represent a checkpoint. The checkpoint XML is modeled heavily after
virDomainSnapshotPtr. See the docs for more details.
Add testsuite coverage for some minimal uses of the XML (bare minimum,
the sample from html, and a full dumpxml). Although use of the
REDEFINE flag will require the <domain> subelement to be present, it
is easier for most of the tests to provide counterpart output produced
with the NO_DOMAIN flag (particularly since synthesizing a valid
<domain> during testing is not trivial).
Signed-off-by: Eric Blake <eblake(a)redhat.com>
---
[...]
diff --git a/docs/formatcheckpoint.html.in
b/docs/formatcheckpoint.html.in
new file mode 100644
index 0000000000..ef5f8a826b
--- /dev/null
+++ b/docs/formatcheckpoint.html.in
@@ -0,0 +1,218 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE html>
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <body>
+ <h1>Checkpoint XML format</h1>
+
+ <ul id="toc"></ul>
+
+ <h2><a id="CheckpointAttributes">Checkpoint
XML</a></h2>
+
+ <p>
+ One method of capturing domain disk backups is via the use of
+ incremental backups. Right now, incremental backups are only
+ supported for the QEMU hypervisor when using qcow2 disks at the
+ active layer; if other disk formats are in use, capturing disk
+ backups requires different libvirt APIs.
Perhaps add a 'since' to note that all of this was added starting from
5.6.
+ </p>
+ <p>
+ Libvirt is able to facilitate incremental backups by tracking
+ disk checkpoints, which are points in time against which it is
+ easy to compute which portion of the disk has changed. Given a
+ full backup (a backup created from the creation of the disk to a
+ given point in time), coupled with the creation of a disk
+ checkpoint at that time, and an incremental backup (a backup
+ created from just the dirty portion of the disk between the
+ first checkpoint and the second backup operation), it is
+ possible to do an offline reconstruction of the state of the
+ disk at the time of the second backup without having to copy as
+ much data as a second full backup would require. Future API
+ additions will make it possible to create checkpoints in
+ conjunction with a backup
+ via <code>virDomainBackupBegin()</code> or with an external
+ snapshot via <code>virDomainSnapshotCreateXML2</code>; but for
+ now, libvirt exposes enough support to create disk checkpoints
+ independently from a backup operation
+ via <code>virDomainCheckpointCreateXML()</code>.
So it will not happen in this release? Otherwise I don't see a point in
documenting it.
+ </p>
+ <p>
+ Attributes of libvirt checkpoints are stored as child elements
+ of the <code>domaincheckpoint</code> element. At checkpoint
+ creation time, normally only
+ the <code>name</code>, <code>description</code>,
+ and <code>disks</code> elements are settable. The rest of the
+ fields are ignored on creation and will be filled in by libvirt
+ in for informational purposes
+ by <code>virDomainCheckpointGetXMLDesc()</code>. However, when
+ redefining a checkpoint, with
+ the <code>VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE</code> flag
+ of <code>virDomainCheckpointCreateXML()</code>, all of the XML
+ fields described here are relevant on input, even the fields
+ that are normally described as readonly for output.
+ </p>
+ <p>
+ A checkpoint is marked current if the hypervisor is actively
+ updating the checkpoint as guest operations modify the disk
+ after the point where the checkpoint was created. When tracking
+ multiple checkpoints, a hypervisor may choose to either have all
+ relevant checkpoints be current (a guest write updates thea
So to reiterate. By definition all checkpoint should be "current",
because they must be actively updated to be able to track the difference
from the point when they were created until now.
The fact whether it includes update to a bitmap and which bitmaps are
updated does not seem to be relevant for outside users.
In fact we may even opt to change how this is done or perhaps convert
one layout of bitmaps into a different one if we insist.
Thus I'm saying it again. "current" property seems to be wrong in
conjunction with checkpoints.
+ associated tracking for each checkpoint), or to have a single
+ checkpoint be current (older checkpoints stop tracking changes
NACK, internal detail. Don't document it. Also I'll object to this later
in detail.
+ when a newer one is created, and describing all changes since
an
+ older checkpoint then involves libvirt automatically combining
+ the changes from the chain of checkpoints from the current back
+ to the checkpoint in question; the QEMU hypervisor uses this
+ method). Checkpoints are maintained in a hierarchy to facilitate
Nope. All of the above seem to be internal stuff.
+ hypervisors that use this latter method, where libvirt
+ automatically updates the parent-child relationship as well as
+ which checkpoint is current when creating or deleting
+ checkpoints. For now, the creation of checkpoints is not
+ possible when external snapshots exist, but this may be added in
+ the future.
+ </p>
+ <p>
+ The top-level <code>domaincheckpoint</code> element may contain
+ the following elements:
+ </p>
+ <dl>
+ <dt><code>name</code></dt>
+ <dd>The optional name for this checkpoint. If the name is
+ omitted, libvirt will create a name based on the time of the
+ creation.
+ </dd>
+ <dt><code>description</code></dt>
+ <dd>An optional human-readable description of the checkpoint.
+ If the description is omitted when initially creating the
+ checkpoint, then this field will be empty.
+ </dd>
+ <dt><code>disks</code></dt>
+ <dd>On input, this is an optional listing of specific
+ instructions for disk checkpoints; it is needed when making a
+ checkpoint on only a subset of the disks associated with a
+ domain. In particular, since QEMU checkpoints require qcow2
+ disks, this element may be needed on input for excluding guest
+ disks that are not in qcow2 format. If the entire element was
+ omitted on input, then all disks participate in the
+ checkpoint, otherwise, only the disks explicitly listed which
+ do not also use <code>checkpoint='no'</code> will
+ participate. On output, this is the checkpoint state of each
+ of the domain's disks.
+ <dl>
+ <dt><code>disk</code></dt>
+ <dd>This sub-element describes the checkpoint properties of
+ a specific disk with the following attributes:
+ <dl>
+ <dt><code>name</code></dt>
+ <dd>A mandatory attribute which must match either
+ the <code><target dev='name'/></code>
or an
+ unambiguous <code><source
file='name'/></code>
+ of one of
+ the <a href="formatdomain.html#elementsDisks">disk
+ devices</a> specified for the domain at the time of
+ the checkpoint.</dd>
+ <dt><code>checkpoint</code></dt>
+ <dd>An optional attribute; possible values
+ are <code>no</code> when the disk does not participate
+ in this checkpoint; or <code>bitmap</code> if the disk
+ will track all changes since the creation of this
+ checkpoint via a bitmap.</dd>
+ <dt><code>bitmap</code></dt>
+ <dd>The attribute <code>bitmap</code> is only valid
+ if <code>checkpoint='bitmap'</code>; it describes
the
+ name of the tracking bitmap (defaulting to the
+ checkpoint name).</dd>
+ <dt><code>size</code></dt>
+ <dd>The attribute <code>size</code> is ignored on
input;
+ on output, it is only present if
+ the <code>VIR_DOMAIN_CHECKPOINT_XML_SIZE</code> flag
+ was used to perform a dynamic query of the estimated
+ size in bytes of the changes made since the checkpoint
+ was created.</dd>
+ </dl>
+ </dd>
+ </dl>
+ </dd>
+ <dt><code>creationTime</code></dt>
+ <dd>A readonly representation of the time this checkpoint was
+ created. The time is specified in seconds since the Epoch,
+ UTC (i.e. Unix time).
+ </dd>
+ <dt><code>parent</code></dt>
+ <dd>An optional readonly representation of the parent of this
+ checkpoint. If present, this element contains exactly one
+ child element, <code>name</code>.
This reads weird and does not seem to say much.
+ </dd>
+ <dt><code>current</code></dt>
+ <dd>A readonly integer, 1 if this checkpoint is current (that
+ is, actively tracking guest changes) or 0 if the checkpoint is
+ inactive because a child checkpoint is now tracking changes.
+ </dd>
See below in the schema. The contents of this element are wrong.
+ <dt><code>domain</code></dt>
+ <dd>A readonly representation of the
+ inactive <a href="formatdomain.html">domain
configuration</a>
+ at the time the checkpoint was created. This element may be
+ omitted for output brevity by supplying
+ the <code>VIR_DOMAIN_CHECKPOINT_XML_NO_DOMAIN</code> flag, but
+ the resulting XML is no longer viable for use with
+ the <code>VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE</code> flag
+ of <code>virDomainCheckpointCreateXML()</code>. The domain
+ will have security-sensitive information omitted unless the
+ flag <code>VIR_DOMAIN_CHECKPOINT_XML_SECURE</code> is provided
+ on a read-write connection.
+ </dd>
+ </dl>
+
+ <h2><a id="example">Examples</a></h2>
+
+ <p>Using this XML to create a checkpoint of just vda on a qemu
+ domain with two disks and a prior checkpoint:</p>
+ <pre>
+<domaincheckpoint>
+ <description>Completion of updates after OS
install</description>
+ <disks>
+ <disk name='vda' checkpoint='bitmap'/>
+ <disk name='vdb' checkpoint='no'/>
+ </disks>
+</domaincheckpoint></pre>
+
+ <p>will result in XML similar to this from
+ <code>virDomainCheckpointGetXMLDesc()</code>:</p>
+ <pre>
+<domaincheckpoint>
+ <name>1525889631</name>
+ <description>Completion of updates after OS
install</description>
+ <parent>
+ <name>1525111885</name>
+ </parent>
+ <creationTime>1525889631</creationTime>
+ <disks>
+ <disk name='vda' checkpoint='bitmap'
bitmap='1525889631'/>
+ <disk name='vdb' checkpoint='no'/>
+ </disks>
+ <domain type='qemu'>
+ <name>fedora</name>
+ <uuid>93a5c045-6457-2c09-e56c-927cdf34e178</uuid>
+ <memory>1048576</memory>
+ ...
+ <devices>
+ <disk type='file' device='disk'>
+ <driver name='qemu' type='qcow2'/>
+ <source file='/path/to/file1'/>
+ <target dev='vda' bus='virtio'/>
+ </disk>
+ <disk type='file' device='disk'
snapshot='external'>
+ <driver name='qemu' type='raw'/>
+ <source file='/path/to/file2'/>
+ <target dev='vdb' bus='virtio'/>
+ </disk>
+ ...
+ </devices>
+ </domain>
+</domaincheckpoint></pre>
+
+ <p>With that checkpoint created, the qcow2 image is now tracking
+ all changes that occur in the image since the checkpoint via
+ the persistent bitmap named <code>1525889631</code>.
+ </p>
+ </body>
+</html>
diff --git a/docs/formatsnapshot.html.in b/docs/formatsnapshot.html.in
index 92cc566467..2bfb69cf49 100644
--- a/docs/formatsnapshot.html.in
+++ b/docs/formatsnapshot.html.in
@@ -91,7 +91,9 @@
sets that snapshot as current, and the prior current snapshot is
the parent of the new snapshot. Branches in the hierarchy can
be formed by reverting to a snapshot with a child, then creating
- another snapshot.
+ another snapshot. For now, the creation of external snapshots
+ when checkpoints exist is forbidden, although future work will
+ make it possible to integrate these two concepts.
</p>
<p>
The top-level <code>domainsnapshot</code> element may contain
I'd slightly prefer if the snapshot-interlock stuff is in a separate
patch.
[...]
diff --git a/docs/schemas/domaincheckpoint.rng
b/docs/schemas/domaincheckpoint.rng
new file mode 100644
index 0000000000..ceb5436aa3
--- /dev/null
+++ b/docs/schemas/domaincheckpoint.rng
@@ -0,0 +1,95 @@
+<?xml version="1.0"?>
+<!-- A Relax NG schema for the libvirt domain checkpoint properties XML format
-->
+<grammar xmlns="http://relaxng.org/ns/structure/1.0">
+ <start>
+ <ref name='domaincheckpoint'/>
+ </start>
+
+ <include href='domaincommon.rng'/>
+
+ <define name='domaincheckpoint'>
+ <element name='domaincheckpoint'>
+ <interleave>
+ <optional>
+ <element name='name'>
+ <text/>
This should be something more sensible than '<text/>' we have plenty of
types for names which exclude newlines and '/'.
+ </element>
+ </optional>
+ <optional>
+ <element name='description'>
+ <text/>
+ </element>
+ </optional>
+ <optional>
+ <element name='creationTime'>
+ <text/>
This is supposed to be a number.
+ </element>
+ </optional>
+ <optional>
+ <element name='current'>
+ <choice>
+ <value>0</value>
+ <value>1</value>
This looks weird. I see 3 options to make it better:
1) put it as an attribute of 'domaincheckpoint' (and use
'yes'/'no' as
values)
2) encode it as presence of the element
3) at least use 'yes'/'no' as values.
+ </choice>
+ </element>
+ </optional>
+ <optional>
+ <element name='disks'>
+ <oneOrMore>
+ <ref name='diskcheckpoint'/>
+ </oneOrMore>
+ </element>
+ </optional>
+ <optional>
+ <!-- Nested grammar ensures that any of our overrides of
+ storagecommon/domaincommon defines do not conflict
+ with any domain.rng overrides. -->
+ <grammar>
+ <include href='domain.rng'/>
+ </grammar>
+ </optional>
+ <optional>
+ <element name='parent'>
+ <element name='name'>
+ <text/>
Again, this is a 'name'
+ </element>
+ </element>
+ </optional>
+ </interleave>
+ </element>
+ </define>
+
+ <define name='diskcheckpoint'>
+ <element name='disk'>
+ <attribute name='name'>
+ <choice>
+ <ref name='diskTarget'/>
+ <ref name='absFilePath'/>
+ </choice>
+ </attribute>
+ <choice>
+ <attribute name='checkpoint'>
+ <value>no</value>
+ </attribute>
+ <group>
+ <optional>
+ <attribute name='checkpoint'>
+ <value>bitmap</value>
+ </attribute>
+ </optional>
+ <optional>
+ <attribute name='bitmap'>
+ <text/>
This is also a 'name'
+ </attribute>
+ </optional>
+ <optional>
+ <attribute name='size'>
+ <ref name="unsignedLong"/>
+ </attribute>
+ </optional>
+ </group>
+ </choice>
+ </element>
+ </define>
+
+</grammar>
ACK if you sanitize the schema and avoid documenting internals in the
public docs. I think that also the bitmap name field should be
documented as internal.