1:08 p.m.
Hello,
After our discussions about the snapshot API last week, I went ahead and implemented
quite a bit of the API. I also went back to the ESX, Virtualbox, and QEMU API's to
try and make sure our API's matched up. What's below is my revised API based on
that survey. Following my revised API are notes that I took regarding how the
libvirt API matches up to the various API's, and some questions about semantics that
I had while doing the survey. More comments and questions are welcome.
/* NOTE: struct _virDomainSnapshot is a private structure, ala
* struct _virDomain.
*/
typedef struct _virDomainSnapshot virDomainSnapshot;
/* Take a snapshot of the current VM state. Throws an error if
* the VM is not currently running */
virDomainSnapshotPtr virDomainSnapshotCreateXML(virDomainPtr domain,
const char *xmlDesc,
unsigned int flags);
/* Dump the XML of a snapshot */
/* NOTE: see below for proposed XML */
char *virDomainSnapshotGetXMLDesc(virDomainSnapshotPtr snapshot,
unsigned int flags);
/* Return the number of snapshots for this domain */
int virDomainSnapshotNum(virDomainPtr domain, unsigned int flags);
/* Get the names of all snapshots for this domain */
int virDomainListSnapshotNames(virDomainPtr domain, char **names, int nameslen,
unsigned int flags);
/* Get a handle to a named snapshot */
virDomainSnapshotPtr virDomainSnapshotLookupByName(virDomainPtr domain,
const char *name,
unsigned int flags);
/* Get a handle to the current in-use snapshot for the domain */
virDomainSnapshotPtr virDomainSnapshotCurrent(virDomainPtr domain,
unsigned int flags);
/* Start the guest from the snapshot "snapshot" */
int virDomainCreateFromSnapshot(virDomainSnapshotPtr snapshot,
unsigned int flags);
/* Rename the snapshot */
/* Do we really need this? In theory we could use
* virsh snapshot-edit <domain> <name> and then detect
* name changes, but that will require a UUID, which may
* or may not be overkill
*/
int virDomainSnapshotRename(virDomainSnapshotPtr snapshot,
char *newname,
unsigned int flags);
/* Delete a snapshot - with no flags, the snapshot is not used anymore,
* but also not removed. With a MERGE flag, it merges the snapshot into
* the parent snapshot (or the base image, if there is no parent snapshot).
* Note that if other snapshots would be discarded because of this
* MERGE action, this operation will fail. If that is really what is intended,
* use MERGE_FORCE.
*
* With a DISCARD flag, it deletes the snapshot. Note that if children snapshots
* would be discarded because of this delete action, this operation will
* fail. If this is really what is intended, use DISCARD_FORCE.
*
* MERGE, MERGE_FORCE, DISCARD, and DISCARD_FORCE are mutually-exclusive.
*
* Note that this operation can happen when the domain is running or shut
* down, though this is hypervisor specific */
typedef enum {
VIR_DOMAIN_SNAPSHOT_DELETE_MERGE,
VIR_DOMAIN_SNAPSHOT_DELETE_MERGE_FORCE,
VIR_DOMAIN_SNAPSHOT_DELETE_DISCARD,
VIR_DOMAIN_SNAPSHOT_DELETE_DISCARD_FORCE,
} virDomainSnapshotDelete;
int virDomainSnapshotDelete(virDomainSnapshotPtr snapshot,
unsigned int flags);
int virDomainSnapshotFree(virDomainSnapshotPtr snapshot);
NOTE: During snapshot creation, *none* of the fields are required. That is,
you can call virDomainSnapshotCreateXML() with an XML of
"<domainsnapshot/>".
In this case, the individual driver will make up a <name> and <uuid> for you,
the <creationdate> will be set to the current time+date, <description> will
be
empty, <state> will be the current state of the VM, and <parent> will
be set to the current snapshot (if any). If you do want to specify some
fields during virDomainSnapshotCreateXML(), note that the only ones that are
settable are <name>, <uuid>, and <description>;
the rest are ignored, and filled in by the driver when the snapshot is
actually created.
NOTE: <state> refers to the state of the VM when the snapshot was taken.
<domainsnapshot>
<name>XYZ</name>
<creationdate>...</creationdate>
<description>...</description>
<state>RUNNING</state>
<domain>
<uuid>XXXXX-XXXX-XXXX-XXXX-XXXXXXXXX</uuid>
</domain>
<parent>
<name>ABC</name>
</parent>
</domainsnapshot>
The virsh commands will be:
virsh snapshot-create <dom> <xmlfile>
virsh snapshot-list <dom>
virsh snapshot-dumpxml <dom> <name>
virsh start-with-snapshot <dom> <snapshotname>
virsh snapshot-delete <dom> <snapshotname>
[--merge|--mergeforce|--delete|--deleteforce]
virsh snapshot-delete-all <dom>
Possible issues:
1) I don't see a way to support "managed" save/restore and snapshotting
with
this API. I think we'll have to have a separate API for managed save/restore.
2) What is the semantic for deleting snapshots from a running domain?
Virtualbox seems to not allow you to manipulate snapshots while the domain is
running. Qemu does allow this, but it's currently unclear what the exact
semantics are. VMware seems to allow manipulation of snapshots while the
domain is running.
3) Do we need a snapshot UUID? Virtualbox allows you to have multiple snapshots
with the same name, differentiated by UUID. Confusingly, they also have a
"FindByName" method that returns the first depth-first search snapshot that
matches
a given name. For qemu, if you specify the same name twice it overwrites the previous
one with the new one. I don't know what ESX does here.
Mapping of our interface to various hypervisors:
+-------------------------------+-----------------+-------------------+------------------------------+
| Libvirt | Qemu | Virtualbox | ESX
|
+-------------------------------+-----------------+-------------------+------------------------------+
| virDomainSnapshotCreateXML | monitor command | takeSnapshot |
CreateSnapshot_task |
| | "savevm"; if | Snapshots can | takes a
name, description, |
| | snapshot name | be taken on | memory
(true/false) and |
| | is already in | powered off, | quiesce
(true/false). |
| | use, replaces | saved, running, | What does
"memory" mean? |
| | the previous | or paused VMs. | Should we model
"quiesce" |
| | snapshot. Also | The snapshot is | Trees of snapshots
are |
| | qemu-img | always taken | supported. What
happens |
| | snapshot -c can | against the | on a duplicate
name? What |
| | be used to | current snapshot. | state(s) can a VM
be in |
| | create a | What happens on | when calling this?
Does |
| | disk-only | a duplicate | a VM get paused
when this |
| | snapshot. What | name? Trees of | is called?
|
| | happens if the | snapshots are |
|
| | VM is running | not currently |
|
| | when you do | supported. |
|
| | this? Trees of | Taking a snapshot |
|
| | snapshots seem | of a running VM |
|
| | to be supported | pauses the VM |
|
| | VM gets paused | before taking the |
|
| | while this is | snapshot. |
|
| | happening. What | |
|
| | states can the | |
|
| | VM be in? | |
|
+-------------------------------+-----------------+-------------------+------------------------------+
| virDomainSnapshotGetXMLDesc | Libvirt qemu | GetId | VirtualMachine
object-> |
| | snapshot | GetDescription | snapshot->
rootSnapshotList |
| | metadata | GetTimeStamp | [i].createTime
|
| | | GetParent | .description
|
| | | GetCurrentSnapshot| .name
|
| | | | .id
|
| | | | .state
|
| | | | .quiesced
|
| | | | .vm
|
+-------------------------------+-----------------+-------------------+------------------------------+
| virDomainSnapshotNum | Libvirt qemu | GetSnapshotCount | VirtualMachine
object-> |
| | snapshot | | snapshot->
rootSnapshotList |
| | metadata | |
|
+-------------------------------+-----------------+-------------------+------------------------------+
| virDomainListSnapshotNames | Libvirt qemu | GetSnapshotCount | VirtualMachine
object-> |
| | snapshot | GetChildren | snapshot->
rootSnapshotList |
| | metadata, or | |
|
| | listvm monitor | |
|
| | command or | |
|
| | qemu-img | |
|
| | snapshot -l | |
|
| | <file>. If the | |
|
| | VM has multiple | |
|
| | disks and the | |
|
| | disks have | |
|
| | different | |
|
| | snapshots, what | |
|
| | do you do? | |
|
+-------------------------------+-----------------+-------------------+------------------------------+
| virDomainSnapshotLookupByName | " | findSnapshot |
VirtualMachine object-> |
| | | Takes a name and | snapshot->
rootSnapshotList |
| | | returns a |
|
| | | snapshot object. |
|
| | | In case of |
|
| | | multiple snapshots|
|
| | | with the same |
|
| | | name, it returns |
|
| | | the first object |
|
| | | from a depth |
|
| | | first search. |
|
+-------------------------------+-----------------+-------------------+------------------------------+
| virDomainCreateFromSnapshot | qemu-img | restoreSnapshot |
RevertToSnapshot_Task |
| | snapshot -a | Takes a snapshot | Changes execution
state |
| | <snapname> | object, and | of VM to
state of this |
| | <file>. What | resets the VM's |
snapshot. Takes a |
| | happens if the | state to that of | snapshot object,
an |
| | VM has multiple | the snapshot. | optional host, and
|
| | files with | If this is a | suppressPowerOn,
which |
| | different | snapshot taken | forces the VM to
the off |
| | snapshots? | against a running | state regardless
of the |
| | | machine, then the | state when the
snapshot |
| | | memory is | was taken.
Implies that |
| | | restored as well. | without this flag,
VM |
| | | Does *not* start | starts in whatever
|
| | | the VM. The VM | <state> was
when snapshot |
| | | must be off for | was taken.
|
| | | this operation to |
|
| | | succeed. |
|
+-------------------------------+-----------------+-------------------+------------------------------+
| virDomainSnapshotDelete | monitor command | deleteSnapshot |
RemoveSnapshot_Task |
| | "delvm". What | deletes the | removes
this snapshot and |
| | happens if the | specified | deletes any
associated |
| | snapshot is in | snapshot. Takes | storage. Operates
on a |
| | use? What | an ID. The VM |
VirtualMachineSnapshot |
| | states can the | must be off. | object. What
states can |
| | VM be in? Also | Differences to | the VM be in?
What |
| | qemu-img | children | happens if this
snapshot |
| | snapshot -d | snapshots will be | is in-use? What
happens |
| | <name> <file> | merged with the | to
parents and children? |
| | command can be | children to keep |
|
| | used. What | children valid. |
|
| | happens if the | Parent for this |
|
| | disk is in-use? | snapshot will |
|
| | What happens to | become parent of |
|
| | parents and | any children |
|
| | children? | snapshots. |
|
| | How do we | |
|
| | handle merges? | |
|
+-------------------------------+-----------------+-------------------+------------------------------+
| virDomainSnapshotCurrent | Libvirt qemu | currentSnapshot | VirtualMachine
object-> |
| | snapshot | | snapshot->
currentSnapshot |
| | metadata | |
|
+-------------------------------+-----------------+-------------------+------------------------------+
| virDomainSnapshotRename | Libvirt qemu | ISnapshot->name | RenameSnapshot
|
| | snapshot | |
|
| | metadata | |
|
+-------------------------------+-----------------+-------------------+------------------------------+
| virDomainSnapshotDelete with | Libvirt qemu | deleteSnapshot |
RemoveAllSnapshots_Task |
| DISCARD_FORCE flag against | snapshot | with manual depth |
|
| root snapshot | metadata | deletion of |
|
| | | children |
|
+-------------------------------+-----------------+-------------------+------------------------------+
| | | |
RevertToCurrentSnapshot_Task |
+-------------------------------+-----------------+-------------------+------------------------------+
Attribute mapping:
+----------------+----------+-------------+----------------------------+
| Libvirt | Qemu | Virtualbox | ESX |
+----------------+----------+-------------+----------------------------+
| <name> | TAG | name | name |
+----------------+----------+-------------+----------------------------+
| <creationdate> | DATE | timeStamp | createTime |
+----------------+----------+-------------+----------------------------+
| <description> | Libvirt | description | description |
| | qemu | | |
| | metadata | | |
+----------------+----------+-------------+----------------------------+
| <state> | Libvirt | online | state (this is the *power* |
| | qemu | | state of the VM when this |
| | metadata | | snapshot was taken) |
+----------------+----------+-------------+----------------------------+
| <domain><uuid> | Libvirt | machine | vm |
| | qemu | | |
| | metadata | | |
+----------------+----------+-------------+----------------------------+
| <parent><name> | Libvirt | parent | N/A |
| | qemu | | |
| | metadata | | |
+----------------+----------+-------------+----------------------------+
| N/A | N/A | id (uuid) | id |
+----------------+----------+-------------+----------------------------+
| N/A | N/A | children | childSnapshotList |
+----------------+----------+-------------+----------------------------+
| N/A | N/A | N/A | memory |
+----------------+----------+-------------+----------------------------+
| N/A | N/A | N/A | quiesced |
+----------------+----------+-------------+----------------------------+
| N/A | N/A | N/A | backupManifest |
+----------------+----------+-------------+----------------------------+
| N/A | N/A | N/A | replaySupported |
+----------------+----------+-------------+----------------------------+
--
Chris Lalancette
3:40 p.m.
Hi.
...
/* NOTE: struct _virDomainSnapshot is a private structure, ala
* struct _virDomain.
*/
typedef struct _virDomainSnapshot virDomainSnapshot;
/* Take a snapshot of the current VM state. Throws an error if
* the VM is not currently running */
virDomainSnapshotPtr virDomainSnapshotCreateXML(virDomainPtr domain,
const char *xmlDesc,
unsigned int flags);
This is probably a leftover from previous versions, but... why do we restrict
this API only for running VMs?
...
/* Delete a snapshot - with no flags, the snapshot is not used
anymore,
* but also not removed. With a MERGE flag, it merges the snapshot into
* the parent snapshot (or the base image, if there is no parent snapshot).
* Note that if other snapshots would be discarded because of this
* MERGE action, this operation will fail. If that is really what is intended,
* use MERGE_FORCE.
*
* With a DISCARD flag, it deletes the snapshot. Note that if children snapshots
* would be discarded because of this delete action, this operation will
* fail. If this is really what is intended, use DISCARD_FORCE.
*
* MERGE, MERGE_FORCE, DISCARD, and DISCARD_FORCE are mutually-exclusive.
*
* Note that this operation can happen when the domain is running or shut
* down, though this is hypervisor specific */
typedef enum {
VIR_DOMAIN_SNAPSHOT_DELETE_MERGE,
VIR_DOMAIN_SNAPSHOT_DELETE_MERGE_FORCE,
VIR_DOMAIN_SNAPSHOT_DELETE_DISCARD,
VIR_DOMAIN_SNAPSHOT_DELETE_DISCARD_FORCE,
} virDomainSnapshotDelete;
Merging a snapshot into its parent is probably not the best semantics for
MERGE flag as hypervisors differ in the way merging is implemented. As you
also mention below, VirtualBox merges into all children instead of a parent.
We should allow for both cases. However it influences several things. Firstly,
it makes MERGE_FORCE unnecessary for child merging, which is not a big deal as
it can just be treated in the same way as MERGE. Secondly, it makes a huge
difference when deleting a snapshot with no child. In one case it results in
changes being merged and in other case it results changes begin dropped.
One option is to refine the semantics to something like:
- MERGE: merge changes into other snapshot(s) and fail if it would require any
snapshot to be discarded (even the one which was supposed to be merged)
- MERGE_FORCE: really merge even discarding other snapshots but fail if the
snapshot itself would actually be discarded
- DISCARD: discard the snapshot and fail if other snapshots would be discarded
- DISCARD_FORCE: discard, no matter what
Another option would be to introduce several different APIs for merging into
children, merging into parent, and discarding. That would allow drivers to
implement only supported methods. Even all of them for a very flexible
hypervisor.
And the third option I see would be distinguishing merge direction using new
flags.
Personally, I like the second option best as it provides the easiest way for
application to detect unsupported behavior.
...
Possible issues:
1) I don't see a way to support "managed" save/restore and snapshotting
with
this API. I think we'll have to have a separate API for managed save/restore.
2) What is the semantic for deleting snapshots from a running domain?
Virtualbox seems to not allow you to manipulate snapshots while the domain is
running. Qemu does allow this, but it's currently unclear what the exact
semantics are. VMware seems to allow manipulation of snapshots while the
domain is running.
3) Do we need a snapshot UUID? Virtualbox allows you to have multiple snapshots
with the same name, differentiated by UUID. Confusingly, they also have a
"FindByName" method that returns the first depth-first search snapshot that
matches
a given name. For qemu, if you specify the same name twice it overwrites the previous
one with the new one. I don't know what ESX does here.
Libvirt uses/generates UUIDs for almost everything (networks, vms, ...) so it
might be more consistent to have UUID in snapshot as well.
Jirka
3:52 p.m.
2010/3/30 Jiri Denemark <jdenemar(a)redhat.com>:
Hi.
...
> /* NOTE: struct _virDomainSnapshot is a private structure, ala
> * struct _virDomain.
> */
> typedef struct _virDomainSnapshot virDomainSnapshot;
>
> /* Take a snapshot of the current VM state. Throws an error if
> * the VM is not currently running */
> virDomainSnapshotPtr virDomainSnapshotCreateXML(virDomainPtr domain,
> const char *xmlDesc,
> unsigned int flags);
This is probably a leftover from previous versions, but... why do we restrict
this API only for running VMs?
Yep, if a domain it not running you'll just get a disk snapshot
without a memory snapshot.
...
> Possible issues:
> 1) I don't see a way to support "managed" save/restore and
snapshotting with
> this API. I think we'll have to have a separate API for managed save/restore.
> 2) What is the semantic for deleting snapshots from a running domain?
> Virtualbox seems to not allow you to manipulate snapshots while the domain is
> running. Qemu does allow this, but it's currently unclear what the exact
> semantics are. VMware seems to allow manipulation of snapshots while the
> domain is running.
> 3) Do we need a snapshot UUID? Virtualbox allows you to have multiple snapshots
> with the same name, differentiated by UUID. Confusingly, they also have a
> "FindByName" method that returns the first depth-first search snapshot that
matches
> a given name. For qemu, if you specify the same name twice it overwrites the
previous
> one with the new one. I don't know what ESX does here.
Libvirt uses/generates UUIDs for almost everything (networks, vms, ...) so it
might be more consistent to have UUID in snapshot as well.
ESX snapshots don't have a UUID, and there is no easy way to store an
UUID-to-snapshot mapping.
Matthias
4:26 p.m.
On 03/30/2010 02:52 PM, Matthias Bolte wrote:
2010/3/30 Jiri Denemark <jdenemar(a)redhat.com>:
> Hi.
>
> ...
>> /* NOTE: struct _virDomainSnapshot is a private structure, ala
>> * struct _virDomain.
>> */
>> typedef struct _virDomainSnapshot virDomainSnapshot;
>>
>> /* Take a snapshot of the current VM state. Throws an error if
>> * the VM is not currently running */
>> virDomainSnapshotPtr virDomainSnapshotCreateXML(virDomainPtr domain,
>> const char *xmlDesc,
>> unsigned int flags);
>
> This is probably a leftover from previous versions, but... why do we restrict
> this API only for running VMs?
>
Yep, if a domain it not running you'll just get a disk snapshot
without a memory snapshot.
Is taking a halted disk snapshot something we might want to allow with a
flag, though? For that matter, is this API useful for taking a disk
snapshot and disregarding a memory snapshot even of a running VM?
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
5:39 p.m.
On 03/30/2010 05:26 PM, Eric Blake wrote:
On 03/30/2010 02:52 PM, Matthias Bolte wrote:
> 2010/3/30 Jiri Denemark <jdenemar(a)redhat.com>:
>> Hi.
>>
>> ...
>>> /* NOTE: struct _virDomainSnapshot is a private structure, ala
>>> * struct _virDomain.
>>> */
>>> typedef struct _virDomainSnapshot virDomainSnapshot;
>>>
>>> /* Take a snapshot of the current VM state. Throws an error if
>>> * the VM is not currently running */
>>> virDomainSnapshotPtr virDomainSnapshotCreateXML(virDomainPtr domain,
>>> const char *xmlDesc,
>>> unsigned int flags);
>>
>> This is probably a leftover from previous versions, but... why do we restrict
>> this API only for running VMs?
>>
>
> Yep, if a domain it not running you'll just get a disk snapshot
> without a memory snapshot.
Is taking a halted disk snapshot something we might want to allow with a
flag, though? For that matter, is this API useful for taking a disk
snapshot and disregarding a memory snapshot even of a running VM?
The problem with disgregarding a memory snapshot of a running guest is that it
is very easy to get inconsistent snapshots; that is, the guest could
have data cached in memory that is not on disk when you snapshot, and then you
have completely inconsistent results on disk.
ESX allows you to do a disk only snapshot of a running guest, but they cheat;
they also have a way to quiesce the guest (force writes), thus ensuring a
consistent snapshot.
Virtualbox doesn't give you a choice in the matter; if the guest is off when
you take a snapshot, you get a disk snapshot, and if it's on when you take a
snapshot, you get a disk+memory snapshot.
Qemu allows both usages, although it must have the consistency problems I
mentioned above for a disk-only snapshot on a running guest.
For now, I think we can stick with the disk-only semantic for shutoff guests,
and the disk+memory semantic for running guests. If it turns out that there
is a need for a disk-only semantic for running guests, this should be easy
to add later via a flag to virDomainSnapshotCreateXML.
--
Chris Lalancette
3:04 a.m.
On Tue, Mar 30, 2010 at 06:39:02PM -0400, Chris Lalancette wrote:
On 03/30/2010 05:26 PM, Eric Blake wrote:
> On 03/30/2010 02:52 PM, Matthias Bolte wrote:
>> Yep, if a domain it not running you'll just get a disk snapshot
>> without a memory snapshot.
>
> Is taking a halted disk snapshot something we might want to allow with a
> flag, though? For that matter, is this API useful for taking a disk
> snapshot and disregarding a memory snapshot even of a running VM?
>
The problem with disgregarding a memory snapshot of a running guest is that it
is very easy to get inconsistent snapshots; that is, the guest could
have data cached in memory that is not on disk when you snapshot, and then you
have completely inconsistent results on disk.
ESX allows you to do a disk only snapshot of a running guest, but they cheat;
they also have a way to quiesce the guest (force writes), thus ensuring a
consistent snapshot.
Virtualbox doesn't give you a choice in the matter; if the guest is off when
you take a snapshot, you get a disk snapshot, and if it's on when you take a
snapshot, you get a disk+memory snapshot.
Qemu allows both usages, although it must have the consistency problems I
mentioned above for a disk-only snapshot on a running guest.
For now, I think we can stick with the disk-only semantic for shutoff guests,
and the disk+memory semantic for running guests. If it turns out that there
is a need for a disk-only semantic for running guests, this should be easy
to add later via a flag to virDomainSnapshotCreateXML.
yes, that sounds a reasonable approach to me.
Daniel
--
Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/
daniel(a)veillard.com | Rpmfind RPM search engine http://rpmfind.net/
http://veillard.com/ | virtualization library http://libvirt.org/
5:13 p.m.
On 03/30/2010 04:40 PM, Jiri Denemark wrote:
Hi.
...
> /* NOTE: struct _virDomainSnapshot is a private structure, ala
> * struct _virDomain.
> */
> typedef struct _virDomainSnapshot virDomainSnapshot;
>
> /* Take a snapshot of the current VM state. Throws an error if
> * the VM is not currently running */
> virDomainSnapshotPtr virDomainSnapshotCreateXML(virDomainPtr domain,
> const char *xmlDesc,
> unsigned int flags);
This is probably a leftover from previous versions, but... why do we restrict
this API only for running VMs?
Oops, yeah, you are right, I just forgot to change the comment. It now says:
/* Take a snapshot of the current VM state. */
...
> /* Delete a snapshot - with no flags, the snapshot is not used anymore,
> * but also not removed. With a MERGE flag, it merges the snapshot into
> * the parent snapshot (or the base image, if there is no parent snapshot).
> * Note that if other snapshots would be discarded because of this
> * MERGE action, this operation will fail. If that is really what is intended,
> * use MERGE_FORCE.
> *
> * With a DISCARD flag, it deletes the snapshot. Note that if children snapshots
> * would be discarded because of this delete action, this operation will
> * fail. If this is really what is intended, use DISCARD_FORCE.
> *
> * MERGE, MERGE_FORCE, DISCARD, and DISCARD_FORCE are mutually-exclusive.
> *
> * Note that this operation can happen when the domain is running or shut
> * down, though this is hypervisor specific */
> typedef enum {
> VIR_DOMAIN_SNAPSHOT_DELETE_MERGE,
> VIR_DOMAIN_SNAPSHOT_DELETE_MERGE_FORCE,
> VIR_DOMAIN_SNAPSHOT_DELETE_DISCARD,
> VIR_DOMAIN_SNAPSHOT_DELETE_DISCARD_FORCE,
> } virDomainSnapshotDelete;
Merging a snapshot into its parent is probably not the best semantics for
MERGE flag as hypervisors differ in the way merging is implemented. As you
Yeah, you are right, we can't just declare this. It's also an open question
to me what qemu does about merging (though bugs with loadvm/delvm are preventing
me from testing this at the moment).
also mention below, VirtualBox merges into all children instead of a
parent.
We should allow for both cases. However it influences several things. Firstly,
it makes MERGE_FORCE unnecessary for child merging, which is not a big deal as
it can just be treated in the same way as MERGE. Secondly, it makes a huge
difference when deleting a snapshot with no child. In one case it results in
changes being merged and in other case it results changes begin dropped.
One option is to refine the semantics to something like:
- MERGE: merge changes into other snapshot(s) and fail if it would require any
snapshot to be discarded (even the one which was supposed to be merged)
- MERGE_FORCE: really merge even discarding other snapshots but fail if the
snapshot itself would actually be discarded
- DISCARD: discard the snapshot and fail if other snapshots would be discarded
- DISCARD_FORCE: discard, no matter what
The problem with declaring these semantics is that they are somewhat confusing, so
application developers will probably get them wrong. On the other hand it does
allow us to declare a semantic that all 3 hypervisors probably can support,
unlike the options below.
Another option would be to introduce several different APIs for merging into
children, merging into parent, and discarding. That would allow drivers to
implement only supported methods. Even all of them for a very flexible
hypervisor.
The problem with this one is that it will be difficult for application writers
to write a single application that handles all of the hypervisors. Imagine trying
to write a GUI around this, and you'll see what I mean. If we were to add 3 new
API's like this, we would also probably want to add some data to the capabilities
XML for the particular hypervisors so you could grey out specific options in a GUI.
On the other hand, it does solve the problem with merging parents vs. children, and
also diffuses Paolo's concern about the "SnapshotDelete" API sometimes
deleting data
and sometimes modifying the base image.
And the third option I see would be distinguishing merge direction using new
flags.
This one is like the second option, in that you don't know which particular
directions a particular hypervisor supports. You'd still need to add
capabilities XML for each hypervisor.
I have to say that after thinking about these 3 options, I like the first
option the best. While it's slightly confusing, it is a good semantic. I'll
update the documentation for this.
Personally, I like the second option best as it provides the easiest way for
application to detect unsupported behavior.
...
> Possible issues:
> 1) I don't see a way to support "managed" save/restore and
snapshotting with
> this API. I think we'll have to have a separate API for managed save/restore.
> 2) What is the semantic for deleting snapshots from a running domain?
> Virtualbox seems to not allow you to manipulate snapshots while the domain is
> running. Qemu does allow this, but it's currently unclear what the exact
> semantics are. VMware seems to allow manipulation of snapshots while the
> domain is running.
> 3) Do we need a snapshot UUID? Virtualbox allows you to have multiple snapshots
> with the same name, differentiated by UUID. Confusingly, they also have a
> "FindByName" method that returns the first depth-first search snapshot that
matches
> a given name. For qemu, if you specify the same name twice it overwrites the
previous
> one with the new one. I don't know what ESX does here.
Libvirt uses/generates UUIDs for almost everything (networks, vms, ...) so it
might be more consistent to have UUID in snapshot as well.
Yeah, that is true, which is why I'm waffling with it. In general it seems like
superflous
information, except in the one case of virtualbox duplicate names (ESX doesn't allow
duplicate names, I don't think, and qemu blows away duplicates). My inclination is
to declare the semantics of duplicate names to be undefined, since it doesn't seem
to be a very useful feature.
--
Chris Lalancette
7:29 p.m.
2010/3/31 Chris Lalancette <clalance(a)redhat.com>:
On 03/30/2010 04:40 PM, Jiri Denemark wrote:
> Hi.
>
> ...
>> /* NOTE: struct _virDomainSnapshot is a private structure, ala
>> * struct _virDomain.
>> */
>> typedef struct _virDomainSnapshot virDomainSnapshot;
>>
>> /* Take a snapshot of the current VM state. Throws an error if
>> * the VM is not currently running */
>> virDomainSnapshotPtr virDomainSnapshotCreateXML(virDomainPtr domain,
>> const char *xmlDesc,
>> unsigned int flags);
>
> This is probably a leftover from previous versions, but... why do we restrict
> this API only for running VMs?
Oops, yeah, you are right, I just forgot to change the comment. It now says:
/* Take a snapshot of the current VM state. */
>
> ...
>> /* Delete a snapshot - with no flags, the snapshot is not used anymore,
>> * but also not removed. With a MERGE flag, it merges the snapshot into
>> * the parent snapshot (or the base image, if there is no parent snapshot).
>> * Note that if other snapshots would be discarded because of this
>> * MERGE action, this operation will fail. If that is really what is intended,
>> * use MERGE_FORCE.
>> *
>> * With a DISCARD flag, it deletes the snapshot. Note that if children
snapshots
>> * would be discarded because of this delete action, this operation will
>> * fail. If this is really what is intended, use DISCARD_FORCE.
>> *
>> * MERGE, MERGE_FORCE, DISCARD, and DISCARD_FORCE are mutually-exclusive.
>> *
>> * Note that this operation can happen when the domain is running or shut
>> * down, though this is hypervisor specific */
>> typedef enum {
>> VIR_DOMAIN_SNAPSHOT_DELETE_MERGE,
>> VIR_DOMAIN_SNAPSHOT_DELETE_MERGE_FORCE,
>> VIR_DOMAIN_SNAPSHOT_DELETE_DISCARD,
>> VIR_DOMAIN_SNAPSHOT_DELETE_DISCARD_FORCE,
>> } virDomainSnapshotDelete;
>
> Merging a snapshot into its parent is probably not the best semantics for
> MERGE flag as hypervisors differ in the way merging is implemented. As you
Yeah, you are right, we can't just declare this. It's also an open question
to me what qemu does about merging (though bugs with loadvm/delvm are preventing
me from testing this at the moment).
And as described in my other response ESX does merge-into-children
too, but I didn't realize this until now. I just used the wrong words
to describe it at first.
So, after I thought about this in detail, I'd like to ask: Is there
something like a merge-into-parent semantic at all?
> also mention below, VirtualBox merges into all children instead
of a parent.
> We should allow for both cases. However it influences several things. Firstly,
> it makes MERGE_FORCE unnecessary for child merging, which is not a big deal as
> it can just be treated in the same way as MERGE. Secondly, it makes a huge
> difference when deleting a snapshot with no child. In one case it results in
> changes being merged and in other case it results changes begin dropped.
>
> One option is to refine the semantics to something like:
>
> - MERGE: merge changes into other snapshot(s) and fail if it would require any
> snapshot to be discarded (even the one which was supposed to be merged)
> - MERGE_FORCE: really merge even discarding other snapshots but fail if the
> snapshot itself would actually be discarded
> - DISCARD: discard the snapshot and fail if other snapshots would be discarded
> - DISCARD_FORCE: discard, no matter what
The problem with declaring these semantics is that they are somewhat confusing, so
application developers will probably get them wrong. On the other hand it does
allow us to declare a semantic that all 3 hypervisors probably can support,
unlike the options below.
>
> Another option would be to introduce several different APIs for merging into
> children, merging into parent, and discarding. That would allow drivers to
> implement only supported methods. Even all of them for a very flexible
> hypervisor.
The problem with this one is that it will be difficult for application writers
to write a single application that handles all of the hypervisors. Imagine trying
to write a GUI around this, and you'll see what I mean. If we were to add 3 new
API's like this, we would also probably want to add some data to the capabilities
XML for the particular hypervisors so you could grey out specific options in a GUI.
On the other hand, it does solve the problem with merging parents vs. children, and
also diffuses Paolo's concern about the "SnapshotDelete" API sometimes
deleting data
and sometimes modifying the base image.
Again, I think there is nothing like "merging parents vs. children".
The only possibility is merging into children. Or am I totally
confused now?
>
> And the third option I see would be distinguishing merge direction using new
> flags.
This one is like the second option, in that you don't know which particular
directions a particular hypervisor supports. You'd still need to add
capabilities XML for each hypervisor.
I have to say that after thinking about these 3 options, I like the first
option the best. While it's slightly confusing, it is a good semantic. I'll
update the documentation for this.
>
> Personally, I like the second option best as it provides the easiest way for
> application to detect unsupported behavior.
>
> ...
>> Possible issues:
>> 1) I don't see a way to support "managed" save/restore and
snapshotting with
>> this API. I think we'll have to have a separate API for managed
save/restore.
>> 2) What is the semantic for deleting snapshots from a running domain?
>> Virtualbox seems to not allow you to manipulate snapshots while the domain is
>> running. Qemu does allow this, but it's currently unclear what the exact
>> semantics are. VMware seems to allow manipulation of snapshots while the
>> domain is running.
>> 3) Do we need a snapshot UUID? Virtualbox allows you to have multiple
snapshots
>> with the same name, differentiated by UUID. Confusingly, they also have a
>> "FindByName" method that returns the first depth-first search snapshot
that matches
>> a given name. For qemu, if you specify the same name twice it overwrites the
previous
>> one with the new one. I don't know what ESX does here.
>
> Libvirt uses/generates UUIDs for almost everything (networks, vms, ...) so it
> might be more consistent to have UUID in snapshot as well.
Yeah, that is true, which is why I'm waffling with it. In general it seems like
superflous
information, except in the one case of virtualbox duplicate names (ESX doesn't allow
duplicate names, I don't think, and qemu blows away duplicates). My inclination is
to declare the semantics of duplicate names to be undefined, since it doesn't seem
to be a very useful feature.
As said in another response: ESX 4.0 allows duplicate names. It
distinguishes snapshots based on their ID. But this ID was added in
ESX 4.0. I think ESX 3.5 doesn't allow duplicate names.
With ESX 4.0 we could use the ID to derive a read-only UUID from it.
With ESX 3.5 we would need to abuse the name filed of a snapshot on
the ESX side to store a UUID per snapshot.
Matthias
3:14 a.m.
On Tue, Mar 30, 2010 at 06:13:07PM -0400, Chris Lalancette wrote:
On 03/30/2010 04:40 PM, Jiri Denemark wrote:
[...]
> also mention below, VirtualBox merges into all children instead
of a parent.
> We should allow for both cases. However it influences several things. Firstly,
> it makes MERGE_FORCE unnecessary for child merging, which is not a big deal as
> it can just be treated in the same way as MERGE. Secondly, it makes a huge
> difference when deleting a snapshot with no child. In one case it results in
> changes being merged and in other case it results changes begin dropped.
>
> One option is to refine the semantics to something like:
>
> - MERGE: merge changes into other snapshot(s) and fail if it would require any
> snapshot to be discarded (even the one which was supposed to be merged)
> - MERGE_FORCE: really merge even discarding other snapshots but fail if the
> snapshot itself would actually be discarded
> - DISCARD: discard the snapshot and fail if other snapshots would be discarded
> - DISCARD_FORCE: discard, no matter what
The problem with declaring these semantics is that they are somewhat confusing, so
application developers will probably get them wrong. On the other hand it does
allow us to declare a semantic that all 3 hypervisors probably can support,
unlike the options below.
Well I think if we documents the 4 different flags with small graphic
examples showing what happen to hierarchies of snapshot on the
operation, I don't see why the users would be confused.
The only other option to avoid user confusion is to reduce
capabilities drastically, and I don't think we want to go there.
> Libvirt uses/generates UUIDs for almost everything (networks,
vms, ...) so it
> might be more consistent to have UUID in snapshot as well.
Yeah, that is true, which is why I'm waffling with it. In general it seems like
superflous
information, except in the one case of virtualbox duplicate names (ESX doesn't allow
duplicate names, I don't think, and qemu blows away duplicates). My inclination is
to declare the semantics of duplicate names to be undefined, since it doesn't seem
to be a very useful feature.
IMHO uuid allows to name things uniquely when we can't check for
uniqueness at creation (e.g. the domain has been moving around and
we don't have a shared storage for snaphots). That's an edge case
but if using also UUID is cheap I would go for it.
Daniel
--
Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/
daniel(a)veillard.com | Rpmfind RPM search engine http://rpmfind.net/
http://veillard.com/ | virtualization library http://libvirt.org/
7:14 p.m.
2010/3/30 Chris Lalancette <clalance(a)redhat.com>:
Hello,
After our discussions about the snapshot API last week, I went ahead and implemented
quite a bit of the API. I also went back to the ESX, Virtualbox, and QEMU API's to
try and make sure our API's matched up. What's below is my revised API based on
that survey. Following my revised API are notes that I took regarding how the
libvirt API matches up to the various API's, and some questions about semantics that
I had while doing the survey. More comments and questions are welcome.
/* Start the guest from the snapshot "snapshot" */
int virDomainCreateFromSnapshot(virDomainSnapshotPtr snapshot,
unsigned int flags);
Will it be enforced that the domain is shutdown in order to call this function?
ESX doesn't have such a restriction. Not sure about other hypervisors.
/* Rename the snapshot */
/* Do we really need this? In theory we could use
* virsh snapshot-edit <domain> <name> and then detect
* name changes, but that will require a UUID, which may
* or may not be overkill
*/
int virDomainSnapshotRename(virDomainSnapshotPtr snapshot,
char *newname,
unsigned int flags);
/* Delete a snapshot - with no flags, the snapshot is not used anymore,
* but also not removed. With a MERGE flag, it merges the snapshot into
* the parent snapshot (or the base image, if there is no parent snapshot).
Note that "merging into the parent" seems to be the wrong term, even
in case of ESX. See detailed discussion below.
* Note that if other snapshots would be discarded because of this
* MERGE action, this operation will fail. If that is really what is intended,
* use MERGE_FORCE.
*
* With a DISCARD flag, it deletes the snapshot. Note that if children snapshots
* would be discarded because of this delete action, this operation will
* fail. If this is really what is intended, use DISCARD_FORCE.
*
* MERGE, MERGE_FORCE, DISCARD, and DISCARD_FORCE are mutually-exclusive.
*
* Note that this operation can happen when the domain is running or shut
* down, though this is hypervisor specific */
typedef enum {
VIR_DOMAIN_SNAPSHOT_DELETE_MERGE,
VIR_DOMAIN_SNAPSHOT_DELETE_MERGE_FORCE,
VIR_DOMAIN_SNAPSHOT_DELETE_DISCARD,
VIR_DOMAIN_SNAPSHOT_DELETE_DISCARD_FORCE,
} virDomainSnapshotDelete;
int virDomainSnapshotDelete(virDomainSnapshotPtr snapshot,
unsigned int flags);
int virDomainSnapshotFree(virDomainSnapshotPtr snapshot);
NOTE: During snapshot creation, *none* of the fields are required. That is,
you can call virDomainSnapshotCreateXML() with an XML of
"<domainsnapshot/>".
In this case, the individual driver will make up a <name> and <uuid> for you,
Does <uuid> here refer to a snapshot UUID? As said before, there is no
easy way have a UUID per snapshot with ESX. Well, we could store
<uuid>:<name> in the name field on the ESX side, but that's not a
really good way to do it.
the <creationdate> will be set to the current time+date,
<description> will be
empty, <state> will be the current state of the VM, and <parent> will
be set to the current snapshot (if any). If you do want to specify some
fields during virDomainSnapshotCreateXML(), note that the only ones that are
settable are <name>, <uuid>, and <description>;
the rest are ignored, and filled in by the driver when the snapshot is
actually created.
NOTE: <state> refers to the state of the VM when the snapshot was taken.
<domainsnapshot>
<name>XYZ</name>
<creationdate>...</creationdate>
<description>...</description>
<state>RUNNING</state>
<domain>
<uuid>XXXXX-XXXX-XXXX-XXXX-XXXXXXXXX</uuid>
</domain>
<parent>
<name>ABC</name>
</parent>
</domainsnapshot>
The virsh commands will be:
virsh snapshot-create <dom> <xmlfile>
virsh snapshot-list <dom>
virsh snapshot-dumpxml <dom> <name>
virsh start-with-snapshot <dom> <snapshotname>
virsh snapshot-delete <dom> <snapshotname>
[--merge|--mergeforce|--delete|--deleteforce]
virsh snapshot-delete-all <dom>
Possible issues:
1) I don't see a way to support "managed" save/restore and snapshotting
with
this API. I think we'll have to have a separate API for managed save/restore.
What's "managed" save/restore and snapshotting?
3) Do we need a snapshot UUID? Virtualbox allows you to have
multiple snapshots
with the same name, differentiated by UUID. Confusingly, they also have a
"FindByName" method that returns the first depth-first search snapshot that
matches
a given name. For qemu, if you specify the same name twice it overwrites the previous
one with the new one. I don't know what ESX does here.
ESX 4.0 allows multiple snapshots with the same name. I think this is
because ESX 4.0 has an integer ID per snapshot. I'm not sure if ESX
3.5 allows multiple snapshots with the same name, because the ID field
was added in ESX 4.0. I assume ESX 3.5 doesn't allow multiple
snapshots with the same name, but I have currently no ESX 3.5 at hand
to test.
We could use this integer ID and convert it to UUID format, but you
won't be able to set the UUID, it'll be read-only and only available
on ESX 4.0 and above.
Mapping of our interface to various hypervisors:
+-------------------------------+-----------------+-------------------+------------------------------+
| Libvirt | Qemu | Virtualbox | ESX
|
+-------------------------------+-----------------+-------------------+------------------------------+
| virDomainSnapshotCreateXML | monitor command | takeSnapshot |
CreateSnapshot_task |
| | "savevm"; if | Snapshots can | takes a
name, description, |
| | snapshot name | be taken on | memory
(true/false) and |
| | is already in | powered off, | quiesce
(true/false). |
| | use, replaces | saved, running, | What does
"memory" mean? |
If memory is true, ESX snapshots the memory of the domain too,
otherwise only a disk snapshot is created.
Creating a disk-only snapshot is nearly instant, while creating a
memory snapshot also requires a notable amount of time to write the
memory image to disk.
| | the previous | or paused VMs.
| Should we model "quiesce" |
The vSphere API docs give a good description what the quiesce option does:
"If TRUE and the virtual machine is powered on when the snapshot is
taken, VMware Tools is used to quiesce the file system in the virtual
machine. This assures that a disk snapshot represents a consistent
state of the guest file systems. If the virtual machine is powered off
or VMware Tools are not available, the quiesce flag is ignored."
I assume "quiesce the file system" means to flush write caches and
stuff like that.
This option is important if you want to create a disk-only snapshot of
a running domain.
| | snapshot. Also | The snapshot is
| Trees of snapshots are |
| | qemu-img | always taken | supported. What
happens |
| | snapshot -c can | against the | on a duplicate
name? What |
| | be used to | current snapshot. | state(s) can a VM
be in |
| | create a | What happens on | when calling
this? Does |
| | disk-only | a duplicate | a VM get paused
when this |
| | snapshot. What | name? Trees of | is called?
|
In case of ESX the domain can be in any state when a snapshot is created.
If the domain is running when you create a snapshot then the domain is
_not_ paused during the snapshot creation.
I tested it and the memory snapshot represents the state at the time
the snapshot command was issued.
| | happens if the | snapshots are
| |
| | VM is running | not currently |
|
| | when you do | supported. |
|
| | this? Trees of | Taking a snapshot |
|
| | snapshots seem | of a running VM |
|
| | to be supported | pauses the VM |
|
| | VM gets paused | before taking the |
|
| | while this is | snapshot. |
|
| | happening. What | |
|
| | states can the | |
|
| | VM be in? | |
|
+-------------------------------+-----------------+-------------------+------------------------------+
+-------------------------------+-----------------+-------------------+------------------------------+
| virDomainSnapshotDelete | monitor command | deleteSnapshot |
RemoveSnapshot_Task |
| | "delvm". What | deletes the | removes
this snapshot and |
| | happens if the | specified | deletes any
associated |
| | snapshot is in | snapshot. Takes | storage.
Operates on a |
| | use? What | an ID. The VM |
VirtualMachineSnapshot |
| | states can the | must be off. | object. What
states can |
| | VM be in? Also | Differences to | the VM be in?
What |
| | qemu-img | children | happens if this
snapshot |
| | snapshot -d | snapshots will be | is in-use? What
happens |
| | <name> <file> | merged with the | to
parents and children? |
| | command can be | children to keep |
|
| | used. What | children valid. |
|
| | happens if the | Parent for this |
|
| | disk is in-use? | snapshot will |
|
| | What happens to | become parent of |
|
| | parents and | any children |
|
| | children? | snapshots. |
|
| | How do we | |
|
| | handle merges? | |
|
+-------------------------------+-----------------+-------------------+------------------------------+
The domain can be in any state when deleting a snapshot, even if you
delete the current snapshot. VMware has some documentation about how a
snapshot is merged into its parent:
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&...
And some more general docs about snapshots:
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&...
Regarding what get's merged and where, I should define the terms I'm
using first.
A <--1-- B <--2-- C <--3-- current
+ <--4-- D
I intentionally draw the arrows directed from child to parent.
A, B, C, D are what I call a snapshot, a point in "time" I can switch
to. The disk differences between these points are stored in COW sparse
images, here shown as 1, 2, 3, 4. The current state of the domain is
denoted by the "current" item.
Each snapshot is associated with a disk image: A is associated with
the base image, B with sparse image 1, C with 2 and so on. A special
case is sparse image 3, it's not associated with a snapshot, but with
the current state. Also each snapshot can be associated with a memory
image (not shown here).
The current snapshot in this case is C. If the domain writes changes
to disk, these changes get stored in sparse image 3. If you switch to
another snapshot from here then the changes in 3 are lost, because you
cannot go back to a point where you could access the changes in 3
again.
Now lets delete B. In this case the memory images associated with B is
just discarded and 1 and 2 are merged into 5. That's what I was
referring to when I said ESX merges snapshots into the parent.
A <------5------- C <--3-- current
+ <--4-- D
But this only happens for snapshots like B, that have a parent and a
child (C is such a snapshot too, even if its child isn't an actual
snapshot). If you delete D in this example, then the changes in sparse
image 4 are discarded, because there is no place where they could be
merged. Merging 4 in the base image would alter A, merging 4 and 5
would alter C.
Now as I think of this in detail, it seems that the term "merging into
the parent" is wrong.
In the next example we have snapshot E with parent B.
A <--1-- B <--2-- C <--3-- current
+ <--6-- E
Now what's going to happen if we delete B? In order to preserve C and
E, the changes in 1 need to be merged into 2 and 6, this results in 1
+ 2 = 5 and 1 + 6 = 7. Or rephrased: B is merged in toits children C
and E.
A <------5------- C <--3-- current
+ <------7------- E
So, virDomainSnapshotDelete's semantic for VirtualBox and ESX seems to
be the same. I just used the wrong words to describe it at first.
Sorry for that.
Matthias
12:16 p.m.
On 03/30/2010 08:14 PM, Matthias Bolte wrote:
2010/3/30 Chris Lalancette <clalance(a)redhat.com>:
> Hello,
> After our discussions about the snapshot API last week, I went ahead and
implemented
> quite a bit of the API. I also went back to the ESX, Virtualbox, and QEMU API's
to
> try and make sure our API's matched up. What's below is my revised API based
on
> that survey. Following my revised API are notes that I took regarding how the
> libvirt API matches up to the various API's, and some questions about semantics
that
> I had while doing the survey. More comments and questions are welcome.
> /* Start the guest from the snapshot "snapshot" */
> int virDomainCreateFromSnapshot(virDomainSnapshotPtr snapshot,
> unsigned int flags);
Will it be enforced that the domain is shutdown in order to call this function?
ESX doesn't have such a restriction. Not sure about other hypervisors.
Heh, I was just going through that myself. No, it's not required to be
shutdown in general; qemu supports both modes. I've updated the documentation
for this call.
<snip>
> * Note that if other snapshots would be discarded because of
this
> * MERGE action, this operation will fail. If that is really what is intended,
> * use MERGE_FORCE.
> *
> * With a DISCARD flag, it deletes the snapshot. Note that if children snapshots
> * would be discarded because of this delete action, this operation will
> * fail. If this is really what is intended, use DISCARD_FORCE.
> *
> * MERGE, MERGE_FORCE, DISCARD, and DISCARD_FORCE are mutually-exclusive.
> *
> * Note that this operation can happen when the domain is running or shut
> * down, though this is hypervisor specific */
> typedef enum {
> VIR_DOMAIN_SNAPSHOT_DELETE_MERGE,
> VIR_DOMAIN_SNAPSHOT_DELETE_MERGE_FORCE,
> VIR_DOMAIN_SNAPSHOT_DELETE_DISCARD,
> VIR_DOMAIN_SNAPSHOT_DELETE_DISCARD_FORCE,
> } virDomainSnapshotDelete;
> int virDomainSnapshotDelete(virDomainSnapshotPtr snapshot,
> unsigned int flags);
>
> int virDomainSnapshotFree(virDomainSnapshotPtr snapshot);
>
> NOTE: During snapshot creation, *none* of the fields are required. That is,
> you can call virDomainSnapshotCreateXML() with an XML of
"<domainsnapshot/>".
> In this case, the individual driver will make up a <name> and <uuid> for
you,
Does <uuid> here refer to a snapshot UUID? As said before, there is no
easy way have a UUID per snapshot with ESX. Well, we could store
<uuid>:<name> in the name field on the ESX side, but that's not a
really good way to do it.
Yeah, agreed, that was a leftover I forgot to edit out. See my reply to
Jiri Denemark, but essentially I'm content to declare duplicate names
unsupported/undefined, and not deal with UUID's at all. I've removed
mention of UUID's from the documentation now.
<snip>
> The virsh commands will be:
> virsh snapshot-create <dom> <xmlfile>
> virsh snapshot-list <dom>
> virsh snapshot-dumpxml <dom> <name>
> virsh start-with-snapshot <dom> <snapshotname>
> virsh snapshot-delete <dom> <snapshotname>
[--merge|--mergeforce|--delete|--deleteforce]
> virsh snapshot-delete-all <dom>
>
> Possible issues:
> 1) I don't see a way to support "managed" save/restore and
snapshotting with
> this API. I think we'll have to have a separate API for managed save/restore.
What's "managed" save/restore and snapshotting?
Oops, yeah, that's a personal note that I didn't really expound upon. One of the
reasons that I originally started down the path of implementing snapshotting was
to implement save/restore for guest during host shutdown
and startup. Because of the way autostart works within libvirt, we can't have
an external script (ala xendomains) do this; it needs to be handled inside the
libvirt daemon itself, and our current save/restore API is not sufficient for
this. That being said, after all of the discussions we have had about
this snapshotting API, I don't think it will be appropriate to shoehorn this
"managed"
save/restore into this API, and we'll need a separate API for that.
> 3) Do we need a snapshot UUID? Virtualbox allows you to have multiple snapshots
> with the same name, differentiated by UUID. Confusingly, they also have a
> "FindByName" method that returns the first depth-first search snapshot that
matches
> a given name. For qemu, if you specify the same name twice it overwrites the
previous
> one with the new one. I don't know what ESX does here.
ESX 4.0 allows multiple snapshots with the same name. I think this is
because ESX 4.0 has an integer ID per snapshot. I'm not sure if ESX
3.5 allows multiple snapshots with the same name, because the ID field
was added in ESX 4.0. I assume ESX 3.5 doesn't allow multiple
snapshots with the same name, but I have currently no ESX 3.5 at hand
to test.
We could use this integer ID and convert it to UUID format, but you
won't be able to set the UUID, it'll be read-only and only available
on ESX 4.0 and above.
Yeah, again, I'm happy to drop UUID and declare duplicate names unsupported
unless there is a good use case.
> Mapping of our interface to various hypervisors:
>
+-------------------------------+-----------------+-------------------+------------------------------+
> | Libvirt | Qemu | Virtualbox | ESX
|
>
+-------------------------------+-----------------+-------------------+------------------------------+
> | virDomainSnapshotCreateXML | monitor command | takeSnapshot |
CreateSnapshot_task |
> | | "savevm"; if | Snapshots can |
takes a name, description, |
> | | snapshot name | be taken on | memory
(true/false) and |
> | | is already in | powered off, | quiesce
(true/false). |
> | | use, replaces | saved, running, | What does
"memory" mean? |
If memory is true, ESX snapshots the memory of the domain too,
otherwise only a disk snapshot is created.
Creating a disk-only snapshot is nearly instant, while creating a
memory snapshot also requires a notable amount of time to write the
memory image to disk.
Sorry, I misread the documentation yesterday. That's fairly clear.
What's less clear to me is what happens when you take a disk-only snapshot,
and then try to RevertToSnapshot from a running VM. What happens in that case?
> | | the previous | or paused VMs. | Should we
model "quiesce" |
The vSphere API docs give a good description what the quiesce option does:
"If TRUE and the virtual machine is powered on when the snapshot is
taken, VMware Tools is used to quiesce the file system in the virtual
machine. This assures that a disk snapshot represents a consistent
state of the guest file systems. If the virtual machine is powered off
or VMware Tools are not available, the quiesce flag is ignored."
I assume "quiesce the file system" means to flush write caches and
stuff like that.
This option is important if you want to create a disk-only snapshot of
a running domain.
Exactly. I'm not sure this is going to be possible in general (and
I guess it's not even really possible in ESX unless you install VMware
Tools inside the guest). I'm inclined not to model it at the moment,
although I could be convinced otherwise.
> | | snapshot. Also | The snapshot is | Trees of
snapshots are |
> | | qemu-img | always taken | supported.
What happens |
> | | snapshot -c can | against the | on a
duplicate name? What |
> | | be used to | current snapshot. | state(s) can
a VM be in |
> | | create a | What happens on | when calling
this? Does |
> | | disk-only | a duplicate | a VM get
paused when this |
> | | snapshot. What | name? Trees of | is called?
|
In case of ESX the domain can be in any state when a snapshot is created.
If the domain is running when you create a snapshot then the domain is
_not_ paused during the snapshot creation.
I tested it and the memory snapshot represents the state at the time
the snapshot command was issued.
OK, great. I'll update these notes about that.
> | | happens if the | snapshots are |
|
> | | VM is running | not currently |
|
> | | when you do | supported. |
|
> | | this? Trees of | Taking a snapshot |
|
> | | snapshots seem | of a running VM |
|
> | | to be supported | pauses the VM |
|
> | | VM gets paused | before taking the |
|
> | | while this is | snapshot. |
|
> | | happening. What | |
|
> | | states can the | |
|
> | | VM be in? | |
|
>
+-------------------------------+-----------------+-------------------+------------------------------+
>
+-------------------------------+-----------------+-------------------+------------------------------+
> | virDomainSnapshotDelete | monitor command | deleteSnapshot |
RemoveSnapshot_Task |
> | | "delvm". What | deletes the |
removes this snapshot and |
> | | happens if the | specified | deletes any
associated |
> | | snapshot is in | snapshot. Takes | storage.
Operates on a |
> | | use? What | an ID. The VM |
VirtualMachineSnapshot |
> | | states can the | must be off. | object. What
states can |
> | | VM be in? Also | Differences to | the VM be in?
What |
> | | qemu-img | children | happens if
this snapshot |
> | | snapshot -d | snapshots will be | is in-use?
What happens |
> | | <name> <file> | merged with the |
to parents and children? |
> | | command can be | children to keep |
|
> | | used. What | children valid. |
|
> | | happens if the | Parent for this |
|
> | | disk is in-use? | snapshot will |
|
> | | What happens to | become parent of |
|
> | | parents and | any children |
|
> | | children? | snapshots. |
|
> | | How do we | |
|
> | | handle merges? | |
|
>
+-------------------------------+-----------------+-------------------+------------------------------+
The domain can be in any state when deleting a snapshot, even if you
delete the current snapshot. VMware has some documentation about how a
snapshot is merged into its parent:
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&...
And some more general docs about snapshots:
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&...
Regarding what get's merged and where, I should define the terms I'm
using first.
A <--1-- B <--2-- C <--3-- current
+ <--4-- D
I intentionally draw the arrows directed from child to parent.
A, B, C, D are what I call a snapshot, a point in "time" I can switch
to. The disk differences between these points are stored in COW sparse
images, here shown as 1, 2, 3, 4. The current state of the domain is
denoted by the "current" item.
Each snapshot is associated with a disk image: A is associated with
the base image, B with sparse image 1, C with 2 and so on. A special
case is sparse image 3, it's not associated with a snapshot, but with
the current state. Also each snapshot can be associated with a memory
image (not shown here).
The current snapshot in this case is C. If the domain writes changes
to disk, these changes get stored in sparse image 3. If you switch to
another snapshot from here then the changes in 3 are lost, because you
cannot go back to a point where you could access the changes in 3
again.
Now lets delete B. In this case the memory images associated with B is
just discarded and 1 and 2 are merged into 5. That's what I was
referring to when I said ESX merges snapshots into the parent.
A <------5------- C <--3-- current
+ <--4-- D
But this only happens for snapshots like B, that have a parent and a
child (C is such a snapshot too, even if its child isn't an actual
snapshot). If you delete D in this example, then the changes in sparse
image 4 are discarded, because there is no place where they could be
merged. Merging 4 in the base image would alter A, merging 4 and 5
would alter C.
Now as I think of this in detail, it seems that the term "merging into
the parent" is wrong.
In the next example we have snapshot E with parent B.
A <--1-- B <--2-- C <--3-- current
+ <--6-- E
Now what's going to happen if we delete B? In order to preserve C and
E, the changes in 1 need to be merged into 2 and 6, this results in 1
+ 2 = 5 and 1 + 6 = 7. Or rephrased: B is merged in toits children C
and E.
A <------5------- C <--3-- current
+ <------7------- E
So, virDomainSnapshotDelete's semantic for VirtualBox and ESX seems to
be the same. I just used the wrong words to describe it at first.
Sorry for that.
OK, that's very interesting to know. So VirtualBox and ESX seem to do the
same thing here. This is the last thing I have to do testing on with qemu
to get it's semantic; I'll get to that today, and then we can look again
at the semantics of the flags to virDomainSnapshotDelete.
--
Chris Lalancette
12:33 p.m.
2010/3/31 Chris Lalancette <clalance(a)redhat.com>:
On 03/30/2010 08:14 PM, Matthias Bolte wrote:
> 2010/3/30 Chris Lalancette <clalance(a)redhat.com>:
>> Hello,
>> After our discussions about the snapshot API last week, I went ahead and
implemented
>> quite a bit of the API. I also went back to the ESX, Virtualbox, and QEMU
API's to
>> try and make sure our API's matched up. What's below is my revised API
based on
>> that survey. Following my revised API are notes that I took regarding how the
>> libvirt API matches up to the various API's, and some questions about
semantics that
>> I had while doing the survey. More comments and questions are welcome.
>> Mapping of our interface to various hypervisors:
>>
+-------------------------------+-----------------+-------------------+------------------------------+
>> | Libvirt | Qemu | Virtualbox | ESX
|
>>
+-------------------------------+-----------------+-------------------+------------------------------+
>> | virDomainSnapshotCreateXML | monitor command | takeSnapshot |
CreateSnapshot_task |
>> | | "savevm"; if | Snapshots can |
takes a name, description, |
>> | | snapshot name | be taken on | memory
(true/false) and |
>> | | is already in | powered off, | quiesce
(true/false). |
>> | | use, replaces | saved, running, | What does
"memory" mean? |
>
> If memory is true, ESX snapshots the memory of the domain too,
> otherwise only a disk snapshot is created.
>
> Creating a disk-only snapshot is nearly instant, while creating a
> memory snapshot also requires a notable amount of time to write the
> memory image to disk.
Sorry, I misread the documentation yesterday. That's fairly clear.
What's less clear to me is what happens when you take a disk-only snapshot,
and then try to RevertToSnapshot from a running VM. What happens in that case?
If the domain is running and you revert to a disk-only snapshot then
the domain gets shutdown. Or rephrased: the power state associated
with a disk-only snapshot is always powered-off, even if the domain
was running while the disk-only snapshot was created.
>
>> | | the previous | or paused VMs. | Should we
model "quiesce" |
>
> The vSphere API docs give a good description what the quiesce option does:
>
> "If TRUE and the virtual machine is powered on when the snapshot is
> taken, VMware Tools is used to quiesce the file system in the virtual
> machine. This assures that a disk snapshot represents a consistent
> state of the guest file systems. If the virtual machine is powered off
> or VMware Tools are not available, the quiesce flag is ignored."
>
> I assume "quiesce the file system" means to flush write caches and
> stuff like that.
>
> This option is important if you want to create a disk-only snapshot of
> a running domain.
Exactly. I'm not sure this is going to be possible in general (and
I guess it's not even really possible in ESX unless you install VMware
Tools inside the guest). I'm inclined not to model it at the moment,
although I could be convinced otherwise.
Yes, quiesce requires VMware Tools to be installed in the guest.
Matthias
11:16 a.m.
...
/* Get a handle to the current in-use snapshot for the domain */
virDomainSnapshotPtr virDomainSnapshotCurrent(virDomainPtr domain,
unsigned int flags);
This call needs to be changed a bit to allow distinguishing between an error
and a domain with no snapshots (i.e., no current snapshot):
int virDomainSnapshotCurrent(virDomainPtr domain,
virDomainSnapshotPtr *snapshot,
unsigned int flags);
Jirka
11:24 a.m.
On Wed, Mar 31, 2010 at 06:16:12PM +0200, Jiri Denemark wrote:
...
> /* Get a handle to the current in-use snapshot for the domain */
> virDomainSnapshotPtr virDomainSnapshotCurrent(virDomainPtr domain,
> unsigned int flags);
This call needs to be changed a bit to allow distinguishing between an error
and a domain with no snapshots (i.e., no current snapshot):
int virDomainSnapshotCurrent(virDomainPtr domain,
virDomainSnapshotPtr *snapshot,
unsigned int flags);
I think it'd be nicer to keep the first syntax, but add a way to query if
a guest has a current snapshot
int virDomainSnapshotHasCurrent(virDomainPtr dom);
That way, virDomainSnapshotCurrent() has no need to distingush between the
'no snapshots' error and other types of errors.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
5349
days inactive
5350
days old
13 comments
6 participants
participants (6)
-
Chris Lalancette -
Daniel P. Berrange -
Daniel Veillard -
Eric Blake -
Jiri Denemark -
Matthias Bolte