[libvirt] question about save/restore domain
Hi, my name is Jovanka Gulicoska and I'm SoC student. I'm working on implementing save/load of VM in Gnome-Boxes. Something confused me about snapshots. In the documentation about life cycle, in section about Save/Restore is said that domain saved with virDomainSave/virDomainSaveFlags can be restored only once. But in the documentation about Snapshot XML format there is no information about this in VM state snapshots. So is it better just to use virDomainSnapshotCreateXML instead of virDomainSave/virDomainSaveFlags? I've created bindings for this functions in libvirt-glib but I wanted to be sure before I send them for review. Thanks, Jovanka (gulic)
On 07/03/2012 06:42 AM, Jovanka Gulicoska wrote:
Hi,
my name is Jovanka Gulicoska and I'm SoC student. I'm working on implementing save/load of VM in Gnome-Boxes. Something confused me about snapshots. In the documentation about life cycle, in section about Save/Restore is said that domain saved with virDomainSave/virDomainSaveFlags can be restored only once.
Can you point out the URL to that documentation? It's slightly incorrect, and worth patching to make clearer if that would help the next person to read them. virDomainRestore can be used to restore a domain as many times as you want, _provided_ that before each restore, you _manually_ restore the disk state back to what it was at the time of the virDomainSave. virDomainManagedSave uses virDomainSave under the hood to create a save image which is then autoloaded at the next virDomainStart, but because it is auto-managed, it can only be restored once (on restoring a managedsave domain, the managed state is discarded).
But in the documentation about Snapshot XML format there is no information about this in VM state snapshots.
virDomainSnapshotCreateXML is able to create a system checkpoint of both disk and RAM state at once, such that virDomainSnapshotRevert then restores disk state and memory state in one operation, instead of leaving the disk state to manual operation.
So is it better just to use virDomainSnapshotCreateXML instead of virDomainSave/virDomainSaveFlags?
It all depends on what your end goal is with doing a save/load of a VM. Do you need fast saves, or is it okay if the save takes several seconds? Will you be loading the state exactly once, or do you plan to make it something that the user can revert to multiple times in a row?
I've created bindings for this functions in libvirt-glib but I wanted to be sure before I send them for review.
Feel free to ask more questions on the topic; I'll do my best to answer them. -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Tue, Jul 3, 2012 at 6:01 PM, Eric Blake <eblake@redhat.com> wrote:
On 07/03/2012 06:42 AM, Jovanka Gulicoska wrote:
Hi,
my name is Jovanka Gulicoska and I'm SoC student. I'm working on implementing save/load of VM in Gnome-Boxes. Something confused me about snapshots. In the documentation about life cycle, in section about Save/Restore is said that domain saved with virDomainSave/virDomainSaveFlags can be restored only once.
Can you point out the URL to that documentation? It's slightly incorrect, and worth patching to make clearer if that would help the next person to read them.
This is in the draft of the documentation: http://libvirt.org/guide/html-single/#Application_Development_Guide-Lifecycl... --- "For basic usage this implies that a guest can only be restored once from any given saved state image."
virDomainRestore can be used to restore a domain as many times as you want, _provided_ that before each restore, you _manually_ restore the disk state back to what it was at the time of the virDomainSave.
virDomainManagedSave uses virDomainSave under the hood to create a save image which is then autoloaded at the next virDomainStart, but because it is auto-managed, it can only be restored once (on restoring a managedsave domain, the managed state is discarded).
I've also created bindings for virDomainRestore and virDomainManagedSave is already implemented.
But in the documentation about Snapshot XML format there is no information about this in VM state snapshots.
virDomainSnapshotCreateXML is able to create a system checkpoint of both disk and RAM state at once, such that virDomainSnapshotRevert then restores disk state and memory state in one operation, instead of leaving the disk state to manual operation.
I understood this and I've also implemented it.
So is it better just to use virDomainSnapshotCreateXML instead of virDomainSave/virDomainSaveFlags?
It all depends on what your end goal is with doing a save/load of a VM. Do you need fast saves, or is it okay if the save takes several seconds? Will you be loading the state exactly once, or do you plan to make it something that the user can revert to multiple times in a row?
The main idea is to allow the user to save the virtual machine and be able to load it when he wants and be able to share it with others who uses Boxes, so the other person can also load it on his machine (in Boxes). I was not sure which patches to include to the review and which functions should I use in Boxes. I'm testing this now.
I've created bindings for this functions in libvirt-glib but I wanted to be sure before I send them for review.
Feel free to ask more questions on the topic; I'll do my best to answer them.
Thanks for your response. And please correct me if I misunderstood something. Jovanka
-- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On 07/03/2012 11:35 AM, Jovanka Gulicoska wrote:
On Tue, Jul 3, 2012 at 6:01 PM, Eric Blake <eblake@redhat.com> wrote:
On 07/03/2012 06:42 AM, Jovanka Gulicoska wrote:
Hi,
my name is Jovanka Gulicoska and I'm SoC student. I'm working on implementing save/load of VM in Gnome-Boxes. Something confused me about snapshots. In the documentation about life cycle, in section about Save/Restore is said that domain saved with virDomainSave/virDomainSaveFlags can be restored only once.
Can you point out the URL to that documentation? It's slightly incorrect, and worth patching to make clearer if that would help the next person to read them.
This is in the draft of the documentation: http://libvirt.org/guide/html-single/#Application_Development_Guide-Lifecycl... --- "For basic usage this implies that a guest can only be restored once from any given saved state image."
Ah, and I think reading that sentence in context clarifies the situation: "Thus when the guest is restored, the underlying guest storage must be in exactly the same state as it was when the guest was initially saved. For basic usage this implies that a guest can only be restored once from any given saved state image. To allow a guest to be restored from the same saved state multiple times, the application must also have taken a snapshot of the guest storage at time of saving, and explicitly revert to this storage snapshot when restoring." Either you manually revert the storage back to match the save image state before restoring, or you use things in a more basic manner (never touch the storage manually, and therefore restore works only once because the storage was unchanged between the save/restore cycle but is now altered after the successful restore). But you're still welcome to provide a patch; the source code for that page can be found at this repo: http://libvirt.org/git/?p=libvirt-appdev-guide.git;a=summary
So is it better just to use virDomainSnapshotCreateXML instead of virDomainSave/virDomainSaveFlags?
It all depends on what your end goal is with doing a save/load of a VM. Do you need fast saves, or is it okay if the save takes several seconds? Will you be loading the state exactly once, or do you plan to make it something that the user can revert to multiple times in a row?
The main idea is to allow the user to save the virtual machine and be able to load it when he wants and be able to share it with others who uses Boxes, so the other person can also load it on his machine (in Boxes).
Letting _another_ person load the saved image sounds like some degree of cloning. Are you trying to clone from an offline state (in which case, it is just as easy to create a new domain from XML pointing to the cloned disks), or are you trying to clone from an online state (where the other user gets to load the domain already booted to the point that the first user had it)? The former is already possible, the latter is on my wishlist of cool things where it would be nice to add better libvirt support. Another thing to worry about with the feature request of online cloning is what happens if the guest has saved state related to the original (for example, dmidecode information in the guest can see the UUID assigned by the host, but cloning implies creating a new domain with a new UUID). -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Tue, Jul 3, 2012 at 9:18 PM, Eric Blake <eblake@redhat.com> wrote:
On 07/03/2012 11:35 AM, Jovanka Gulicoska wrote:
On Tue, Jul 3, 2012 at 6:01 PM, Eric Blake <eblake@redhat.com> wrote:
On 07/03/2012 06:42 AM, Jovanka Gulicoska wrote:
Hi,
my name is Jovanka Gulicoska and I'm SoC student. I'm working on implementing save/load of VM in Gnome-Boxes. Something confused me about snapshots. In the documentation about life cycle, in section about Save/Restore is said that domain saved with virDomainSave/virDomainSaveFlags can be restored only once.
Can you point out the URL to that documentation? It's slightly incorrect, and worth patching to make clearer if that would help the next person to read them.
This is in the draft of the documentation: http://libvirt.org/guide/html-single/#Application_Development_Guide-Lifecycl... --- "For basic usage this implies that a guest can only be restored once from any given saved state image."
Ah, and I think reading that sentence in context clarifies the situation:
"Thus when the guest is restored, the underlying guest storage must be in exactly the same state as it was when the guest was initially saved. For basic usage this implies that a guest can only be restored once from any given saved state image. To allow a guest to be restored from the same saved state multiple times, the application must also have taken a snapshot of the guest storage at time of saving, and explicitly revert to this storage snapshot when restoring."
Either you manually revert the storage back to match the save image state before restoring, or you use things in a more basic manner (never touch the storage manually, and therefore restore works only once because the storage was unchanged between the save/restore cycle but is now altered after the successful restore).
But you're still welcome to provide a patch; the source code for that page can be found at this repo: http://libvirt.org/git/?p=libvirt-appdev-guide.git;a=summary
So is it better just to use virDomainSnapshotCreateXML instead of virDomainSave/virDomainSaveFlags?
It all depends on what your end goal is with doing a save/load of a VM. Do you need fast saves, or is it okay if the save takes several seconds? Will you be loading the state exactly once, or do you plan to make it something that the user can revert to multiple times in a row?
The main idea is to allow the user to save the virtual machine and be able to load it when he wants and be able to share it with others who uses Boxes, so the other person can also load it on his machine (in Boxes).
Letting _another_ person load the saved image sounds like some degree of cloning. Are you trying to clone from an offline state (in which case, it is just as easy to create a new domain from XML pointing to the cloned disks), or are you trying to clone from an online state (where the other user gets to load the domain already booted to the point that the first user had it)?
Yeah, this feature is about cloning the VM but using a single file that has everything about the domain: disk, ram and the config.
The former is already possible, the latter is on my wishlist of cool things where it would be nice to add better libvirt support. Another thing to worry about with the feature request of online cloning is what happens if the guest has saved state related to the original (for example, dmidecode information in the guest can see the UUID assigned by the host, but cloning implies creating a new domain with a new UUID).
The plan was to do the latter (online cloning) but since its not already possible and IMO it might be too much work for Jovanka to handle within the GSoC timeline (keeping in mind that half of the time is already gone), we can drop the "saving of RAM/state" part for now at least. Which API should we be using for offline cloning? Recall that we want to have it all in one file. If thats not possible, I guess we have to manually put the config and disk in a tarball and on loading machine, we create the new domain with that config and put the disk image in the right place? -- Regards, Zeeshan Ali (Khattak) FSF member#5124
On 07/03/2012 02:44 PM, Zeeshan Ali (Khattak) wrote:
Yeah, this feature is about cloning the VM but using a single file that has everything about the domain: disk, ram and the config.
Unfortunately, there is no single-file format that represents the state of a qemu domain; it really is the combination of the libvirt XML plus the state of all the disk files, so you may indeed need to create your own tarball or other file format that wraps all the pieces of state into a larger collection that can be passed as a single file.
The former is already possible, the latter is on my wishlist of cool things where it would be nice to add better libvirt support. Another thing to worry about with the feature request of online cloning is what happens if the guest has saved state related to the original (for example, dmidecode information in the guest can see the UUID assigned by the host, but cloning implies creating a new domain with a new UUID).
For offline cloning, we have libguestfs that has the ability to scrub a disk image to remove some of these artifacts of a clone needing to start from different state than the parent (things such as clearing out the random number seed, the hardware MAC address, and so on); see virt-sysprep for some ideas on the sort of things worth altering in a clone. One of the problems of online cloning, though, is that it becomes extremely difficult to say which things in the guest memory correspond to file contents that an offline scrub was able to clean. If you simplify the problem slightly, by saying that the clone can continue to run from the same UUID and name but under a different qemu:///session (that is, the UUID is no longer globally unique, but it still remains unique between the distinct qemu:///session instances that are managing the original and the clone; and similarly, that user-space networking is set up so that reuse of the same hardware MAC between the two qemu:///session instances will not collide on the real network), then I suppose that save/restore could be used as a form of online cloning, without quite as many hurdles as what it would take to do a live clone all within the same libvirtd instance.
The plan was to do the latter (online cloning) but since its not already possible and IMO it might be too much work for Jovanka to handle within the GSoC timeline (keeping in mind that half of the time is already gone), we can drop the "saving of RAM/state" part for now at least.
Which API should we be using for offline cloning? Recall that we want to have it all in one file. If thats not possible, I guess we have to manually put the config and disk in a tarball and on loading machine, we create the new domain with that config and put the disk image in the right place?
For online cloning: If you are going to simplify the problem by using separate qemu:///session instances for the original and the clone, then an online clone is indeed possible by transferring the save image created by 'virsh save' as well as a copy of all the disk images in use at the time of the save. If you are using all qcow2 images, and don't mind dealing with internal snapshots yourself (since libvirt does not yet have virStorageVol* API for nicely dealing with internal snapshots when cloning a qcow2 image), it might also be possible to use 'virsh snapshot-create' to create a system checkpoint, then 'virsh snapshot-dumpxml' to dump the snapshot, which you can then modify in order to redefine it into a snapshot tied to a new domain, then 'virsh snapshot-revert' to load that snapshot (or use the underlying virDomainSnapshot* API that the various virsh commands are wrapping); but I haven't tried doing it myself to see how many tweaks are really required to get the pieces to work. If you are looking at offline cloning, you may want to look into how virt-manager has already solved the problem. It involves cloning each of the disk images and creating a new domain pointing to each of the clones. If you have the premise of shared storage for backing files, it might also be possible to take a disk snapshot of a domain, then have both the original and the clone use new wrapping qcow2 files that both point to the same backing file, so that you don't have to copy the entire disk image but just manage the two different deltas built atop the same common base. -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Tue, Jul 03, 2012 at 11:44:16PM +0300, Zeeshan Ali (Khattak) wrote:
On Tue, Jul 3, 2012 at 9:18 PM, Eric Blake <eblake@redhat.com> wrote:
On 07/03/2012 11:35 AM, Jovanka Gulicoska wrote:
On Tue, Jul 3, 2012 at 6:01 PM, Eric Blake <eblake@redhat.com> wrote:
On 07/03/2012 06:42 AM, Jovanka Gulicoska wrote:
Hi,
my name is Jovanka Gulicoska and I'm SoC student. I'm working on implementing save/load of VM in Gnome-Boxes. Something confused me about snapshots. In the documentation about life cycle, in section about Save/Restore is said that domain saved with virDomainSave/virDomainSaveFlags can be restored only once.
Can you point out the URL to that documentation? It's slightly incorrect, and worth patching to make clearer if that would help the next person to read them.
This is in the draft of the documentation: http://libvirt.org/guide/html-single/#Application_Development_Guide-Lifecycl... --- "For basic usage this implies that a guest can only be restored once from any given saved state image."
Ah, and I think reading that sentence in context clarifies the situation:
"Thus when the guest is restored, the underlying guest storage must be in exactly the same state as it was when the guest was initially saved. For basic usage this implies that a guest can only be restored once from any given saved state image. To allow a guest to be restored from the same saved state multiple times, the application must also have taken a snapshot of the guest storage at time of saving, and explicitly revert to this storage snapshot when restoring."
Either you manually revert the storage back to match the save image state before restoring, or you use things in a more basic manner (never touch the storage manually, and therefore restore works only once because the storage was unchanged between the save/restore cycle but is now altered after the successful restore).
But you're still welcome to provide a patch; the source code for that page can be found at this repo: http://libvirt.org/git/?p=libvirt-appdev-guide.git;a=summary
So is it better just to use virDomainSnapshotCreateXML instead of virDomainSave/virDomainSaveFlags?
It all depends on what your end goal is with doing a save/load of a VM. Do you need fast saves, or is it okay if the save takes several seconds? Will you be loading the state exactly once, or do you plan to make it something that the user can revert to multiple times in a row?
The main idea is to allow the user to save the virtual machine and be able to load it when he wants and be able to share it with others who uses Boxes, so the other person can also load it on his machine (in Boxes).
Letting _another_ person load the saved image sounds like some degree of cloning. Are you trying to clone from an offline state (in which case, it is just as easy to create a new domain from XML pointing to the cloned disks), or are you trying to clone from an online state (where the other user gets to load the domain already booted to the point that the first user had it)?
Yeah, this feature is about cloning the VM but using a single file that has everything about the domain: disk, ram and the config.
I don't really see that having a single file is at all relevant to locally cloning VMs. If, however, you are actually trying to share & distribute VM images between users, this is an entirely separate concept from save/restore and cloning, and different factors come into play. For a start you do not want to be distributing the normal libvirt XML for the VM. The libvirt XML represents the host-specific config for a specific instantiation of the VM. When distributing images you have to have a config format that describes the host-agnostic requirements of the VM. This is the distinction between libvirt XML and something like the OVF appliance metadata format. You'll find that anyone distributing VM appliances uses a "single file" simply by virtue of distributing a ZIP/tar.gz file containing the disk image + OVF metadata + whatever else they desire. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Tue, Jul 03, 2012 at 11:44:16PM +0300, Zeeshan Ali (Khattak) wrote:
On Tue, Jul 3, 2012 at 9:18 PM, Eric Blake <eblake@redhat.com> wrote: The plan was to do the latter (online cloning) but since its not already possible and IMO it might be too much work for Jovanka to handle within the GSoC timeline (keeping in mind that half of the time is already gone), we can drop the "saving of RAM/state" part for now at least.
I think "saving of RAM/state" in an appliance (something that is distributed to other people) does not go well with <cpu mode="host-model"> since the other user may have a totally different CPU in his machine. One more reason to focus on offline appliances for now ;) Christophe
On Wed, Jul 04, 2012 at 12:15:20PM +0200, Christophe Fergeau wrote:
On Tue, Jul 03, 2012 at 11:44:16PM +0300, Zeeshan Ali (Khattak) wrote:
On Tue, Jul 3, 2012 at 9:18 PM, Eric Blake <eblake@redhat.com> wrote: The plan was to do the latter (online cloning) but since its not already possible and IMO it might be too much work for Jovanka to handle within the GSoC timeline (keeping in mind that half of the time is already gone), we can drop the "saving of RAM/state" part for now at least.
I think "saving of RAM/state" in an appliance (something that is distributed to other people) does not go well with <cpu mode="host-model"> since the other user may have a totally different CPU in his machine. One more reason to focus on offline appliances for now ;)
host-model is probably the least of your worries. More critical is that unless the recipient has the exact same QEMU version, they may well not even be able to load the saved image. While QEMU does strive for save restore compatibility across versions, in practice is has been terribly unreliable & crash-prone. You'd be well served to stay far away from the idea of passing saved-VM images between people. It can only lead to pain & suffering. As mentioned elsewhere in this thread, you don't even want to be passing around the libvirt XML really. You want to use the disk image in combination with an appliance format like OVF to describe its general requirements, rather than the config for a specific instantiation of the image which is what libvirt XML gives you. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
participants (5)
-
Christophe Fergeau -
Daniel P. Berrange -
Eric Blake -
Jovanka Gulicoska -
Zeeshan Ali (Khattak)