Re: [libvirt-users] Snapshot system: really confusing.
Hello, thank you for your fast reply! To help in comprehension I'll explain you a bit what am I trying to realize so the picture will be more clear. Basically it's a platform for debug, what I need is to access to the memory dump image and to the FS ones; I don't need any reverting support. The common lifecycle of a domain is: a) Given a backing store disk (qcow2) I create a new disk image: my_image.qcow2 b) I start this image and play around based on a persistent domain. c) I take several pictures (snapshots, dumps) of the VE state: I need at least readable pictures of the FileSystem and the RAM. d) I shutdown the guest. e) I extract valuable information from the pictures. This is the critical phase where all my doubts on libvirt platform come from. f) I store those information. g) I wipe out everything else. h) Ready for a new test, return to point a). libvirt is a great platform! And documentation is not bad at all. The only nebulous part is the snapshot part (I guessed was on hard development anyway). Now I'll answer to your points.
Greetings,
Hello,
I am developing a platform for test and debug use.
The typical scenario I want to realize is the following:
I run a .qcow2 image via QEMU-KVM, the running OS is not relevant. I want to take several snapshots of both Disk and RAM status; same file, separate files doesn't matter. I just need a way to have consistent information about them (better with XML description) and the data as well. I want to store these snapshots on my storage devices to use them whenever and wherever I want.
Hopefully libvirt can supply what you are interested in; and be aware that it is a work in progress (that is, unreleased qemu 1.1 and libvirt 0.9.12 will be adding yet more features to the picture, in the form of live block copy).
I knew the situation, I saw a RFC of yours that is really interesting as it introduces snapshots of single storage volumes: virStorageVolSnapshotPtr. Really interesting under my point of view.
Is it possible to store a single snapshot providing both the memory and disks state in a file (maybe a .qcow2 file)?
Single file: no. Single snapshot: yes. The virDomainSnapshotCreateXML API (exposed by virsh snapshot-create[-as]) is able to create a single XML representation that libvirt uses to track the multiple files that make up a snapshot including both memory and disk state.
This would be enough, as long as I'm able to read those information.
Is there any way to get a unique interface which handles my snapshots?
I was used to use the virDomainSnapshotCreateXML() defining the destination file in the XML with <disk> fields.
Right now, you have a choice:
1. Don't use the DISK_ONLY flag. That means that you can't use <disk> in the snapshot XML. The snapshot then requires qcow2 files, the VM state is saved (it happens to be in the first qcow2 disk), and the disk state is saved via qcow2 internal snapshots.
That's good to know, I just got yesterday, reading your mails in the dev mailing list, where snapshots were stored; I always tried to look for a new file, unsuccessfully.
2. Use the DISK_ONLY flag. Then you can set up external snapshot file names (basically, another layer of qcow2 files, where your original file name is now the snapshot backing the new layer), and you get no VM state directly. Here is where the <disk> element of the snapshot comes into play. If you _also_ want to save VM state, you can use 'virsh save' (virDomainSaveFlags) to save just VM state; but then you have to coordinate things so that the disk snapshot and the VM state correspond to the same point in guest time by pausing the guest before taking the disk snapshot.
After updating libvirt it was not working anymore, I thought was a bug but then I realized it was intentional. The function complains about the fact that the <disk> parameter is not accepted anymore.
What versions of libvirt are you playing with? <disk> was unrecognized (and ignored) prior to 0.9.5; after that point, it can only be used with the DISK_ONLY flag, but then you have to take the VM state separately.
I guessed that the <disk> tag was just ignored, I inherited the code from a previous project and I spent the first weeks struggling with what was not working. I was using libvirt 0.8.3-5 from debian squeeze, I migrated to wheezy to be able to access to libguestfs features, now I'm running libvirt 0.9.11.
Someday, I'd like to submit more patches to allow <disk> to be mixed with live VM state, but I'm not there yet.
So I started guessing how to solve reading the API documentation and I fall in a completely nebulous world.
For what I got: - virDomainSnapshotCreateXML(): According to flags can take system checkpoints (really useful) and disks snapshots. System checkpoints: What I need but I didn't find any way to retrieve the storage file; I'm only able to get the snapshot pointer, quite useless as from its pointer I can only print the XML description.
The storage is in internal snapshots of your qcow2 file. Try 'qemu-img info /path/to/file' to see those internal snapshots. You can also use
qemu-img convert -s snapname /path/to/image /path/to/output
Great! This is the information I needed: how to access to those damned snapshot in a readable way. I still believe anyway that an interface provided by libvirt would be really valuable. I am trying to stay as much as I can on libvirt to work on a higher level of abstraction, this is really important for my architecture. Using directly qemu is quite annoying but if at the moment is the only solution I'll move to that.
as a way to extract that snapshot into a file that you can use independently (but only while your guest is not running); alas, qemu doesn't yet provide a way to extract this information from a running domain, nor have I yet had time to map this functionality into libvirt API. But I have requested qemu enhancements to allow it (perhaps by qemu 1.2), as well as have a vision of where I plan to take the libvirt API in the next year or so to make this more useful.
ATM I don't need "on the fly" extraction, everything is done after turning off the guest, what I need is a way to get its state whenever I want, reading it is another business. But indeed this is something that in a future may be greatly valuable.
Disk snapshot: here through the XML description I can specify a target file where to write the information. But I still need the memory status. - virDomainSaveFlags(): No matter how I set the flags, it halts the guest; I need it still running afterward and I think that reverting the domain and restarting for my scenarios is unaffordable.
Yeah, that's another thing on my list, to add a flag to virDomainSaveFlags that will keep the domain running after the save is complete, and/or enhance virDomainSnapshotCreateXML to incorporate a domain save to an external file alongside disk snapshots at the same point in guest time. I'd also like to play with background migration (right now, virDomainSaveFlags does a foreground migrate-to-file, which pauses the guest up front; but a minimal downtime solution would use the live migration path which results in a larger output file but much less downtime).
- virDomainCoreDump(): Does what I need for the memory, but doesn't give any XML description and doesn't supply any useful interface to handle it, I just get back an int that represent the exit status of the routine.
virDomainCoreDump() and virDomainSaveFlags() both do a migration to file, it's just that the core dump version isn't designed for reverting like domain save. And there have been proposals on list for making core dump management better (such as allowing you to get at a core dump file from a remote machine; right now, the core dump can only be stored on the same machine as the guest being dumped).
I think this is bad, why not merging these functions? Under an API point of view (I'm aware of architectural difficulties on the background) this is useless and confusing. Better a virDomainSaveFlags() with several configuration than two different functions. This is a complex system-management API, so anyone who'll use it will be conscious of that; I don't think that many flags and XML files will be confusing as long as they're clearly documented. But having multiple interfaces (alias functions/methods) that basically do the same in a different way with slightly different results is really confusing. Better have a single method configurable with XML files and flags than two without. (This is a point of view of mine as a developer).
(other functions really similar)
The question is: why all this confusion?
Different qemu capabilities ('savevm', 'migrate', 'blockdev-snapshot-sync'), different needs at the time each feature was first added, etc. Ultimately, virDomainSnapshotCreateXML is the most powerful interface, and I have plans to make it be a superset of virDomainSaveFlags(), but I'm not there yet.
Indeed is a powerful interface! But again: why those functions do something similar? Isn't better (always under an API point of view) having: - Two different interfaces that handle separately disk and state; if I want to revert/migrate I need to give both the results of those interfaces. - One interface that gives revert/migrate capability and one interface that, via flags or XML, gives the separate components. - Other combinations: may still be done as long as they're clear. Here we have: virDomainSnapshotCreateXML(): Takes complete snapshots, disk snapshots, no single-memory snapshots. It does it keeping the domain alive (paused). It stores information internally if flags=A, externally if flags=B. virDomainSaveFlags(): No complete snapshots, no disk snapshots, takes memory snapshots. It does it stopping the domain. It stores information externally. Under my point of view this can be done in a more clear way. Separation of duties: many small functions that do a single thing and a global one that wraps everything giving a complete package is a good example.
I absolutely understand the problematic that realizing a multiplatform snapshots management raises; but I think that for an API purpose what is implemented here is completely confusing the developer.
Is there anything I can do to help improve the documentation? I know that's an area where a picture can speak a thousand words; and as more features get added into the snapshot picture, it probably becomes more important to accurately display the various APIs and the advantages for using each.
What's basically missing is data flow and representation, you don't need to realize it by scratch (data format is strongly bound to qemu) but give an idea and provide references. What makes a documentation relevant is the way the reader has to access to information, must be easy! If snapshots reliy to qemu ones just link their documentation as well, so if I don't find enough clues I can go deeper as much as needed. If help is needed I can contribute, I just need to know where to look at when I need something. The platforms I'm working on are really time-consuming but this technology is really important under my point of view as it will reduce the time to maintain them. Thanks again for answering so fast! NoxDaFox PS: I forgot to CC the libvirt-users mailing list, sorry for the spam.
On 04/25/2012 03:25 AM, NoxDaFox wrote:
Hello,
thank you for your fast reply! To help in comprehension I'll explain you a bit what am I trying to realize so the picture will be more clear. Basically it's a platform for debug, what I need is to access to the memory dump image and to the FS ones; I don't need any reverting support. The common lifecycle of a domain is:
a) Given a backing store disk (qcow2) I create a new disk image: my_image.qcow2 b) I start this image and play around based on a persistent domain. c) I take several pictures (snapshots, dumps) of the VE state: I need at least readable pictures of the FileSystem and the RAM. d) I shutdown the guest. e) I extract valuable information from the pictures. This is the critical phase where all my doubts on libvirt platform come from. f) I store those information. g) I wipe out everything else. h) Ready for a new test, return to point a).
Thanks; that helps in knowing what you are trying to accomplish - it sounds like you are intentionally forking machine state from a known point in time (the offline backing store disk), and then want to compare the running state of those forks by snooping both the RAM and disk state at the time of the snapshot series you took along each branch of the fork.
libvirt is a great platform! And documentation is not bad at all. The only nebulous part is the snapshot part (I guessed was on hard development anyway).
Always nice to hear from happy customers, and yes, we are trying to improve things. And as always with open source projects, patches are welcome, even for things as mundane as documentation.
Now I'll answer to your points.
Hopefully libvirt can supply what you are interested in; and be aware that it is a work in progress (that is, unreleased qemu 1.1 and libvirt 0.9.12 will be adding yet more features to the picture, in the form of live block copy).
I knew the situation, I saw a RFC of yours that is really interesting as it introduces snapshots of single storage volumes: virStorageVolSnapshotPtr. Really interesting under my point of view.
Yep, but unfortunately still several months down the road. I basically envision a way to map between virDomainPtr and virStorageVolPtr (given a domain, return a list of the storage volumes it is using; and given a storage volume, return a list of any domains referring to that volume). Once that mapping is in place, then we can expose more power to the various virStorageVol APIs, such as inspecting backing chains or internal snapshots.
Is it possible to store a single snapshot providing both the memory and disks state in a file (maybe a .qcow2 file)?
Single file: no. Single snapshot: yes. The virDomainSnapshotCreateXML API (exposed by virsh snapshot-create[-as]) is able to create a single XML representation that libvirt uses to track the multiple files that make up a snapshot including both memory and disk state.
This would be enough, as long as I'm able to read those information.
The 'system checkpoint' created by virDomainSnapshotCreateXML saves VM state and disk state, but saves the VM state into an internal section of a qcow2 file; it is done by the 'savevm' monitor command of qemu. I'm not sure if that is directly accessible (that is, I have no idea if qemu-img exposes any way to read the VM state, which may mean that the only way to read that state is to load the VM back into a fresh qemu instance). On the other hand, the VM-only save done by 'virsh save' uses the migration to file, which is the same format used by 'virsh dump', and that external file can be loaded up in crash analysis tools in order to inspect RAM state at the time of the save, without having to worry about extracting the VM state from an internal segment of a qcow2 disk file.
Is there any way to get a unique interface which handles my snapshots?
I was used to use the virDomainSnapshotCreateXML() defining the destination file in the XML with <disk> fields.
Right now, you have a choice:
1. Don't use the DISK_ONLY flag. That means that you can't use <disk> in the snapshot XML. The snapshot then requires qcow2 files, the VM state is saved (it happens to be in the first qcow2 disk), and the disk state is saved via qcow2 internal snapshots.
That's good to know, I just got yesterday, reading your mails in the dev mailing list, where snapshots were stored; I always tried to look for a new file, unsuccessfully.
There's multiple files involved in a snapshot: libvirt maintains an XML file (this currently happens to live in /etc/libvirt/, but you shouldn't directly be probing for this file, so much as using the libvirt API to get at the XML); the disk state is done with an internal snapshot of each of the qcow2 disks, and the VM state is done with an internal section of the first qcow2 disk. 'qemu-img info $first_disk' will show you the size of the internal state, although like I said, I don't know how to probe for the contents of that state short of starting a new qemu instance.
2. Use the DISK_ONLY flag. Then you can set up external snapshot file names (basically, another layer of qcow2 files, where your original file name is now the snapshot backing the new layer), and you get no VM state directly. Here is where the <disk> element of the snapshot comes into play. If you _also_ want to save VM state, you can use 'virsh save' (virDomainSaveFlags) to save just VM state; but then you have to coordinate things so that the disk snapshot and the VM state correspond to the same point in guest time by pausing the guest before taking the disk snapshot.
After updating libvirt it was not working anymore, I thought was a bug but then I realized it was intentional. The function complains about the fact that the <disk> parameter is not accepted anymore.
What versions of libvirt are you playing with? <disk> was unrecognized (and ignored) prior to 0.9.5; after that point, it can only be used with the DISK_ONLY flag, but then you have to take the VM state separately.
I guessed that the <disk> tag was just ignored, I inherited the code from a previous project and I spent the first weeks struggling with what was not working. I was using libvirt 0.8.3-5 from debian squeeze, I migrated to wheezy to be able to access to libguestfs features, now I'm running libvirt 0.9.11.
Then that explains the change in behavior :) Libvirt is a fast-moving target, and while I normally like the slow and cautious approach of debian stable, it sure makes using new virtualization features difficult.
Someday, I'd like to submit more patches to allow <disk> to be mixed with live VM state, but I'm not there yet.
So I started guessing how to solve reading the API documentation and I fall in a completely nebulous world.
For what I got: - virDomainSnapshotCreateXML(): According to flags can take system checkpoints (really useful) and disks snapshots. System checkpoints: What I need but I didn't find any way to retrieve the storage file; I'm only able to get the snapshot pointer, quite useless as from its pointer I can only print the XML description.
The storage is in internal snapshots of your qcow2 file. Try 'qemu-img info /path/to/file' to see those internal snapshots. You can also use
qemu-img convert -s snapname /path/to/image /path/to/output
Great! This is the information I needed: how to access to those damned snapshot in a readable way.
That accesses just the disk information, not the VM state, but it's better than nothing.
I still believe anyway that an interface provided by libvirt would be really valuable.
So would I, which is why it is on my to-do list.
I am trying to stay as much as I can on libvirt to work on a higher level of abstraction, this is really important for my architecture. Using directly qemu is quite annoying but if at the moment is the only solution I'll move to that.
as a way to extract that snapshot into a file that you can use independently (but only while your guest is not running); alas, qemu doesn't yet provide a way to extract this information from a running domain, nor have I yet had time to map this functionality into libvirt API. But I have requested qemu enhancements to allow it (perhaps by qemu 1.2), as well as have a vision of where I plan to take the libvirt API in the next year or so to make this more useful.
ATM I don't need "on the fly" extraction, everything is done after turning off the guest, what I need is a way to get its state whenever I want, reading it is another business.
Good to know, as qemu-img is a bit more powerful. Also, knowing which qemu-img features would be useful when wrapped by libvirt API will help me in prioritizing which libvirt API I should be working on.
virDomainCoreDump() and virDomainSaveFlags() both do a migration to file, it's just that the core dump version isn't designed for reverting like domain save. And there have been proposals on list for making core dump management better (such as allowing you to get at a core dump file from a remote machine; right now, the core dump can only be stored on the same machine as the guest being dumped).
I think this is bad, why not merging these functions? Under an API point of view (I'm aware of architectural difficulties on the background) this is useless and confusing. Better a virDomainSaveFlags() with several configuration than two different functions.
There is a proposal on list to make virDomainCoreDump() dump _just_ memory, in ELF format; whereas virDomainSaveFlags() will always dump all VM state (RAM but also device state) needed to restart a VM from the same state. Just because the current implementation of the two functions uses migrate-to-file as the underlying glue does not mean that this will always be the case, as the two functions really are for different purposes (saving a VM in order to resume it, vs. dumping just the memory state of a VM for analysis).
This is a complex system-management API, so anyone who'll use it will be conscious of that; I don't think that many flags and XML files will be confusing as long as they're clearly documented. But having multiple interfaces (alias functions/methods) that basically do the same in a different way with slightly different results is really confusing.
Unfortunately true, but we're stuck with historical naming - libvirt will not ever remove a function that has been previously exported in the API. It is made worse by the fact that some of the older functions are not easily extensible (such as lacking a flags argument), so we have to add new API to fill in the gaps. But when I do add new API, I do try to cross-reference the similar functions; any documentation that points in both directions between the older and simpler name and the newer and more powerful name is worthwhile.
Better have a single method configurable with XML files and flags than two without. (This is a point of view of mine as a developer).
(other functions really similar)
The question is: why all this confusion?
Different qemu capabilities ('savevm', 'migrate', 'blockdev-snapshot-sync'), different needs at the time each feature was first added, etc. Ultimately, virDomainSnapshotCreateXML is the most powerful interface, and I have plans to make it be a superset of virDomainSaveFlags(), but I'm not there yet.
Indeed is a powerful interface! But again: why those functions do something similar? Isn't better (always under an API point of view) having: - Two different interfaces that handle separately disk and state; if I want to revert/migrate I need to give both the results of those interfaces.
Not necessarily, since having 2 API instead of 1 means you then have to worry about atomicity (you don't want the guest to be running between your two API calls, otherwise, you can't grab a combined picture of the machine at one point in time).
- One interface that gives revert/migrate capability and one interface that, via flags or XML, gives the separate components. - Other combinations: may still be done as long as they're clear.
Here we have: virDomainSnapshotCreateXML(): Takes complete snapshots, disk snapshots, no single-memory snapshots.
No single-memory snapshots, yet. It's coming.
It does it keeping the domain alive (paused). It stores information internally if flags=A, externally if flags=B.
It will someday also be possible to keep the domain alive and running (as is the case with live migration between machines).
virDomainSaveFlags(): No complete snapshots, no disk snapshots, takes memory snapshots. It does it stopping the domain. It stores information externally.
Under my point of view this can be done in a more clear way. Separation of duties: many small functions that do a single thing and a global one that wraps everything giving a complete package is a good example.
I absolutely understand the problematic that realizing a multiplatform snapshots management raises; but I think that for an API purpose what is implemented here is completely confusing the developer.
Is there anything I can do to help improve the documentation? I know that's an area where a picture can speak a thousand words; and as more features get added into the snapshot picture, it probably becomes more important to accurately display the various APIs and the advantages for using each.
What's basically missing is data flow and representation, you don't need to realize it by scratch (data format is strongly bound to qemu) but give an idea and provide references. What makes a documentation relevant is the way the reader has to access to information, must be easy! If snapshots reliy to qemu ones just link their documentation as well, so if I don't find enough clues I can go deeper as much as needed.
If help is needed I can contribute, I just need to know where to look at when I need something. The platforms I'm working on are really time-consuming but this technology is really important under my point of view as it will reduce the time to maintain them.
Thanks again for answering so fast!
NoxDaFox
PS: I forgot to CC the libvirt-users mailing list, sorry for the spam.
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
-- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On 27/04/12 17:56, Eric Blake wrote:
Hello,
thank you for your fast reply! To help in comprehension I'll explain you a bit what am I trying to realize so the picture will be more clear. Basically it's a platform for debug, what I need is to access to the memory dump image and to the FS ones; I don't need any reverting support. The common lifecycle of a domain is:
a) Given a backing store disk (qcow2) I create a new disk image: my_image.qcow2 b) I start this image and play around based on a persistent domain. c) I take several pictures (snapshots, dumps) of the VE state: I need at least readable pictures of the FileSystem and the RAM. d) I shutdown the guest. e) I extract valuable information from the pictures. This is the critical phase where all my doubts on libvirt platform come from. f) I store those information. g) I wipe out everything else. h) Ready for a new test, return to point a). Thanks; that helps in knowing what you are trying to accomplish - it sounds like you are intentionally forking machine state from a known
On 04/25/2012 03:25 AM, NoxDaFox wrote: point in time (the offline backing store disk), and then want to compare the running state of those forks by snooping both the RAM and disk state at the time of the snapshot series you took along each branch of the fork.
That's it! Really simple. From a known state I want to run a scenario (whatever piece of code) and track down the differences on disk and memory state. I need a way to track down those states and to access in a comfortable way to the deltas. I think that a valuable feature that virtualization technology could bring, is the capability to study code behavior for debug or reverse engineering purpose. The problem is that there aren't yet interfaces capable of doing so. All the available solutions just provide snapshots to revert system to a certain state; not to use snapshots for analysis purpose. This could be something that can differentiate libvirt from other products. To be more clear imagine this situation: I have a software (MySoft) and, as it works on Windows registers, I need to control that it won't break the OS. MySoft once run does windows registry cleanup, write a logs and stops. The scenario I want to represent is the installation of MySoft and one execution of itself. The information I need are: which files are created/modified once MySoft is installed, the memory movements once MySoft is running, which files are modified/removed once MySoft has run and the final log file. Here's the test's lifecycle: - I set up a disk: Windows Vista Home Edition SP1 fully updated. - I put MySoft installer in the disk's FS. - I start the guest. - Once windows is up and running I take a disk state snapshot (DISK#1). - I run the installer; once MySoft is installed I take a second disk snapshots (DISK#2). - Windows is idle, I take a memory state snapshot (MEM#1). - I start MySoft, I take 3 other memory snapshots (MEM#2, MEM#3, MEM#4). - MySoft closed gracefully, I take both memory and disk snapshots (DISK#3, MEM#5). - I shut down the guest. - I release the resources. At this point I have several pictures of MySoft behavior: I can see which files are created/modified after installation mounting DISK#2 snapshot and comparing it with DISK#1. I can see which files are modified/removed after MySoft execution comparing DISK#3 and DISK#2. I can check if all the memory resources are given back to the OS comparing MEM#5 and MEM#1. I can look at the memory evolution through MEM#2, MEM#3, MEM#4. I can extract the log file from DISK#3. I hope this example may be useful to understand what I want to realize. I think that using virtualization in such a way may be a really useful for developers!
libvirt is a great platform! And documentation is not bad at all. The only nebulous part is the snapshot part (I guessed was on hard development anyway). Always nice to hear from happy customers, and yes, we are trying to improve things. And as always with open source projects, patches are welcome, even for things as mundane as documentation.
Now I'll answer to your points.
Hopefully libvirt can supply what you are interested in; and be aware that it is a work in progress (that is, unreleased qemu 1.1 and libvirt 0.9.12 will be adding yet more features to the picture, in the form of live block copy). I knew the situation, I saw a RFC of yours that is really interesting as it introduces snapshots of single storage volumes: virStorageVolSnapshotPtr. Really interesting under my point of view. Yep, but unfortunately still several months down the road. I basically envision a way to map between virDomainPtr and virStorageVolPtr (given a domain, return a list of the storage volumes it is using; and given a storage volume, return a list of any domains referring to that volume). Once that mapping is in place, then we can expose more power to the various virStorageVol APIs, such as inspecting backing chains or internal snapshots. The easiest way to track down state differences at the moment is, given a snapshot and the image from which has been generated, create a disk image, mount it and compare it with the original backing one. Consistency problems may be solved flushing all the buffers on the disk and pausing the guest before snapshotting.
Is it possible to store a single snapshot providing both the memory and disks state in a file (maybe a .qcow2 file)? Single file: no. Single snapshot: yes. The virDomainSnapshotCreateXML API (exposed by virsh snapshot-create[-as]) is able to create a single XML representation that libvirt uses to track the multiple files that make up a snapshot including both memory and disk state. This would be enough, as long as I'm able to read those information. The 'system checkpoint' created by virDomainSnapshotCreateXML saves VM state and disk state, but saves the VM state into an internal section of a qcow2 file; it is done by the 'savevm' monitor command of qemu. I'm not sure if that is directly accessible (that is, I have no idea if qemu-img exposes any way to read the VM state, which may mean that the only way to read that state is to load the VM back into a fresh qemu instance). This is really bad as I need to inspect this memory. As I said: taking disk and memory states separately or together is not relevant at the moment; I just need a way to elegantly implement it. Something easy to use programmatically (I won't be the final user of the
What I miss now is a libvirt interface that allows me to pass from a snapshot to a volume. Would be fantastic to map in such a way that, from a disk snapshot pointer, through the volumes pool I generate a new volume that I can track with the pool itself. This will allow a system to be really easy to maintain: I have a snapshot list, I choose the desired ones, I generate new volumes, I compare those volumes using libguestfs (or whatever else tool), I collect the desired information, I clean up the pool and I destroy the snapshots list. platform).
On the other hand, the VM-only save done by 'virsh save' uses the migration to file, which is the same format used by 'virsh dump', and that external file can be loaded up in crash analysis tools in order to inspect RAM state at the time of the save, without having to worry about extracting the VM state from an internal segment of a qcow2 disk file.
So I guess the virDomainSave doesn't use the qemu savevm command. But anyway this function is useless under my point of view because it stops the guest.
Is there any way to get a unique interface which handles my snapshots?
I was used to use the virDomainSnapshotCreateXML() defining the destination file in the XML with<disk> fields. Right now, you have a choice:
1. Don't use the DISK_ONLY flag. That means that you can't use<disk> in the snapshot XML. The snapshot then requires qcow2 files, the VM state is saved (it happens to be in the first qcow2 disk), and the disk state is saved via qcow2 internal snapshots. That's good to know, I just got yesterday, reading your mails in the dev mailing list, where snapshots were stored; I always tried to look for a new file, unsuccessfully. There's multiple files involved in a snapshot: libvirt maintains an XML file (this currently happens to live in /etc/libvirt/, but you shouldn't directly be probing for this file, so much as using the libvirt API to get at the XML); the disk state is done with an internal snapshot of each of the qcow2 disks, and the VM state is done with an internal section of the first qcow2 disk. 'qemu-img info $first_disk' will show you the size of the internal state, although like I said, I don't know how to probe for the contents of that state short of starting a new qemu instance. The point is that this XML doesn't track those involved files so I have no clue where to get them. And I still just need to analyze the memory state, no other uses. ... <cut> ... virDomainCoreDump() and virDomainSaveFlags() both do a migration to file, it's just that the core dump version isn't designed for reverting like domain save. And there have been proposals on list for making core dump management better (such as allowing you to get at a core dump file from a remote machine; right now, the core dump can only be stored on the same machine as the guest being dumped).
I think this is bad, why not merging these functions? Under an API point of view (I'm aware of architectural difficulties on the background) this is useless and confusing. Better a virDomainSaveFlags() with several configuration than two different functions. There is a proposal on list to make virDomainCoreDump() dump _just_ memory, in ELF format; whereas virDomainSaveFlags() will always dump all VM state (RAM but also device state) needed to restart a VM from the same state. Just because the current implementation of the two functions uses migrate-to-file as the underlying glue does not mean that this will always be the case, as the two functions really are for different purposes (saving a VM in order to resume it, vs. dumping just the memory state of a VM for analysis). With devices you mean disks or other devices (network cards, usb sticks)? For the moment my first priority are the disk (my guests run on a single virtual disk, I don't need to represent multiple disks or partitions in my scenario) and the memory state. Then we can also think about buffers, devices and so on; but this is more complex and less important.
The important think is having something clear to use for analysis, with function A I get a disk readable state, with B I get a memory one. For what I understood now the virDomainCoreDump() is what I need, as it gives an ELF image whit the memory state, right?
This is a complex system-management API, so anyone who'll use it will be conscious of that; I don't think that many flags and XML files will be confusing as long as they're clearly documented. But having multiple interfaces (alias functions/methods) that basically do the same in a different way with slightly different results is really confusing. Unfortunately true, but we're stuck with historical naming - libvirt will not ever remove a function that has been previously exported in the API. It is made worse by the fact that some of the older functions are not easily extensible (such as lacking a flags argument), so we have to add new API to fill in the gaps. But when I do add new API, I do try to cross-reference the similar functions; any documentation that points in both directions between the older and simpler name and the newer and more powerful name is worthwhile.
Fully understandable, you cannot brutally remove a deprecated function, but in the documentation underlining it as "deprecate, use XXX instead" would be more clear. Basically my idea is to use libvirt as an analysis tool but for the moment the project is focusing only on the migration and reverting point of view. This is fully understandable as all the virtualization tools are basing their business on those features, but I think that introducing also analysis support may be really valuable for this product (my 2 cents of course). NoxDaFox.
I played around a bit more with storages, pools and snapshots and I arrived at one conclusion. In my case I am talking about: Directory pools: I am managing only this kind of pools. qcow2 image files: They may be new and empty, populated with a fresh guest installation or with a full lifecycle (several snapshots etc.) I need to handle them both with copy on write and without. These files may be everywhere in every kind of FS: NFS, on a USB stick... The pool conception is pretty valuable, easy to use and gives the capability of managing disk resources in a good way. But the pool management itself is quite confusing. To create a pool I need to call many functions in the following order: - virStoragePoolDefineXML(): coherent with persistent domain definition, OK - virStoragePoolBuild(): to create the proper directory, can be easily merged in the next one function. - virStoragePoolCreate(): coherent again with persistent domain , OK To delete a pool: - virStoragePoolDelete() - virStoragePoolDestroy() - virStoragePoolFree() - virStoragePoolUndefine() Too many functions, not documented at all, not coherent with domain management. The result is confusion, if they're not called in the right order they don't work. The most important thing is, in my opinion, to have clear ideas whether to manage FS resources or not. With the virStoragePoolBuild() libvirt allows to create a new folder, then remove it is quite complicated: even if the cleanup is made properly (removal of all the storage files and removal of the pool in the right way) it fails if in the directory there are still some files. If libvirt wants to offer the capability of manage files and directories (as volumes and pool obviously) it must then offer a complete and clear interface. Really useful missing functions for example are the virStoragePoolGetPath() and virStoragePoolLookupByPath(), I can create the folders but then I need to track them down manually. In this case better to leave the developer the responsibility of creating the FS resources and just link them to the libvirt layer. Same problems with volume management. I ran in this really tedious issue: I create a disk from a snapshot through qemu-img (external process) as it is missing a libvirt feature that allows me to do so; I create it in the pool directory but it is not tracked by the pool itself so I need to "manually" add it through virStorageVolCreateXML(). Then, in the same directory, I copy through a UNIX terminal command another image, again I need to run virStorageVolCreateXML() to add it to the pool. Now with my code I copy (I'm using python bindings so through any python command) a third image in the pool folder; used to the two cases I listed I run virStorageVolCreateXML(); in this case I get: libvirtError: Storage volume not found: storage vol already exists I spent much time figuring out the problem debugging my code and eventually I arrived at the conclusion that was not my code creating this problem. In conclusion what is missing in libvirt, considering only the directory pool management (every resource as a file or directory), is a realisting mapping through FS objects and libvirt pointers or resources. In my opinion before trying to manage FS resources libvirt should be able to: - from a (.qcow2, .img, .raw) file path ----> virStorageVolPtr and vice versa - from a path in which is mapped a pool ----> virStoragePoolPtr and vice versa - from a virDomainSnapshotPtr ----> virStoragePoolPtr ----> (.qcow2, .img, .raw) file path In a clear way and at any moment of the lifecycle of libvirt. The FS management itself should be removed if not complete and clear, a programmer can always prepare the resources beforehand. If I can suggest this interface: - virStoragePoolDefineXML(): it defines a persistent pool like a domain. The dir path in the XML must already exist with correct permissions. - virStoragePoolCreate(): it creates and start the pool. - virStoragePoolDestroy(): unlink all the volume storages, stops the pool and keep persistent information - virStoragePoolFree(): free the pointer - virStoragePoolUndefine(): remove persistent information - virStoragePoolGetPath() - virStoragePoolLookupByPath() For the volumes: - virStorageVolPtr virStorageVolCreateXML(virStoragePoolPtr pool, const char * xmldesc, virStorageVolPtr clonevol, unsigned int flags) Creates a new storage volume according to the XML description, clonevol is optional, if present the volume will be cloned by the clonevol one. - virStorageVolPtr virStorageVolAdd(virStoragePoolPtr pool, const char * path, unsigned int flags) Add the storage volume IF NOT ALREADY PRESENT. If it is used by other pools or even worse: by other domains the function will fail (in the future may be added a forcing flag). For the snapshots: http://www.redhat.com/archives/libvir-list/2011-June/msg00761.html
participants (2)
-
Eric Blake -
NoxDaFox