Re: [libvirt] [PATCH] docs: Add detailed notes snapshots, blockcommit, blockpull

On 01/23/2013 01:55 AM, Kashyap Chamarthy wrote: > Eric, > > I wonder if you still have some time to take a look at this. Thanks for the reminder. And sorry that I lost your original email when my disk crashed last November; this ping makes it easier for me to dig up your message than hunting for it in list archives. > > Thanks. > > On 10/23/2012 03:28 PM, Kashyap Chamarthy wrote: >> More elaborate notes on snapshots, blockpull, blockcommit. Much of this >> is derived from various dicussions with Eric Blake, Jeff Cody, Kevin Wolf >> (thanks a lot!) & several others on IRC and mailing lists and a lot of adhoc >> testing. I didn't wanted this to get lost. >> >> I also plan to add notes for 'blockcopy' once I complete testing with upstream >> libvirt/qemu git. >> >> NOTE: This document is formatted using reStructuredText. And can be trivially >> converted to HTML using: >> # rst2html snapshots-blockcommit-blockpull.rst > snapshots-blockcommit-blockpull.html Since we already use html.in in the rest of our documentation, I will probably just do the conversion and then make the canonical copy be html after that point. >> >> ('rst2html' is part of python-docutils package.) >> >> I didn't send an html PATCH directly, as I thought, this'd be more readable. It may be more readable in isolation, but using rst in the libvirt.git would add one more prereq tool to the chain. But don't worry too much about that point. >> >> Any comments, criticisms more than welcome. >> >> >> --- >> docs/snapshots-blockcommit-blockpull.rst | 646 ++++++++++++++++++++++++++++++ >> 1 files changed, 646 insertions(+), 0 deletions(-) >> create mode 100644 docs/snapshots-blockcommit-blockpull.rst >> >> diff --git a/docs/snapshots-blockcommit-blockpull.rst b/docs/snapshots-blockcommit-blockpull.rst >> new file mode 100644 >> index 0000000000000000000000000000000000000000..99c30223a004ee5291e2914b788ac7fe04eee3c8 >> --- /dev/null >> +++ b/docs/snapshots-blockcommit-blockpull.rst >> @@ -0,0 +1,646 @@ >> +.. ---------------------------------------------------------------------- >> + Note: All these tests were performed with latest qemu-git,libvirt-git (as of >> + 20-Oct-2012 on a Fedora-18 alpha machine >> +.. ---------------------------------------------------------------------- Hmm - I guess it makes sense to call out which versions were tested; but probably makes more sense to call out names like libvirt 1.0.2 and qemu 1.4 than it does to call out testing dates (as then the reader has to research what qemu version was available on that date). Also, now that a few months have elapsed since your first post, and Peter and I did some work on snapshots in the meantime, there may be some tweaks to make to this. >> + >> + >> +Introduction >> +============ >> + >> +A virtual machine snapshot is a view of a virtual machine(its OS & all its s/machine(its/machine (its/ In general, you should always have space before ( in English prose. >> +applications) at a given point in time. So that, one can revert to a known sane >> +state, or take backups while the guest is running live. So, before we dive into >> +snapshots, let's have an understanding of backing files and overlays. >> + >> + >> + >> +QCOW2 backing files & overlays >> +------------------------------ >> + >> +In essence, QCOW2(Qemu Copy-On-Write) gives you an ability to create a base-image, >> +and create several 'disposable' copy-on-write overlay disk images on top of the >> +base image(also called backing file). Backing files and overlays are >> +extremely useful to rapidly instantiate thin-privisoned virtual machines(more on s/privisoned/provisioned/ >> +it below). Especially quite useful in development & test environments, so that >> +one could quickly revert to a known state & discard the overlay. >> + >> +**Figure-1** >> + >> +:: >> + >> + .--------------. .-------------. .-------------. .-------------. >> + | | | | | | | | >> + | RootBase |<---| Overlay-1 |<---| Overlay-1A <--- | Overlay-1B | >> + | (raw/qcow2) | | (qcow2) | | (qcow2) | | (qcow2) | >> + '--------------' '-------------' '-------------' '-------------' >> + >> +The above figure illustrates - RootBase is the backing file for Overlay-1, which >> +in turn is backing file for Overlay-2, which in turn is backing file for >> +Overlay-3. Probably worth figuring out how to create a png for this picture; I'm not a guru on docs, or how we have done it on other pages, so an initial text-only version is a good start if someone else can help out. >> + >> +**Figure-2** >> +:: >> + >> + .-----------. .-----------. .------------. .------------. .------------. >> + | | | | | | | | | | >> + | RootBase |<--- Overlay-1 |<--- Overlay-1A <--- Overlay-1B <--- Overlay-1C | >> + | | | | | | | | | (Active) | >> + '-----------' '-----------' '------------' '------------' '------------' >> + ^ ^ >> + | | >> + | | .-----------. .------------. >> + | | | | | | >> + | '-------| Overlay-2 |<---| Overlay-2A | >> + | | | | (Active) | >> + | '-----------' '------------' >> + | >> + | >> + | .-----------. .------------. >> + | | | | | >> + '------------| Overlay-3 |<---| Overlay-3A | >> + | | | (Active) | >> + '-----------' '------------' >> + >> +The above figure is just another representation which indicates, we can use a >> +'single' backing file, and create several overlays -- which can be used further, >> +to create overlays on top of them. >> + >> + >> +**NOTE**: Backing files are always opened **read-only**. In other words, once >> + an overlay is created, its backing file should not be modified(as the >> + overlay depends on a particular state of the backing file). Refer >> + below ('blockcommit' section) for relevant info on this. >> + >> + >> +**Example** : >> + >> +:: >> + >> + [FedoraBase.img] ----- <- [Fedora-guest-1.qcow2] <- [Fed-w-updates.qcow2] <- [Fedora-guest-with-updates-1A] >> + \ >> + \--- <- [Fedora-guest-2.qcow2] <- [Fed-w-updates.qcow2] <- [Fedora-guest-with-updates-2A] >> + >> +(Arrow to be read as Fed-w-updates.qcow2 has Fedora-guest-1.qcow2 as its backing file.) >> + >> +In the above example, say, *FedoraBase.img* has a freshly installed Fedora-17 OS on it, >> +and let's establish it as our backing file. Now, FedoraBase can be used as a >> +read-only 'template' to quickly instantiate two(or more) thinly provisioned >> +Fedora-17 guests(say Fedora-guest-1.qcow2, Fedora-guest-2.qcow2) by creating >> +QCOW2 overlay files pointing to our backing file. Also, the example & *Figure-2* >> +above illustrate that a single root-base image(FedoraBase.img) can be used >> +to create multiple overlays -- which can subsequently have their own overlays. >> + >> + >> + To create two thinly-provisioned Fedora clones(or overlays) using a single >> + backing file, we can invoke qemu-img as below: :: >> + >> + >> + # qemu-img create -b /export/vmimages/RootBase.img -f qcow2 \ >> + /export/vmimages/Fedora-guest-1.qcow2 >> + >> + # qemu-img create -b /export/vmimages/RootBase.img -f qcow2 \ >> + /export/vmimages/Fedora-guest-2.qcow2 >> + >> + Now, both the above images *Fedora-guest-1* & *Fedora-guest-2* are ready to >> + boot. Continuting with our example, say, now you want to instantiate a s/Continuting/Continuing/ >> + Fedora-17 guest, but this time, with full Fedora updates. This can be >> + accomplished by creating another overlay(Fedora-guest-with-updates-1A) - but >> + this overly would point to 'Fed-w-updates.qcow2' as its backing file (which >> + has the full Fedora updates) :: >> + >> + # qemu-img create -b /export/vmimages/Fed-w-updates.qcow2 -f qcow2 \ >> + /export/vmimages/Fedora-guest-with-updates-1A.qcow2 >> + >> + >> + Information about a disk image, like virtual size, disk size, backing file(if it >> + exists) can be obtained by using 'qemu-img' as below: >> + :: >> + >> + # qemu-img info /export/vmimages/Fedora-guest-with-updates-1A.qcow2 While qemu-img info can indeed be used to inspect an image, it would be much nicer to use _only_ libvirt API to describe the setup. Anywhere we have to resort to qemu-img points to a hole in our implementation (not your fault, though). On the other hand, for people familiar with qemu-img, documenting _both_ the libvirt and qemu-img counterparts may help them come up to speed with what is going on (that is, I don't mind mentioning a libvirt usage first, then as a footnote mentioning that it maps to a certain qemu-img operation). Overall, the goal is that other hypervisors can fit into the same framework, even if they don't use qemu-img. >> + >> + NOTE: With latest qemu, an entire backing chain can be recursively >> + enumerated by doing: >> + :: >> + >> + # qemu-img info --backing-chain /export/vmimages/Fedora-guest-with-updates-1A.qcow2 >> + >> + >> + >> +Snapshot Terminology: >> +--------------------- Shoot, I ran out of review time today. I'll resume next week.