[libvirt] [PATCH] docs: Add detailed notes snapshots, blockcommit, blockpull
by Kashyap Chamarthy
More elaborate notes on snapshots, blockpull, blockcommit. Much of this
is derived from various dicussions with Eric Blake, Jeff Cody, Kevin Wolf
(thanks a lot!) & several others on IRC and mailing lists and a lot of adhoc
testing. I didn't wanted this to get lost.
I also plan to add notes for 'blockcopy' once I complete testing with upstream
libvirt/qemu git.
NOTE: This document is formatted using reStructuredText. And can be trivially
converted to HTML using:
# rst2html snapshots-blockcommit-blockpull.rst > snapshots-blockcommit-blockpull.html
('rst2html' is part of python-docutils package.)
I didn't send an html PATCH directly, as I thought, this'd be more readable.
Any comments, criticisms more than welcome.
---
docs/snapshots-blockcommit-blockpull.rst | 646 ++++++++++++++++++++++++++++++
1 files changed, 646 insertions(+), 0 deletions(-)
create mode 100644 docs/snapshots-blockcommit-blockpull.rst
diff --git a/docs/snapshots-blockcommit-blockpull.rst b/docs/snapshots-blockcommit-blockpull.rst
new file mode 100644
index 0000000000000000000000000000000000000000..99c30223a004ee5291e2914b788ac7fe04eee3c8
--- /dev/null
+++ b/docs/snapshots-blockcommit-blockpull.rst
@@ -0,0 +1,646 @@
+.. ----------------------------------------------------------------------
+ Note: All these tests were performed with latest qemu-git,libvirt-git (as of
+ 20-Oct-2012 on a Fedora-18 alpha machine
+.. ----------------------------------------------------------------------
+
+
+Introduction
+============
+
+A virtual machine snapshot is a view of a virtual machine(its OS & all its
+applications) at a given point in time. So that, one can revert to a known sane
+state, or take backups while the guest is running live. So, before we dive into
+snapshots, let's have an understanding of backing files and overlays.
+
+
+
+QCOW2 backing files & overlays
+------------------------------
+
+In essence, QCOW2(Qemu Copy-On-Write) gives you an ability to create a base-image,
+and create several 'disposable' copy-on-write overlay disk images on top of the
+base image(also called backing file). Backing files and overlays are
+extremely useful to rapidly instantiate thin-privisoned virtual machines(more on
+it below). Especially quite useful in development & test environments, so that
+one could quickly revert to a known state & discard the overlay.
+
+**Figure-1**
+
+::
+
+ .--------------. .-------------. .-------------. .-------------.
+ | | | | | | | |
+ | RootBase |<---| Overlay-1 |<---| Overlay-1A <--- | Overlay-1B |
+ | (raw/qcow2) | | (qcow2) | | (qcow2) | | (qcow2) |
+ '--------------' '-------------' '-------------' '-------------'
+
+The above figure illustrates - RootBase is the backing file for Overlay-1, which
+in turn is backing file for Overlay-2, which in turn is backing file for
+Overlay-3.
+
+**Figure-2**
+::
+
+ .-----------. .-----------. .------------. .------------. .------------.
+ | | | | | | | | | |
+ | RootBase |<--- Overlay-1 |<--- Overlay-1A <--- Overlay-1B <--- Overlay-1C |
+ | | | | | | | | | (Active) |
+ '-----------' '-----------' '------------' '------------' '------------'
+ ^ ^
+ | |
+ | | .-----------. .------------.
+ | | | | | |
+ | '-------| Overlay-2 |<---| Overlay-2A |
+ | | | | (Active) |
+ | '-----------' '------------'
+ |
+ |
+ | .-----------. .------------.
+ | | | | |
+ '------------| Overlay-3 |<---| Overlay-3A |
+ | | | (Active) |
+ '-----------' '------------'
+
+The above figure is just another representation which indicates, we can use a
+'single' backing file, and create several overlays -- which can be used further,
+to create overlays on top of them.
+
+
+**NOTE**: Backing files are always opened **read-only**. In other words, once
+ an overlay is created, its backing file should not be modified(as the
+ overlay depends on a particular state of the backing file). Refer
+ below ('blockcommit' section) for relevant info on this.
+
+
+**Example** :
+
+::
+
+ [FedoraBase.img] ----- <- [Fedora-guest-1.qcow2] <- [Fed-w-updates.qcow2] <- [Fedora-guest-with-updates-1A]
+ \
+ \--- <- [Fedora-guest-2.qcow2] <- [Fed-w-updates.qcow2] <- [Fedora-guest-with-updates-2A]
+
+(Arrow to be read as Fed-w-updates.qcow2 has Fedora-guest-1.qcow2 as its backing file.)
+
+In the above example, say, *FedoraBase.img* has a freshly installed Fedora-17 OS on it,
+and let's establish it as our backing file. Now, FedoraBase can be used as a
+read-only 'template' to quickly instantiate two(or more) thinly provisioned
+Fedora-17 guests(say Fedora-guest-1.qcow2, Fedora-guest-2.qcow2) by creating
+QCOW2 overlay files pointing to our backing file. Also, the example & *Figure-2*
+above illustrate that a single root-base image(FedoraBase.img) can be used
+to create multiple overlays -- which can subsequently have their own overlays.
+
+
+ To create two thinly-provisioned Fedora clones(or overlays) using a single
+ backing file, we can invoke qemu-img as below: ::
+
+
+ # qemu-img create -b /export/vmimages/RootBase.img -f qcow2 \
+ /export/vmimages/Fedora-guest-1.qcow2
+
+ # qemu-img create -b /export/vmimages/RootBase.img -f qcow2 \
+ /export/vmimages/Fedora-guest-2.qcow2
+
+ Now, both the above images *Fedora-guest-1* & *Fedora-guest-2* are ready to
+ boot. Continuting with our example, say, now you want to instantiate a
+ Fedora-17 guest, but this time, with full Fedora updates. This can be
+ accomplished by creating another overlay(Fedora-guest-with-updates-1A) - but
+ this overly would point to 'Fed-w-updates.qcow2' as its backing file (which
+ has the full Fedora updates) ::
+
+ # qemu-img create -b /export/vmimages/Fed-w-updates.qcow2 -f qcow2 \
+ /export/vmimages/Fedora-guest-with-updates-1A.qcow2
+
+
+ Information about a disk image, like virtual size, disk size, backing file(if it
+ exists) can be obtained by using 'qemu-img' as below:
+ ::
+
+ # qemu-img info /export/vmimages/Fedora-guest-with-updates-1A.qcow2
+
+ NOTE: With latest qemu, an entire backing chain can be recursively
+ enumerated by doing:
+ ::
+
+ # qemu-img info --backing-chain /export/vmimages/Fedora-guest-with-updates-1A.qcow2
+
+
+
+Snapshot Terminology:
+---------------------
+
+ - **Internal Snapshots** -- A single qcow2 image file holds both the saved state
+ & the delta since that saved point. This can be further classified as :-
+
+ (1) **Internal disk snapshot**: The state of the virtual disk at a given
+ point in time. Both the snapshot & delta since the snapshot are
+ stored in the same qcow2 file. Can be taken when the guest is 'live'
+ or 'offline'.
+
+ - Libvirt uses QEMU's 'qemu-img' command when the guest is 'offline'.
+ - Libvirt uses QEMU's 'savevm' command when the guest is 'live'.
+
+ (2) **Internal system checkpoint**: RAM state, device state & the
+ disk-state of a running guest, are all stored in the same originial
+ qcow2 file. Can be taken when the guest is running 'live'.
+
+ - Libvirt uses QEMU's 'savevm' command when the guest is 'live'
+
+
+ - **External Snapshots** -- Here, when a snapshot is taken, the saved state will
+ be stored in one file(from that point, it becomes a read-only backing
+ file) & a new file(overlay) will track the deltas from that saved state.
+ This can be further classified as :-
+
+ (1) **External disk snapshot**: The snapshot of the disk is saved in one
+ file, and the delta since the snapshot is tracked in a new qcow2
+ file. Can be taken when the guest is 'live' or 'offline'.
+
+ - Libvirt uses QEMU's 'transaction' cmd under the hood, when the
+ guest is 'live'.
+
+ - Libvirt uses QEMU's 'qemu-img' cmd under the hood when the
+ guest is 'offline'(this implementation is in progress, as of
+ writing this).
+
+ (2) **External system checkpoint**: Here, the guest's disk-state will be
+ saved in one file, its RAM & device-state will be saved in another
+ new file (This implementation is in progress upstream libvirt, as of
+ writing this).
+
+
+
+ - **VM State**: Saves the RAM & device state of a running guest(not 'disk-state') to
+ a file, so that it can be restored later. This simliar to doing hibernate
+ of the system. (NOTE: The disk-state should be unmodified at the time of
+ restoration.)
+
+ - Libvirt uses QEMU's 'migrate' (to file) cmd under the hood.
+
+
+
+Creating snapshots
+==================
+ - Whenever an 'external' snapshot is issued, a /new/ overlay image is
+ created to facilitate guest writes, and the previous image becomes a
+ snapshot.
+
+ - **Create a disk-only internal snapshot**
+
+ (1) If I have a guest named 'f17vm1', to create an offline or online
+ 'internal' snapshot called 'snap1' with description 'snap1-desc' ::
+
+ # virsh snapshot-create-as f17vm1 snap1 snap1-desc
+
+ (2) List the snapshot ; and query using *qemu-img* tool to view
+ the image info & its internal snapshot details ::
+
+ # virsh snapshot-list f17vm1
+ # qemu-img info /home/kashyap/vmimages/f17vm1.qcow2
+
+
+
+ - **Create a disk-only external snapshot** :
+
+ (1) List the block device associated with the guest. ::
+
+ # virsh domblklist f17-base
+ Target Source
+ ---------------------------------------------
+ vda /export/vmimages/f17-base.qcow2
+
+ #
+
+ (2) Create external disk-only snapshot (while the guest is *running*). ::
+
+ # virsh snapshot-create-as --domain f17-base snap1 snap1-desc \
+ --disk-only --diskspec vda,snapshot=external,file=/export/vmimages/sn1-of-f17-base.qcow2 \
+ --atomic
+ Domain snapshot snap1 created
+ #
+
+ * Once the above command is issued, the original disk-image
+ of f17-base will become the backing_file & a new overlay
+ image is created to track the new changes. Here on, libvirt
+ will use this overlay for further write operations(while
+ using the original image as a read-only backing_file).
+
+ (3) Now, list the block device associated(use cmd from step-1, above)
+ with the guest,again, to ensure it reflects the new overlay image as
+ the current block device in use. ::
+
+ # virsh domblklist f17-base
+ Target Source
+ ----------------------------------------------------
+ vda /export/vmimages/sn1-of-f17-base.qcow2
+
+ #
+
+
+
+
+Reverting to snapshots
+======================
+As of writing this, reverting to 'Internal Snapshots'(system checkpoint or
+disk-only) is possible.
+
+ To revert to a snapshot named 'snap1' of domain f17vm1 ::
+
+ # virsh snapshot-revert --domain f17vm1 snap1
+
+Reverting to 'external disk snapshots' using *snapshot-revert* is a little more
+tricky, as it involves slightly complicated process of dealing with additional
+snapshot files - whether to merge 'base' images into 'top' or to merge other way
+round ('top' into 'base').
+
+That said, there are a couple of ways to deal with external snapshot files by
+merging them to reduce the external snapshot disk image chain by performing
+either a **blockpull** or **blockcommit** (more on this below).
+
+Further improvements on this front is in work upstream libvirt as of writing
+this.
+
+
+
+Merging snapshot files
+======================
+External snapshots are incredibly useful. But, with plenty of external snapshot
+files, there comes a problem of maintaining and tracking all these inidivdual
+files. At a later point in time, we might want to 'merge' some of these snapshot
+files (either backing_files into overlays or vice-versa) to reduce the length of
+the image chain. To accomplish that, there are two mechanisms:
+
+ + blockcommit: merges data from **top** into **base** (in other
+ words, merge overlays into backing files).
+
+
+ + blockpull: Populates a disk image with data from its backing file. Or
+ merges data from **base** into **top** (in other words, merge backing files
+ into overlays).
+
+
+blockcommit
+-----------
+
+Block Commit allows you to merge from a 'top' image(within a disk backing file
+chain) into a lower-level 'base' image. To rephrase, it allows you to
+merge overlays into backing files. Once the **blockcommit** operation is finished,
+any portion that depends on the 'top' image, will now be pointing to the 'base'.
+
+This is useful in flattening(or collapsing or reducing) backing file chain
+length after taking several external snapshots.
+
+
+Let's understand with an illustration below:
+
+We have a base image called 'RootBase', which has a disk image chain with 4
+external snapshots. With 'Active' as the current active-layer, where 'live' guest
+writes happen. There are a few possibilities of resulting image chains that we
+can end up with, using 'blockcommit' :
+
+ (1) Data from Snap-1, Snap-2 and Snap-3 can be merged into 'RootBase'
+ (resulting in RootBase becoming the backing_file of 'Active', and thus
+ invalidating Snap-1, Snap-2, & Snap-3).
+
+ (2) Data from Snap-1 and Snap-2 can be merged into RootBase(resulting in
+ Rootbase becoming the backing_file of Snap-3, and thus invalidating
+ Snap-1 & Snap-2).
+
+ (3) Data from Snap-1 can be merged into RootBase(resulting in RootBase
+ becoming the backing_file of Snap-2, and thus invalidating Snap-1).
+
+ (4) Data from Snap-2 can be merged into Snap-1(resulting in Snap-1 becoming
+ the backing_file of Snap-3, and thus invalidating Snap-2).
+
+ (5) Data from Snap-3 can be merged into Snap-2(resulting in Snap-2 becoming
+ the backing_file for 'Active', and thus invalidating Snap-3).
+
+ (6) Data from Snap-2 and Snap-3 can be merged into Snap-1(resulting in
+ Snap-1 becoming the backing_file of 'Active', and thus invalidating
+ Snap-2 & Snap-3).
+
+ NOTE: Eventually(not supported in qemu as of writing this), we can also
+ merge down the 'Active' layer(the top-most overlay) into its
+ backing_files. Once it is supported, the 'top' argument can become
+ optional, and default to active layer.
+
+
+(The below figure illustrates case (6) from the above)
+
+**Figure-3**
+::
+
+ .------------. .------------. .------------. .------------. .------------.
+ | | | | | | | | | |
+ | RootBase <--- Snap-1 <--- Snap-2 <--- Snap-3 <--- Snap-4 |
+ | | | | | | | | | (Active) |
+ '------------' '------------' '------------' '------------' '------------'
+ / |
+ / |
+ / commit data |
+ / |
+ / |
+ / |
+ v commit data |
+ .------------. .------------. <--------------------' .------------.
+ | | | | | |
+ | RootBase <--- Snap-1 |<---------------------------------| Snap-4 |
+ | | | | Backing File | (Active) |
+ '------------' '------------' '------------'
+
+For instance, if we have the below scenario:
+
+ Actual: [base] <- sn1 <- sn2 <- sn3 <- sn4(this is active)
+
+ Desired: [base] <- sn1 <- sn4 (thus invalidating sn2,sn3)
+
+ Any of the below two methods is valid (as of 17-Oct-2012 qemu-git). With
+ method-a, operation will be faster & correct if we don't care about
+ sn2(because, it'll be invalidated). Note that, method-b is slower, but sn2
+ will remain valid. (Also note that, the guest is 'live' in all these cases).
+
+ **(method-a)**:
+ ::
+
+ # virsh blockcommit --domain f17 vda --base /export/vmimages/sn1.qcow2 --top /export/vmimages/sn3.qcow2 --wait --verbose
+
+ [OR]
+
+ **(method-b)**:
+ ::
+
+ # virsh blockcommit --domain f17 vda --base /export/vmimages/sn2.qcow2 --top /export/vmimages/sn3.qcow2 --wait --verbose
+ # virsh blockcommit --domain f17 vda --base /export/vmimages/sn1.qcow2 --top /export/vmimages/sn2.qcow2 --wait --verbose
+
+ NOTE: If we had to do manually with *qemu-img* cmd, we can only do method-b at the moment.
+
+
+**Figure-4**
+::
+
+ .------------. .------------. .------------. .------------. .------------.
+ | | | | | | | | | |
+ | RootBase <--- Snap-1 <--- Snap-2 <--- Snap-3 <--- Snap-4 |
+ | | | | | | | | | (Active) |
+ '------------' '------------' '------------' '------------' '------------'
+ / | |
+ / | |
+ / | |
+ commit data / commit data | |
+ / | |
+ / | commit data |
+ v | |
+ .------------.<----------------------|-------------' .------------.
+ | |<----------------------' | |
+ | RootBase | | Snap-4 |
+ | |<-------------------------------------------------| (Active) |
+ '------------' Backing File '------------'
+
+
+The above figure is another representation of reducing the disk image chain
+using blockcommit. Data from Snap-1, Snap-2, Snap-3 are merged(/committed)
+into RootBase, & now the current 'Active' image now pointing to 'RootBase' as its
+backing file(instead of Snap-3, which was the case *before* blockcommit). Note
+that, now intermediate images Snap-1, Snap-1, Snap-3 will be invalidated(as they were
+dependent on a particular state of RootBase).
+
+blockpull
+---------
+Block Pull(also called 'Block Stream' in QEMU's paralance) allows you to merge
+into 'base' from a 'top' image(within a disk backing file chain). To rephrase it
+allows merging backing files into an overlay(active). This works in the
+opposite side of 'blockcommit' to flatten the snapshot chain. At the moment,
+**blockpull** can pull only into the active layer(the top-most image). It's
+worth noting here that, intermediate images are not invalidated once a blockpull
+operation is complete (while blockcommit, invalidates them).
+
+
+Consider the below illustration:
+
+**Figure-5**
+::
+
+ .------------. .------------. .------------. .------------. .------------.
+ | | | | | | | | | |
+ | RootBase <--- Snap-1 <--- Snap-2 <--- Snap-3 <--- Snap-4 |
+ | | | | | | | | | (Active) |
+ '------------' '------------' '------------' '------------' '------------'
+ | | \
+ | | \
+ | | \
+ | | \ stream data
+ | | stream data \
+ | stream data | \
+ | | v
+ .------------. | '---------------> .------------.
+ | | '---------------------------------> | |
+ | RootBase | | Snap-4 |
+ | | <---------------------------------------- | (Active) |
+ '------------' Backing File '------------'
+
+
+
+The above figure illustrates that, using block-copy we can pull data from
+Snap-1, Snap-2 and Snap-3 into the 'Active' layer, resulting in 'RootBase'
+becoming the backing file for the 'Active' image (instead of 'Snap-3', which was
+the case before doing the blockpull operation).
+
+The command flow would be:
+ (1) Assuming a external disk-only snapshot was created as mentioned in
+ *Creating Snapshots* section:
+
+ (2) A blockpull operation can be issued this way, to achieve the desired
+ state of *Figure-5*-- [RootBase] <- [Active]. ::
+
+ # virsh blockpull --domain RootBase --path var/lib/libvirt/images/active.qcow2 --base /var/lib/libvirt/images/RootBase.qcow2 --wait --verbose
+
+
+ As a follow up, we can do the below to clean-up the snapshot *tracking*
+ metadata by libvirt (note: the below does not 'remove' the files, it
+ just cleans up the snapshot tracking metadata). ::
+
+ # virsh snapshot-delete --domain RootBase Snap-3 --metadata
+ # virsh snapshot-delete --domain RootBase Snap-2 --metadata
+ # virsh snapshot-delete --domain RootBase Snap-1 --metadata
+
+
+
+
+**Figure-6**
+::
+
+ .------------. .------------. .------------. .------------. .------------.
+ | | | | | | | | | |
+ | RootBase <--- Snap-1 <--- Snap-2 <--- Snap-3 <--- Snap-4 |
+ | | | | | | | | | (Active) |
+ '------------' '------------' '------------' '------------' '------------'
+ | | | \
+ | | | \
+ | | | \ stream data
+ | | | stream data \
+ | | | \
+ | | stream data | \
+ | stream data | '------------------> v
+ | | .--------------.
+ | '---------------------------------> | |
+ | | Snap-4 |
+ '----------------------------------------------------> | (Active) |
+ '--------------'
+ 'Standalone'
+ (w/o backing
+ file)
+
+The above figure illustrates, once blockpull operation is complete, by
+pulling/streaming data from RootBase, Snap-1, Snap-2, Snap-3 into 'Active', all
+the backing files can be discarded and 'Active' now will be a standalone image
+without any backing files.
+
+Command flow would be:
+ (0) Assuming 4 external disk-only (live) snapshots were created as
+ mentioned in *Creating Snapshots* section,
+
+ (1) Let's check the snapshot overlay images size *before* blockpull operation (note the image of 'Active'):
+ ::
+
+ # ls -lash /var/lib/libvirt/images/RootBase.img
+ 608M -rw-r--r--. 1 qemu qemu 1.0G Oct 11 17:54 /var/lib/libvirt/images/RootBase.img
+
+ # ls -lash /var/lib/libvirt/images/*Snap*
+ 840K -rw-------. 1 qemu qemu 896K Oct 11 17:56 /var/lib/libvirt/images/Snap-1.qcow2
+ 392K -rw-------. 1 qemu qemu 448K Oct 11 17:56 /var/lib/libvirt/images/Snap-2.qcow2
+ 456K -rw-------. 1 qemu qemu 512K Oct 11 17:56 /var/lib/libvirt/images/Snap-3.qcow2
+ 2.9M -rw-------. 1 qemu qemu 3.0M Oct 11 18:10 /var/lib/libvirt/images/Active.qcow2
+
+ (2) Also, check the disk image information of 'Active'. It can noticed that
+ 'Active' has Snap-3 as its backing file. ::
+
+ # qemu-img info /var/lib/libvirt/images/Active.qcow2
+ image: /var/lib/libvirt/images/Active.qcow2
+ file format: qcow2
+ virtual size: 1.0G (1073741824 bytes)
+ disk size: 2.9M
+ cluster_size: 65536
+ backing file: /var/lib/libvirt/images/Snap-3.qcow2
+
+ (3) Do the **blockpull** operation. ::
+
+ # virsh blockpull --domain ptest2-base --path /var/lib/libvirt/images/Active.qcow2 --wait --verbose
+ Block Pull: [100 %]
+ Pull complete
+
+ (4) Let's again check the snapshot overlay images size *after*
+ blockpull operation. It can be noticed, 'Active' is now considerably larger. ::
+
+ # ls -lash /var/lib/libvirt/images/*Snap*
+ 840K -rw-------. 1 qemu qemu 896K Oct 11 17:56 /var/lib/libvirt/images/Snap-1.qcow2
+ 392K -rw-------. 1 qemu qemu 448K Oct 11 17:56 /var/lib/libvirt/images/Snap-2.qcow2
+ 456K -rw-------. 1 qemu qemu 512K Oct 11 17:56 /var/lib/libvirt/images/Snap-3.qcow2
+ 1011M -rw-------. 1 qemu qemu 3.0M Oct 11 18:29 /var/lib/libvirt/images/Active.qcow2
+
+
+ (5) Also, check the disk image information of 'Active'. It can now be
+ noticed that 'Active' is a standalone image without any backing file -
+ which is the desired state of *Figure-6*.::
+
+ # qemu-img info /var/lib/libvirt/images/Active.qcow2
+ image: /var/lib/libvirt/images/Active.qcow2
+ file format: qcow2
+ virtual size: 1.0G (1073741824 bytes)
+ disk size: 1.0G
+ cluster_size: 65536
+
+ (6) We can now clean-up the snapshot tracking metadata by libvirt to
+ reflect the new reality ::
+
+ # virsh snapshot-delete --domain RootBase Snap-3 --metadata
+
+ (7) Optionally, one can check, the guest disk contents by invoking
+ *guestfish* tool(part of *libguestfs*) **READ-ONLY** (*--ro* option
+ below does it) as below ::
+
+ # guestfish --ro -i -a /var/lib/libvirt/images/Active.qcow2
+
+
+Deleting snapshots (and 'offline commit')
+=========================================
+
+Deleting (live/offline) *Internal Snapshots* (where the originial & all the named snapshots
+are stored in a single QCOW2 file), is quite straight forward. ::
+
+ # virsh snapshot-delete --domain f17vm --snapshotname snap6
+
+ [OR]
+
+ # virsh snapshot-delete f17vm snap6
+
+Deleting External snapshots (offline), Libvirt has not acquired the capability.
+But, it can be done via *qemu-img* manipulation.
+
+Say, we have this image chain(the guest is *offline* here): **base <- sn1 <- sn2 <- sn3**
+(arrow to be read as 'sn3 has sn2 as its backing file').
+
+
+And, we want to delete the second snapshot(sn2). It's possible to do it in two
+ways:
+
+
+ - **Method (1)**: **base <- sn1 <- sn3** (by copying sn2 into sn1)
+ - **Method (2)**: **base <- sn1 <- sn3** (by copying sn2 into sn3)
+
+Method (1)
+----------
+To end up with this image chain : **base <- sn1 <- sn3** (by copying *sn2* into *sn1*)
+
+**NOTE**: This is only possible *if* sn1 isn't used by more images as their backing
+file, or they'd get corrupted!!
+
+ (a) We're doing an *offline commit* (similar to what *blockcommit* can do
+ to an *online* guest). ::
+
+ # qemu-img commit sn2.qcow2
+
+ - This will *commit* the changes from sn2 into its backing file(which is
+ sn1).
+
+ (b) Now that we've comitted changes from sn2 into sn1, let's change the
+ backing file link in sn3 to point to sn1. ::
+
+ # qemu-img rebase -u -b sn1.qcow2 sn3.qcow2
+
+ - **NOTE**: This is 'Unsafe mode' -- in this mode, only the backing file
+ name is changed w/o any checks on the file contents. The user must
+ take care of specifying the correct new backing file, or the
+ guest-visible. This mode is useful for renaming or moving the
+ backing file to somewhere else. It can be used without an
+ accessible old backing file, i.e. you can use it to fix an image
+ whose backing file has already been moved/renamed.
+
+
+ (c) Now, we can delete the sn2 disk image(as the changes are now committed
+ to sn1). ::
+
+ # rm sn2.qcow2
+
+
+Method (2)
+----------
+To end up with this image chain : **base <- sn1 <- sn3** (by copying *sn2* into *sn3*)
+
+ (a) Copy contents of sn2(the old backing file) into sn3, and change the backing file link of sn3 to sn1::
+
+ # qemu-img rebase -b sn1.qcow2 sn3.qcow2
+
+ - Apart from changing backing file link of sn3 to sn1, the above cmd
+ will it also /copy/ the contents from sn2 into sn3).
+
+ - In other words: This is 'Safe mode', which is the default --
+ any clusters that differ between the new backing_file(in this
+ case, sn1) and the old backing file(in this case, sn2) of
+ filename(in this case, sn3) are merged into filename(sn3), before
+ actually changing the backing file.
+
+ (b) Now, we can delete the sn2 disk image(as the changes are now committed to
+ sn1). ::
+
+ # rm sn2.qcow2
+
--
1.7.7.6
11 years, 11 months
[libvirt] Google Summer of Code 2013 ideas wiki open
by Stefan Hajnoczi
I have created the Google Summer of Code 2013 wiki page where you can
add project ideas:
http://wiki.qemu.org/Google_Summer_of_Code_2013
Please add project ideas you are willing to mentor. If you have an
idea but cannot mentor this year, feel free to add it but please try
to find a mentor for it.
If you want to be a mentor, please see
http://wiki.qemu.org/Google_Summer_of_Code_2013#Information_for_mentors.
Background on Google Summer of Code:
Google Summer of Code is a scheme where Google funds university
students to work on open source projects for 12 weeks during the
summer. It has been held annually since 2005.
GSoC is an opportunity for students to experience open source
development. Organizations sometimes find long-term contributors, and
either way it's a fun experience.
Open source organizations can apply to take part. Those chosen will
be able to take on one or more students for the summer. Each student
is paired with a mentor who helps them and evaluates their progress.
Last year QEMU was not accepted but we were successful in years before
that. I believe Google will announce GSoC again this year (there is
no guarantee though) and I have created the wiki page so we can begin
organizing project ideas that students can choose from.
This way we'll be ready to put together a good organization application.
Stefan
11 years, 11 months
[libvirt] [PATCH 0/5] VirtualBox version 4.2 support for libvirt vbox driver
by ryan woodsmall
This patch set adds VirtualBox 4.2 initial support for the libvirt vbox driver.
I've tested enough to check capabilities, create a VM, destroy it, etc. Five
patches total:
- Patch 1 is the C API header file from Oracle, cleaned up for libvirt.
- Patch 2 is the version specific source file for version dependency.
- Patch 3 is the src/Makefile.am change to pick up the two new files.
- Patch 4 is the vbox driver support for the new VirtualBox API/version.
- Patch 5 is the vbox_tmpl.c template support for the new version.
A few things have changed in the VirtualBox API - some small (capitalizations
of things in function names like Ip to IP and Dhcp to DHCP) and some much larger
(FindMedium is superceded by OpenMedium). The biggest change for the sake of this
patch set is the signature of CreateMachine is quite a bit different. Using the
Oracle source as a guide, to spin up a VM with a given UUID, it looks like a text
flag has to be passed in a new argument to CreateMachine. This flag is built in the
VirtualBox 4.2 specific ifdefs and is kind of ugly but works. Additionally, there
is now (unused) VM groups support in CreateMachine and the previous 'osTypeId' arg
is currently set to nsnull as in the Oracle code.
The FindMedium to OpenMedium changes were more straightforward and are pretty clear.
The rest of the vbox template changes are basically spelling/capitalization changes
from the looks of things.
This probably isn't perfect, but it works on git and patched against 0.10.2 for a
few quick tests. Not currently on the list, so ping me directly if you need any
other info on these, or if anything could use additional work. Thanks! -r
ryan woodsmall (5):
vbox C API header for VirtualBox v4.2
vbox version-specific C file for VirtualBox v4.2
Makefile.am additions for VirtualBox v4.2
vbox driver support for VirtualBox v4.2
vbox template support for VirtualBox v4.2
src/Makefile.am | 3 +-
src/vbox/vbox_CAPI_v4_2.h | 8855 +++++++++++++++++++++++++++++++++++++++++++++
src/vbox/vbox_V4_2.c | 13 +
src/vbox/vbox_driver.c | 8 +
src/vbox/vbox_tmpl.c | 90 +-
5 files changed, 8958 insertions(+), 11 deletions(-)
create mode 100644 src/vbox/vbox_CAPI_v4_2.h
create mode 100644 src/vbox/vbox_V4_2.c
11 years, 11 months
[libvirt] [PATCH] Release VM lock before acquiring virDomainObjListPtr lock
by Daniel P. Berrange
From: "Daniel P. Berrange" <berrange(a)redhat.com>
When removing a VM from the virDomainObjListPtr, we must not
be holding the VM lock while acquiring the list lock. Re-order
code to ensure that we can release the VM lock early.
---
src/conf/domain_conf.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 5e16ddf..d92e54a 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -2115,11 +2115,10 @@ void virDomainObjListRemove(virDomainObjListPtr doms,
{
char uuidstr[VIR_UUID_STRING_BUFLEN];
- virObjectLock(doms);
virUUIDFormat(dom->def->uuid, uuidstr);
-
virObjectUnlock(dom);
+ virObjectLock(doms);
virHashRemoveEntry(doms->objs, uuidstr);
virObjectUnlock(doms);
}
--
1.8.1
11 years, 12 months
[libvirt] [libvirt-php PATCH 0/1] use LDFLAGS for libvirt-php.so
by stefan.kuhn＠foss-group.ch
I'm getting this warning:
* QA Notice: Files built without respecting LDFLAGS have been detected
* Please include the following list of files in your report:
* /usr/lib64/php5.3/lib/extensions/no-debug-non-zts-20090626/libvirt-php.so
The attached patch seems reasonable to me, please check it.
Thank you very much for the quick responses and for adding me to the authors! :-)
Stefan
Stefan Kuhn (1):
src/Makefile.am: use LDFLAGS for libvirt-php.so
src/Makefile.am | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--
1.7.12.4
11 years, 12 months
[libvirt] [libvirt-php PATCH 0/1] Fix: mutlithreading compilation: set_error needs 2 params
by stefan.kuhn＠foss-group.ch
= Info =
Someone reports errors withing "PHP_FUNCTION(libvirt_domain_migrate)" due to "set_error":
libvirt-php.c: In function ‘zif_libvirt_domain_migrate’:
libvirt-php.c:5456:3: error: too few arguments to function ‘set_error’
= Background =
Michal Novotny already reacted to Gentoos bug:
Bug 437624 - =dev-php/libvirt-php-0.4.6: too few arguments to function ‘set_error’
https://bugs.gentoo.org/show_bug.cgi?id=437624
with commit:
bfca998ae921ab897f95025344fad71d9633ccf6
http://libvirt.org/git/?p=libvirt-php.git;a=commit;h=bfca998ae921ab897f95...
but some calls to "set_error" still lacked the 2nd param "TSRMLS_CC", especially in the "PHP_FUNCTION(libvirt_domain_migrate)".
= Patch =
Add 2nd param "TSRMLS_CC" to all calls of "set_error" where only 1 param was supplied.
Stefan Kuhn (1):
Fix: mutlithreading compilation: set_error needs 2 params
src/libvirt-php.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
--
1.7.12.4
11 years, 12 months
[libvirt] libvirt<->QEMU interfaces for CPU models
by Eduardo Habkost
Hi,
After a long time trying to figure out the proper modelling inside QEMU,
I believe the plans are now clearer in QEMU, so it's time to coordinate
more closely with libvirt to try to make use of the new stuff.
I tried to enumerate the libvirt requirements and current problems, and
how we should be able to solve those problems using the X86CPU
subclasses and properties, on the following wiki page:
http://wiki.qemu.org/Features/CPUModels#Interfaces.2Frequirements_from_li...
I am pasting the section contents below, to facilitate discussion:
= Ensuring predictable set of guest features =
Requirement: libvirt needs to ensure all features required on the command-line
are present and exposed to the guest.
Current problem: libvirt doesn't use the "enforce" flag so it can't guarantee
that a given feature will be actually exposed to the guest.
Solution: use the "enforce" flag on the "-cpu" option.
Limitation: no proper machine-friendly interface to report which features
are missing.
Workaround: See "querying for host capabilities" below.
== Future plans ==
Machine-friendly reporting of missing host features/capabilities.
= Listing CPU models =
Requirement: libvirt needs to know which CPU models are available to be used
with the "-cpu" option.
Current problem: libvirt relies on help output parsing for that.
Solution: use QMP qom-list-types command.
Dependency: X86CPU subclasses.
Limitation: needs a live QEMU process for the query.
Solution: use QMP query-cpu-definitions command.
Limitation: needs a live QEMU process for the query.
== Future plans ==
It would be interesting to get rid of the requirement for a live QEMU process
(with a complete machine being created) to be already running.
= Getting information about CPU models =
Requirement: libvirt uses the predefined CPU models from QEMU, but it needs to
be able to query for CPU model details, to find out how it can create a VM that
matches what was requested by the user.
Current problem: libvirt has a copy of the CPU model definitions on its
cpu_map.xml file, and the copy can be out of sync in case CPU models in QEMU
change. libvirt also assumes that the set of features on each model is always
the same on all machine-types, which is not true.
Challenge: the resulting CPU features depend on lots of factors, including
the machine-type.
Workaround: start a paused VM and query for the CPU device information
after the CPU was created.
Solution: start a paused VM with no devices, but with the right
machine-type and right CPU model. Use QMP QOM commands to query for CPU
flags (especially the properties starting with the "f-" prefix).
Dependency: X86CPU feature properties ("f-*" properties).
Limitation: requires a live QEMU process with the right machine-type/
CPU-model to be started, to make the query.
Requirement: libvirt needs to know if a specific CPU model can be used in the
current host.
See "Ensuring predictable set of guest features" above
See "Querying host capabilities" below
= Querying host capabilities =
Requirement: libvirt needs to know which feature can really be enabled, before
it tries to start a VM, and before it tries to start a live-migration process.
The set of available capabilities depend on:
• Host CPU (hardware) capabilities;
• Kernel capabilities (reported by GET_SUPPORTED_CPUID);
• QEMU capabilities;
• Specific configuration options (e.g. in-kernel IRQ chip is required for
some features).
Current problem: libvirt uses the CPUID intruction directly and assumes that
the presence of a feature in the host CPU means it can be enabled and exposed
to the guest. This breaks when virtualization of a feature requires:
• Additional hardware support (e.g. INVPCID);
• Additional host kernel code (this applies to _all_ CPU features, that need
to be reported as supported by GET_SUPPORTED_CPUID);
• Additional QEMU-side code;
• Specific configuration options (e.g. in-kernel IRQ chip).
Challenge: QEMU doesn't have a generic capability-querying interface, and host
capability querying depends on KVM to be initialized.
Workaround: start a paused VM using the "host" CPU model, that has every
single CPU feature supported by the host enabled by default, and query for
the information about the CPU though QMP, using the QOM commands.
Current solution: start a paused VM with no devices but with "host" CPU model
and use QMP QOM commands to query for the enabled CPU features.
Dependency: X86CPU feature properties
== Future plans ==
It would be interesting to have a more generic capability-querying interface
that doesn't require starting a whole machine with a live QEMU process.
See also: -query-capabilities RFC series from Anthony
Message-Id: <1332169763-30665-9-git-send-email-aliguori(a)us.ibm.com>
--
Eduardo
12 years
[libvirt] Problems with <filesystem type='block'>
by Lars Kellogg-Stedman
Using libvirt 1.0.1, I'm trying to start an LXC container using the
'<filesytem type="block">' syntax, like this:
<filesystem type="block" accessmode="passthrough">
<source dev="/dev/vg_files/vm-foobar-root" />
<target dir="/" />
</filesystem>
The specified block device exists:
# ls -lL /dev/vg_files/vm-foobar-root
brw-rw---- 1 root disk 253, 19 Dec 21 22:23 /dev/vg_files/vm-foobar-root
If I start the domain, it appears to start without any errors...
# virsh start foobar
Domain foobar started
...but it's not actually running. The log files (with loglevel=2)
don't seem to be very interesting; this is everything from the
instance log file:
2012-12-22 04:10:57.862+0000: starting up
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin LIBVIRT_DEBUG=2 LIBVIRT_LOG_OUTPUTS=2:stderr /usr/lib/libvirt/libvirt_lxc --name foobar --console 22 --security=none --handshake 25 --background --veth veth1
2012-12-22 04:10:57.967+0000: 1468: info : libvirt version: 1.0.1
2012-12-22 04:10:57.967+0000: 1468: info : lxcCapsInit:151 : No driver, not initializing security driver
PATH=/bin:/sbin TERM=linux container=lxc-libvirt container_uuid=9041e32e-1df2-00c2-4660-dfa5b41510b7 LIBVIRT_LXC_UUID=9041e32e-1df2-00c2-4660-dfa5b41510b7 LIBVIRT_LXC_NAME=foobar /sbin/init
2012-12-22 04:10:58.198+0000: 1: warning : lxcContainerDropCapabilities:1788 : libcap-ng support not compiled in, unable to clear capabilities
2012-12-22 04:10:58.198+0000: 1493: warning : lxcControllerClearCapabilities:679 : libcap-ng support not compiled in, unable to clear capabilities
Running an "strace" on the libvirtd process (strace -p <libvirtd_pid>
-f ...), it doesn't look like libvirt is ever trying to mount the
referenced filesystem.
Is this supposed to work? It seems like the support for having
libvirt mount the block device is relatively recent, and I haven't had
much luck finding examples of other folks using this capability.
Thanks,
-- Lars
12 years
[libvirt] [PATCH 0/4] Multiple problems with saving to block devices
by Daniel P. Berrange
This patch series makes it possible to save to a block device,
instead of a plain file. There were multiple problems
- WHen save failed, we might de-reference a NULL pointer
- When save failed, we unlinked the device node !!
- The approach of using >> to append, doesn't work with block devices
- CGroups was blocking QEMU access to the block device when enabled
One remaining problem is not in libvirt, but rather QEMU. The QEMU
exec: based migration often fails to detect failure of the command
and will thus hang forever attempting a migration that'll never
succeed! Fortunately you can now work around this in libvirt using
the virsh domjobabort command
12 years
