[libvirt-users] Looks like blockpull does not accept a subset of the entire chain of backing files
Hello If I'm not terribly mistaken, looks like libvirt 1.2.1 does not provide ability of merging only a subset of the entire chain of backing files. So, if I have a chain like this: root <- a <-b <- c <- d <- active ... and I'd like to obtain a chain like this: root <- c <- d <- active ... looks like it's not supported, since I'm trying the command / //virsh blockpull domain --path /path/to/c --bandwidth 30 --base /path/to/root// (1) / ... but it complains because /--path /path/to/c/ is not recognized, since only /--path /path/to/active/ is acceptable: / //virsh blockpull domain --path /path/to/active --bandwidth 30 --base /path/to/root// (2) / So, command (1) fails but command (2) succeeds. The point is: How could I obtain the results I'm trying to achieve via command (1) ? I'm new to libvirt, but the article below made me think that what I'm trying to do would be possible: http://kashyapc.fedorapeople.org/virt/lc-2012/snapshots-handout.html ( see section on /blockpull/ ) Thoughts? For your information, my environment is: Compiled against library: libvirt 1.2.1 Using library: libvirt 1.2.1 Using API: QEMU 1.2.1 Running hypervisor: QEMU 1.7.0 Running against daemon: 1.2.1 Thanks -- Richard Gomes http://rgomes.info http://www.linkedin.com/in/rgomes mobile: +44(77)9955-6813 inum <http://www.inum.net/>: +883(5100)0800-9804 sip:rgomes@ippi.fr
On 01/29/2014 07:51 AM, Richard Gomes wrote:
Hello
If I'm not terribly mistaken, looks like libvirt 1.2.1 does not provide ability of merging only a subset of the entire chain of backing files.
Correct - qemu doesn't provide full functionality for merging in all directions. And even though upstream qemu has been adding more pieces (for example, they recently added the ability to commit the active image), libvirt still has to be taught to exercise that.
So, if I have a chain like this:
root <- a <-b <- c <- d <- active
... and I'd like to obtain a chain like this:
root <- c <- d <- active
Are you trying to squash a and b into root (commit direction, in which case, the existing block-commit command works just fine for this scenario) or into c (pull direction, in which case, qemu doesn't support it yet)?
The point is: How could I obtain the results I'm trying to achieve via command (1) ?
In the current blockpull implementation, qemu refuses to pull into anything but the active image (true even for your use of qemu 1.7). I've been asking for the ability to pull into intermediate images, but it isn't there yet.
I'm new to libvirt, but the article below made me think that what I'm trying to do would be possible:
That said, it IS possible to fake the same effect, using a series of block-rebase and raw qemu-img commands (not heavily tested, so you may still have to tweak the actions you use). Also, block mirroring doesn't yet work with persistent disks (because qemu doesn't yet have a way to resume an operation across restarts), so you temporarily have to make your domain transient (virsh undefine $dom); you can redefine it at the end of the sequence. Start from root <- a <- b <- c <- d <- active Create c1 with 'cp c c1' for this tree: root <- a <- b \- c <- d <- active - c1 Merge a and b into c1 with 'qemu-img rebase -b base c1' for this tree: root \- a <- b <- c <- d <- active ------------c1 Create d1 with 'cp d d1' for this tree: root \- a <- b <- c \- d <- active ------------c1 - d1 Use 'qemu-img rebase -u -b c1 d1' for this tree: root \- a <- b <- c <- d <- active ------------c1 <- d1 Create a blank destination active file wrapping d1, with 'qemu-img create -f qcow2 -o backing_file=d1,backing_fmt=qcow2 active1' for this tree: root \- a <- b <- c <- d <- active ------------c1 <- d1 <- active1 Set up a block mirroring, and pivot over to the new chain once it is stable, with 'virsh blockcopy $dom active --dest active1 --shallow --reuse-external --wait --pivot', for this tree: root \------------c1 <- d1 <- active1 - a <- b <- c <- d <- active Clean up - a, b, c, d, and active are now no longer in use, and both qemu and libvirt see the shorter chain (albeit with new file names), with virtually no guest downtime. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Hello Eric, Absolutely enlightening! Thanks a lot :) Richard Gomes http://rgomes.info http://www.linkedin.com/in/rgomes mobile: +44(77)9955-6813 inum <http://www.inum.net/>: +883(5100)0800-9804 sip:rgomes@ippi.fr On 29/01/14 16:46, Eric Blake wrote:
Hello
If I'm not terribly mistaken, looks like libvirt 1.2.1 does not provide ability of merging only a subset of the entire chain of backing files. Correct - qemu doesn't provide full functionality for merging in all
On 01/29/2014 07:51 AM, Richard Gomes wrote: directions. And even though upstream qemu has been adding more pieces (for example, they recently added the ability to commit the active image), libvirt still has to be taught to exercise that.
So, if I have a chain like this:
root <- a <-b <- c <- d <- active
... and I'd like to obtain a chain like this:
root <- c <- d <- active Are you trying to squash a and b into root (commit direction, in which case, the existing block-commit command works just fine for this scenario) or into c (pull direction, in which case, qemu doesn't support it yet)?
The point is: How could I obtain the results I'm trying to achieve via command (1) ? In the current blockpull implementation, qemu refuses to pull into anything but the active image (true even for your use of qemu 1.7). I've been asking for the ability to pull into intermediate images, but it isn't there yet.
I'm new to libvirt, but the article below made me think that what I'm trying to do would be possible: That said, it IS possible to fake the same effect, using a series of block-rebase and raw qemu-img commands (not heavily tested, so you may still have to tweak the actions you use). Also, block mirroring doesn't yet work with persistent disks (because qemu doesn't yet have a way to resume an operation across restarts), so you temporarily have to make your domain transient (virsh undefine $dom); you can redefine it at the end of the sequence.
Start from root <- a <- b <- c <- d <- active
Create c1 with 'cp c c1' for this tree: root <- a <- b \- c <- d <- active - c1
Merge a and b into c1 with 'qemu-img rebase -b base c1' for this tree: root \- a <- b <- c <- d <- active ------------c1
Create d1 with 'cp d d1' for this tree: root \- a <- b <- c \- d <- active ------------c1 - d1
Use 'qemu-img rebase -u -b c1 d1' for this tree: root \- a <- b <- c <- d <- active ------------c1 <- d1
Create a blank destination active file wrapping d1, with 'qemu-img create -f qcow2 -o backing_file=d1,backing_fmt=qcow2 active1' for this tree: root \- a <- b <- c <- d <- active ------------c1 <- d1 <- active1
Set up a block mirroring, and pivot over to the new chain once it is stable, with 'virsh blockcopy $dom active --dest active1 --shallow --reuse-external --wait --pivot', for this tree: root \------------c1 <- d1 <- active1 - a <- b <- c <- d <- active
Clean up - a, b, c, d, and active are now no longer in use, and both qemu and libvirt see the shorter chain (albeit with new file names), with virtually no guest downtime.
hello, I'm trying to use macvtap on Debian Wheezy. Actually, I've installed a recent version of libvirt and qemu from wheezy-backports. $ virsh version Compiled against library: libvirt 1.2.1 Using library: libvirt 1.2.1 Using API: QEMU 1.2.1 Running hypervisor: QEMU 1.7.0 I'm trying to configure a macvtap interface like this: <interface type='direct'> <mac address='52:54:00:7b:05:cd'/> <source dev='eth1' mode='passthrough'/> <model type='rtl8139'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </interface> The interface eth1 is used exclusively by this VM, so as far as I understand, mode 'passthrough' is enough since I'd like to "plug" the VM straight onto a public static IP. On my /etc/networks/interfaces, I have this: # start :: define eth1 iface eth1 inet manual # end :: define eth1 It does not matter whether eth1 is up or down, when I try to start the VM, I get the following error: Error starting domain: Cannot create macvlan devices on this platform: Function not implemented Traceback (most recent call last): File "/usr/share/virt-manager/virtManager/asyncjob.py", line 45, in cb_wrapper callback(asyncjob, *args, **kwargs) File "/usr/share/virt-manager/virtManager/asyncjob.py", line 66, in tmpcb callback(*args, **kwargs) File "/usr/share/virt-manager/virtManager/domain.py", line 1114, in startup self._backend.create() File "/usr/lib/python2.7/dist-packages/libvirt.py", line 866, in create if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self) libvirtError: Cannot create macvlan devices on this platform: Function not implemented Trying to solve the issue, I found this: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=737097 I've tried to rebuild the package from Debian sources, but apparently, macvtap is already defined, as I've reported to another mailing list. For your information:
I'm new to building packages from Debian sources, so I followed instructions from
https://wiki.debian.org/HowToPackageForDebian#Building_Debian_packages
$ apt-get -t wheezy-backports source libvirt $ cd libvirt-0.9.12.3/ $ sudo apt-get build-dep libvirt $ debuild -i -us -uc -b
So... yes, it works and builds without any errors.
Next step now is applying the changes mentioned. But I'm confused because /apparently/ debian/rules is already the way it should be.
ifneq (,$(findstring $(DEB_HOST_ARCH_OS), linux)) WITH_STORAGE_LVM = --with-storage-lvm WITH_STORAGE_ISCSI = --with-storage-iscsi WITH_STORAGE_DISK = --with-storage-disk WITH_UDEV = --with-udev --without-hal WITH_CAPNG = --with-capng WITH_POLKIT = --with-polkit * WITH_MACVTAP = --with-macvtap** ** WITH_NETWORK = --with-network** * WITH_QEMU = --with-qemu WITH_OPENVZ = --with-openvz WITH_NETCF = --with-netcf ifneq (,$(findstring $(DEB_HOST_ARCH), amd64 i386 ia64 mips mipsel powerpc)) WITH_NUMA = --with-numactl else WITH_NUMA = --without-numactl endif ifneq (,$(findstring $(DEB_HOST_ARCH), ia64)) WITH_LXC = --without-lxc else WITH_LXC = --with-lxc endif else WITH_STORAGE_LVM = --without-storage-lvm WITH_STORAGE_ISCSI = --without-storage-iscsi WITH_STORAGE_DISK = --without-storage-disk WITH_UDEV = --without-udev --with-hal WITH_CAPNG = --without-capng WITH_POLKIT = --without-polkit * WITH_MACVTAP = --without-macvtap** ** WITH_NETWORK = --without-network** * WITH_QEMU = --without-qemu WITH_LXC = --without-lxc WITH_NUMA = --without-numactl WITH_NETCF = --without-netcf endif
I suppose that nothing has to be changed, correct? ... since the _/then/_ branch already has the change mentioned and the _/else/_ branch probably means that nothing related to networking would be enabled anyway.
Am I missing something ?
Thoughts? Thanks a lot :) Richard Gomes http://rgomes.info http://www.linkedin.com/in/rgomes mobile: +44(77)9955-6813 inum <http://www.inum.net/>: +883(5100)0800-9804 sip:rgomes@ippi.fr
On 01/31/2014 10:56 AM, Richard Gomes wrote:
hello,
I'm trying to use macvtap on Debian Wheezy. Actually, I've installed a recent version of libvirt and qemu from wheezy-backports.
Your mail has nothing to do with your subject line. It would have been better to start a new thread instead of hijacking an unrelated one. I don't personally know enough about building on Debian to be able to answer your question. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On 01/29/2014 08:21 PM, Richard Gomes wrote:
Hello
If I'm not terribly mistaken, looks like libvirt 1.2.1 does not provide ability of merging only a subset of the entire chain of backing files.
So, if I have a chain like this:
root <- a <-b <- c <- d <- active
... and I'd like to obtain a chain like this:
root <- c <- d <- active
... looks like it's not supported, since I'm trying the command
/ //virsh blockpull domain --path /path/to/c --bandwidth 30 --base /path/to/root// (1) / ... but it complains because /--path /path/to/c/ is not recognized, since only /--path /path/to/active/ is acceptable:
/ //virsh blockpull domain --path /path/to/active --bandwidth 30 --base /path/to/root// (2) / So, command (1) fails but command (2) succeeds.
The point is: How could I obtain the results I'm trying to achieve via command (1) ?
I'm new to libvirt, but the article below made me think that what I'm trying to do would be possible:
http://kashyapc.fedorapeople.org/virt/lc-2012/snapshots-handout.html ( see section on /blockpull/ )
I still have to make to time to submit a V2 of that that patch[1] with reflecting latest upstream status. Apart from what Eric said in painstakingly-detailed way, here's some old notes from a previous discussion with Eric on mailing lists/IRC. Eric, please point out if something is wrong below. - "blockpull" can only pull into the *active* layer. - Support to pull into an intermediate/arbitrary layer is in progress. - NOTES: - Pulling into intermediate images requires opening the intermediate file read/write, as well as ensuring that any reads done in the active layer get correct data. - Case of 'reading data' - In the chain base <- snap1 <- active if you pull base into snap1, reading active has to know whether the data comes from base or from snap1 . But if you pull into active, you already have the destination read/write, and once the data is pulled, you don't have to worry about keeping the chain consistent. - Case of 'writing data' -- If you pull into the active layer, then a write eliminates the need to pull that cluster from the backing file, thanks to copy-on-write semantics. But, if you pull into an intermediate layer, a write at the active layer doesn't affect the fac t that you still have to pull into the intermediate layer. Even though the active layer won't use that cluster from the intermediate layer. - At the moment, virsh blockpull cannot be done 'offline' (although, 'qemu-img' can do that.) [1] http://www.redhat.com/archives/libvir-list/2013-January/msg01903.html -- /kashyap
Hi Kashyap, Thanks a lot for clarifying. If there's an open issue for this improvement, please let me so that I will subscribe to it. Cheers :) Richard Gomes http://rgomes.info http://www.linkedin.com/in/rgomes mobile: +44(77)9955-6813 inum <http://www.inum.net/>: +883(5100)0800-9804 sip:rgomes@ippi.fr On 31/01/14 18:31, Kashyap Chamarthy wrote:
On 01/29/2014 08:21 PM, Richard Gomes wrote:
Hello
If I'm not terribly mistaken, looks like libvirt 1.2.1 does not provide ability of merging only a subset of the entire chain of backing files.
So, if I have a chain like this:
root <- a <-b <- c <- d <- active
... and I'd like to obtain a chain like this:
root <- c <- d <- active
... looks like it's not supported, since I'm trying the command
/ //virsh blockpull domain --path /path/to/c --bandwidth 30 --base /path/to/root// (1) / ... but it complains because /--path /path/to/c/ is not recognized, since only /--path /path/to/active/ is acceptable:
/ //virsh blockpull domain --path /path/to/active --bandwidth 30 --base /path/to/root// (2) / So, command (1) fails but command (2) succeeds.
The point is: How could I obtain the results I'm trying to achieve via command (1) ?
I'm new to libvirt, but the article below made me think that what I'm trying to do would be possible:
http://kashyapc.fedorapeople.org/virt/lc-2012/snapshots-handout.html ( see section on /blockpull/ ) I still have to make to time to submit a V2 of that that patch[1] with reflecting latest upstream status.
Apart from what Eric said in painstakingly-detailed way, here's some old notes from a previous discussion with Eric on mailing lists/IRC. Eric, please point out if something is wrong below.
- "blockpull" can only pull into the *active* layer.
- Support to pull into an intermediate/arbitrary layer is in progress.
- NOTES:
- Pulling into intermediate images requires opening the intermediate file read/write, as well as ensuring that any reads done in the active layer get correct data.
- Case of 'reading data' - In the chain base <- snap1 <- active if you pull base into snap1, reading active has to know whether the data comes from base or from snap1 . But if you pull into active, you already have the destination read/write, and once the data is pulled, you don't have to worry about keeping the chain consistent.
- Case of 'writing data' -- If you pull into the active layer, then a write eliminates the need to pull that cluster from the backing file, thanks to copy-on-write semantics. But, if you pull into an intermediate layer, a write at the active layer doesn't affect the fac t that you still have to pull into the intermediate layer. Even though the active layer won't use that cluster from the intermediate layer.
- At the moment, virsh blockpull cannot be done 'offline' (although, 'qemu-img' can do that.)
[1] http://www.redhat.com/archives/libvir-list/2013-January/msg01903.html
participants (5)
-
Eric Blake -
Kashyap Chamarthy -
Richard Gomes -
Richard Gomes -
Richard Gomes