On 02/01/2013 11:20 AM, Jamie Fargen wrote: > I am searching for directions for using live block migration to copy > running vm's to a different storage pool. > > > Example: VM1 running on Host1, the image(s) for VM1 are stored in > /var/lib/libvirt/images. I'd like to copy the disk image(s) that VM1 > is using to /nfs/images. Without stopping/pausing/powering down the > VM. > > Do you have any examples or documentation of how to accomplish this process? Unfortunately, we probably need to do a better job on documentation or examples; there has been a patch proposed that might help: https://www.redhat.com/archives/libvir-list/2013-January/msg01637.html But it is indeed possible to migrate storage with qemu 1.3 or later, with a guest downtime of only a fraction of a second. True avoidance of guest downtime during a storage migration requires support for persistent dirty bitmaps, which won't be added until at least qemu 1.5; likewise, the lack of persistent dirty bitmaps means the current libvirt will only let you do live storage migration on a transient guest (the hope is that the persistent bitmap of qemu 1.5 will let libvirt allow block mirroring across both guest and libvirtd restarts, and therefore I can add code to libvirt to make blockcopy work for a persistent guest). Thankfully, it is possible to make a guest transient, do the storage migration, then make the guest persistent again. So, the way I have done live block migration is as follows: # save off the persistent definition for later virsh dumpxml --inactive $dom > $dom.xml # make the guest transient virsh undefine $dom # remind myself which disks need migration virsh domblklist $dom # for each disk (such as "vda"), do a migration virsh blockcopy $dom $disk /path/to/new --wait --verbose --pivot # make the guest persistent again virsh define $dom.xml Note that while my example was blocking for each disk, you can also do migration in parallel for multiple disks at once, if you avoid the '--wait' parameter to 'virsh blockcopy' and instead use 'virsh blockjob $dom $disk --pivot' to do the pivots. An attempt to pivot will fail until the disk has reached mirroring state; but it does mean that you have to set up your own polling loop to see when each disk is ready. Also, a shallow block copy can result in less downtime in waiting for a disk to reach mirroring state. That is, given the following disk layout prior to the migration (such as by using 'virsh snapshot-create-as $dom --disk-only): base <- snapshot then avoiding --shallow has to transfer the entire disk image, creating: copy while using --shallow only has to transfer the contents of snapashot, creating: base <- copy When using --shallow, libvirt will NOT copy the base file; and since qemu will prefer to create backing chains that used absolute names to the base files, the copied leaf image defaults to pointing to the original base image. If you need to migrate the entire chain including the base file, then you have to pre-create a blank destination qcow2 file that points to the new desired base image (using qemu-img create), then use the --reuse-external flag when telling libvirt to do the blockcopy, and avoid doing the pivot operation until the base file has been copied to its new location (it is possible to copy the base image in parallel with waiting for libvirt to reach full mirroring phase of the shallow blockcopy of the leaf image, and wait do manually do the pivot until both operations are ready, regardless of which finished first). Further use of blockpull or blockcommit can reduce the length of a backing chain without any guest downtime (although as of qemu 1.4, while blockpull can be used to collapse down to a single element, blockcommit only works for collapsing a chain of 3 or more files down to a minimum of 2). I'm happy to answer more questions you may have. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org

On 02/12/2013 06:22 AM, Jamie Fargen wrote: > Eric- [Please don't top-post on technical lists] > > Why wouldn't a 'virsh blockcopy --pivot domain src dest' be sufficient to > migrate the volumes to a new storage pool? It IS sufficient, as part of the sequence I documented here: >> >> # save off the persistent definition for later >> virsh dumpxml --inactive $dom > $dom.xml >> # make the guest transient >> virsh undefine $dom >> # remind myself which disks need migration >> virsh domblklist $dom >> # for each disk (such as "vda"), do a migration >> virsh blockcopy $dom $disk /path/to/new --wait --verbose --pivot >> # make the guest persistent again >> virsh define $dom.xml Or maybe I'm misunderstanding your question. Are you asking why blockcopy doesn't work with a persistent domain? And the answer to that is because as of at least qemu 1.4, there is no support for remembering the progress of a block copy across a domain restart; there have been patches proposed for qemu 1.5 that add a persistent bitmap, and there are plans to teach libvirt to use this new feature to allow 'virsh blockcopy' even for a persistent domain, so that you don't have to juggle through the steps of going to a transient domain just to do the blockcopy. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org

Eric, Following along with your direction I get stuck at virsh blockcopy. I receive an error "unsupported configuration: block copy is not supported with this QEMU binary." As I originally explained this host is running Red Hat EL6.3. Is live blockcopy supported in EL6.3?

Packages: qemu-kvm-0.12.1.2-2.295.el6_3.2.x86_64 libvirt-client-0.9.10-21.el6_3.4.x86_64 Regards, Jamie Ian Fargen On Tue, Feb 12, 2013 at 10:12 AM, Eric Blake <eblake(a)redhat.com <mailto:eblake@redhat.com>> wrote: On 02/12/2013 06:22 AM, Jamie Fargen wrote: > Eric- [Please don't top-post on technical lists] > > Why wouldn't a 'virsh blockcopy --pivot domain src dest' be sufficient to > migrate the volumes to a new storage pool? It IS sufficient, as part of the sequence I documented here: >> >> # save off the persistent definition for later >> virsh dumpxml --inactive $dom > $dom.xml >> # make the guest transient >> virsh undefine $dom >> # remind myself which disks need migration >> virsh domblklist $dom >> # for each disk (such as "vda"), do a migration >> virsh blockcopy $dom $disk /path/to/new --wait --verbose --pivot >> # make the guest persistent again >> virsh define $dom.xml Or maybe I'm misunderstanding your question. Are you asking why blockcopy doesn't work with a persistent domain? And the answer to that is because as of at least qemu 1.4, there is no support for remembering the progress of a block copy across a domain restart; there have been patches proposed for qemu 1.5 that add a persistent bitmap, and there are plans to teach libvirt to use this new feature to allow 'virsh blockcopy' even for a persistent domain, so that you don't have to juggle through the steps of going to a transient domain just to do the blockcopy. -- Eric Blake eblake redhat com +1-919-301-3266 <tel:%2B1-919-301-3266> Libvirt virtualization library http://libvirt.org _______________________________________________ libvirt-users mailing list libvirt-users(a)redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users