[libvirt-users] Proper way to reference a disk
Hi, We are running on CentOS 7.0. We want to move our VMs (kvm/qemu) from qcow/image files to logical volumes under LVM. I have moved one of them but I had a little trouble editing the disk properly and I wonder what is the correct way to do it. Specifically, in the XML under the devices/disk section why does the source say "source file=", and not "source dev="? The reason I ask is that I have a number of other machines to move and I don't want to mess them up. The source file entry for the disk was like this in our xml file: <source file='/var/lib/libvirt/images/pxetest.qcow2'/> I got the vm to work by converting the image to raw, then dumping via dd into the LV, and editing the XML using virsh as follows: <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='none'/> <*source file*='/dev/kvm_temp/pxetest'/> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> By comparison I have a test machine on another real host upon which I loaded a VM and put its disk into a logical volume. Its disk section looks like this: <disk type='block' device='disk'> <driver name='qemu' type='raw' cache='none' io='native'/> <*source dev*='/dev/kvm_hosts/pxetest2'/> <target dev='hda' bus='ide'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> The source file / source dev thing really has me wondering. *- Mike Schwager* * Linux Network Engineer, Mocho Trading LLC* -- This message is for the named person(s) use only. It may contain confidential proprietary or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of it and notify the sender. You must not, directly or indirectly use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. Mocho Trading LLC reserves the right to monitor all e-mail communications through its networks. Any views expressed in this message are those of the individual sender, except where the message states otherwise and the sender is authorized to state them to be the views of any such entity.
On 05/06/2015 02:05 PM, Michael Schwager wrote:
Hi, We are running on CentOS 7.0. We want to move our VMs (kvm/qemu) from qcow/image files to logical volumes under LVM.
I have moved one of them but I had a little trouble editing the disk properly and I wonder what is the correct way to do it. Specifically, in the XML under the devices/disk section why does the source say "source file=", and not "source dev="? The reason I ask is that I have a number of other machines to move and I don't want to mess them up.
The source file entry for the disk was like this in our xml file: <source file='/var/lib/libvirt/images/pxetest.qcow2'/>
I got the vm to work by converting the image to raw, then dumping via dd into the LV, and editing the XML using virsh as follows: <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='none'/> <*source file*='/dev/kvm_temp/pxetest'/>
The key for what <source> looks like is the <disk type='...'> attribute a couple lines earlier. When type='file', then <source> lists a file='...' absolute path, and libvirt treats it as a file (which sort of works, except when querying for how much of the file is actually allocated, because it assumes that stat() alone is good enough)
By comparison I have a test machine on another real host upon which I loaded a VM and put its disk into a logical volume. Its disk section looks like this:
<disk type='block' device='disk'> <driver name='qemu' type='raw' cache='none' io='native'/> <*source dev*='/dev/kvm_hosts/pxetest2'/>
And here, you used type='block', so <source> must list dev='...' and give an absolute path to a block device (and here, when libvirt is computing how much of the device is allocated, it relies on more than just stat(), which can give more accurate numbers when doing things like qcow2 format atop an LVM block device).
<target dev='hda' bus='ide'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk>
The source file / source dev thing really has me wondering.
Hopefully I answered it. As a rule of thumb, if it is a block device, use type='block' (or type='lun' if you need sgio passthrough support) and <source dev='...'>); otherwise use type='file' and <source file='...'. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Wed, May 6, 2015 at 4:14 PM, Eric Blake <eblake@redhat.com> wrote:
The source file / source dev thing really has me wondering. Hopefully I answered it.
Yes you did, thanks. Furthermore,
which can give more accurate numbers when doing things like qcow2 format atop an LVM block device).
...This has me intrigued. We are running qcow2 format in files, and the instructions I've seen have you converting qcow2 to raw and then dd into LVM. But is there an advantage to running qcow2 on LVM? Is there an example of this in action? Thanks again. *- Mike Schwager* * Linux Network Engineer, Mocho Trading LLC* * 312-646-4783 Phone 312-637-0011 Cell 312-957-9804 Fax* -- This message is for the named person(s) use only. It may contain confidential proprietary or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of it and notify the sender. You must not, directly or indirectly use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. Mocho Trading LLC reserves the right to monitor all e-mail communications through its networks. Any views expressed in this message are those of the individual sender, except where the message states otherwise and the sender is authorized to state them to be the views of any such entity.
On 05/06/2015 03:27 PM, Michael Schwager wrote:
which can give more accurate numbers when doing things like qcow2 format atop an LVM block device).
...This has me intrigued. We are running qcow2 format in files, and the instructions I've seen have you converting qcow2 to raw and then dd into LVM. But is there an advantage to running qcow2 on LVM? Is there an example of this in action?
VDSM does this. You gain the advantages of qcow2 (storage snapshots, efficient zero-cluster representations) coupled with less overhead than what you would get with qcow2 on a file system (cutting out the filesystem layer) and guaranteed host resources (you know the lvm size is reserved for the guest and won't grow without your action, unlike a file system where automatic growth of a qcow2 file competes for space quotas with every other file in the file system). It's still less efficient than running raw, but as close as you can get to bare-metal speeds while still gaining benefits from the format. When using qcow2-on-lvm, you are responsible for making sure the LVM volume is large enough: often (thanks to qcow2 having efficient 0-cluster representation, and particularly true if your guest supports TRIM on clusters that its file system is not using) the guest will see more virtual space than what the actual lvm volume exposes (a guest that needs 10G of storage but isn't fully using it can sometimes get by with a 5G lvm partition, for example); although the converse is also possible (a guest that can only see 10G of storage, but where you have lots of qcow2 snapshots, may require 20G of lvm storage to track all the qcow2 metadata). Another use case for using qcow2 on lvm is that you can initially oversubscribe your storage: qemu has a mode where it will gracefully halt the guest if it runs out of space required for managing the qcow2 data, at which point you can then enlarge the lvm volume and resume the guest; the guest will never know that you were shuffling the size of the host storage (note that this applies for twp separate cases - one where you are merely giving the host more room to store qcow2 metadata, but leaving the guest size unchanged, such as when you start a guest that sees a 10G disk on an lvm partition that is initially 5G large, then enlarge the lvm partition to 6G but the guest still sees a 10G disk; the other where you are actually resizing the amount of storage allocated to the guest as if you had hot-swapped in a larger disk, and where some guests can then resize their file system on the fly to take advantage of the larger disk). Related to this, I'm currently working on a patch for libvirt that will take advantage of a recent qemu feature of setting a threshold for event triggering, such that you could then request notification that the lvm volume is X% full and give yourself time to enlarge the lvm volume even before you reach the point where the guest would halt because the lvm volume was full. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
participants (2)
-
Eric Blake -
Michael Schwager