Hi, I am using virStorageVolGetInfo to get the volume size virStorageVol.download to download a normal file for VDSM project. I want to add support for block devices as well, I was able to see the size via virDomainBlockInfo and download via virDomain.blockPeek which is works fine for both file and block device. 1. Can I depend on virDomainBlockInfo for non block devices? 2. Does the virDomainBlockInfo return the correct physical size for normal file? 3. Performance wise do we have a difference between virStorageVol.download and virDomain.blockPeek? Assuming the VMs are down, this is the code I intend to use for all formats: vm = con.lookupByName(options.vmname) info = vm.blockInfo(src) physical = info[2] off = 0 size = 0 with open(dest, 'w+') as f: while off < physical: if (physical - off) < bufsize: size = physical - off else: size = bufsize buf = vm.blockPeek(src, off, size) f.write(buf) off += size

On 12/05/2016 05:27 AM, Shahar Havivi wrote: > Ping > Sorry for the delay... Lots of investigation, patches, and other work. > On Wed, Nov 30, 2016 at 3:13 PM, Shahar Havivi <shavivi(a)redhat.com > <mailto:shavivi@redhat.com>> wrote: > > Hi, > > I am using virStorageVolGetInfo to get the volume size > virStorageVol.download to download a > normal file for VDSM project. Normal as in a 'raw' fully allocated file? As opposed to a sparse and/or container (e.g. qcow2) file? > > I want to add support for block devices as well, I was able to see > the size > via virDomainBlockInfo and download via virDomain.blockPeek which is > works > fine for both file and block device. As described in the bz, but for others benefit... While looking into this I found a commit Eric Blake made: http://libvirt.org/git/?p=libvirt.git;a=commit;h= 0282ca45a0a4c384226d526d4878cacd77936ef4 which more or less summarizes to: capacity: logical size in bytes of the image (how much storage the guest will see) allocation: host storage in bytes occupied by the image (such as highest allocated extent if there are no holes, similar to 'du') physical: host physical size in bytes of the image container (last offset, similar to 'ls') The data virDomainGetBlockInfo gets when the domain is not active is the same data that virStorageVolGetInfo gets (which I know you couldn't use because it didn't have <physical>). IOW: In the long run for libvirt, they use the same API's to get the data. For a "file" backed <disk> the API's use 'fstat()' while for a "block" backed <disk> the API's use 'lseek()' to the end. Unfortunately there were some differences that have crept into virDomainGetBlockInfo for active domain as a result of changes made to support virConnectGetAllDomainStats. Fortunately those differences are mostly related to domain disk devices with sources that are sparse files, containers (qcow2, qed for example), or block devices (perhaps an iSCSI /dev/sdX device) that's using some sort of container file. The virDomainBlockPeek strictly uses the lseek(), which I think should return the same answer as fstat() - although I could be wrong. > > 1. Can I depend on virDomainBlockInfo for non block devices? You should be able to for an inactive domain. If it doesn't provide you with the right or expected answer, then investigation could be done. The API "works" using the <target dev='nnX' bus='$BUS'/> such that "block" is how the guest is viewing things - e.g. /dev/nnX if nnX were valid. Typically for bus='virtio', it's dev='vdX' where X is a, b, c, etc. Still it the <source ...> part of the domain XML which will get the answer for an inactive domain, while for an active domain there's matching going on from a qemu call to get the block info from the domain. > 2. Does the virDomainBlockInfo return the correct physical size for > normal file? I hope so... It's supposed to. Of course what's normal these days? I would think sparse files are the norm. After all who wants to require 15-20G when they could get away with 5G knowing that up to some 15-20G could be used. There's also block devices using a logical backend where we rely on the logical code to provide the "right answer". Again - if something isn't providing the right answer, then more details would be necessary. > 3. Performance wise do we have a difference between > virStorageVol.download and virDomain.blockPeek? > Well blockPeek is pretty quick - it's looking at lseek() data. Not sure if you meant to compare vs. GetBlockInfo which uses fstat or lseek. The download function is a bit different, but IIRC from the bz is your actual consumer. > > Assuming the VMs are down, this is the code I intend to use for all > formats: > > vm = con.lookupByName(options.vmname) > info = vm.blockInfo(src) > physical = info[2] > > off = 0 > size = 0 > with open(dest, 'w+') as f: > while off < physical: > if (physical - off) < bufsize: > size = physical - off > else: > size = bufsize > buf = vm.blockPeek(src, off, size) > f.write(buf) > off += size > > That would seem to work with the caveat about the <disk type='block' device='disk'> with "<source dev='/dev/sdX'/> where the /dev/sdX (in my case) in an iSCSI target that was creating using "qemu-img create -t qcow2 file.qcow2 1G" and the resulting lseek() is not 1G, but something quite a bit smaller. For that type of <disk> device, the "capacity" returns 1G. I assume you have a test bed of various formats so you can ensure you get the "expected" results? John