On 12/05/2016 05:27 AM, Shahar Havivi wrote:
> Ping
>

Sorry for the delay... Lots of investigation, patches, and other work.

> On Wed, Nov 30, 2016 at 3:13 PM, Shahar Havivi <shavivi@redhat.com
> <mailto:shavivi@redhat.com>> wrote:
>
>     Hi,
>
>     I am using virStorageVolGetInfo to get the volume size
>     virStorageVol.download to download a
>     normal file for VDSM project.

Normal as in a 'raw' fully allocated file? As opposed to a sparse and/or
container (e.g. qcow2) file?

>
>     I want to add support for block devices as well, I was able to see
>     the size
>     via virDomainBlockInfo and download via virDomain.blockPeek which is
>     works
>     fine for both file and block device.

As described in the bz, but for others benefit... While looking into
this I found a commit Eric Blake made:

http://libvirt.org/git/?p=libvirt.git;a=commit;h=0282ca45a0a4c384226d526d4878cacd77936ef4

which more or less summarizes to:

     capacity: logical size in bytes of the image (how much storage
               the guest will see)
     allocation: host storage in bytes occupied by the image (such
                 as highest allocated extent if there are no holes,
                 similar to 'du')
     physical: host physical size in bytes of the image container
               (last offset, similar to 'ls')

The data virDomainGetBlockInfo gets when the domain is not active is the
same data that virStorageVolGetInfo gets (which I know you couldn't use
because it didn't have <physical>).  IOW: In the long run for libvirt,
they use the same API's to get the data.

For a "file" backed <disk> the API's use 'fstat()' while for a "block"
backed <disk> the API's use 'lseek()' to the end.

Unfortunately there were some differences that have crept into
virDomainGetBlockInfo for active domain as a result of changes made to
support virConnectGetAllDomainStats. Fortunately those differences are
mostly related to domain disk devices with sources that are sparse
files, containers (qcow2, qed for example), or block devices (perhaps an
iSCSI /dev/sdX device) that's using some sort of container file.

The virDomainBlockPeek strictly uses the lseek(), which I think should
return the same answer as fstat() - although I could be wrong.

>
>     1. Can I depend on virDomainBlockInfo for non block devices?

You should be able to for an inactive domain. If it doesn't provide you
with the right or expected answer, then investigation could be done.

The API "works" using the <target dev='nnX' bus='$BUS'/> such that
"block" is how the guest is viewing things - e.g. /dev/nnX if nnX were
valid.  Typically for bus='virtio', it's dev='vdX' where X is a, b, c,
etc.  Still it the <source ...> part of the domain XML which will get
the answer for an inactive domain, while for an active domain there's
matching going on from a qemu call to get the block info from the domain.

>     2. Does the virDomainBlockInfo return the correct physical size for
>     normal file?

I hope so...  It's supposed to. Of course what's normal these days? I
would think sparse files are the norm. After all who wants to require
15-20G when they could get away with 5G knowing that up to some 15-20G
could be used.

There's also block devices using a logical backend where we rely on the
logical code to provide the "right answer".

Again - if something isn't providing the right answer, then more details
would be necessary.

>     3. Performance wise do we have a difference between
>     virStorageVol.download and virDomain.blockPeek?
>

Well blockPeek is pretty quick - it's looking at lseek() data. Not sure
if you meant to compare vs. GetBlockInfo which uses fstat or lseek. The
download function is a bit different, but IIRC from the bz is your
actual consumer.

>
>     Assuming the VMs are down, this is the code I intend to use for all
>     formats:
>
>     vm = con.lookupByName(options.vmname)
>     info = vm.blockInfo(src)
>     physical = info[2]
>
>     off = 0
>     size = 0
>     with open(dest, 'w+') as f:
>         while off < physical:
>             if (physical - off) < bufsize:
>                 size = physical - off
>             else:
>                 size = bufsize
>             buf = vm.blockPeek(src, off, size)
>             f.write(buf)
>             off += size
>
>

That would seem to work with the caveat about the <disk type='block'
device='disk'> with "<source dev='/dev/sdX'/> where the /dev/sdX (in my
case) in an iSCSI target that was creating using "qemu-img create -t
qcow2 file.qcow2 1G" and the resulting lseek() is not 1G, but something
quite a bit smaller. For that type of <disk> device, the "capacity"
returns 1G.

I assume you have a test bed of various formats so you can ensure you
get the "expected" results?


John