11:01 p.m.
Hi There,
There is a case I met about the destination file's disk size after do
blockcopy
Details as below:
Env:
# rpm -q libvirt qemu-kvm-rhev
libvirt-1.1.1-29.el7.x86_64
qemu-kvm-rhev-1.5.3-60.el7ev_0.2.x86_64
1. Check source file
# qemu-img info /var/lib/libvirt/images/rhel6.img
image: /var/lib/libvirt/images/rhel6.img
file format: qcow2
virtual size: 5.0G (5368709120 bytes)
disk size: 1.2G
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
# ll -lash /var/lib/libvirt/images/rhel6.img
1.2G -rw-r--r--. 1 qemu qemu 1.2G Jul 22 11:04
/var/lib/libvirt/images/rhel6.img
Here: Disk Size is "1.2G"
2. Do blockcopy
# virsh domblklist rhel6
Target Source
------------------------------------------------
vda /var/lib/libvirt/images/rhel6.img
# time virsh blockcopy rhel6 vda /var/lib/libvirt/images/copy.img --wait
--verbose
Block Copy: [100 %]
Now in mirroring phase
real 0m21.285s
user 0m0.038s
sys 0m0.018s
3. Check destination file's disk size
# qemu-img info /var/lib/libvirt/images/copy.img
image: /var/lib/libvirt/images/copy.img
file format: qcow2
virtual size: 5.0G (5368709120 bytes)
disk size: 2.0G
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
# ll -lash /var/lib/libvirt/images/copy.img
2.1G -rw-------. 1 qemu qemu 1.2G Jul 22 11:10
/var/lib/libvirt/images/copy.img
Here: Disk Size is "2.0G"
So my question is why the "disk size" is different between source file
and destination file ? (Actually, the destination always change after I
try many times, sometime it is same size as source, sometimes it is
lager than source.)
4. Do --pivot
After --pivot, the disk size is same size as after blockcopy
--
Regards
shyu
7:52 a.m.
On 07/21/2014 10:01 PM, shyu wrote:
# rpm -q libvirt qemu-kvm-rhev
libvirt-1.1.1-29.el7.x86_64
qemu-kvm-rhev-1.5.3-60.el7ev_0.2.x86_64
These are downstream builds. Can you reproduce your situation with
upstream libvirt 1.2.6 and qemu 2.1-rc2? It may be that you are hitting
behavior that was introduced by downstream backports.
1. Check source file
# qemu-img info /var/lib/libvirt/images/rhel6.img
image: /var/lib/libvirt/images/rhel6.img
file format: qcow2
virtual size: 5.0G (5368709120 bytes)
disk size: 1.2G
Disk size tracks how much of the qcow2 file has been allocated, NOT how
much guest data has been allocated.
3. Check destination file's disk size
# qemu-img info /var/lib/libvirt/images/copy.img
image: /var/lib/libvirt/images/copy.img
file format: qcow2
virtual size: 5.0G (5368709120 bytes)
disk size: 2.0G
The thing to remember here is that blockcopy defaults to doing a cluster
at a time, even if the guest has not yet touched every sector within the
cluster. It may be that you are hitting cases where the copy operation
ends up writing an entire cluster in the destination where only a
partial cluster had been allocated in the source. But that does not
necessarily mean the copy is flawed, only that the default granularity
was large enough to inflate the destination with redundant all-zero
sectors in the interest of speeding up the operation, or that the
destination is not as sparse as the source. Qemu offers the
'granularity' parameter to the 'drive-mirror' command to alter the
granularity, but libvirt is not (currently) exposing this knob to the
user so for now libvirt is just relying on qemu defaults.
It may also be a factor of how much copy-on-write dirtying is happening.
If the guest is actively hammering on the disk during the copy
operation, the same cluster may be marked dirty multiple times; if qemu
allocates a new destination cluster for each pass through the dirty
bitmap, it may result in some inflation in size due to clusters that are
written early then discarded as they are later rewritten in a new
allocation. I'm not familiar enough with qemu block handling to know if
this is happening, or even whether qemu could be patched to do better
garbage collection of clusters left unused if it is happening.
There is nothing that libvirt can do about this. I don't think it is a
bug, but you may want to ask on the qemu list, since it is up to qemu
whether or not the copy will be inflated in host size. But inflation is
not a bad thing in itself - the real question is whether the copy
contains the same guest contents as the original at the time the copy
completed - as long as that is the case, even if the host sizes are
different, then the copy is reliable.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
9:28 p.m.
Hi Eric,
On 07/22/2014 08:52 PM, Eric Blake wrote:
On 07/21/2014 10:01 PM, shyu wrote:
> # rpm -q libvirt qemu-kvm-rhev
> libvirt-1.1.1-29.el7.x86_64
> qemu-kvm-rhev-1.5.3-60.el7ev_0.2.x86_64
These are downstream builds. Can you reproduce your situation with
upstream libvirt 1.2.6 and qemu 2.1-rc2? It may be that you are hitting
behavior that was introduced by downstream backports.
I tried with
qemu-kvm-2.1.0-0.5.rc3.fc20.x86_64 and libvirt built from
latest libvirt.git, I am able to reproduce that.
> 1. Check source file
> # qemu-img info /var/lib/libvirt/images/rhel6.img
> image: /var/lib/libvirt/images/rhel6.img
> file format: qcow2
> virtual size: 5.0G (5368709120 bytes)
> disk size: 1.2G
Disk size tracks how much of the qcow2 file has been allocated, NOT how
much guest data has been allocated.
> 3. Check destination file's disk size
>
> # qemu-img info /var/lib/libvirt/images/copy.img
> image: /var/lib/libvirt/images/copy.img
> file format: qcow2
> virtual size: 5.0G (5368709120 bytes)
> disk size: 2.0G
Thanks very much for your explanation
The thing to remember here is that blockcopy defaults to doing a
cluster
at a time, even if the guest has not yet touched every sector within the
cluster. It may be that you are hitting cases where the copy operation
ends up writing an entire cluster in the destination where only a
partial cluster had been allocated in the source. But that does not
necessarily mean the copy is flawed, only that the default granularity
was large enough to inflate the destination with redundant all-zero
sectors in the interest of speeding up the operation, or that the
destination is not as sparse as the source. Qemu offers the
'granularity' parameter to the 'drive-mirror' command to alter the
granularity, but libvirt is not (currently) exposing this knob to the
user so for now libvirt is just relying on qemu defaults.
It may also be a factor of how much copy-on-write dirtying is happening.
If the guest is actively hammering on the disk during the copy
operation, the same cluster may be marked dirty multiple times; if qemu
allocates a new destination cluster for each pass through the dirty
bitmap, it may result in some inflation in size due to clusters that are
written early then discarded as they are later rewritten in a new
allocation. I'm not familiar enough with qemu block handling to know if
this is happening, or even whether qemu could be patched to do better
garbage collection of clusters left unused if it is happening.
There is nothing that libvirt can do about this. I don't think it is a
bug, but you may want to ask on the qemu list, since it is up to qemu
whether or not the copy will be inflated in host size. But inflation is
not a bad thing in itself - the real question is whether the copy
contains the same guest contents as the original at the time the copy
completed - as long as that is the case, even if the host sizes are
different, then the copy is reliable.
--
Regards
shyu
9:33 p.m.
On 07/27/2014 10:28 AM, shyu wrote:
Hi Eric,
On 07/22/2014 08:52 PM, Eric Blake wrote:
> On 07/21/2014 10:01 PM, shyu wrote:
>
>> # rpm -q libvirt qemu-kvm-rhev
>> libvirt-1.1.1-29.el7.x86_64
>> qemu-kvm-rhev-1.5.3-60.el7ev_0.2.x86_64
> These are downstream builds. Can you reproduce your situation with
> upstream libvirt 1.2.6 and qemu 2.1-rc2? It may be that you are hitting
> behavior that was introduced by downstream backports.
I tried with qemu-kvm-2.1.0-0.5.rc3.fc20.x86_64 and libvirt built from
latest libvirt.git, I am able to reproduce that.
Oops, I am unable to reproduce that. :)
>> 1. Check source file
>> # qemu-img info /var/lib/libvirt/images/rhel6.img
>> image: /var/lib/libvirt/images/rhel6.img
>> file format: qcow2
>> virtual size: 5.0G (5368709120 bytes)
>> disk size: 1.2G
> Disk size tracks how much of the qcow2 file has been allocated, NOT how
> much guest data has been allocated.
>
>> 3. Check destination file's disk size
>>
>> # qemu-img info /var/lib/libvirt/images/copy.img
>> image: /var/lib/libvirt/images/copy.img
>> file format: qcow2
>> virtual size: 5.0G (5368709120 bytes)
>> disk size: 2.0G
Thanks very much for your explanation
> The thing to remember here is that blockcopy defaults to doing a cluster
> at a time, even if the guest has not yet touched every sector within the
> cluster. It may be that you are hitting cases where the copy operation
> ends up writing an entire cluster in the destination where only a
> partial cluster had been allocated in the source. But that does not
> necessarily mean the copy is flawed, only that the default granularity
> was large enough to inflate the destination with redundant all-zero
> sectors in the interest of speeding up the operation, or that the
> destination is not as sparse as the source. Qemu offers the
> 'granularity' parameter to the 'drive-mirror' command to alter the
> granularity, but libvirt is not (currently) exposing this knob to the
> user so for now libvirt is just relying on qemu defaults.
>
> It may also be a factor of how much copy-on-write dirtying is happening.
> If the guest is actively hammering on the disk during the copy
> operation, the same cluster may be marked dirty multiple times; if qemu
> allocates a new destination cluster for each pass through the dirty
> bitmap, it may result in some inflation in size due to clusters that are
> written early then discarded as they are later rewritten in a new
> allocation. I'm not familiar enough with qemu block handling to know if
> this is happening, or even whether qemu could be patched to do better
> garbage collection of clusters left unused if it is happening.
>
> There is nothing that libvirt can do about this. I don't think it is a
> bug, but you may want to ask on the qemu list, since it is up to qemu
> whether or not the copy will be inflated in host size. But inflation is
> not a bad thing in itself - the real question is whether the copy
> contains the same guest contents as the original at the time the copy
> completed - as long as that is the case, even if the host sizes are
> different, then the copy is reliable.
>
--
Regards
shyu
3807
days inactive
3812
days old
3 comments
2 participants
participants (2)
-
Eric Blake -
shyu