10:57 a.m.
Hello,
Thanks for all the information, I think my last remaining question is
why the image created with virt-install shows the full apparent size
and the image from virt-sparsify does not? (The image testimage1.qcow2
created with default options).
As they are both qcow2 I would expect them both to have the same
behavior as far as apparent size goes. Or is there an option I am
missing in virt-sparsify that would keep the full apparent size? Or
perhaps one in virt-install to set apperent and real size the same?
On Thu, 23 Jun 2016 09:39:52 +0200
Andrea Bolognani <abologna(a)redhat.com> wrote:
The '--format' option is to specify the image format for the
input image. If you want the *output* image to be raw, you'll
have to use '--convert raw'.
On Thu, 23 Jun 2016 08:52:35 +0100
"Richard W.M. Jones" <rjones(a)redhat.com> wrote:
This is incorrect usage. You've told virt-sparsify that the
input
image is raw. But it's not raw, it's qcow2.
That makes sense, I won't try and tell it the input is raw anymore
when its not.
Keelan
On Thu, 23 Jun 2016 08:52:35 +0100
"Richard W.M. Jones" <rjones(a)redhat.com> wrote:
> On Wed, Jun 22, 2016 at 05:56:09PM -0500,
> libvirt_users(a)skagitattic.com wrote:
> > > Delete what you've done and start from the beginning. Describe
> > > exactly how you created the guest. Use 'qemu-img info' to show
> > > the format of the input file. Show precisely the virt-sparsify
> > > command you are running. And use 'qemu-img info' on the output
> > > file too.
> >
> > Okay, here goes. Start off with an install
> >
> > root@testingbox:~ # virt-install --name testimage --memory 512
> > -l
> > 'http://ftp.nl.debian.org/debian/dists/jessie/main/installer-amd64/'
> > --disk size=50 --nographics -x "console=ttyS0"
> >
> > Starting install...
> > Retrieving file
> > MANIFEST...
> > | 3.3 kB 00:00 ... Retrieving file
> > linux...
> > | 6.0 MB 00:03 ... Retrieving file
> > initrd.gz...
> > | 29 MB 00:09 ... Allocating
> > 'testimage.qcow2'
> > | 50 GB 00:00 Creating
> > domain...
> > | 0 B 00:00 Connected to domain testimage Escape
> > character is [ 0.000000] Initializing cgroup subsys cpuset
> > [ 0.000000] Initializing cgroup subsys cpu [ 0.000000]
> > Initializing cgroup subsys cpuacct [ 0.000000] Linux version
> > 3.16.0-4-amd64 (debian-kernel(a)lists.debian.org) (gcc version 4.8.4
> > (Debian 4.8.4-1) ) #1 SMP Debian 3.16.7-ckt25-2 (2016-04-08)
> >
> > .... Many more lines here and it turns interactive. Suffice it to
> > say I did a basic minimal install following the prompts. Nothing
> > special here.
> >
> > This created an image that is 2.5G with a 50G apparent size and
> > identifies as qcow2 as shown below.
> >
> > # ls -slh testimage.qcow2
> > 2.5G -rw-r--r-- 1 root root 51G Jun 22 20:22 testimage.qcow2
> >
> > # qemu-img info testimage.qcow2
> > image: testimage.qcow2
> > file format: qcow2
> > virtual size: 50G (53687091200 bytes)
> > disk size: 2.4G
> > cluster_size: 65536
> > Format specific information:
> > compat: 1.1
> > lazy refcounts: true
> >
> > Now we sparsify (/bigtmp is on a larger partition then / to give
> > virt-sparsify enough room to work with so it does not complain).
> >
> > root@testingbox: 08:59 PM # virt-sparsify testimage.qcow2
> > testimage1.qcow2 --tmp /bigtmp Input disk virtual size =
> > 53687091200 bytes (50.0G) Create overlay file in /bigtmp to protect
> > source disk ... Examine source disk ...
> > 100%
> >
⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧
> > --:-- Fill free space in /dev/sda1 with zero ... 100%
> >
⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧
> > 00:00 Clearing Linux swap on /dev/sda5 ... 100%
> >
⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧
> > --:-- Copy to destination and make sparse ...
> >
> > Sparsify operation completed with no errors. Before deleting
> > the old disk, carefully check that the target disk boots and works
> > correctly.
> > root@testingbox: 09:04 PM #
> >
> > This image reports still reports as qcow2 however the apparent and
> > real size is the same.
> >
> > # ls -slh testimage1.qcow2
> > 1.1G -rw-r--r-- 1 root root 1.1G Jun 22 21:04 testimage1.qcow2
> >
> > # qemu-img info testimage1.qcow2
> > image: testimage1.qcow2
> > file format: qcow2
> > virtual size: 50G (53687091200 bytes)
> > disk size: 1.1G
> > cluster_size: 65536
> > Format specific information:
> > compat: 1.1
> > lazy refcounts: false
>
> virt-sparsify saved about 1.3 GB of disk space (2.4G -> 1.1G).
>
> > If we try it again but specify raw its MUCH faster
> >
> > root@testingbox: 09:26 PM # virt-sparsify testimage.qcow2
> > testimage2.qcow2 --tmp /bigtmp --format raw
>
This is incorrect usage. You've told virt-sparsify that the
input
image is raw. But it's not raw, it's qcow2.
>
> In any case, virt-sparsify opened the image as raw, couldn't make any
> sense of it (it appears to virt-sparsify to be random data, not a disk
> image), and so it cannot sparsify it properly. This is "quicker" only
> because virt-sparsify didn't do any sparsification of filesystems,
> because it couldn't see the filesystems.
>
> > This time it takes up more space and reports real and apparent size
> > differently. It still reports as qcow2 with qemu-img.
> >
> > # ls -slh testimage2.qcow2
> > 1.7G -rw-r--r-- 1 root root 51G Jun 22 21:27 testimage2.qcow2
> > # qemu-img info testimage2.qcow2
> > image: testimage2.qcow2
> > file format: qcow2
> > virtual size: 50G (53687091200 bytes)
> > disk size: 1.7G
>
> Looks like virt-sparsify still found some clusters containing all
> zeroes which it could trim, even though it didn't know how to read the
> disk.
>
> Anyway, the moral is, don't misinform virt-sparsify of the format
> of your input image.
>
> If what you really intended to do was to convert the format from qcow2
> to raw, you should use '--convert raw' instead.
>
> > The only things I see different from the finished files is that the
> > raw one uses a bit more space, the apparent size is not set to the
> > virtual size on the default sparsify one and the lazy refcounts is
> > different.
> >
> > Is there something more detailed then "qemu-img info" that I could
> > tell the difference with?
>
> qemu-img info is the correct tool to use.
>
> > > Raw has fewer features, especially no simple snapshotting or
> > > support for backing files.
>
> I wrote this thinking you were talking about actual raw format disks,
> not about qcow2 but where you've misinformed virt-sparsify about the
> format.
>
> > Is simple snapshotting using qemu-img snapshot or something else? I
> > tried making a snapshot with both test images and they both
> > appeared to work.
>
> http://wiki.qemu.org/Documentation/CreateSnapshot
>
> > # qemu-img snapshot -c test testimage1.qcow2
> > # qemu-img snapshot -c test testimage2.qcow2
> > # qemu-img info testimage2.qcow2
> > image: testimage2.qcow2
> > file format: qcow2
> > virtual size: 50G (53687091200 bytes)
> > disk size: 1.7G
> > cluster_size: 65536
> > Snapshot list:
> > ID TAG VM SIZE DATE
> > VM CLOCK 1 test 0 2016-06-22 22:41:52
> > 00:00:00.000 Format specific information:
> > compat: 1.1
> > lazy refcounts: true
>
> This works because testimage2.qcow2 is still in qcow2 format.
>
> Anyway, use the correct --format option.
>
> Rich.
>