Hello, Thanks for all the information, I think my last remaining question is why the image created with virt-install shows the full apparent size and the image from virt-sparsify does not? (The image testimage1.qcow2 created with default options). As they are both qcow2 I would expect them both to have the same behavior as far as apparent size goes. Or is there an option I am missing in virt-sparsify that would keep the full apparent size? Or perhaps one in virt-install to set apperent and real size the same? On Thu, 23 Jun 2016 09:39:52 +0200 Andrea Bolognani <abologna@redhat.com> wrote:
The '--format' option is to specify the image format for the input image. If you want the *output* image to be raw, you'll have to use '--convert raw'.
On Thu, 23 Jun 2016 08:52:35 +0100 "Richard W.M. Jones" <rjones@redhat.com> wrote:
This is incorrect usage. You've told virt-sparsify that the input image is raw. But it's not raw, it's qcow2.
That makes sense, I won't try and tell it the input is raw anymore when its not. Keelan On Thu, 23 Jun 2016 08:52:35 +0100 "Richard W.M. Jones" <rjones@redhat.com> wrote:
On Wed, Jun 22, 2016 at 05:56:09PM -0500, libvirt_users@skagitattic.com wrote:
Delete what you've done and start from the beginning. Describe exactly how you created the guest. Use 'qemu-img info' to show the format of the input file. Show precisely the virt-sparsify command you are running. And use 'qemu-img info' on the output file too.
Okay, here goes. Start off with an install
root@testingbox:~ # virt-install --name testimage --memory 512 -l 'http://ftp.nl.debian.org/debian/dists/jessie/main/installer-amd64/' --disk size=50 --nographics -x "console=ttyS0"
Starting install... Retrieving file MANIFEST... | 3.3 kB 00:00 ... Retrieving file linux... | 6.0 MB 00:03 ... Retrieving file initrd.gz... | 29 MB 00:09 ... Allocating 'testimage.qcow2' | 50 GB 00:00 Creating domain... | 0 B 00:00 Connected to domain testimage Escape character is [ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Initializing cgroup subsys cpuacct [ 0.000000] Linux version 3.16.0-4-amd64 (debian-kernel@lists.debian.org) (gcc version 4.8.4 (Debian 4.8.4-1) ) #1 SMP Debian 3.16.7-ckt25-2 (2016-04-08)
.... Many more lines here and it turns interactive. Suffice it to say I did a basic minimal install following the prompts. Nothing special here.
This created an image that is 2.5G with a 50G apparent size and identifies as qcow2 as shown below.
# ls -slh testimage.qcow2 2.5G -rw-r--r-- 1 root root 51G Jun 22 20:22 testimage.qcow2
# qemu-img info testimage.qcow2 image: testimage.qcow2 file format: qcow2 virtual size: 50G (53687091200 bytes) disk size: 2.4G cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: true
Now we sparsify (/bigtmp is on a larger partition then / to give virt-sparsify enough room to work with so it does not complain).
root@testingbox: 08:59 PM # virt-sparsify testimage.qcow2 testimage1.qcow2 --tmp /bigtmp Input disk virtual size = 53687091200 bytes (50.0G) Create overlay file in /bigtmp to protect source disk ... Examine source disk ... 100% ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧ --:-- Fill free space in /dev/sda1 with zero ... 100% ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧ 00:00 Clearing Linux swap on /dev/sda5 ... 100% ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧ --:-- Copy to destination and make sparse ...
Sparsify operation completed with no errors. Before deleting the old disk, carefully check that the target disk boots and works correctly. root@testingbox: 09:04 PM #
This image reports still reports as qcow2 however the apparent and real size is the same.
# ls -slh testimage1.qcow2 1.1G -rw-r--r-- 1 root root 1.1G Jun 22 21:04 testimage1.qcow2
# qemu-img info testimage1.qcow2 image: testimage1.qcow2 file format: qcow2 virtual size: 50G (53687091200 bytes) disk size: 1.1G cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false
virt-sparsify saved about 1.3 GB of disk space (2.4G -> 1.1G).
If we try it again but specify raw its MUCH faster
root@testingbox: 09:26 PM # virt-sparsify testimage.qcow2 testimage2.qcow2 --tmp /bigtmp --format raw
This is incorrect usage. You've told virt-sparsify that the input image is raw. But it's not raw, it's qcow2.
In any case, virt-sparsify opened the image as raw, couldn't make any sense of it (it appears to virt-sparsify to be random data, not a disk image), and so it cannot sparsify it properly. This is "quicker" only because virt-sparsify didn't do any sparsification of filesystems, because it couldn't see the filesystems.
This time it takes up more space and reports real and apparent size differently. It still reports as qcow2 with qemu-img.
# ls -slh testimage2.qcow2 1.7G -rw-r--r-- 1 root root 51G Jun 22 21:27 testimage2.qcow2 # qemu-img info testimage2.qcow2 image: testimage2.qcow2 file format: qcow2 virtual size: 50G (53687091200 bytes) disk size: 1.7G
Looks like virt-sparsify still found some clusters containing all zeroes which it could trim, even though it didn't know how to read the disk.
Anyway, the moral is, don't misinform virt-sparsify of the format of your input image.
If what you really intended to do was to convert the format from qcow2 to raw, you should use '--convert raw' instead.
The only things I see different from the finished files is that the raw one uses a bit more space, the apparent size is not set to the virtual size on the default sparsify one and the lazy refcounts is different.
Is there something more detailed then "qemu-img info" that I could tell the difference with?
qemu-img info is the correct tool to use.
Raw has fewer features, especially no simple snapshotting or support for backing files.
I wrote this thinking you were talking about actual raw format disks, not about qcow2 but where you've misinformed virt-sparsify about the format.
Is simple snapshotting using qemu-img snapshot or something else? I tried making a snapshot with both test images and they both appeared to work.
http://wiki.qemu.org/Documentation/CreateSnapshot
# qemu-img snapshot -c test testimage1.qcow2 # qemu-img snapshot -c test testimage2.qcow2 # qemu-img info testimage2.qcow2 image: testimage2.qcow2 file format: qcow2 virtual size: 50G (53687091200 bytes) disk size: 1.7G cluster_size: 65536 Snapshot list: ID TAG VM SIZE DATE VM CLOCK 1 test 0 2016-06-22 22:41:52 00:00:00.000 Format specific information: compat: 1.1 lazy refcounts: true
This works because testimage2.qcow2 is still in qcow2 format.
Anyway, use the correct --format option.
Rich.