On Mon, Apr 30, 2018 at 08:42:56PM +0200, Gionatan Danti wrote: qcow2 is widely used in production at large scale in general. Just not with internal snapshots - almost everything uses external snapshots, aka backing file chains. The QEMU community still tends to discourage use of internal snapshots. There are not even any QMP monitor commands to use them - you are forced to use the legacy HMP interface to QEMU for mgmt. All of the workaround providing interesting block storage mgmt is focused on external snapshots (aka the backing_file option). There are some technical downsides to internal snapshots IIUC, such as inability to free the space used by the internal snapshot when it is deleted, loading/saving snapshots blocks execution of the guest OS, and probably more I've forgotten about. The only nice thing about internal snapshots is simplicity of mgmt, and that is a very nice thing indeed, which is why virt-manager has code to support that - it was much easier to add that code for external snapshots. Just a shame about all the downsides :-( Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On 05/01/2018 03:11 PM, Gionatan Danti wrote: That's fairly accurate. Also, mixing internal and external snapshots at the same time is likely to trigger some known data-loss problems, if you are not extremely careful, so for now, just pick one or the other and stick to it (I hope to someday enhance qemu to refuse operations that would risk data loss, or to perform a slower version of the operation with the same result instead of its current fast but wrong implementations, when dealing with mixed-internal/external backing chains). https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg00865.html -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org

On 01/05/2018 22:49, Gionatan Danti wrote: Any hope to get virsh external snapshot creation/deletion fixed? Or, even better, to have virt-manager support for external snapshots? Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti(a)assyoma.it - info(a)assyoma.it GPG public key ID: FF5F32A8

On 04/05/2018 10:26, Kashyap Chamarthy wrote: True. Reading old mailing list posts, I was under impression that some work was put to automatically and trasparently issue the best blockpull/blockcommit command. If not - fair enough ;) Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti(a)assyoma.it - info(a)assyoma.it GPG public key ID: FF5F32A8

On 05/01/2018 04:17 PM, Paul O'Rorke wrote: This is not an insurmountable difficulty, just one that no one has spent time coding up. If you have no internal snapshots, you can do a 'qemu-img convert' to copy just the portion of the image that is actively in use; the copy will use less disk space than the original because it got rid of the now-unused space. 'virt-sparsify' from libguestfs takes this one step further, by also removing unused space within the guest filesystem itself. In fact, even if you do have internal snapshots, there is probably a sequence of 'qemu-img convert' invocations that can ultimately convert all of your internal snapshots into an external chain of snapshots; but I don't have a ready formula off-hand to point to (experiment on an image you don't care about, before doing it on your production image). In a managed environment, external snapshots probably have the most support for creating and later merging portions of a chain of snapshots, although we could still improve libvirt to make this feel like more of a first class citizen. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org

Thanks for the suggestions Eric, great stuff and most appreciated. I must play in my sandbox... *Paul O'Rorke* *Tracker Software Products (Canada) Limited * www.tracker-software.com <http://www.tracker-software.com/> Tel: +1 (250) 324 1621 Fax: +1 (250) 324 1623 <http://www.tracker-software.com/> Support: http://www.tracker-software.com/support Download latest Releases http://www.tracker-software.com/downloads/ On 2018-05-01 02:31 PM, Eric Blake wrote:

On 05/22/2018 01:53 PM, Paul O'Rorke wrote: To make sure I understand, /dev/trk-kvm-02-vg/rt44 was using qcow2 on top of a block device, both before and after you used qemu-img convert to compact out the wasted space? Also, do you know if you were using qcow2v2 or v3 prior to running out of space? qcow2v2 was the default in CentOS 6 for historical reasons, but qemu doesn't support efficient space reclamation on those older images the way it does on qcow2v3. Did you save the full image anywhere, so we can do additional postmortem analysis on it? Not that I'm asking to see the image itself, in case that would give away private information; but even something like running 'qcheck' from https://github.com/jnsnow/qcheck.git might shed some insight into the storage being used by the image that reached capacity. The highest write offset (block.<num>.allocation in virConnectGetAllDomainStats()) should tell you how much of the underlying block device is "in use", that is, the highest offset that is being written to. It doesn't count for holes earlier in the image, especially where space could be reclaimed by reusing those holes, but does give a good indication when you might need to resize a storage volume to account for growing amounts of metadata that aren't being cleared. And it may be that qemu still needs some patches to more fully trim and reuse the space previously occupied by a deleted internal snapshot. qcow2 is a mapped file format. Resizing the LVM volume does NOT change the amount of disk space seen by the guest. Clusters may appear in a different order in the host's raw storage than the order in which they are visited in the guest ('qemu-img map' can show you the mapping), but the guest does not care because it is always presented a logical view of a linear array of storage, regardless of how clusters are fragmented across the host device. Right now, qcow2 does not have any way to constrain the clusters used by guest data to appear at any particular offset (although there has been talk of adding a new mode to qemu to operate on a fully-preallocated image where all guest clusters except maybe the first occur in linear order at the beginning of the file, and all qcow2 metadata except for the leading header be placed at the end of the file at offsets higher than the guest clusters - in such a layout, as long as you don't use internal snapshots, then all further qcow2 metadata writes would be beyond the linear region reserved for guest-visible clusters). virt-resize from libguestfs-tools is your friend! -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org

On Tue, May 1, 2018 at 10:45 AM, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: BTW: in virt-manager of CentOS 7.4 by default qcow2 disks (see below for zzz3.qcow2) are created with compatibility 1.1 (probably indirectly because of its lazy refcounts feature enabled, as Daniel wrote): [root@c7client images]# qemu-img info zzz3.qcow2 image: zzz3.qcow2 file format: qcow2 virtual size: 1.0G (1073741824 bytes) disk size: 516K cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: true [root@c7client images]# The same if I create a volume with a backing file from virt-manager (not important if this is 0.10 or 1.1): [root@c7client images]# qemu-img info zzz4.qcow2 image: zzz4.qcow2 file format: qcow2 virtual size: 1.0G (1073741824 bytes) disk size: 196K cluster_size: 65536 backing file: /var/lib/libvirt/images/zzz2.qcow2 backing file format: qcow2 Format specific information: compat: 1.1 lazy refcounts: true [root@c7client images]# But I don't know if internally virt-manager uses virsh or qemu-img actually.... My tests with: libvirt-client-3.2.0-14.el7_4.9.x86_64 virt-manager-1.4.1-7.el7.noarch The same default to 0.10 format seems true with virsh from Fedora 27 and libvirt-client-3.7.0-4.fc27.x86_64 Gianluca