Hi everybody! I'm using a snapshot purely for writes and was wondering if it's possible to clear/empty the snapshot after it reaches a certain size? Created with: virsh snapshot-create-as --disk-only I tried deleting and recreating it but get an error at deletion: virsh snapshot-delete Error: source for disk 'vda' is not a regular file I also tried virsh snapshot-create-as --no-metadata --disk-only But the same error as above pops up when trying to delete. Any ideas?
On Mon, Nov 29, 2021 at 13:05:40 +0100, Elias Mobery wrote:
Hi everybody!
I'm using a snapshot purely for writes and was wondering if it's possible to clear/empty the snapshot after it reaches a certain size?
Created with: virsh snapshot-create-as --disk-only
So this creates a so-called external snapshot, which for disks means an overlay image.
I tried deleting and recreating it but get an error at deletion:
virsh snapshot-delete
but unfortunately deleting external snapshots is not yet implemented. The usual approach is to do a block commit operation to merge the overlay images back into the backing image manually. The basic syntax via virsh is: virsh blockcommit --active --pivot --verbose $VM $DISK This merges any overlay images into the backing image. Backing image is the most deepest nested image when you look at <backingStore> in the XML.
Error: source for disk 'vda' is not a regular file
This error doesn't make much sense though. The error should state that deletion of external snapshots is nto supported.
I also tried virsh snapshot-create-as --no-metadata --disk-only
This creates a snapshot without libvirt metadata. This means that libvirt just creates the overlay, but doesn't create any internal state related to it. It saves you the step of deleting metadata, but you still need to manually merge the image.
But the same error as above pops up when trying to delete.
This makes absolutely no sense though. Are you sure you are copying the correct error message? 'virsh snapshot-delete' works _only_ with snapshots which do have metadata, so if you create a snapshot without there's nothing to delete.
Hello Peter, thank you so much for that detailed info! Sorry, you were right, when trying to delete my external snapshot via snapshot-delete, the error says "deletion of external snapshots unsupported" I can't merge the snapshot because the VM image is in a read-only filesystem. Sorry I should've said, it's a live system. So the image is in the read-only squashfs and external snapshot in overlay is used for writing. Now I would like the snapshot emptied or deleted/recreated when it reaches 4GB. Is there even a way to do this with the image being read-only? Thanks again for your advice, I really love this community. On Tue, Nov 30, 2021, 9:42 AM Peter Krempa <pkrempa@redhat.com> wrote:
On Mon, Nov 29, 2021 at 13:05:40 +0100, Elias Mobery wrote:
Hi everybody!
I'm using a snapshot purely for writes and was wondering if it's possible to clear/empty the snapshot after it reaches a certain size?
Created with: virsh snapshot-create-as --disk-only
So this creates a so-called external snapshot, which for disks means an overlay image.
I tried deleting and recreating it but get an error at deletion:
virsh snapshot-delete
but unfortunately deleting external snapshots is not yet implemented.
The usual approach is to do a block commit operation to merge the overlay images back into the backing image manually.
The basic syntax via virsh is:
virsh blockcommit --active --pivot --verbose $VM $DISK
This merges any overlay images into the backing image. Backing image is the most deepest nested image when you look at <backingStore> in the XML.
Error: source for disk 'vda' is not a regular file
This error doesn't make much sense though. The error should state that deletion of external snapshots is nto supported.
I also tried virsh snapshot-create-as --no-metadata --disk-only
This creates a snapshot without libvirt metadata. This means that libvirt just creates the overlay, but doesn't create any internal state related to it.
It saves you the step of deleting metadata, but you still need to manually merge the image.
But the same error as above pops up when trying to delete.
This makes absolutely no sense though. Are you sure you are copying the correct error message? 'virsh snapshot-delete' works _only_ with snapshots which do have metadata, so if you create a snapshot without there's nothing to delete.
On Tue, Nov 30, 2021 at 14:51:54 +0100, Elias Mobery wrote:
Hello Peter, thank you so much for that detailed info!
Sorry, you were right, when trying to delete my external snapshot via snapshot-delete, the error says "deletion of external snapshots unsupported"
I can't merge the snapshot because the VM image is in a read-only filesystem. Sorry I should've said, it's a live system. So the image is in the read-only squashfs and external snapshot in overlay is used for writing.
Okay, that changes the situation quite a bit:
Now I would like the snapshot emptied or deleted/recreated when it reaches 4GB.
Is there even a way to do this with the image being read-only?
If the base image is on a read-only filesystem you obviously can't commit to it. Now the question is what should happen to the data in the overlay. Discarding/recreating the overlay image is possible only once you turn off the VM because it basically rolls back the state of the disk back to the time when the overlay was created. This means that everything written to the disk will be lost. Filesystems obviously can't handle that so that's why it simply won't be possible to do live with the root image. You can have a second disk, which you hot-unplug, wipe the overlay and plug it back. Another possibility is to enable trim/discard and just simply delete the data in the VM which was added after the overlay was created. When trim/discard is enabled on all layers incluging the guest filesystem, then deleting stuff inside the VM will also mean that the overlay will shrink again.
Hey Peter, thanks for the super quick reply, you're awesome! So you mean the first option would be to create a second disk instead of using snapshot for write, when it gets too big I can then unplug the disk, delete it (or wipe entire overlay fs) and recreate it to start fresh? OK I will try that, thanks for the tip! Option 2 with trim/discard I have to do some research, this seems a bit more challenging. Thank you SO much for the great advice. I will try everything out and report back. :-) On Tue, Nov 30, 2021, 3:25 PM Peter Krempa <pkrempa@redhat.com> wrote:
On Tue, Nov 30, 2021 at 14:51:54 +0100, Elias Mobery wrote:
Hello Peter, thank you so much for that detailed info!
Sorry, you were right, when trying to delete my external snapshot via snapshot-delete, the error says "deletion of external snapshots unsupported"
I can't merge the snapshot because the VM image is in a read-only filesystem. Sorry I should've said, it's a live system. So the image is in the read-only squashfs and external snapshot in overlay is used for writing.
Okay, that changes the situation quite a bit:
Now I would like the snapshot emptied or deleted/recreated when it
reaches
4GB.
Is there even a way to do this with the image being read-only?
If the base image is on a read-only filesystem you obviously can't commit to it.
Now the question is what should happen to the data in the overlay.
Discarding/recreating the overlay image is possible only once you turn off the VM because it basically rolls back the state of the disk back to the time when the overlay was created. This means that everything written to the disk will be lost.
Filesystems obviously can't handle that so that's why it simply won't be possible to do live with the root image.
You can have a second disk, which you hot-unplug, wipe the overlay and plug it back.
Another possibility is to enable trim/discard and just simply delete the data in the VM which was added after the overlay was created. When trim/discard is enabled on all layers incluging the guest filesystem, then deleting stuff inside the VM will also mean that the overlay will shrink again.
Hey man, I tried both suggestions, thanks again. Option 1.) Create/attach a second qcow2 disk & unplug when wiping. This did not work unfortunately. After wiping the overlay filesystem on the host (where the snapshots are sitting), I cannot create another snapshot, a similar error message like before appears. Option 2.) Enabling Discard Now I think this worked, but I'm not sure. I read all the docs, and enabled discard like so on the VM domain: <driver name='qemu' type='qcow2' discard='unmap'> I tried both discard=unmap and discard=on (same thing) Then I enabled fstrim in the guest like this: (Not entirely sure if I am missing smth) sudo fstrim -av Or permanently sudo systemctl enable --now fstrim.timer On Tue, Nov 30, 2021, 4:08 PM Elias Mobery <eliasmobery@gmail.com> wrote:
Hey Peter, thanks for the super quick reply, you're awesome!
So you mean the first option would be to create a second disk instead of using snapshot for write, when it gets too big I can then unplug the disk, delete it (or wipe entire overlay fs) and recreate it to start fresh?
OK I will try that, thanks for the tip!
Option 2 with trim/discard I have to do some research, this seems a bit more challenging.
Thank you SO much for the great advice. I will try everything out and report back. :-)
On Tue, Nov 30, 2021, 3:25 PM Peter Krempa <pkrempa@redhat.com> wrote:
On Tue, Nov 30, 2021 at 14:51:54 +0100, Elias Mobery wrote:
Hello Peter, thank you so much for that detailed info!
Sorry, you were right, when trying to delete my external snapshot via snapshot-delete, the error says "deletion of external snapshots unsupported"
I can't merge the snapshot because the VM image is in a read-only filesystem. Sorry I should've said, it's a live system. So the image is in the read-only squashfs and external snapshot in overlay is used for writing.
Okay, that changes the situation quite a bit:
Now I would like the snapshot emptied or deleted/recreated when it
reaches
4GB.
Is there even a way to do this with the image being read-only?
If the base image is on a read-only filesystem you obviously can't commit to it.
Now the question is what should happen to the data in the overlay.
Discarding/recreating the overlay image is possible only once you turn off the VM because it basically rolls back the state of the disk back to the time when the overlay was created. This means that everything written to the disk will be lost.
Filesystems obviously can't handle that so that's why it simply won't be possible to do live with the root image.
You can have a second disk, which you hot-unplug, wipe the overlay and plug it back.
Another possibility is to enable trim/discard and just simply delete the data in the VM which was added after the overlay was created. When trim/discard is enabled on all layers incluging the guest filesystem, then deleting stuff inside the VM will also mean that the overlay will shrink again.
On Wed, Dec 01, 2021 at 18:35:44 +0100, Elias Mobery wrote:
Hey man, I tried both suggestions, thanks again.
Option 1.) Create/attach a second qcow2 disk & unplug when wiping. This did not work unfortunately. After wiping the overlay filesystem on the host (where the snapshots are sitting), I cannot create another snapshot, a similar error message like before appears.
Okay, that's weird. Could you please post your exact steps? Additionally if you have a new enough libvirt you can actually possibly even try using a disk with <transient/>. This transparently creates a overlay (in the same path as the image though, not sure if it suits your setup) and automatically discards it when unplugging or shutting down the VM. In such case you could simply unplug and replug the disk, but note that this feature was implemented only this year IIRC, so you need a fairly recent libvirt to do so.
Option 2.) Enabling Discard
Now I think this worked, but I'm not sure. I read all the docs, and enabled discard like so on the VM domain:
<driver name='qemu' type='qcow2' discard='unmap'>
This is the correct setting. Unmap means that the qcow2 image should unmap the blocks which were discarded.
I tried both discard=unmap and discard=on (same thing)
Then I enabled fstrim in the guest like this: (Not entirely sure if I am missing smth)
sudo fstrim -av
Or permanently
sudo systemctl enable --now fstrim.timer
Yup, both should do fine. Now if you delete something in the VM (which would result in freeing blocks in the overlay; deleting something from the base image won't help obviously) the overlay image should shrink.
OK, so I tried both ways. (The VM image is in the read-only squashfs.) 1.) Editing VM domain XML: <disk> <transient/> </disk> Error: Internal Error - Could not open /var/lib/libvirt/images/vm.snapshot1 : Permission Denied I chowned everything to libvirt but no changes. Maybe a read-only conflict? 2.) Adding second disk to VM domain XML (shortened ) <disk> source /var/lib..../images/vm.qcow2 target dev=vda </disk> <disk> source /var/lib...../images/vm.qcow2 target dev=vdb <transient/> </disk> Error: unsupported configuration: cannot create external snapshot for disk vdb - collision with disk vdb. After googling I found that a lot of people get permission denied errors using transient. I will keep looking, really not sure. On Thu, Dec 2, 2021, 9:30 AM Peter Krempa <pkrempa@redhat.com> wrote:
Hey man, I tried both suggestions, thanks again.
Option 1.) Create/attach a second qcow2 disk & unplug when wiping. This did not work unfortunately. After wiping the overlay filesystem on
On Wed, Dec 01, 2021 at 18:35:44 +0100, Elias Mobery wrote: the
host (where the snapshots are sitting), I cannot create another snapshot, a similar error message like before appears.
Okay, that's weird. Could you please post your exact steps?
Additionally if you have a new enough libvirt you can actually possibly even try using a disk with <transient/>. This transparently creates a overlay (in the same path as the image though, not sure if it suits your setup) and automatically discards it when unplugging or shutting down the VM.
In such case you could simply unplug and replug the disk, but note that this feature was implemented only this year IIRC, so you need a fairly recent libvirt to do so.
Option 2.) Enabling Discard
Now I think this worked, but I'm not sure. I read all the docs, and
enabled
discard like so on the VM domain:
<driver name='qemu' type='qcow2' discard='unmap'>
This is the correct setting. Unmap means that the qcow2 image should unmap the blocks which were discarded.
I tried both discard=unmap and discard=on (same thing)
Then I enabled fstrim in the guest like this: (Not entirely sure if I am missing smth)
sudo fstrim -av
Or permanently
sudo systemctl enable --now fstrim.timer
Yup, both should do fine. Now if you delete something in the VM (which would result in freeing blocks in the overlay; deleting something from the base image won't help obviously) the overlay image should shrink.
On Thu, Dec 02, 2021 at 16:27:00 +0100, Elias Mobery wrote:
OK, so I tried both ways. (The VM image is in the read-only squashfs.)
1.) Editing VM domain XML: <disk>
<transient/> </disk>
Error: Internal Error - Could not open /var/lib/libvirt/images/vm.snapshot1 : Permission Denied
I chowned everything to libvirt but no changes. Maybe a read-only conflict?
Yes, the overlay is created in the same path as the original image, just with a suffix, so if that is a read-only FS it will not work. I've thought about adding a possibility to specify the location for the overlay but didn't ever get to implementing it actually.
2.) Adding second disk to VM domain XML (shortened )
<disk> source /var/lib..../images/vm.qcow2 target dev=vda </disk>
<disk> source /var/lib...../images/vm.qcow2 target dev=vdb <transient/> </disk>
Error: unsupported configuration: cannot create external snapshot for disk vdb - collision with disk vdb.
Could you please post the unabbreviated steps? I don't really know what's going on based on this.
After googling I found that a lot of people get permission denied errors using transient. I will keep looking, really not sure.
participants (2)
-
Elias Mobery -
Peter Krempa