[libvirt-users] Libvirt pool cannot see or create rbd clones
Hello everyone, To increase my odds of finding an answer I also wanted to ask here. This is my post from serverfault[1] in verbatim: While trying to get a cloned disk running from my OS snapshot I run into the problem that Libvirt cannot see existing images cloned from a snapshot. Created via: # rbd -p vmdisks clone vmdisks/coreos_2023@base vmdisks/coreos00.disk The base image has the one snapshot 'base' and is protected. The cloned disk is created just fine: # rbd -p vmdisks info coreos00.disk rbd image 'coreos00.disk': size 8.49GiB in 2174 objects order 22 (4MiB objects) block_name_prefix: rbd_data.48a99c6b8b4567 format: 2 features: layering flags: create_timestamp: Thu Apr 25 14:46:52 2019 parent: vmdisks/coreos_2023@base overlap: 8.49GiB I temporarily have Libvirt configured with a rbd pool that uses the ceph admin user. But I cannot see the cloned disk. Just the parent: virsh # vol-list --pool rbd_image_root Name Path --------------------------------------------------------------------- --------- coreos_2023 vmdisks/coreos_2023 If I try to create the cloned image from within virsh I run into the following issue: virsh # vol-clone --pool rbd_image_root coreos_2023 coreos00.disk error: Failed to clone vol from coreos_2023 error: failed to iterate RBD snapshot coreos_2023@base: Operation not permitted Note that this pool uses the Ceph admin user which makes the Operation not permitted a tad odd. Am I missing a configuration option here that would allow for the pool to use clones? I can't find any information on this in the documentation so far. And the source code of libvirt looks like it should support both features. Versions: Libvirt Machine: Ubuntu 18.04 Compiled against library: libvirt 4.0.0 Using library: libvirt 4.0.0 Using API: QEMU 4.0.0 Running hypervisor: QEMU 2.11.1 Ceph Machine: openSUSE Leap 42.3 Ceph 12.2.5 [1] https://serverfault.com/questions/964586/libvirt-pool-cannot-see-or-create-r...
Some additional information that I have found since yesterday: When I create a volume like this: <volume> <name>coreos00.disk</name> <capacity unit="bytes">9116319744</capacity> <target> <format type="raw"></format> <permissions> <mode>644</mode> </permissions> </target> <backingStore> <path>vmdisks/coreos_2023</path> <format type="raw"></format> </backingStore> </volume> Which does not seem to work with the backing store either as it creates an empty volume without the underlying parent. But when I now delete the volume via rbd and create a new clone it shows up in libvirt and can be used by the machine. # rbd -p vmdisks rm coreos00.disk # rbd clone vmdisks/coreos_2023@basis vmdisks/coreos00.disk Sadly this is not a very efficient way to create the clones I would say. Anyone got any, like literally any, ideas what might be wrong in this setup?
On 4/26/19 11:53 AM, Weller, Lennart wrote:
Hello everyone,
To increase my odds of finding an answer I also wanted to ask here. This is my post from serverfault[1] in verbatim:
While trying to get a cloned disk running from my OS snapshot I run into the problem that Libvirt cannot see existing images cloned from a snapshot. Created via:
# rbd -p vmdisks clone vmdisks/coreos_2023@base vmdisks/coreos00.disk
The base image has the one snapshot 'base' and is protected. The cloned disk is created just fine:
# rbd -p vmdisks info coreos00.disk rbd image 'coreos00.disk': size 8.49GiB in 2174 objects order 22 (4MiB objects) block_name_prefix: rbd_data.48a99c6b8b4567 format: 2 features: layering flags: create_timestamp: Thu Apr 25 14:46:52 2019 parent: vmdisks/coreos_2023@base overlap: 8.49GiB
I temporarily have Libvirt configured with a rbd pool that uses the ceph admin user. But I cannot see the cloned disk. Just the parent:
virsh # vol-list --pool rbd_image_root Name Path --------------------------------------------------------------------- --------- coreos_2023 vmdisks/coreos_2023
If I try to create the cloned image from within virsh I run into the following issue:
virsh # vol-clone --pool rbd_image_root coreos_2023 coreos00.disk error: Failed to clone vol from coreos_2023 error: failed to iterate RBD snapshot coreos_2023@base: Operation not permitted
Note that this pool uses the Ceph admin user which makes the Operation not permitted a tad odd.
Am I missing a configuration option here that would allow for the pool to use clones? I can't find any information on this in the documentation so far. And the source code of libvirt looks like it should support both features.
Versions:
Libvirt Machine: Ubuntu 18.04 Compiled against library: libvirt 4.0.0 Using library: libvirt 4.0.0 Using API: QEMU 4.0.0 Running hypervisor: QEMU 2.11.1
Ceph Machine: openSUSE Leap 42.3 Ceph 12.2.5
I wonder if this is a libvirt bug. Looks like it. The error comes from rbd_diff_iterate() (or rbd_diff_iterate2 if compiled against newer librbd). Anyway, a callback is passed to the itreate() function and from librbd code it looks like if the callback returns a negative value then an error is signalized from the iterate() function. Well, we're passing a callback that does nothing but return an error. This might be a possible fix then: diff --git i/src/storage/storage_backend_rbd.c w/src/storage/storage_backend_rbd.c index f8c968e682..08f8123678 100644 --- i/src/storage/storage_backend_rbd.c +++ w/src/storage/storage_backend_rbd.c @@ -1038,7 +1038,7 @@ virStorageBackendRBDIterateCb(uint64_t offset ATTRIBUTE_UNUSED, * searching any further. */ *(int*) arg = 1; - return -1; + return 0; } static int Can you please give it a try? Thanks, Michal
participants (2)
-
Michal Privoznik -
Weller, Lennart