Hi, I wanted to get your advice on a patch I'm preparing for libvirt. It touches the code-path that allows using LUKS encryption on top of an RBD image. We recently added LUKS and LUKS2 encryption support in Ceph's librbd. We exposed this in qemu in a recent patch by adding new optional "encrypt" member to BlockdevOptionsRbd. This patch was included in the recent release of qemu 6.1. To enable libvirt users to use librbd encryption, we need libvirt to use this new "encrypt" when it builds the blockdev options for RBD. The interesting question is how to define the libvirt XML syntax that will trigger the use of librbd encryption. My thought was to use the already existing <encryption> tag. In that case, we just need to add a new format VIR_STORAGE_ENCRYPTION_FORMAT_LUKS2 to the enum virStorageEncryptionFormatType. This type will be checked in qemuBlockStorageSourceGetRBDProps.

The problem with this approach is that it only works for LUKS2. librbd encryption also supports LUKS1. We want to allow the user to choose between the qemu LUKS implementation and the librbd one. One reason to keep support both is that on the one hand librbd only supports XTS mode. On the other hand, qemu implementation will not support a chain of uniquely encrypted RBD images (each serving as a backing store for the previous one).

So we need a way in the XML API to support both implementations. Our current thought is to add a new "engine" attribute to the encryption tag. By default, encryption will use the QEMU LUKS implementation, unless <encryption engine='rbd' ...> is specified. To make this more general, we can have engine='backend' instead of engine='rbd' to denote that the encryption is to be delegated to the backend storage properties (instead of the format properties).

"Daniel P. Berrangé" <berrange(a)redhat.com&gt; wrote on 09/02/2021 15:23:14: > From: "Daniel P. Berrangé" <berrange(a)redhat.com&gt; > To: "Or Ozeri" <ORO(a)il.ibm.com&gt; > Cc: libvir-list(a)redhat.com, pkrempa(a)redhat.com, "Danny Harnik" > <DANNYH(a)il.ibm.com&gt; > Date: 09/02/2021 15:23 > Subject: [EXTERNAL] Re: RBD encryption support in libvirt > > On Thu, Sep 02, 2021 at 05:55:20AM +0000, Or Ozeri wrote: > > Hi, > > > > > > I wanted to get your advice on a patch I'm preparing for libvirt. > > It touches the code-path that allows using LUKS encryption on top > of an RBD image. > > > > > > We recently added LUKS and LUKS2 encryption support in Ceph's librbd. > > We exposed this in qemu in a recent patch by adding new optional > > "encrypt" member to BlockdevOptionsRbd. > > This patch was included in the recent release of qemu 6.1. > > To enable libvirt users to use librbd encryption, we need libvirt > > to use this new "encrypt" when it builds the blockdev options for RBD. > > > > > > The interesting question is how to define the libvirt XML syntax that > > will trigger the use of librbd encryption. > > My thought was to use the already existing <encryption> tag. > > In that case, we just need to add a new format > VIR_STORAGE_ENCRYPTION_FORMAT_LUKS2 to the enum > virStorageEncryptionFormatType. > > This type will be checked in qemuBlockStorageSourceGetRBDProps. > > I don't think we want to switch impls based on luks1 vs luks2, > because that will create trouble in future when/if QEMU's native > impl gains Luksv2 support. So I think we need an explicit way > to request librbd encryption, even for luks2. > > > The problem with this approach is that it only works for LUKS2. > > librbd encryption also supports LUKS1. > > We want to allow the user to choose between the qemu LUKS > implementation and the librbd one. > > One reason to keep support both is that on the one hand librbd > only supports XTS mode. > > On the other hand, qemu implementation will not support a chain of > uniquely encrypted RBD images (each serving as a backing store for > the previous one). > > I think I asked this before on qemu-devel, but I can't find the > answer, but can you explain the RBB backing store chain problem ? > > QEMUs' LUKS driver can be layered on top of any QEMU block protocol > driver. So if there are multiple RBD layers in the qemu -blockdev > config, we can layer LUKS over each one independantly. > RBD has an implementation of clones that is similar to backing chains in QCOW2, One can create a thinly provisioned clone of a snapshot using this mechanism. However the distinction between parent (source snapshot) and child (new clone) volumes is internal to RBD and is not visible to Qemu.Hence using Qemu LUKS on top of each layer is not a viable solution. Instead, we implemented the ability to support varying encryption keys between parent and child. Much like your qcow2-LUKS implementation in QEMU. We took some different design choices (e.g. the LUKS header is always part of the data payload), but the idea is very similar.

> I don't think we'll ever do it, but conceptually libvirt could even > integrate with host kernel cryptsetup tools. Having an 'engine' > attribute feels like a decent enough idea. > > Maybe engine='host|qemu|builtin' > > - host - the cryptosetup/kernel driver > - qemu - qemu's custom luks driver > - builtin - the librbd (or equiv) builtin driver "builtin" is a bit general. Why not be explicit and say "librbd"? I can envision scenarios in which you have two layers of encryption that are both builtin... So being specific would resolve possible confusion.