Re: [libvirt] specifying rbd images in libvirt

On Thu, 1 Sep 2011, Sage Weil wrote: > On Thu, 1 Sep 2011, Daniel P. Berrange wrote: > > On Sat, Aug 27, 2011 at 12:19:33PM -0700, Sage Weil wrote: > > > Hi all, > > > > > > Currently, you can specify an rbd (or nbd, sheepdog) image with xml > > > that looks like so: > > > > > > <disk type='network' device='disk'> > > > <driver name='qemu' type='raw' cache='writeback'/> > > > <source protocol='rbd' name='mypool/myimage'> > > > <host name='monhost1.mydomain.com' port='6789'/> > > > <host name='monhost2.mydomain.com' port='6789'/> > > > <host name='monhost3.mydomain.com' port='6789'/> > > > </source> > > > <target dev='vda' bus='virtio'/> > > > </disk> > > > > > > This works okay if you have authentication disabled and all of the default > > > settings are okay. Usually, though, there are other options you need to > > > specify to librbd to make it do what you want. > > > > > > The current schema can be abused by adding options after the image name > > > like so: > > > > > > <disk type='network' device='disk'> > > > <driver name='qemu' type='raw' cache='writeback'/> > > > <source protocol='rbd' name='mypool/myimage:conf=/etc/ceph/ceph.conf:id=admin:this=that:foo=bar'> > > > <host name='monhost1.mydomain.com' port='6789'/> > > > <host name='monhost2.mydomain.com' port='6789'/> > > > <host name='monhost3.mydomain.com' port='6789'/> > > > </source> > > > <target dev='vda' bus='virtio'/> > > > </disk> > > > > > > This works only because that's what the qemu incantation looks like. In > > > general, though, this is ugly. I also doesn't generalize well to the > > > kernel-level rbd driver, which we'd like to also support, as that will > > > work with hypervisors other than qemu. > > > > Also, we should be doing some validation on the content of > > the 'name' attribute to prevent abuse like that. > > > > > What about something more like this? > > > > > > <disk type='network' device='disk'> > > > <driver name='qemu' type='raw' cache='writeback'/> > > > <source protocol='rbd' name='mypool/myimage'> > > > <option name='conf'>/etc/ceph/ceph.conf</option> > > > <option name='id'>myusername</option> > > > <option name='foo'>bar</option> > > > <host name='monhost1.mydomain.com' port='6789'/> > > > <host name='monhost2.mydomain.com' port='6789'/> > > > <host name='monhost3.mydomain.com' port='6789'/> > > > </source> > > > <target dev='vda' bus='virtio'/> > > > </disk> > > > > > > I'm not married to any particular syntax/schema, as long as there is a > > > generic way to specify name/value pairs to configure the driver. I think > > > the above would generalize well to other network block devices as well, > > > which presumably also want a way to feed in information other than a > > > server address (e.g. for authentication). > > > > > > Does that look reasonable? If there are no objections we can work up some > > > patches and send them along! > > > > We generally prefer to add explicit models for attributes, rather than > > just do a generic key/value passthrough. > > Fair enough. For the qemu/librbd and kernel drivers, the key fields we > need are: > > id - ceph userid to authenticate as > snap - snapshot name (optional; used if mapping a read-only snapshot) > > The goal would be to have libvirt describe rbd images in a generic way, > and let you managed them as a storage pool via librbd. Then, when it > comes time to spin up an actual VM, either use the native qemu support or > map a kernel block device. To control that it might make sense to have an > option like > > driver - librbd (qemu-only), kernel, or auto (default) > > to let users control which driver to use in the qemu case. > > > For authentication credentials, we also need to consider security > > implications of keeping them in the XML. For qcow2 encryption > > keys, we leverage the separate libvirt secrets management APIs > > to provide the keys outside the XML. IMHO we should likely do the > > same for any keys required to login to network block devices. > > Usually we keep secrets in a file on disk and reference the file when > starting up librbd or mapping a kernel rbd device. It's possible to > specify it explicitly but that is discouraged. So by specifying id=foo > above, librbd would normally look for a key called foo in its default > keyring file. > > We can definitely use the libvirt secrets API to store these keys there, > too. In that case, we would need something like > > keyname - name of the libvirt secret to use for authentication > > What do you think? Shall I take silence as general agreement and start putting together patches?

I don't think there's anything controversial here. I suspect the main concern is that these fields are named in a sufficiently general fashion so that they can be used for other backend storage types.