Re: [libvirt] Using external ceph.conf for RBD pools and disks

On 11/01/2013 07:42 AM, Eric Blake wrote: This will break OpenStack's usage of libvirt + rbd in Grizzly and earlier releases, which relied on loading ceph.conf for the monitor addresses. This is fixed in OpenStack Havana, but I wanted to note that applications are relying on this behavior. Passing conf=/dev/null removes the last remaining way of specifying arbitrary ceph options for rbd devices, which is backwards-incompatible in some setups even with well-behaved applications. In general it may break setups using non-default options that libvirt is not aware of. For example, ceph has an option to require messages to be signed. This is off by default for backwards compatibility with older ceph clients, but it can be enabled for qemu right now by adding an option to /etc/ceph/ceph.conf. If libvirt passes conf=/dev/null, guests are less secure since they may get their data from an untrusted source that does not sign messages. Ceph is a fast-moving complex project, and there are many options (and will be more in the future) that affect security, performance tuning, run-time introspection, logging, etc. I don't think libvirt should remove the ability to configure these settings without having a way to add them via xml. It doesn't seem feasible to make libvirt (and all applications using it) aware of all existing and new options, especially since many of them are quite ceph-specific. Instead, I'd like to propose a mechanism for passing through generic key/value pairs to configure block devices. Concretely, this could be something like: <disk type='network'> <driver name='qemu' type='raw' cache='writeback'/> <source protocol='rbd' name='pool/image'> <host name='mon1.example.org'/> <option name="cephx require signatures" value="true"/> <option name="rbd cache size" value="131768"/> <option name="rbd cache max dirty" value="131768"/> <option name="rbd cache max dirty age" value="1.5"/> <option name="rbd balance snap reads" value="true"/> <option name="debug ms" value="0/0"/> <option name="debug auth" value="0/0"/> <option name="debug rados" value="0/0"/> </source> </disk> I don't care about the particular format, just that there's a way to set these kinds of settings. It's much easier for users of libvirt and ceph if these are treated as opaque strings by libvirt, since they can ugrade ceph and use new options without upgrading libvirt and any applications using it as well. I'm happy to provide patches if this approach is acceptable. Josh