Re: [libvirt] [PATCH 0/9] Add support for (qcow*) volume encryption

On Fri, Jul 24, 2009 at 07:25:54AM -0400, Miloslav Trmac wrote: I did not mean to imply that libvirtd on a node should have access to *all* secrets. Each libvirtd daemon has its own identity, and when talking to a keys server it would authenticate and the key server would only allow it access to some sub-set of keys. ie, only the keys neccessary for VMs it needs to run. If you include the secrets in the XML for a guest, that does not ensure those secrets are only accessible to that host. For example, when performing migration, the XML doc for a guest is passed directly from the source libvirtd to the destination libvirtd. So an admin could ssh into a node, run 'virsh migrate' and the guest & secrets would be transferred to another host. If you have each libvirtd requesting secrets directly from the keystore, at the time it starts a guest, then should an admin issue a migrate command manually, the destination libvirtd would still be unable to access the secrets of the incoming VM, since the keystore will not have been configured to allow it access. We also have to bear in mind how the MGMT server communicates with the libvirt daemon. One likely option is using a messaging service, which offers authenticity but not secrecy. ie, libvirt receiving a request off the message bus can be sure the request came from the mgmtm server, but cannot be sure it wasn't seen by other users during transport. Thus by including secrets in the XML, you could be exposing them to the message bus adminstrator. Taking your diagram, I think I would generalize it still further to allow for mgmt server to be optional separate from key server, and to introduce a "message bus" between MGMT server & libvirt. Really pushing the limits of ASCII art... MGMT client | V MGMT server <---> Key server | ^ V | message bus | | | V | libvirt daemon <----/ | V QEMU To start a VM, the sequence of steps would be: 1. MGMT client says 'boot vm X on node Y' to MGMT server 2. MGMT server says 'allow node Y access to secrets for VM X' to key server 3. MGMT server puts 'start vm X' message o nmessage bus to node Y 4. libvirt on node Y says 'give secrets for vm X' to key server 5. libvirt spawns QEMU, passing secrets If node E intercepts the 'start vm X' message from the bus it cannot see the secrets directly. If node 'E' asks the key server for secrets for VM X, it will be refused, since the MGMT server has not authorized it to see them. If sysadmin on node N, tries to migrate the VM to node E, node E will not be able to start the VM, since it again has not been authorized to fetch the secrets. To actually allow a migration, the sequence of steps would be 1. MGMT client says 'migrate vm X from node Y to node Z' to MGMT server 2. MGMT server says 'allow node Z access to secrets S' to key server 3. MGMT server puts 'migrate vm X to node Z' message on message bus to node Y 4. libvirt on node Y issues the migration operation, passing XML for the VM to node Z 5. libvirt on node Z says 'give secrets for vm X' to key server 6. libvirt on node Z spawns QEMU, passing secrets 7. MGMT server says 'deny node Y access to secrets for vm X' to key server I believe this deals with all the use cases, specifically - A node can only see secrets for VMs it is configured to run - MGMT server does not need to ever see the secrets itself. It merely controls access for which nodes can see them, and can request generation of new secrets - Messages between the MGMT server & libvirtd do not need to be encrypted, since they don't include secrets - Other users who authenticate to libvirt on a node cannot move a VM to an unauthorized node, since it can't see secrets - VM save memory images (which include the fully XML) do not ever expose the secrets. So VM save image cannot be restored on an unauthorized node. I think the outline above should address your concern here. libvirt is intended to deal with resources on a single node, and provide a means for command & control of that node. I don't think anything I have said suggests / requires that libvirt needs full knowledge of the whole site. What we're dealing with here is a layered architecture, that can be deployed in a variety of different scenarios. The MGMT server authenticates & authorizes connections from MGMT clients. Libvirtd can be made to authenticate & authorize connections from the MGMT server and possibly other clients apps - that's a deployment decision of the specific MGMT application / adminstrator. libvirtd does not need to know the secrets of all VMs for all nodes, it only needs the ability to use key service which can provide secrets it needs. That key service may be local to the node, it may be on storage shared by several nodes, or it may be over the network on a remote node. A remote key service should of course authenticate / authorize each libvirtd that connects and requests access to keys, to ensure a node cannot access keys for VMs on another node. By providing a pluggable key service backend for libvirt I believe we can satisfy all these different deployment scenarios that have been discussed thus far, without needing to favour one. I think it is very important that all these deployment scenarios can be supported without including the secrets in the XML config, since there are faaaar too many ways in which the XML config itself may be exposed to undesirable places. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|