All, I've been looking at implementing secure migration for libvirt. I've taken a look at two approaches; I'll outline them, and then tell you which one I think we should do, and why. Note that in the discussion below, I'm talking mostly about the qemu migration protocol; I believe both of the approaches will lend themselves to, say, Xen, but the details will be different. For the sake of the outlines, I'm using three specific terms: "src" is used to refer to the source of the migration, that is, where the VM is running *before* migration begins; "dst" is used to refer to the destination of the migration, that is, where we want the VM to be at the end; and "controller" refers to the controlling program (usually virsh), which can be run on an entirely separate machine. -------------------------------------------------------------------------------- 1) virsh on the controller connects to the src, and initiates the migration command. In turn, this causes the controller to also connect to the dst. Now, during the "Prepare" step on the dst, we setup a qemu container to listen to some port on localhost. Next, the "Perform" step is called on the src machine. This causes an external program to be forked on the src; this external program connects to the dst (using virConnectOpen), then waits for migration data from the qemu instance. As the migration data comes in, it uses a new RPC call (something like "writeMigrationData") to pass the data to the dst. The dst then takes that data, and writes it out to the waiting qemu container. Pros: Uses existing RPC mechanism, so you don't have to open a new port on the destination Cons: There is a "hidden" dependency between the src and dst, which may be difficult for users to get right (more on this below). 2) virsh on the controller connects to the src, and initiates the migration command. In turn, this causes the controller to also connect to the dst. Now, during the "Prepare" step on the dst, we setup a qemu container to listen to some port (call it 1234) on localhost. It also forks an external program (or a thread) to listen for an incoming gnutls connection. Next, the "Perform" step is call on the src machine. This forks an external program (or thread) to listen for incoming data from a localhost migration, do the gnutls handshake with the dst, and dump the data over the gnutls connection to the dst. Pros: Works mostly transparently; the user doesn't have to set anything up different than they do today for "unsecure" migration Cons: Requires opening a new (but still well-known) port in the firewall on the dst side. -------------------------------------------------------------------------------- More about the "hidden" dependencies for 1). There are quite a few examples of where the controller might have access to both machines, but the src libvirtd does not. A few examples below: a) libvirtd is using TLS + the "tls_allowed_dn_list" in /etc/libvirt/libvirtd.conf on the dst machine. They have it configured so that they can access the machine via TLS on the controller machine, but not from the src machine. b) libvirtd is using SASL digest-md5 on the dst machine. When the src machine tries to connect, it needs a name and password to do so, which it doesn't have. (technically, I guess we could proxy that response back to the controller, and have the user fill it there, and then return back to the src, and then the dst, but it seems roundabout) c) libvirtd is using SASL gssapi on the dst machine. When the src machine tries to connect to the dst, it needs to have the right configuration (i.e. /etc/krb5.conf and /etc/sasl2/libvirt.conf need to work), and it also has to get some kind of principal from the kerberos server (but which principal?). Because of the hidden dependency problem, I think solution 2) is actually more viable; yes, it also has a dependency (opening a hole in the firewall), but that can be documented and will work no matter what other authentication you are doing between the controller and src and dst machines. However, I am open to being convinced otherwise. Thoughts? -- Chris Lalancette