Re: [libvirt] [Qemu-devel] spec, RFC: TLS support for NBDµ

Hi Markus, On Tue, Oct 21, 2014 at 10:17:17AM +0200, Markus Armbruster wrote: [...] My advice is to give both client and server the ability to have TLS switched on or off, and possibly (but not necessarily so, and certainly not by default) also the _ability_ to negotiate TLS if the other side supports it, while not aborting if it doesn't. No, I can't, at least not if I want to continue to use an assigned port without breaking backwards compatibility. When 10809 was assigned to NBD, IANA made it perfectly clear that I wouldn't get a new port number for a "secure" variant. Well, to some extent, every software issue is merely an implementation restriction... The problem is that in order to clear memory when your swap device is on the other side of a network connection, you need to write to said swap device, which causes TCP traffic, which means the kernel needs to read TCP ACK packets (or in the case of NBD, the reply packet to the NBD_CMD_WRITE request that you sent out) to ensure that the traffic has in fact reached its destination. Unfortunately, you can't impose ordering on incoming network traffic, so that means you may need to read through a whole lot of youtube MPEG traffic or some such in order to find your one TCP ACK that tells you your swapout has been processed correctly and you can now clear the memory which you wrote out to swap. It's fairly obvious how that could lead to a deadlock if you don't know that this one data stream is not related to swapspace while this other one here is. That's why a PF_MEMALLOC kernel-space socket option was created; sockets marked with that option are related to a swapdevice, so when the kernel is low on memory, it will throw away all incoming network traffic _except_ for packets destined for a socket marked with that option, in the hope that this will allow us to clear up some memory. Since it's a kernel-space only thing, that means you can't mark sockets as such from userspace; so if the NBD traffic is wrapped in some other traffic from which it needs to be unwrapped in userspace, then the userspace component would basically be a proxy with two sockets -- one towards the server, one towards the kernel. The socket towards the kernel would have the PF_MEMALLOC socket option set, but the one towards the server would not, and *that* is the one which actually has data flowing in over the network. There's your deadlock again. Since, with the current state of affairs, the NBD handshake is done in user space with the actual data pushing stuff later being done in kernel space, that means you do actually need to unwrap the TLS traffic in userspace. I can't think of many ways that would allow us to get around that issue: - Figure out a way to move negotiated keys from user space to kernel space, and handle the TLS in kernel space rather than user space. I'm not even sure if you _can_ do TLS in kernel space (let alone whether it's a good idea). - Move the handshake to kernel space from user space. I'm not a fan of that idea. It would also still require much of the TLS in kernel space. - Patch the kernel so that userspace can mark a particular socket with PF_MEMALLOC. And mlock() the whole TLS stack into memory (including keys etc). All three of those require "patch the kernel", so at any rate going there would require "wait until the kernel supports it before we can do TLS", which I think is not a good option. True. -- It is easy to love a country that is famous for chocolate and beer -- Barack Obama, speaking in Brussels, Belgium, 2014-03-26