Pass-through security model in QEMU 9p server needs root privilege to do few file operations (like chown, chmod to any mode/uid:gid). There are two issues in pass-through security model 1) TOCTTOU vulnerability: Following symbolic links in the server could provide access to files beyond 9p export path. 2) When libvirt is configured to run qemu as non-root user (for example, if qemu is configured to run as normal user 'qemu'), running file operations on pass-through security model would fail because it needs root privileges. To overcome above issues, following approach is suggested: A new filesytem type 'proxy' is introduced. Proxy FS uses chroot + socket combination for securing the vulnerability known with following symbolic links. Intention of adding a new filesystem type is to allow qemu to run in non-root mode, but doing privileged operations using socket IO. A new binary (known as proxy helper) will be provided as part of qemu. Proxy helper will chroot into 9p export path and create a socket pair or a named socket based on the command line parameter. Qemu and proxy helper will communicate using this socket. We need following changes in the libvirt code to accomodate new 'proxy' filesystem type: If qemu 9p server is configured to use 'proxy' FS, libvirt will do * Create a socket pair * invoke proxy_helper binary with one of the socket id from the pair as command line parameters to it with root privilege * invoke qemu with one of socket id from the pair as paramter to qemu virtfs after dropping to the configured user privilege. ie, libvirt will invoke proxy_helper as: proxy_helper -i <socket_fd_from_socket_pair> -p <9p-path-to-export> and qemu will be invoked with following virtfs parameter: -virtfs proxy,id=<id>,sock_fd=<socket_fd_from_socket_pair> ,path=/tmp/,security_model=prox,mount_tag=v_pass

On Wednesday, September 28, 2011 08:29:06 PM Daniel P. Berrange wrote: > On Wed, Sep 28, 2011 at 07:49:34PM +0530, M. Mohan Kumar wrote: > > Pass-through security model in QEMU 9p server needs root privilege to do > > few file operations (like chown, chmod to any mode/uid:gid). There are > > two issues in pass-through security model > > > > 1) TOCTTOU vulnerability: Following symbolic links in the server could > > provide access to files beyond 9p export path. > > > > 2) When libvirt is configured to run qemu as non-root user (for example, > > if qemu is configured to run as normal user 'qemu'), running file > > operations on pass-through security model would fail because it needs > > root privileges. > > > > To overcome above issues, following approach is suggested: A new > > filesytem type 'proxy' is introduced. Proxy FS uses chroot + socket > > combination for securing the vulnerability known with following symbolic > > links. Intention of adding a new filesystem type is to allow qemu to run > > in non-root mode, but doing privileged operations using socket IO. > > > > A new binary (known as proxy helper) will be provided as part of qemu. > > Proxy helper will chroot into 9p export path and create a socket pair or > > a named socket based on the command line parameter. Qemu and proxy > > helper will communicate using this socket. > > > > We need following changes in the libvirt code to accomodate new 'proxy' > > filesystem type: > > If qemu 9p server is configured to use 'proxy' FS, libvirt will do > > * Create a socket pair > > * invoke proxy_helper binary with one of the socket id from the pair as > > command line parameters to it with root privilege > > * invoke qemu with one of socket id from the pair as paramter to qemu > > virtfs after dropping to the configured user privilege. > > > > ie, libvirt will invoke proxy_helper as: > > proxy_helper -i <socket_fd_from_socket_pair> -p <9p-path-to-export> > > > > and qemu will be invoked with following virtfs parameter: > > -virtfs proxy,id=<id>,sock_fd=<socket_fd_from_socket_pair> > > > > ,path=/tmp/,security_model=prox,mount_tag=v_pass > Thank you Daniel for the quick response, I was on leave and I could not respond to you immediately. > Interesting proposal. Explicitly comparing the security characteristics > of running QEMU as root, vs using the proxy helper > > * QEMU run as root > > - QEMU is root, with full capabilities > - QEMU has read/write any file/dir, regardless of whether it is > exported via 9p > > * QEMU run as non-root, with proxy_helper root > > - QEMU is non-root, with no capabilities > - QEMU has write to only files with matching UID/GID > - proxy_helper is root, with full capabilities > - proxy_helper has read/write to any file/dir > > Since QEMU can send arbitrary FS calls to the proxy_helper, the overall > security of the system clearly depends on the security of the proxy_helper > process. QEMU can't send arbitrary FS calls to the proxy helper. All interfaces are defined and if qemu sends arbitrary FS command, proxy helper would return an EIO. For example T_OPEN is one of the supported interface between 9p qemu server and proxy helper. For more information please look at my chroot patchset that I posted to qemu-devel few weeks ago. http://lists.gnu.org/archive/html/qemu-devel/2011-09/msg00671.html

> If we assume that QEMU gets exploited, and that QEMU can find some flaw > in the proxy_helper that it can exploit, what damage can the proxy_helper > do ? Clearly we want it to not be able to read/write any files other > than those exported, even when it becomes compromised, ideally also > without requiring SELinux/AppArmour to make it safe. If proxy_helper > is running as root with full capabilities, it can trivially escape > the chroot[1], so this isn't all that nice for overall system security. When proxy helper process is forked, its made sure that all open file descriptor are closed except the socket descriptor, so that proxy helper can't escape chroot jail.

Main aim of providing proxy helper is to 1) Access beyond 9p export path should not be given 2) Remove the limitation that qemu should be run by root user to use pass- through security model

> The attacker simply needs to find a vulnerability in QEMU and in the > proxy_helper, instead of just in QEMU. We want a stronger guarantee > than that. > > In order to be able to chown/chmod files etc, the proxy_helper is going > to require some elevated privileges, however, this does not actually > imply that proxy_helper needs to run as root. You don't want the > proxy_helper to be allowed todo various other things that root would be > allowed, even if it is inside a chroot. eg you don't want it being allowed > to reboot, mlock, change network setting, mount volumes, make device > nodes, and quite alot more. Proxy helper operations are categorized as: 1) Open/create file and return the descriptor 2) Create directory, link, symbolic link, device nodes, FIFOs etc 3) Remove/rename filesystem object 4) Return stat information 5) Return readlink() information > > Looking at the man page 'capabilities(7)', I'd venture to suggest that > the proxy_helper does not in fact need to run as the 'root' UID/GID at > all. All it needs is to have the CAP_CHOWN, CAP_DAC_OVERRIDE, > CAP_DAC_READ_SEARCH, CAP_FOWNER capabilities, at which point it could > still run as the 'qemu' user ID (or perhaps another 'qemuhelper' user > ID). IMHO, proxy helper can run as root because it will run only predefined file operations.

On Thu, Sep 29, 2011 at 11:42:47PM +0530, M. Mohan Kumar wrote: > On Wednesday, September 28, 2011 08:29:06 PM Daniel P. Berrange wrote:

> > If we assume that QEMU gets exploited, and that QEMU can find some flaw > > in the proxy_helper that it can exploit, what damage can the proxy_helper > > do ? Clearly we want it to not be able to read/write any files other > > than those exported, even when it becomes compromised, ideally also > > without requiring SELinux/AppArmour to make it safe. If proxy_helper > > is running as root with full capabilities, it can trivially escape > > the chroot[1], so this isn't all that nice for overall system security. > > When proxy helper process is forked, its made sure that all open file > descriptor are closed except the socket descriptor, so that proxy helper can't > escape chroot jail. Unfortunately that is not sufficient. Even if you have closed all file descriptors except the socket, you can still escape the chroot jail. There is a nice demo program for escaping from a chroot on this webpage: http://www.bpfh.net/simes/computing/chroot-break.html

Basically if the process inside the chroot, is still running as root, then it is not securely contained.