[libvirt] [RFC PATCH] Add proxy FS (9p) support to libvirt

From: "M. Mohan Kumar" <mohan@in.ibm.com> This patch *) adds new sub element <virtfs-proxy-helper> under <device> element. This sub-element is used to specify the virtfs-proxy-helper binary (part of QEMU) to be executed when proxy FS driver is used. *) invokes proxy_helper binary specified by the <virtfs-proxy-helper> sub-element with passing one of the created socket pair descriptor and adds "-sock_fd" paramter to qemu. Is it okay to add the sub-element "virtfs-proxy-helper" under "devices" element? Proxy helper binary is common for all 9p proxy FS devices, so it can not be placed under "filesystems" element. About Proxy FS driver ====================== Proxy FS driver is added to qemu 9p server to overcome two issues with pass-through security model (needs root privilege) security model. 1) TOCTTOU vulnerability: Following symbolic links in the server could provide access to files beyond 9p export path. 2) Running QEMU with root privilege could be a security issue. Proxy FS uses chroot + socket combination for securing the vulnerability known with following symbolic links. Intention of adding a new filesystem type is to allow qemu to run in non-root mode, but doing privileged operations using socket IO. Proxy helper(a stand alone binary part of qemu) is invoked with root privileges. Proxy helper chroots into 9p export path and creates a socket pair or a named socket based on the command line parameter. Qemu and proxy helper communicate using this socket. QEMU proxy fs driver sends filesystem request to proxy helper and receives the response from it. Proxy helper is designed such a way that it needs only few capabilities related to filesystem operations (such as CAP_DAC_OVERRIDE, CAP_FOWNER, etc) and all other capabilities are dropped (CAP_SYS_CHROOT, etc) Proxy patches http://permalink.gmane.org/gmane.comp.emulators.qemu/128735 Signed-off-by: M. Mohan Kumar <mohan@in.ibm.com> --- docs/formatdomain.html.in | 3 +- src/conf/domain_conf.c | 7 ++++- src/conf/domain_conf.h | 4 +- src/qemu/qemu_capabilities.c | 3 ++ src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 64 ++++++++++++++++++++++++++++++++++++++++-- src/qemu/qemu_command.h | 3 +- tests/qemuhelptest.c | 3 +- 8 files changed, 79 insertions(+), 9 deletions(-) diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 18b7e22..e398779 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -1336,7 +1336,8 @@ This mode also has an optional sub-element <code>driver</code>, with an attribute <code>type='path'</code> - or <code>type='handle'</code> <span class="since">(since + or <code>type='handle'</code> + or <code>type='proxy'</code> <span class="since">(since 0.9.7)</span>. </dd> <dt><code>type='template'</code></dt> diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 0190a81..694c166 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -250,7 +250,8 @@ VIR_ENUM_IMPL(virDomainFS, VIR_DOMAIN_FS_TYPE_LAST, VIR_ENUM_IMPL(virDomainFSDriverType, VIR_DOMAIN_FS_DRIVER_TYPE_LAST, "default", "path", - "handle") + "handle", + "proxy") VIR_ENUM_IMPL(virDomainFSAccessMode, VIR_DOMAIN_FS_ACCESSMODE_LAST, "passthrough", @@ -1465,6 +1466,7 @@ void virDomainDefFree(virDomainDefPtr def) VIR_FREE(def->name); VIR_FREE(def->cpumask); VIR_FREE(def->emulator); + VIR_FREE(def->virtfs_proxy_helper); VIR_FREE(def->description); virBlkioDeviceWeightArrayClear(def->blkio.devices, @@ -7490,6 +7492,9 @@ static virDomainDefPtr virDomainDefParseXML(virCapsPtr caps, goto error; } + def->virtfs_proxy_helper = + virXPathString("string(./devices/virtfs-proxy-helper[1])", ctxt); + /* analysis of the disk devices */ if ((n = virXPathNodeSet("./devices/disk", ctxt, &nodes)) < 0) { goto error; diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 03aa5b6..a7d248a 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -450,7 +450,6 @@ struct _virDomainControllerDef { virDomainDeviceInfo info; }; - /* Two types of disk backends */ enum virDomainFSType { VIR_DOMAIN_FS_TYPE_MOUNT, /* Better named 'bind' */ @@ -466,6 +465,7 @@ enum virDomainFSDriverType { VIR_DOMAIN_FS_DRIVER_TYPE_DEFAULT = 0, VIR_DOMAIN_FS_DRIVER_TYPE_PATH, VIR_DOMAIN_FS_DRIVER_TYPE_HANDLE, + VIR_DOMAIN_FS_DRIVER_TYPE_PROXY, VIR_DOMAIN_FS_DRIVER_TYPE_LAST }; @@ -491,7 +491,6 @@ struct _virDomainFSDef { virDomainDeviceInfo info; }; - /* 5 different types of networking config */ enum virDomainNetType { VIR_DOMAIN_NET_TYPE_USER, @@ -1503,6 +1502,7 @@ struct _virDomainDef { void *namespaceData; virDomainXMLNamespace ns; + char *virtfs_proxy_helper; }; enum virDomainTaintFlags { diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index 43c7578..9c01a3b 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -144,6 +144,7 @@ VIR_ENUM_IMPL(qemuCaps, QEMU_CAPS_LAST, "ich9-ahci", "no-acpi", "fsdev-readonly", + "fsdev-proxy", ); struct qemu_feature_flags { @@ -1083,6 +1084,8 @@ qemuCapsComputeCmdFlags(const char *help, qemuCapsSet(flags, QEMU_CAPS_FSDEV); if (strstr(fsdev, "readonly")) qemuCapsSet(flags, QEMU_CAPS_FSDEV_READONLY); + if (strstr(fsdev, "sock_fd")) + qemuCapsSet(flags, QEMU_CAPS_FSDEV_PROXY); } if (strstr(help, "-smbios type")) qemuCapsSet(flags, QEMU_CAPS_SMBIOS_TYPE); diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h index c759baf..e129e35 100644 --- a/src/qemu/qemu_capabilities.h +++ b/src/qemu/qemu_capabilities.h @@ -117,6 +117,7 @@ enum qemuCapsFlags { QEMU_CAPS_ICH9_AHCI = 77, /* -device ich9-ahci */ QEMU_CAPS_NO_ACPI = 78, /* -no-acpi */ QEMU_CAPS_FSDEV_READONLY =79, /* -fsdev readonly supported */ + QEMU_CAPS_FSDEV_PROXY = 80, /* -fsdev proxy supported */ QEMU_CAPS_LAST, /* this must always be the last item */ }; diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index f2e9cfa..9ac5a4c 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -106,7 +106,8 @@ VIR_ENUM_DECL(qemuDomainFSDriver) VIR_ENUM_IMPL(qemuDomainFSDriver, VIR_DOMAIN_FS_DRIVER_TYPE_LAST, "local", "local", - "handle"); + "handle", + "proxy"); static void @@ -2080,9 +2081,32 @@ error: return NULL; } +/* + * Invokes the Proxy Helper with one of the socketpair as its parameter + * + */ +static int qemuInvokeProxyHelper(const char *binary, int sock, const char *path) +{ + int ret_val, status; + virCommandPtr cmd; + + cmd = virCommandNewArgList(binary, NULL); + virCommandAddArg(cmd, "-f"); + virCommandAddArgFormat(cmd, "%d", sock); + virCommandAddArg(cmd, "-p"); + virCommandAddArgFormat(cmd, "%s", path); + virCommandTransferFD(cmd, sock); + virCommandDaemonize(cmd); + ret_val = virCommandRun(cmd, &status); + if (ret_val < 0) + qemuReportError(VIR_ERR_INTERNAL_ERROR, + _("%s can't execute"), binary); + virCommandFree(cmd); + return ret_val; +} char *qemuBuildFSStr(virDomainFSDefPtr fs, - virBitmapPtr qemuCaps ATTRIBUTE_UNUSED) + virBitmapPtr qemuCaps ATTRIBUTE_UNUSED, int qemuSocket) { virBuffer opt = VIR_BUFFER_INITIALIZER; const char *driver = qemuDomainFSDriverTypeToString(fs->fsdriver); @@ -2108,6 +2132,10 @@ char *qemuBuildFSStr(virDomainFSDefPtr fs, virBufferAddLit(&opt, ",security_model=none"); } virBufferAsprintf(&opt, ",id=%s%s", QEMU_FSDEV_HOST_PREFIX, fs->info.alias); + + if (fs->fsdriver == VIR_DOMAIN_FS_DRIVER_TYPE_PROXY) + virBufferAsprintf(&opt, ",sock_fd=%d", qemuSocket); + virBufferAsprintf(&opt, ",path=%s", fs->src); if (fs->readonly) { @@ -4426,10 +4454,40 @@ qemuBuildCommandLine(virConnectPtr conn, if (qemuCapsGet(qemuCaps, QEMU_CAPS_FSDEV)) { for (i = 0 ; i < def->nfss ; i++) { char *optstr; + int sockets[2] = {-1, -1}; virDomainFSDefPtr fs = def->fss[i]; + /* + * If its a proxy FS, we need to create a socket pair + * and invoke proxy_helper + */ + if (fs->fsdriver == VIR_DOMAIN_FS_DRIVER_TYPE_PROXY) { + if (qemuCapsGet(qemuCaps, QEMU_CAPS_FSDEV_PROXY) < 0) { + qemuReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("proxy helper not supported")); + goto error; + } + if (!def->virtfs_proxy_helper) { + qemuReportError(VIR_ERR_XML_ERROR, "%s", + _("proxy helper not specified")); + goto error; + } + /* create a socket pair */ + if (socketpair(PF_UNIX, SOCK_STREAM, 0, sockets) < 0) { + qemuReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("socketpair failed")); + goto error; + } + virCommandTransferFD(cmd, sockets[1]); + if (qemuInvokeProxyHelper(def->virtfs_proxy_helper, sockets[0], + fs->src) < 0) { + VIR_FORCE_CLOSE(sockets[0]); + VIR_FORCE_CLOSE(sockets[1]); + goto error; + } + } virCommandAddArg(cmd, "-fsdev"); - if (!(optstr = qemuBuildFSStr(fs, qemuCaps))) + if (!(optstr = qemuBuildFSStr(fs, qemuCaps, sockets[1]))) goto error; virCommandAddArg(cmd, optstr); VIR_FREE(optstr); diff --git a/src/qemu/qemu_command.h b/src/qemu/qemu_command.h index de61cf3..8e4f335 100644 --- a/src/qemu/qemu_command.h +++ b/src/qemu/qemu_command.h @@ -87,7 +87,8 @@ char *qemuBuildDriveStr(virConnectPtr conn, bool bootable, virBitmapPtr qemuCaps); char *qemuBuildFSStr(virDomainFSDefPtr fs, - virBitmapPtr qemuCaps); + virBitmapPtr qemuCaps, + int qemuSocket); /* Current, best practice */ char * qemuBuildDriveDevStr(virDomainDiskDefPtr disk, diff --git a/tests/qemuhelptest.c b/tests/qemuhelptest.c index 60155e7..455a7e1 100644 --- a/tests/qemuhelptest.c +++ b/tests/qemuhelptest.c @@ -648,7 +648,8 @@ mymain(void) QEMU_CAPS_PCI_ROMBAR, QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_NO_ACPI, - QEMU_CAPS_FSDEV_READONLY); + QEMU_CAPS_FSDEV_READONLY, + QEMU_CAPS_FSDEV_PROXY); return ret == 0 ? EXIT_SUCCESS : EXIT_FAILURE; } -- 1.7.6

From: "M. Mohan Kumar" <mohan@in.ibm.com> Signed-off-by: M. Mohan Kumar <mohan@in.ibm.com> --- diff --git a/test.xml b/test.xml new file mode 100644 index 0000000..46fc6fc --- /dev/null +++ b/patch/test.xml @@ -0,0 +1,39 @@ +<domain type='kvm'> + <name>test_vm</name> + <uuid>7e359ba6-3833-dfe0-6c31-c09fcfad169b</uuid> + <memory>524288</memory> + <currentMemory>524288</currentMemory> + <vcpu>1</vcpu> + <os> + <type arch='x86_64' machine='pc'>hvm</type> + <kernel>/vm/bzImage</kernel> + <cmdline>root=/dev/vda console=ttyS0 noapic</cmdline> + </os> + <features> + <acpi/> + </features> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/test/bin/qemu-system-x86_64</emulator> + <virtfs-proxy-helper>/test/bin/virtfs-proxy-helper</virtfs-proxy-helper> + + <disk type='file' device='disk'> + <source file='/vm/guest_64'/> + <target dev='vda' bus='virtio'/> + </disk> +<graphics type="vnc" port="5910" listen="127.0.0.1" keymap="en-us"/> + <interface type='user'> + <mac address='52:54:00:38:c4:ea'/> + <model type='virtio'/> + </interface> + <filesystem type='mount'> + <driver type ='proxy'/> + <source dir='/share'/> + <target dir='v_pass'/> + </filesystem> + </devices> +</domain> + -- 1.7.6

On Tue, Jan 17, 2012 at 10:48:15AM +0530, M. Mohan Kumar wrote:
From: "M. Mohan Kumar" <mohan@in.ibm.com>
This patch *) adds new sub element <virtfs-proxy-helper> under <device> element. This sub-element is used to specify the virtfs-proxy-helper binary (part of QEMU) to be executed when proxy FS driver is used.
*) invokes proxy_helper binary specified by the <virtfs-proxy-helper> sub-element with passing one of the created socket pair descriptor and adds "-sock_fd" paramter to qemu.
Is it okay to add the sub-element "virtfs-proxy-helper" under "devices" element? Proxy helper binary is common for all 9p proxy FS devices, so it can not be placed under "filesystems" element.
Hmm, what is the version compatibility like between QEMU and the proxy helper. eg, will we be able to use a version 1.1 QEMU, with a version 1.2 virtfs-proxy-helper, or vica-verca. I'd probably expect that QEMU will always want a precisely matched virtfs-proxy-helper version. It feels to me like we should just form a proxy helper binary path, based on the path to the corresponding QEMU binary. eg, if the guest is using /home/berrange/qemu-git/bin/qemu-system-x86_64, then we should automatically use /home/berrange/qemu-git/libexec/virtfs-proxy-helper Or, alternatively, perhaps QEMU itself should be made to tell us where the helper lives. eg something like # qemu -build-config virtfs-proxy-helper-path=/home/berrange/qemu-git/libexec/virtfs-proxy-helper then libvirt would always be ensured to have the right binary to match QEMU. There is a similar need for the QEMU net device helper program In general I think one of these approachs is better than added anything else to the XML.
+/* + * Invokes the Proxy Helper with one of the socketpair as its parameter + * + */ +static int qemuInvokeProxyHelper(const char *binary, int sock, const char *path) +{ + int ret_val, status; + virCommandPtr cmd; + + cmd = virCommandNewArgList(binary, NULL); + virCommandAddArg(cmd, "-f"); + virCommandAddArgFormat(cmd, "%d", sock); + virCommandAddArg(cmd, "-p"); + virCommandAddArgFormat(cmd, "%s", path); + virCommandTransferFD(cmd, sock); + virCommandDaemonize(cmd); + ret_val = virCommandRun(cmd, &status); + if (ret_val < 0) + qemuReportError(VIR_ERR_INTERNAL_ERROR, + _("%s can't execute"), binary); + virCommandFree(cmd); + return ret_val;
This raises interesting questions wrt sVirt / SELinux integration. ie do we need to run each helper program under a dedicated SELinux context to separate them. I think we probably will need to, but I'll have to thing about this some more Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

On Wednesday, January 18, 2012 03:01:47 AM Daniel P. Berrange wrote:
On Tue, Jan 17, 2012 at 10:48:15AM +0530, M. Mohan Kumar wrote:
Is it okay to add the sub-element "virtfs-proxy-helper" under "devices" element? Proxy helper binary is common for all 9p proxy FS devices, so it can not be placed under "filesystems" element.
Hmm, what is the version compatibility like between QEMU and the proxy helper. eg, will we be able to use a version 1.1 QEMU, with a version 1.2 virtfs-proxy-helper, or vica-verca. I'd probably expect that QEMU will always want a precisely matched virtfs-proxy-helper version.
It feels to me like we should just form a proxy helper binary path, based on the path to the corresponding QEMU binary.
eg, if the guest is using
/home/berrange/qemu-git/bin/qemu-system-x86_64,
then we should automatically use
/home/berrange/qemu-git/libexec/virtfs-proxy-helper
I prefer above approach to determine the virtfs-proxy-helper path (ie based on qemu binary path).
Or, alternatively, perhaps QEMU itself should be made to tell us where the helper lives.
eg something like
# qemu -build-config
virtfs-proxy-helper-path=/home/berrange/qemu-git/libexec/virtfs-proxy-help er
then libvirt would always be ensured to have the right binary to match QEMU. There is a similar need for the QEMU net device helper program
In general I think one of these approachs is better than added anything else to the XML.
This raises interesting questions wrt sVirt / SELinux integration. ie do we need to run each helper program under a dedicated SELinux context to separate them. I think we probably will need to, but I'll have to thing about this some more
Is it required for adding support for 9p proxy to libvirt now? Where I can get more information? -- Regards, M. Mohan Kumar
participants (2)
-
Daniel P. Berrange
-
M. Mohan Kumar