- Devel - libvirt List Archives

[libvirt] Problems with <filesystem type='block'>
by Lars Kellogg-Stedman 24 Mar '13

24 Mar '13

Using libvirt 1.0.1, I'm trying to start an LXC container using the '<filesytem type="block">' syntax, like this: <filesystem type="block" accessmode="passthrough"> <source dev="/dev/vg_files/vm-foobar-root" /> <target dir="/" /> </filesystem> The specified block device exists: # ls -lL /dev/vg_files/vm-foobar-root brw-rw---- 1 root disk 253, 19 Dec 21 22:23 /dev/vg_files/vm-foobar-root If I start the domain, it appears to start without any errors... # virsh start foobar Domain foobar started ...but it's not actually running. The log files (with loglevel=2) don't seem to be very interesting; this is everything from the instance log file: 2012-12-22 04:10:57.862+0000: starting up PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin LIBVIRT_DEBUG=2 LIBVIRT_LOG_OUTPUTS=2:stderr /usr/lib/libvirt/libvirt_lxc --name foobar --console 22 --security=none --handshake 25 --background --veth veth1 2012-12-22 04:10:57.967+0000: 1468: info : libvirt version: 1.0.1 2012-12-22 04:10:57.967+0000: 1468: info : lxcCapsInit:151 : No driver, not initializing security driver PATH=/bin:/sbin TERM=linux container=lxc-libvirt container_uuid=9041e32e-1df2-00c2-4660-dfa5b41510b7 LIBVIRT_LXC_UUID=9041e32e-1df2-00c2-4660-dfa5b41510b7 LIBVIRT_LXC_NAME=foobar /sbin/init 2012-12-22 04:10:58.198+0000: 1: warning : lxcContainerDropCapabilities:1788 : libcap-ng support not compiled in, unable to clear capabilities 2012-12-22 04:10:58.198+0000: 1493: warning : lxcControllerClearCapabilities:679 : libcap-ng support not compiled in, unable to clear capabilities Running an "strace" on the libvirtd process (strace -p <libvirtd_pid> -f ...), it doesn't look like libvirt is ever trying to mount the referenced filesystem. Is this supposed to work? It seems like the support for having libvirt mount the block device is relatively recent, and I haven't had much luck finding examples of other folks using this capability. Thanks, -- Lars

4 8

[libvirt] [PATCH 0/4] Multiple problems with saving to block devices
by Daniel P. Berrange 24 Mar '13

24 Mar '13

This patch series makes it possible to save to a block device, instead of a plain file. There were multiple problems - WHen save failed, we might de-reference a NULL pointer - When save failed, we unlinked the device node !! - The approach of using >> to append, doesn't work with block devices - CGroups was blocking QEMU access to the block device when enabled One remaining problem is not in libvirt, but rather QEMU. The QEMU exec: based migration often fails to detect failure of the command and will thus hang forever attempting a migration that'll never succeed! Fortunately you can now work around this in libvirt using the virsh domjobabort command

4 21

[libvirt] [PATCH RFC 0/2] Report OOM on VIR_ALLOC failure
by Michal Privoznik 23 Mar '13

23 Mar '13

Currently, our code is plenty of following scheme: if (VIR_ALLOC(dummyPtr) < 0) { virReportOOMError(); goto cleanup; } or something similar. What if we just move the OOM reporting into VIR_ALLOC? It would have three nice features: 1) sizeof(code base) gets lower. A lot lower. 2) even for callers which don't follow the schema described above, there is no harm reporting so serious error in the logs. No matter that the callee may fall back and return success. 3) Removing virReportOOMError() from the schema does not need to be done at once, but can be split into several patches. In the worst case scenario - the error gets reported twice. But before I start working on other areas of code, I want to make sure there's an agreement if this is even desired. As an example, how much the code base will lose on weight, I've done the conversion under src/util/: 30 files changed, 81 insertions(+), 221 deletions(-) Michal Privoznik (2): viralloc: Report OOM error on failure util: Don't report OOM twice src/util/iohelper.c | 4 +--- src/util/viralloc.c | 23 ++++++++++++++++++----- src/util/viralloc.h | 13 ++++++++----- src/util/virauthconfig.c | 8 ++------ src/util/vircommand.c | 13 +++---------- src/util/virconf.c | 10 ++-------- src/util/virdnsmasq.c | 19 +++++++------------ src/util/virerror.c | 2 +- src/util/vireventpoll.c | 4 +--- src/util/virfile.c | 4 +--- src/util/virhash.c | 10 ++-------- src/util/virkeyfile.c | 4 +--- src/util/virlockspace.c | 11 +++-------- src/util/virnetdev.c | 4 +--- src/util/virnetdevbandwidth.c | 12 +++--------- src/util/virnetdevmacvlan.c | 4 ++-- src/util/virnetdevvlan.c | 4 +--- src/util/virnetdevvportprofile.c | 4 +--- src/util/virnetlink.c | 8 ++------ src/util/virobject.c | 9 ++++----- src/util/virpci.c | 13 +++---------- src/util/virprocess.c | 4 +--- src/util/virsexpr.c | 4 +--- src/util/virstoragefile.c | 24 ++++++------------------ src/util/virstring.c | 5 +++-- src/util/virsysinfo.c | 8 +++----- src/util/virthreadpool.c | 21 +++++---------------- src/util/virtime.c | 8 ++------ src/util/virtypedparam.c | 36 +++++++++--------------------------- src/util/viruri.c | 2 +- src/util/virusb.c | 8 ++------ src/util/virutil.c | 36 ++++++++---------------------------- src/util/virxml.c | 1 - 33 files changed, 108 insertions(+), 232 deletions(-) -- 1.8.1.5

4 7

[libvirt] [PATCH] storage: fix unlikely memory leak in rbd backend
by Laine Stump 23 Mar '13

23 Mar '13

virStorageBackendRBDRefreshPool() first allocates an array big enough to hold 1024 names, then calls rbd_list(), which returns ERANGE if the array isn't big enough. When that happens, the VIR_ALLOC_N is called again with a larger size. Unfortunately, the original array isn't freed before allocating a new one. --- src/storage/storage_backend_rbd.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/storage/storage_backend_rbd.c b/src/storage/storage_backend_rbd.c index 8a0e517..e815192 100644 --- a/src/storage/storage_backend_rbd.c +++ b/src/storage/storage_backend_rbd.c @@ -317,6 +317,7 @@ static int virStorageBackendRBDRefreshPool(virConnectPtr conn ATTRIBUTE_UNUSED, VIR_WARN("%s", _("A problem occurred while listing RBD images")); goto cleanup; } + VIR_FREE(names); } for (i = 0, name = names; name < names + max_size; i++) { -- 1.7.11.7

3 3

[libvirt] [PATCH] Make virsh support '~' and '$HOME' in interactive mode
by Zhang Xiaohe 23 Mar '13

23 Mar '13

This patch makes '~' and '$HOME' can be recognized by virsh in interactive mode. These two variables are replaced with real path. eg: virsh # pwd /home/libvirt virsh # cd ~/rpmbuild virsh # pwd /root/rpmbuild see https://bugzilla.redhat.com/show_bug.cgi?id=806793 Signed-off-by: Zhang Xiaohe <zhangxh(a)cn.fujitsu.com> --- tools/virsh.c | 25 +++++++++++++++++++++++++ 1 files changed, 25 insertions(+), 0 deletions(-) diff --git a/tools/virsh.c b/tools/virsh.c index b574d7e..5c8df6b 100644 --- a/tools/virsh.c +++ b/tools/virsh.c @@ -1232,6 +1232,27 @@ vshCmddefHelp(vshControl *ctl, const char *cmdname) * --------------- */ static void +vshExpandPath(vshControl *ctl, char **tkdata) +{ + char *argstr = NULL; + char *buf = NULL; + char *p = NULL; + const char *home = getenv("HOME"); + size_t len = strlen(home) + strlen(*tkdata); + + buf = vshMalloc(ctl, len); + p = buf; + buf = virStrcpy(buf, home, len); + argstr = strchr(*tkdata, '/'); + if (argstr) { + buf += strlen(home); + buf = virStrcpy(buf, argstr, strlen(*tkdata)); + } + VIR_FREE(*tkdata); + *tkdata = p; +} + +static void vshCommandOptFree(vshCmdOpt * arg) { vshCmdOpt *a = arg; @@ -1855,6 +1876,10 @@ get_data: /* save option */ vshCmdOpt *arg = vshMalloc(ctl, sizeof(vshCmdOpt)); + /* replace the ~ or $HOME with real path */ + if (tkdata[0] == '~' || STRPREFIX(tkdata, "$HOME")) + vshExpandPath(ctl, &tkdata); + arg->def = opt; arg->data = tkdata; arg->next = NULL; -- 1.7.1

4 9

[libvirt] [PATCH v2] nwfilter: probe for inverted ctdir
by Stefan Berger 23 Mar '13

23 Mar '13

Linux netfilter at some point inverted the meaning of the '--ctdir reply' and newer netfilter implementations now expect '--ctdir original' instead and vice-versa. We probe for this netfilter change via a UDP message over loopback and 3 filtering rules applied to INPUT. If the sent byte arrives, the newer netfilter implementation has been detected. Signed-off-by: Stefan Berger <stefanb(a)linux.vnet.ibm.com> --- v1->v2: - using virSocketAddrParseIPv4 --- src/nwfilter/nwfilter_ebiptables_driver.c | 121 ++++++++++++++++++++++++++++++ 1 file changed, 121 insertions(+) Index: libvirt-acl/src/nwfilter/nwfilter_ebiptables_driver.c =================================================================== --- libvirt-acl.orig/src/nwfilter/nwfilter_ebiptables_driver.c +++ libvirt-acl/src/nwfilter/nwfilter_ebiptables_driver.c @@ -27,6 +27,10 @@ #include <string.h> #include <sys/stat.h> #include <fcntl.h> +#include <arpa/inet.h> +#include <sys/select.h> +#include <sys/time.h> +#include <unistd.h> #include "internal.h" @@ -85,6 +89,12 @@ static char *iptables_cmd_path; static char *ip6tables_cmd_path; static char *grep_cmd_path; +/* + * --ctdir original vs. --ctdir reply's meaning was inverted in netfilter + * at some point. We probe for it. + */ +static bool iptables_ctdir_corrected = false; + #define PRINT_ROOT_CHAIN(buf, prefix, ifname) \ snprintf(buf, sizeof(buf), "libvirt-%c-%s", prefix, ifname) #define PRINT_CHAIN(buf, prefix, ifname, suffix) \ @@ -1262,6 +1272,9 @@ iptablesEnforceDirection(int directionIn virNWFilterRuleDefPtr rule, virBufferPtr buf) { + if (iptables_ctdir_corrected) + directionIn = !directionIn; + if (rule->tt != VIR_NWFILTER_RULE_DIRECTION_INOUT) virBufferAsprintf(buf, " -m conntrack --ctdir %s", (directionIn) ? "Original" @@ -4304,6 +4317,111 @@ ebiptablesDriverTestCLITools(void) return ret; } +static void +ebiptablesDriverProbeCtdir(void) +{ + virBuffer buf = VIR_BUFFER_INITIALIZER; + static const char cmdline[] = + "$IPT -%c INPUT %c -i lo -p udp --dport %hu " + "-m state --state ESTABLISHED -j ACCEPT " CMD_SEPARATOR + "$IPT -%c INPUT %c -i lo -p udp --dport %hu " + "-m conntrack --ctdir original -j ACCEPT " CMD_SEPARATOR + "$IPT -%c INPUT %c -i lo -p udp --dport %hu -j DROP"; + /* + * Above '--ctdir original' gets this test to receive a message on + * 'fixed' netfilter. + */ + unsigned short port; + int ssockfd = -1, csockfd = -1; + virSocketAddr saddr; + struct sockaddr_in *serveraddr = &saddr.data.inet4; + fd_set readfds; + struct timeval timeout = { + .tv_sec = 0, + .tv_usec = 1000 * 200, + }; + int n; + + if (virSocketAddrParseIPv4(&saddr, "127.0.0.1") < 0) { + VIR_ERROR(_("Could not parse IP address")); + goto cleanup; + } + + if ((ssockfd = socket(AF_INET, SOCK_DGRAM, 0)) < 0 || + (csockfd = socket(AF_INET, SOCK_DGRAM, 0)) < 0) { + VIR_ERROR(_("Could not open UDP socket")); + goto cleanup; + } + + for (port = 0xffff; port > 1024; port--) { + serveraddr->sin_port = htons(port); + if (bind(ssockfd, (struct sockaddr *)serveraddr, + sizeof(*serveraddr)) == 0) + break; + } + if (port == 1024) { + VIR_ERROR(_("Could not bind to any UDP socket")); + goto cleanup; + } + + NWFILTER_SET_IPTABLES_SHELLVAR(&buf); + virBufferAsprintf(&buf, cmdline, + 'I', '1', port, + 'I', '2', port, + 'I', '3', port); + + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + + if (ebiptablesExecCLI(&buf, NULL, NULL) < 0) { + VIR_ERROR(_("Could not apply iptables rules")); + goto cleanup_iptables; + } + + if (sendto(csockfd, cmdline, 1, 0, (struct sockaddr *)serveraddr, + sizeof(*serveraddr)) < 0) { + VIR_ERROR(_("Could not send to UDP socket")); + goto cleanup_iptables; + } + + FD_ZERO(&readfds); + FD_SET(ssockfd, &readfds); + + while (true) { + n = select(ssockfd + 1, &readfds, NULL, NULL, &timeout); + if (n < 0) { + if (errno == EINTR) + continue; + VIR_ERROR(_("Select failed")); + goto cleanup_iptables; + } + if (n == 0) { + VIR_INFO("Ctdir probing received no data -- 'old' netfilter"); + goto cleanup_iptables; + } + VIR_INFO("Ctdir probing received data -- 'fixed' netfilter"); + iptables_ctdir_corrected = true; + break; + } + +cleanup_iptables: + virBufferFreeAndReset(&buf); + + NWFILTER_SET_IPTABLES_SHELLVAR(&buf); + virBufferAsprintf(&buf, cmdline, + 'D', ' ', port, + 'D', ' ', port, + 'D', ' ', port); + ebiptablesExecCLI(&buf, NULL, NULL); + +cleanup: + virBufferFreeAndReset(&buf); + VIR_FORCE_CLOSE(ssockfd); + VIR_FORCE_CLOSE(csockfd); +} + static int ebiptablesDriverInit(bool privileged) { @@ -4341,6 +4459,9 @@ ebiptablesDriverInit(bool privileged) return -ENOTSUP; } + if (iptables_cmd_path) + ebiptablesDriverProbeCtdir(); + ebiptables_driver.flags = TECHDRV_FLAG_INITIALIZED; return 0;

3 6

[libvirt] [PATCH 0/5] Refactoring of cgroups usage
by Daniel P. Berrange 23 Mar '13

23 Mar '13

Currently libvirtd creates its basic cgroup hierarchy when it starts up. This is bad because if people mount cgroups after libvirtd it is running it doesn't detect this. The second issue is that the driver code re-creates the virCgroupPtr instance for a domain over & over again, which is inefficient and bloats the code. Finally, if the driver is configured to only use a couple of the cgroups controllers, we are still creating the directories under all of them

2 6

[libvirt] [PATCH] Fix free of uninitialized value in LXC numad setup
by Daniel P. Berrange 23 Mar '13

23 Mar '13

From: "Daniel P. Berrange" <berrange(a)redhat.com> The 'nodeset' variable was never initialized, causing a later VIR_FREE(nodeset) to free uninitialized memory. Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com> --- src/lxc/lxc_controller.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) Pushed under trivial rule diff --git a/src/lxc/lxc_controller.c b/src/lxc/lxc_controller.c index 37e3ce9..bb369e2 100644 --- a/src/lxc/lxc_controller.c +++ b/src/lxc/lxc_controller.c @@ -517,7 +517,7 @@ static int virLXCControllerGetNumadAdvice(virLXCControllerPtr ctrl, virBitmapPtr *mask) { virBitmapPtr nodemask = NULL; - char *nodeset; + char *nodeset = NULL; int ret = -1; /* Get the advisory nodeset from numad if 'placement' of -- 1.7.11.7

2 1

[libvirt] [PATCH] Ensure root filesystem is mounted if a file/block mount.
by Daniel P. Berrange 23 Mar '13

23 Mar '13

From: "Daniel P. Berrange" <berrange(a)redhat.com> For a root filesystem with type=file or type=block, the LXC container was forgetting to actually mount it, before doing the pivot root step. Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com> --- src/lxc/lxc_container.c | 52 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c index be9bc6c..002dba1 100644 --- a/src/lxc/lxc_container.c +++ b/src/lxc/lxc_container.c @@ -105,6 +105,9 @@ struct __lxc_child_argv { int handshakefd; }; +static int lxcContainerMountFSBlock(virDomainFSDefPtr fs, + const char *srcprefix); + /* * reboot(LINUX_REBOOT_CMD_CAD_ON) will return -EINVAL @@ -406,6 +409,51 @@ static int lxcContainerChildMountSort(const void *a, const void *b) # define MS_SLAVE (1<<19) #endif +static int lxcContainerPrepareRoot(virDomainDefPtr def, + virDomainFSDefPtr root) +{ + char *dst; + char *tmp; + + if (root->type == VIR_DOMAIN_FS_TYPE_MOUNT) + return 0; + + if (root->type == VIR_DOMAIN_FS_TYPE_FILE) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Unexpected root filesystem without loop device")); + return -1; + } + + if (root->type != VIR_DOMAIN_FS_TYPE_BLOCK) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Unsupported root filesystem type %s"), + virDomainFSTypeToString(root->type)); + return -1; + } + + if (virAsprintf(&dst, "%s/%s.root", + LXC_STATE_DIR, def->name) < 0) { + virReportOOMError(); + return -1; + } + + tmp = root->dst; + root->dst = dst; + + if (lxcContainerMountFSBlock(root, "") < 0) { + root->dst = tmp; + VIR_FREE(dst); + return -1; + } + + root->dst = tmp; + root->type = VIR_DOMAIN_FS_TYPE_MOUNT; + VIR_FREE(root->src); + root->src = dst; + + return 0; +} + static int lxcContainerPivotRoot(virDomainFSDefPtr root) { int ret; @@ -1926,6 +1974,10 @@ static int lxcContainerSetupPivotRoot(virDomainDefPtr vmDef, if (lxcContainerIdentifyCGroups(&mounts, &nmounts, &cgroupRoot) < 0) goto cleanup; + /* Ensure the root filesystem is mounted */ + if (lxcContainerPrepareRoot(vmDef, root) < 0) + goto cleanup; + /* Gives us a private root, leaving all parent OS mounts on /.oldroot */ if (lxcContainerPivotRoot(root) < 0) goto cleanup; -- 1.7.11.7

3 2

[libvirt] [PATCH] Mount temporary devpts on /var/lib/libvirt/lxc/$NAME.devpts
by Daniel P. Berrange 23 Mar '13

23 Mar '13

From: "Daniel P. Berrange" <berrange(a)redhat.com> Currently the lxc controller sets up the devpts instance on $rootfsdef->src, but this only works if $rootfsdef is using type=mount. To support type=block or type=file for the root filesystem, we must use /var/lib/libvirt/lxc/$NAME.devpts for the temporary devpts mount in the controller --- src/lxc/lxc_container.c | 43 +++++++++++++++++++++++++------------------ src/lxc/lxc_controller.c | 13 ++++--------- 2 files changed, 29 insertions(+), 27 deletions(-) diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c index bf17a38..be9bc6c 100644 --- a/src/lxc/lxc_container.c +++ b/src/lxc/lxc_container.c @@ -627,15 +627,17 @@ static int lxcContainerMountProcFuse(virDomainDefPtr def ATTRIBUTE_UNUSED, } #endif -static int lxcContainerMountFSDevPTS(virDomainFSDefPtr root) +static int lxcContainerMountFSDevPTS(virDomainDefPtr def, + const char *srcprefix) { - char *devpts = NULL; - int rc = -1; + int ret; + char *path = NULL; - if (virAsprintf(&devpts, "/.oldroot%s/dev/pts", root->src) < 0) { - virReportOOMError(); - goto cleanup; - } + if ((ret = virAsprintf(&path, + "%s/%s/%s.devpts", + srcprefix ? srcprefix : "", LXC_STATE_DIR, + def->name)) < 0) + return ret; if (virFileMakePath("/dev/pts") < 0) { virReportSystemError(errno, "%s", @@ -643,19 +645,20 @@ static int lxcContainerMountFSDevPTS(virDomainFSDefPtr root) goto cleanup; } - VIR_DEBUG("Trying to move %s to %s", devpts, "/dev/pts"); - if ((rc = mount(devpts, "/dev/pts", NULL, MS_MOVE, NULL)) < 0) { - virReportSystemError(errno, "%s", - _("Failed to mount /dev/pts in container")); + VIR_DEBUG("Trying to move %s to /dev/pts", path); + + if ((ret = mount(path, "/dev/pts", + NULL, MS_MOVE, NULL)) < 0) { + virReportSystemError(errno, + _("Failed to mount %s on /dev/pts"), + path); goto cleanup; } - rc = 0; - - cleanup: - VIR_FREE(devpts); +cleanup: + VIR_FREE(path); - return rc; + return ret; } static int lxcContainerPopulateDevices(char **ttyPaths, size_t nttyPaths) @@ -1961,7 +1964,7 @@ static int lxcContainerSetupPivotRoot(virDomainDefPtr vmDef, goto cleanup; /* Mounts /dev/pts */ - if (lxcContainerMountFSDevPTS(root) < 0) + if (lxcContainerMountFSDevPTS(vmDef, "/.oldroot") < 0) goto cleanup; /* Populates device nodes in /dev/ */ @@ -2204,7 +2207,11 @@ static int lxcContainerChild(void *data) if (argv->nttyPaths) { if (root) { - if (virAsprintf(&ttyPath, "%s%s", root->src, argv->ttyPaths[0]) < 0) { + const char *tty = argv->ttyPaths[0]; + if (STRPREFIX(tty, "/dev/pts/")) + tty += strlen("/dev/pts/"); + if (virAsprintf(&ttyPath, "%s/%s.devpts/%s", + LXC_STATE_DIR, vmDef->name, tty) < 0) { virReportOOMError(); goto cleanup; } diff --git a/src/lxc/lxc_controller.c b/src/lxc/lxc_controller.c index bb369e2..1d1443c 100644 --- a/src/lxc/lxc_controller.c +++ b/src/lxc/lxc_controller.c @@ -1219,15 +1219,10 @@ virLXCControllerSetupDevPTS(virLXCControllerPtr ctrl) mount_options = virSecurityManagerGetMountOptions(ctrl->securityManager, ctrl->def); - if (!virFileExists(root->src)) { - virReportSystemError(errno, - _("root source %s does not exist"), - root->src); - goto cleanup; - } - - if (virAsprintf(&devpts, "%s/dev/pts", root->src) < 0 || - virAsprintf(&ctrl->devptmx, "%s/dev/pts/ptmx", root->src) < 0) { + if (virAsprintf(&devpts, "%s/%s.devpts", + LXC_STATE_DIR, ctrl->def->name) < 0 || + virAsprintf(&ctrl->devptmx, "%s/%s.devpts/ptmx", + LXC_STATE_DIR, ctrl->def->name) < 0) { virReportOOMError(); goto cleanup; } -- 1.7.11.7

3 2