April 2009 - Devel - libvirt List Archives

[libvirt] [PATCH] lxc: make the pivot_root more robust.
by Serge E. Hallyn 07 Apr '09

07 Apr '09

libvirt/lxc is broken on F11. The pivot_root() call returns -EINVAL. The below is one way we can fix it. I'm also sending another patch which takes the simpler approach of using chroot. However, chroot is trivially escapable (see for instance http://www.linuxsecurity.com/content/view/117632/49/) I don't know whether the typical libvirt user would care. If so, then the below patch is probably the way to go. >From 26cac415771a2d9712af0e1ce60a0bcb41b44665 Mon Sep 17 00:00:00 2001 From: root <root(a)localhost.localdomain> Date: Sat, 4 Apr 2009 22:49:20 -0400 Subject: [PATCH 1/1] lxc: make the pivot_root more robust. The libvirt lxc driver uses pivot_root instead of chroot, because the latter is trivially escapable. However, the pivot_root(2) system call can fail for several subtle reasons. Depending upon your distro init sequence you may get lucky and have the old recipe work, but on a Fedora 11 standard install, for instance, it will fail. Do a few more steps to make pivot_root hopefully always succeed. We mark / as MS_PRIVATE, create an empty tmpfs, and bind-mount the container root onto /new in that fs. In this way, we ensure two reasons for pivot_root to fail - namely old_root->parent being MS_SHARED and old_root and new_root being on the same fs - won't happen. Signed-off-by: Serge Hallyn <serue(a)us.ibm.com> --- src/lxc_container.c | 108 ++++++++++++++++++++++++++++++++++++++++----------- 1 files changed, 85 insertions(+), 23 deletions(-) diff --git a/src/lxc_container.c b/src/lxc_container.c index 3f17b8d..d3959f6 100644 --- a/src/lxc_container.c +++ b/src/lxc_container.c @@ -264,50 +264,113 @@ static int lxcContainerChildMountSort(const void *a, const void *b) return strcmp(*sb, *sa); } +#ifndef MS_REC +#define MS_REC 16384 +#endif + +#ifndef MNT_DETACH +#define MNT_DETACH 0x00000002 +#endif + +#ifndef MS_PRIVATE +#define MS_PRIVATE 1<<18 +#endif + static int lxcContainerPivotRoot(virDomainFSDefPtr root) { int rc; - char *oldroot; + char *oldroot = NULL, *newroot = NULL; - /* First step is to ensure the new root itself is - a mount point */ - if (mount(root->src, root->src, NULL, MS_BIND, NULL) < 0) { - virReportSystemError(NULL, errno, - _("failed to bind new root %s"), - root->src); - return -1; + /* root->parent must be private, so make / private. */ + if (mount("", "/", NULL, MS_PRIVATE|MS_REC, NULL) < 0) { + virReportSystemError(NULL, errno, "%s", + _("failed to make root private")); + goto err; } if (virAsprintf(&oldroot, "%s/.oldroot", root->src) < 0) { virReportOOMError(NULL); - return -1; + goto err; } if ((rc = virFileMakePath(oldroot)) < 0) { virReportSystemError(NULL, rc, _("failed to create %s"), oldroot); - VIR_FREE(oldroot); - return -1; + goto err; + } + + /* Create a tmpfs root since old and new roots must be + * on separate filesystems */ + if (mount("", oldroot, "tmpfs", 0, NULL) < 0) { + virReportSystemError(NULL, errno, + _("failed to mount empty tmpfs at %s"), + oldroot); + goto err; + } + + /* Create a directory called 'new' in tmpfs */ + if (virAsprintf(&newroot, "%s/new", oldroot) < 0) { + virReportOOMError(NULL); + goto err; + } + + if ((rc = virFileMakePath(newroot)) < 0) { + virReportSystemError(NULL, rc, + _("failed to create %s"), + newroot); + goto err; + } + + /* ... and mount our root onto it */ + if (mount(root->src, newroot, NULL, MS_BIND|MS_REC, NULL) < 0) { + virReportSystemError(NULL, errno, + _("failed to bind new root %s into tmpfs"), + root->src); + goto err; + } + + /* Now we chroot into the tmpfs, then pivot into the + * root->src bind-mounted onto '/new' */ + if (chroot(oldroot) < 0) { + virReportSystemError(NULL, errno, "%s", + _("failed to chroot into tmpfs")); + goto err; + } + + if (chdir("/new") < 0) { + virReportSystemError(NULL, errno, "%s", + _("failed to chdir into /new on tmpfs")); + goto err; } /* The old root directory will live at /.oldroot after * this and will soon be unmounted completely */ - if (pivot_root(root->src, oldroot) < 0) { - virReportSystemError(NULL, errno, - _("failed to pivot root %s to %s"), - oldroot, root->src); - VIR_FREE(oldroot); - return -1; + if (pivot_root(".", ".oldroot") < 0) { + virReportSystemError(NULL, errno, "%s", + _("failed to pivot root")); + goto err; } - VIR_FREE(oldroot); /* CWD is undefined after pivot_root, so go to / */ - if (chdir("/") < 0) { - return -1; + if (chdir("/") < 0) + goto err; + + if (umount2(".oldroot", MNT_DETACH) < 0) { + virReportSystemError(NULL, errno, "%s", + _("failed to lazily unmount old root")); + goto err; } + VIR_FREE(oldroot); + VIR_FREE(newroot); + return 0; + +err: + if (oldroot) VIR_FREE(oldroot); + if (newroot) VIR_FREE(newroot); + return -1; } static int lxcContainerPopulateDevices(void) @@ -349,10 +412,9 @@ static int lxcContainerPopulateDevices(void) _("cannot create /dev/pts")); return -1; } - if (mount("/.oldroot/dev/pts", "/dev/pts", NULL, - MS_MOVE, NULL) < 0) { + if (mount("devpts", "/dev/pts", "devpts", 0, NULL) < 0) { virReportSystemError(NULL, errno, "%s", - _("failed to move /dev/pts into container")); + _("failed to mount /dev/pts in container")); return -1; } -- 1.6.2

2 1

[libvirt] [PATCH] lxc: use chroot instead of pivot_root
by Serge E. Hallyn 06 Apr '09

06 Apr '09

This is an alternative to the pivot_root patch which I just sent. It has the advantage of being much simpler. It also won't have a problem with the container's / being a read-only mount. It has the disadvantage, of course, of being escapable. >From a91bca7f60f27e8fbdb4e3bacf3232a6cbb630d3 Mon Sep 17 00:00:00 2001 From: root <root(a)localhost.localdomain> Date: Fri, 3 Apr 2009 23:35:24 -0400 Subject: [PATCH 1/1] lxc: use chroot instead of pivot_root pivot_root is too fragile. For instance, if the container's / is read-only, we can't create .oldroot. Maybe we're happy telling users that if they can't create .oldroot at container create time, then they must make sure it exists ahead of time. Or, maybe we're ok with chroot being escapable, and should just go this simple route. Signed-off-by: Serge Hallyn <serue(a)us.ibm.com> --- src/lxc_container.c | 75 +++++++++++++-------------------------------------- 1 files changed, 19 insertions(+), 56 deletions(-) diff --git a/src/lxc_container.c b/src/lxc_container.c index 3f17b8d..142ed4d 100644 --- a/src/lxc_container.c +++ b/src/lxc_container.c @@ -264,52 +264,6 @@ static int lxcContainerChildMountSort(const void *a, const void *b) return strcmp(*sb, *sa); } -static int lxcContainerPivotRoot(virDomainFSDefPtr root) -{ - int rc; - char *oldroot; - - /* First step is to ensure the new root itself is - a mount point */ - if (mount(root->src, root->src, NULL, MS_BIND, NULL) < 0) { - virReportSystemError(NULL, errno, - _("failed to bind new root %s"), - root->src); - return -1; - } - - if (virAsprintf(&oldroot, "%s/.oldroot", root->src) < 0) { - virReportOOMError(NULL); - return -1; - } - - if ((rc = virFileMakePath(oldroot)) < 0) { - virReportSystemError(NULL, rc, - _("failed to create %s"), - oldroot); - VIR_FREE(oldroot); - return -1; - } - - /* The old root directory will live at /.oldroot after - * this and will soon be unmounted completely */ - if (pivot_root(root->src, oldroot) < 0) { - virReportSystemError(NULL, errno, - _("failed to pivot root %s to %s"), - oldroot, root->src); - VIR_FREE(oldroot); - return -1; - } - VIR_FREE(oldroot); - - /* CWD is undefined after pivot_root, so go to / */ - if (chdir("/") < 0) { - return -1; - } - - return 0; -} - static int lxcContainerPopulateDevices(void) { int i; @@ -349,10 +303,9 @@ static int lxcContainerPopulateDevices(void) _("cannot create /dev/pts")); return -1; } - if (mount("/.oldroot/dev/pts", "/dev/pts", NULL, - MS_MOVE, NULL) < 0) { + if (mount("devpts", "/dev/pts", "devpts", 0, NULL) < 0) { virReportSystemError(NULL, errno, "%s", - _("failed to move /dev/pts into container")); + _("failed to mount /dev/pts in container")); return -1; } @@ -461,15 +414,25 @@ static int lxcContainerUnmountOldFS(void) } -/* Got a FS mapped to /, we're going the pivot_root - * approach to do a better-chroot-than-chroot - * this is based on this thread http://lkml.org/lkml/2008/3/5/29 - */ -static int lxcContainerSetupPivotRoot(virDomainDefPtr vmDef, +/* Got a FS mapped to /, now chroot. + * pivot_root would work, but requires too much hand-holding + * (especially, old_root->parent must not be shared) */ +static int lxcContainerSetupChroot(virDomainDefPtr vmDef, virDomainFSDefPtr root) { - if (lxcContainerPivotRoot(root) < 0) + if (chdir(root->src) < 0) { + virReportSystemError(NULL, errno, + _("failed to chdir to %s"), + root->src); + return -1; + } + + if (chroot(root->src) < 0) { + virReportSystemError(NULL, errno, + _("failed to chroot to %s"), + root->src); return -1; + } if (virFileMakePath("/proc") < 0 || mount("none", "/proc", "proc", 0, NULL) < 0) { @@ -537,7 +500,7 @@ static int lxcContainerSetupMounts(virDomainDefPtr vmDef) } if (root) - return lxcContainerSetupPivotRoot(vmDef, root); + return lxcContainerSetupChroot(vmDef, root); else return lxcContainerSetupExtraMounts(vmDef); } -- 1.6.2

1 0

[libvirt] VM cpuTime discrepancy
by Zvi Dubitzky 06 Apr '09

06 Apr '09

The cpuTime of a VM reported by kvm72 is ok (real seconds ) while that reported by kvm-84 is not. Are you aware of this . Did you talk to kvm and was it fixed in latest kvm releases since 84 ? I access cpuTime via libvirt . (same version in both cases) . thanks Zvi Dubitzky Virtualization and System Architecture Email:dubi@il.ibm.com IBM Haifa Research Laboratory Phone: +972-4-8296182 Haifa, 31905, ISRAEL

1 0

[libvirt] libvirt.org down
by Daniel Veillard 06 Apr '09

06 Apr '09

As Jim found out there is a problem on the libvirt.org box, I don't know exactly what's is going on, and I will go anc check the box tomorrow at 7am EST. Sorry for the inconvenience, and thanks Jim for raising the issue ! Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel(a)veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/

1 1

[libvirt] win32
by Brecht Sanders 06 Apr '09

06 Apr '09

Hi, Is there any way to compile libvirt with xen some support without depending on libxenstore? The reason I ask is that libxen can be built on Windows, but libxenstore seems to depend on the linux kernel, so I doubt it can be built for Windows. Regards Brecht Sanders

3 3

[libvirt] [PATCH] Openvz: how to use a nondefault config
by Anton Protopopov 06 Apr '09

06 Apr '09

2009/4/2 Daniel P. Berrange <berrange(a)redhat.com> > On Thu, Apr 02, 2009 at 06:36:15PM +0400, Evgeniy Sokolov wrote: > > > > Config /etc/vz/conf/ve-XXX.conf-sample define there are many parameters > > like memory, CPU etc. This parameters are not properties of > <filesystem>. > > > > Daniel, usage of pre-defined configs in really usefull. Can we create > > additional tag inside <domain></domain>? > > The trouble with this is that given the XML config of a guest, you can > no longer determine what its configuration is, because a large amount > of configuration is hidden in this opaque 'template'. And in kvm/qemu case there is a lot of hidden information too :) In OpenVZ case we don't loose any info (see below). > > If we're to support templates, I think we need to make them first class > objects, so you can query what templates exist, and see their associated > config settings. > > eg, expose a simple API > > virDomainTemplatePtr; > > int virConnectListDomainTemplates(virConnectPtr conn, const char > **names); > virDomainTemplatePtr virDomainTemplateLookupbyName(virConnectPtr con, > const char *name) > char *virDomainTemplateGetXMLDesc(virDomainTemplatePtr tmpl); > virDomainTemplatePtr virDomainTemplateDefineXML(virConnectPtr conn, const > char *xml) > int virDomainTemplateUndefine(virDomainTemplatePtr tmpl); > int virDomainTemplateFree(virDomainTemplatePtr tmpl); > > > Then, we could either include a <template>$NAME</template> parameter in > the XML for a guest. Or have an alternate API for creating guests > > virDomainCreateXMLWithTemplate(virConnectPtr conn, > const char *xmlDesc, > const char *template) > > It depends whether we'd want to track the template <-> guest association > forever, or just use it once to fill-in the guest XML and then forget > about the template after that. It doesn't matter in OpenVZ case: template (or, "config", in openvz terminology) is used only once, when container is creating. But then per-container "copy" is created, and it is already used by libvirt to keep information about container. Per-container OpenVZ config keeps the name of the original config (in ORIGIN_SAMPLE variable). Is there need for "Template API" in case of other hypervisors?

2 1

[libvirt] Planning and schedule
by Daniel Veillard 06 Apr '09

06 Apr '09

Trying to keep up the promises from last week: - so 0.6.2 is released and out - I still plan for a new release 0.6.3 around Fri 24 April - which mean we should have some kind of feature commit freeze around the 17th April - and the week 20 - 24 focusing mostly on bug fixes Plannned so far for 0.6.3 are: - API for physical host interface - the VirtualBox driver if in shape and in time but that's not an exhaustive list and there is a couple of drivers submitted I need to look at (OpenNebula for example). And I will provide an update next Friday, Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel(a)veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/

2 1

[libvirt] pypxeboot - libvirt
by Gian Mario 06 Apr '09

06 Apr '09

Hi to all, Can anyone tell me on which version libvirt start to support pypxeboot? tnks so much. GMario

1 0

[libvirt] freecell gives out bytes instead of kilobytes
by Gerrit Slomma 03 Apr '09

03 Apr '09

The virsh-command freecell hands out bytes but affixes those with kB. The error ist found in virsh.c on line 1663 and following. I have corrected this and altered the output to the method i chose for virt-manager - in the days of 96 GB per socket (Nehalem-EP) no one cares even about a fraction of a Megabyte. --- a/src/virsh.c 2009-04-02 17:45:50.000000000 +0200 +++ b/src/virsh.c 2009-04-02 23:35:54.000000000 +0200 @@ -1661,10 +1661,23 @@ } if (cell == -1) - vshPrint(ctl, "%s: %llu kB\n", _("Total"), memory); + if (memory < 10*1024*1204*1024) + { + vshPrint(ctl, "%s: %.0f MB\n", _("Total"), (float)memory/1024/1024); + } + else + { + vshPrint(ctl, "%s: %.2f GB\n", _("Total"), (float)memory/1024/1024/1024); + } else - vshPrint(ctl, "%d: %llu kB\n", cell, memory); - + if (memory < 10*1024*1204*1024) + { + vshPrint(ctl, "%d: %.0f MB\n", cell, (float)memory/1024/1024); + } + else + { + vshPrint(ctl, "%d: %.2f GB\n", cell, (float)memory/1024/1024/1024); + } return TRUE; } eg: rr016# virsh freecell; free -m 0: 24 MB total used free shared buffers cached Mem: 3884 3859 24 0 60 1306 -/+ buffers/cache: 2492 1392 Swap: 2047 248 1798 rr017# virsh freecell; free -m 0: 1490 MB total used free shared buffers cached Mem: 2991 1501 1489 0 100 1283 -/+ buffers/cache: 117 2873 Swap: 2047 0 2047 rr019# virsh freecell; free -m 0: 1616 MB total used free shared buffers cached Mem: 3952 2337 1615 0 5 1209 -/+ buffers/cache: 1121 2831 Swap: 2047 0 2047 Is this okay with you? Or just the fix for the output removing the kB-suffix.

4 5

[libvirt] [PATCH] Don't squash file format on volume refresh
by Cole Robinson 03 Apr '09

03 Apr '09

A while back, code was added to storage_backend.c:virStorageBackendUpdateVolTargetInfoFD to try to determine the volume format for iscsi volumes. Unfortunately this function is also called for refreshing file volumes, which has the effect of setting the format as 'raw' whenever a volume is refreshed (ex. after calling virsh vol-info). The attached patch moves the offending code into a wrapper function in the scsi driver. I haven't played with the scsi support yet so this is untested though it's largely code movement. Thanks, Cole

2 2