- Devel - libvirt List Archives

[libvirt] [PATCH] cgroup: Enable memory.use_hierarchy of cgroup for domain
by Ryota Ozaki 06 May '10

06 May '10

Through conversation with Kumar L Srikanth-B22348, I found that the function of getting memory usage (e.g., virsh dominfo) doesn't work for lxc with ns subsystem of cgroup enabled. This is because of features of ns and memory subsystems. Ns creates child cgroup on every process fork and as a result processes in a container are not assigned in a cgroup for domain (e.g., libvirt/lxc/test1/). For example, libvirt_lxc and init (or somewhat specified in XML) are assigned into libvirt/lxc/test1/8839/ and libvirt/lxc/test1/8839/8849/, respectively. On the other hand, memory subsystem accounts memory usage within a group of processes by default, i.e., it does not take any child (and descendent) groups into account. With the two features, virsh dominfo which just checks memory usage of a cgroup for domain always returns zero because the cgroup has no process. Setting memory.use_hierarchy of a group allows to account (and limit) memory usage of every descendent groups of the group. By setting it of a cgroup for domain, we can get proper memory usage of lxc with ns subsystem enabled. (To be exact, the setting is required only when memory and ns subsystems are enabled at the same time, e.g., mount -t cgroup none /cgroup.) --- src/util/cgroup.c | 63 +++++++++++++++++++++++++++++++++++++++++++++++++--- 1 files changed, 59 insertions(+), 4 deletions(-) diff --git a/src/util/cgroup.c b/src/util/cgroup.c index b8b2eb5..f7d6b41 100644 --- a/src/util/cgroup.c +++ b/src/util/cgroup.c @@ -443,7 +443,38 @@ static int virCgroupCpuSetInherit(virCgroupPtr parent, virCgroupPtr group) return rc; } -static int virCgroupMakeGroup(virCgroupPtr parent, virCgroupPtr group, int create) +static int virCgroupSetMemoryUseHierarchy(virCgroupPtr group) +{ + int rc = 0; + unsigned long long value; + const char *filename = "memory.use_hierarchy"; + + rc = virCgroupGetValueU64(group, + VIR_CGROUP_CONTROLLER_MEMORY, + filename, &value); + if (rc != 0) { + VIR_ERROR("Failed to read %s/%s (%d)", group->path, filename, rc); + return rc; + } + + /* Setting twice causes error, so if already enabled, skip setting */ + if (value == 1) + return 0; + + VIR_DEBUG("Setting up %s/%s", group->path, filename); + rc = virCgroupSetValueU64(group, + VIR_CGROUP_CONTROLLER_MEMORY, + filename, 1); + + if (rc != 0) { + VIR_ERROR("Failed to set %s/%s (%d)", group->path, filename, rc); + } + + return rc; +} + +static int virCgroupMakeGroup(virCgroupPtr parent, virCgroupPtr group, + int create, int memory_hierarchy) { int i; int rc = 0; @@ -477,6 +508,20 @@ static int virCgroupMakeGroup(virCgroupPtr parent, virCgroupPtr group, int creat break; } } + /* + * Note that virCgroupSetMemoryUseHierarchy should always be + * called prior to creating subcgroups and attaching tasks. + */ + if (memory_hierarchy && + group->controllers[VIR_CGROUP_CONTROLLER_MEMORY].mountPoint != NULL && + (i == VIR_CGROUP_CONTROLLER_MEMORY || + STREQ(group->controllers[i].mountPoint, group->controllers[VIR_CGROUP_CONTROLLER_MEMORY].mountPoint))) { + rc = virCgroupSetMemoryUseHierarchy(group); + if (rc != 0) { + VIR_FREE(path); + break; + } + } } VIR_FREE(path); @@ -553,7 +598,7 @@ static int virCgroupAppRoot(int privileged, if (rc != 0) goto cleanup; - rc = virCgroupMakeGroup(rootgrp, *group, create); + rc = virCgroupMakeGroup(rootgrp, *group, create, 0); cleanup: virCgroupFree(&rootgrp); @@ -653,7 +698,7 @@ int virCgroupForDriver(const char *name, VIR_FREE(path); if (rc == 0) { - rc = virCgroupMakeGroup(rootgrp, *group, create); + rc = virCgroupMakeGroup(rootgrp, *group, create, 0); if (rc != 0) virCgroupFree(group); } @@ -703,7 +748,17 @@ int virCgroupForDomain(virCgroupPtr driver, VIR_FREE(path); if (rc == 0) { - rc = virCgroupMakeGroup(driver, *group, create); + /* + * Create a cgroup with memory.use_hierarchy enabled to + * surely account memory usage of lxc with ns subsystem + * enabled. (To be exact, memory and ns subsystems are + * enabled at the same time.) + * + * The reason why doing it here, not a upper group, say + * a group for driver, is to avoid overhead to track + * cumulative usage that we don't need. + */ + rc = virCgroupMakeGroup(driver, *group, create, 1); if (rc != 0) virCgroupFree(group); } -- 1.6.5.2

1 0

[libvirt] [PATCH] cgroup: Enable memory.use_hierarchy of cgroup for domain
by Ryota Ozaki 06 May '10

06 May '10

Through conversation with Kumar L Srikanth-B22348, I found that the function of getting memory usage (e.g., virsh dominfo) doesn't work for lxc with ns subsystem of cgroup enabled. This is because of features of ns and memory subsystems. Ns creates child cgroup on every process fork and as a result processes in a container are not assigned in a cgroup for domain (e.g., libvirt/lxc/test1/). For example, libvirt_lxc and init (or somewhat specified in XML) are assigned into libvirt/lxc/test1/8839/ and libvirt/lxc/test1/8839/8849/, respectively. On the other hand, memory subsystem accounts memory usage within a group of processes by default, i.e., it does not take any child (and descendent) groups into account. With the two features, virsh dominfo which just checks memory usage of a cgroup for domain always returns zero because the cgroup has no process. Setting memory.use_hierarchy of a group allows to account (and limit) memory usage of every descendent groups of the group. By setting it of a cgroup for domain, we can get proper memory usage of lxc with ns subsystem enabled. (To be exact, the setting is required only when memory and ns subsystems are enabled at the same time, e.g., mount -t cgroup none /cgroup.) --- src/util/cgroup.c | 49 +++++++++++++++++++++++++++++++++++++++++++++---- 1 files changed, 45 insertions(+), 4 deletions(-) diff --git a/src/util/cgroup.c b/src/util/cgroup.c index b8b2eb5..93cd6a9 100644 --- a/src/util/cgroup.c +++ b/src/util/cgroup.c @@ -443,7 +443,38 @@ static int virCgroupCpuSetInherit(virCgroupPtr parent, virCgroupPtr group) return rc; } -static int virCgroupMakeGroup(virCgroupPtr parent, virCgroupPtr group, int create) +static int virCgroupSetMemoryUseHierarchy(virCgroupPtr group) +{ + int rc = 0; + unsigned long long value; + const char *filename = "memory.use_hierarchy"; + + rc = virCgroupGetValueU64(group, + VIR_CGROUP_CONTROLLER_MEMORY, + filename, &value); + if (rc != 0) { + VIR_ERROR("Failed to read %s/%s (%d)", group->path, filename, rc); + return rc; + } + + /* Setting twice causes error, so if already enabled, skip setting */ + if (value == 1) + return 0; + + VIR_DEBUG("Setting up %s/%s", group->path, filename); + rc = virCgroupSetValueU64(group, + VIR_CGROUP_CONTROLLER_MEMORY, + filename, 1); + + if (rc != 0) { + VIR_ERROR("Failed to set %s/%s (%d)", group->path, filename, rc); + } + + return rc; +} + +static int virCgroupMakeGroup(virCgroupPtr parent, virCgroupPtr group, + int create, int memory_hierarchy) { int i; int rc = 0; @@ -477,6 +508,16 @@ static int virCgroupMakeGroup(virCgroupPtr parent, virCgroupPtr group, int creat break; } } + if (memory_hierarchy && + group->controllers[VIR_CGROUP_CONTROLLER_MEMORY].mountPoint != NULL && + (i == VIR_CGROUP_CONTROLLER_MEMORY || + STREQ(group->controllers[i].mountPoint, group->controllers[VIR_CGROUP_CONTROLLER_MEMORY].mountPoint))) { + rc = virCgroupSetMemoryUseHierarchy(group); + if (rc != 0) { + VIR_FREE(path); + break; + } + } } VIR_FREE(path); @@ -553,7 +594,7 @@ static int virCgroupAppRoot(int privileged, if (rc != 0) goto cleanup; - rc = virCgroupMakeGroup(rootgrp, *group, create); + rc = virCgroupMakeGroup(rootgrp, *group, create, 0); cleanup: virCgroupFree(&rootgrp); @@ -653,7 +694,7 @@ int virCgroupForDriver(const char *name, VIR_FREE(path); if (rc == 0) { - rc = virCgroupMakeGroup(rootgrp, *group, create); + rc = virCgroupMakeGroup(rootgrp, *group, create, 0); if (rc != 0) virCgroupFree(group); } @@ -703,7 +744,7 @@ int virCgroupForDomain(virCgroupPtr driver, VIR_FREE(path); if (rc == 0) { - rc = virCgroupMakeGroup(driver, *group, create); + rc = virCgroupMakeGroup(driver, *group, create, 1); if (rc != 0) virCgroupFree(group); } -- 1.6.5.2

3 3

[libvirt] [PATCH] util: fix va_start usage bug
by Eric Blake 06 May '10

06 May '10

Detected by clang. POSIX requires that the second argument to va_start be the name of the last variable; and in some implementations, passing *path instead of path would dereference bogus memory instead of pulling arguments off the stack. * src/util/util.c (virBuildPathInternal): Use correct argument to va_start. --- I think this falls under the trivial rule, as it silences a compiler warning and is a one-line fix of a real bug, so I pushed it. src/util/util.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/src/util/util.c b/src/util/util.c index 2d32952..c44d012 100644 --- a/src/util/util.c +++ b/src/util/util.c @@ -2799,7 +2799,7 @@ int virBuildPathInternal(char **path, ...) va_list ap; int ret = 0; - va_start(ap, *path); + va_start(ap, path); path_component = va_arg(ap, char *); virBufferAdd(&buf, path_component, -1); -- 1.6.6.1

2 1

[libvirt] Libvirtd in listen mode.
by Kumar L Srikanth-B22348 06 May '10

06 May '10

Hi, If I start the libvirtd service using the command "service libvirtd start", it is not starting in listen mode. I am stopping the libvirtd service, and then starting in listen mode externally, using the following command: "libvirtd -listen". But, how can I start libvirtd service in listen mode by default (when I issue the command "service libvirtd start")? Regards, Srikanth.

2 1

[libvirt] [PATCH] build: silence a clang false positive
by Eric Blake 05 May '10

05 May '10

* src/qemu/qemu_monitor.c (qemuMonitorIOWriteWithFD): Work around recent clang shortcoming in analysis. --- False positive encountered with the latest F-13 clang :( src/qemu/qemu_monitor.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index 443d216..abf1338 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -1,7 +1,7 @@ /* * qemu_monitor.c: interaction with QEMU monitor console * - * Copyright (C) 2006-2009 Red Hat, Inc. + * Copyright (C) 2006-2010 Red Hat, Inc. * Copyright (C) 2006 Daniel P. Berrange * * This library is free software; you can redistribute it and/or @@ -374,6 +374,9 @@ qemuMonitorIOWriteWithFD(qemuMonitorPtr mon, msg.msg_controllen = sizeof(control); cmsg = CMSG_FIRSTHDR(&msg); + /* Some static analyzers, like clang 2.6-0.6.pre2, fail to see + that our use of CMSG_FIRSTHDR will not return NULL. */ + sa_assert(cmsg); cmsg->cmsg_len = CMSG_LEN(sizeof(int)); cmsg->cmsg_level = SOL_SOCKET; cmsg->cmsg_type = SCM_RIGHTS; -- 1.6.6.1

2 2

[libvirt] [PATCH] cgroup: Change virCgroupRemove to remove all child groups at first
by Ryota Ozaki 05 May '10

05 May '10

As same as normal directories, a cgroup cannot be removed if it contains sub groups. This patch changes virCgroupRemove to remove all child groups (subdirectories) of a target group before removing the target group. The handling is required when we run lxc with ns subsystem of cgroup. Ns subsystem automatically creates child cgroups on every process forks, but unfortunately the groups are not removed on process exits, so we have to remove them by ourselves. With this patch, such child groups are surely removed at lxc shutdown, i.e., lxcVmCleanup which calls virCgroupRemove. --- src/util/cgroup.c | 48 ++++++++++++++++++++++++++++++++++++++++++++---- 1 files changed, 44 insertions(+), 4 deletions(-) diff --git a/src/util/cgroup.c b/src/util/cgroup.c index b8b2eb5..531c131 100644 --- a/src/util/cgroup.c +++ b/src/util/cgroup.c @@ -23,6 +23,7 @@ #include <sys/stat.h> #include <sys/types.h> #include <libgen.h> +#include <dirent.h> #include "internal.h" #include "util.h" @@ -561,11 +562,52 @@ cleanup: } #endif +static int virCgroupRemoveRecursively(char *grppath) +{ + DIR *grpdir; + struct dirent *ent; + int rc = 0; + + grpdir = opendir(grppath); + if (grpdir == NULL) { + VIR_ERROR("Unable to open %s (%d)", grppath, errno); + rc = -errno; + return rc; + } + + while ((ent = readdir(grpdir)) != NULL) { + char path[PATH_MAX]; + int ret; + + if (ent->d_name[0] == '.') continue; + if (ent->d_type != DT_DIR) continue; + + ret = snprintf(path, sizeof(path), "%s/%s", grppath, ent->d_name); + if (ret < 0) + break; + rc = virCgroupRemoveRecursively(path); + if (rc != 0) + break; + } + DEBUG("Removing cgroup %s", grppath); + if (rmdir(grppath) != 0 && errno != ENOENT) { + rc = -errno; + VIR_ERROR("Unable to remove %s (%d)", grppath, errno); + } + + return rc; +} + /** * virCgroupRemove: * * @group: The group to be removed * + * It first removes all child groups recursively + * in depth first order and then removes @group + * because the presence of the child groups + * prevents removing @group. + * * Returns: 0 on success */ int virCgroupRemove(virCgroupPtr group) @@ -585,10 +627,8 @@ int virCgroupRemove(virCgroupPtr group) &grppath) != 0) continue; - DEBUG("Removing cgroup %s", grppath); - if (rmdir(grppath) != 0 && errno != ENOENT) { - rc = -errno; - } + DEBUG("Removing cgroup %s and all child cgroups", grppath); + rc = virCgroupRemoveRecursively(grppath); VIR_FREE(grppath); } -- 1.6.5.2

1 0

[libvirt] [PATCH v2] rpmbuild: add ebtables & ip(6)tables dependency for rpm
by Stefan Berger 05 May '10

05 May '10

Add ebtables,iptables & iptables-ipv6 dependency to rpm. Changes from V1 to V2: -passing --without-libpcap to configure script, if libpcap is not to be used Signed-off-by: Stefan Berger <stefanb(a)us.ibm.com> --- libvirt.spec.in | 7 +++++++ 1 file changed, 7 insertions(+) Index: libvirt-acl/libvirt.spec.in =================================================================== --- libvirt-acl.orig/libvirt.spec.in +++ libvirt-acl/libvirt.spec.in @@ -61,6 +61,7 @@ %define with_udev 0%{!?_without_udev:0} %define with_hal 0%{!?_without_hal:0} %define with_yajl 0%{!?_without_yajl:0} +%define with_nwfilter 0%{!?_without_nwfilter:0} %define with_libpcap 0%{!?_without_libpcap:0} # Non-server/HV driver defaults which are always enabled @@ -150,6 +151,7 @@ # Enable libpcap library %if %{with_qemu} +%define with_nwfilter 0%{!?_without_nwfilter:%{server_drivers}} %define with_libpcap 0%{!?_without_libpcap:%{server_drivers}} %endif @@ -195,6 +197,11 @@ Requires: bridge-utils Requires: dnsmasq Requires: iptables %endif +%if %{with_nwfilter} +Requires: ebtables +Requires: iptables +Requires: iptables-ipv6 +%endif # needed for device enumeration %if %{with_hal} Requires: hal @@ -517,6 +524,10 @@ of recent versions of Linux (and other O %define _without_yajl --without-yajl %endif +%if ! %{with_libpcap} +%define _without_libpcap --without-libpcap +%endif + %configure %{?_without_xen} \ %{?_without_qemu} \ %{?_without_openvz} \ @@ -545,6 +556,7 @@ of recent versions of Linux (and other O %{?_without_hal} \ %{?_without_udev} \ %{?_without_yajl} \ + %{?_without_libpcap} \ --with-qemu-user=%{qemu_user} \ --with-qemu-group=%{qemu_group} \ --with-init-script=redhat \

2 2

[libvirt] [PATCH] rpmbuild: add ebtables & ip(6)tables dependency for rpm
by Stefan Berger 05 May '10

05 May '10

Add ebtables,iptables & iptables-ipv6 dependency to rpm. Signed-off-by: Stefan Berger <stefanb(a)us.ibm.com> --- libvirt.spec.in | 7 +++++++ 1 file changed, 7 insertions(+) Index: libvirt-acl/libvirt.spec.in =================================================================== --- libvirt-acl.orig/libvirt.spec.in +++ libvirt-acl/libvirt.spec.in @@ -61,6 +61,7 @@ %define with_udev 0%{!?_without_udev:0} %define with_hal 0%{!?_without_hal:0} %define with_yajl 0%{!?_without_yajl:0} +%define with_nwfilter 0%{!?_without_nwfilter:0} %define with_libpcap 0%{!?_without_libpcap:0} # Non-server/HV driver defaults which are always enabled @@ -150,6 +151,7 @@ # Enable libpcap library %if %{with_qemu} +%define with_nwfilter 0%{!?_without_nwfilter:%{server_drivers}} %define with_libpcap 0%{!?_without_libpcap:%{server_drivers}} %endif @@ -195,6 +197,11 @@ Requires: bridge-utils Requires: dnsmasq Requires: iptables %endif +%if %{with_nwfilter} +Requires: ebtables +Requires: iptables +Requires: iptables-ipv6 +%endif # needed for device enumeration %if %{with_hal} Requires: hal

3 3

[libvirt] libvirt with qemu-kvm, not recognizing NIC model
by Jonathan Hoover 05 May '10

05 May '10

Is this the right list for this question, or should I be elsewhere? I am trying to specify a network card "model type" of "pcnet" (to emulate vmware esxi's network card). No matter what I put for model type in my xml config file, it comes up as an Intel e1000. I ran "qemu-kvm -net nic,model=? /dev/null" and I got back a list of supported models including pcnet,e1000,virtio, and others as expected. No matter which I put in my xml file for /etc/libvirt/qemu/Symantec-bg.xml (a Symantec Brightmail Gateway virtual machine), I just get back that its an Intel card on boot (which doesn't work with Symantec BG). I am running Fedora 12, kernel is 2.6.31.6-166.fc12.x86_64. The version of libvirt is 0.7.1. I am running it on an Intel Core 2 Quad CPU Q8400 @ 2.66 GHz. I have the same problem under Fedora 13 Beta. Thoughts? Jonathan Hoover

2 2

[libvirt] Alternative to XML Creation and Parsing?
by Ganesh Pagade 05 May '10

05 May '10

Hello, We plan to develop a fancy GUI which would help creating and managing VMs/Domains for RHEL 5.4 KVM. However looking at the schemas provided in docs/schemas/ (Ex: domain.rng) generating the XML file from the inputs taken from the GUI and pass it on to virDomainDefineXML() turns out to be a huge tasks in itself. I was wondering if there is an better way for me to provide my configurations to the library and also to read data from it (instead of creating, parsing XMLs)? If an alternate approach exist, are there any limitations with it? Also the schema given in docs/schemas/ would be generic for all hyper visor. How can I identify the tags applicable only for a particular hyper visor? I found: virConnectDomainXMLFromNative(virConnectPtr conn, const char * nativeFormat, const char * nativeConfig, unsigned int flags) in API reference, which could solve my first problem but couldn't figure out what nativeFormat would be. Any inputs/pointers would be highly appreciated. Regards, Ganesh

2 2