February 2014 - Devel - libvirt List Archives

[libvirt] [PATCH 0/4] libxl: fixes related to concurrency improvements
by Jim Fehlig 06 Feb '14

06 Feb '14

While reviving old patches to add job support to the libxl driver, testing revealed some problems that were difficult to encounter in the current, more serialized processing approach used in the driver. The first patch is a bug fix, plugging leaks of libxlDomainObjPrivate objects. The second patch removes the list of libxl timer registrations maintained in the driver - a hack I was never fond of. The third patch moves domain shutdown handling to a thread, instead of doing all the shutdown work in the event handler. The fourth patch fixes an issue wrt child process handling discussed in this thread http://lists.xen.org/archives/html/xen-devel/2014-01/msg01553.html Ian Jackson's latest patches on the libxl side are here http://lists.xen.org/archives/html/xen-devel/2014-02/msg00124.html Jim Fehlig (4): libxl: fix leaking libxlDomainObjPrivate libxl: remove list of timer registrations from libxlDomainObjPrivate libxl: handle domain shutdown events in a thread libxl: improve subprocess handling src/libxl/libxl_conf.h | 5 +- src/libxl/libxl_domain.c | 102 ++++++++--------------------------- src/libxl/libxl_domain.h | 8 +-- src/libxl/libxl_driver.c | 135 +++++++++++++++++++++++++++++++---------------- 4 files changed, 115 insertions(+), 135 deletions(-) -- 1.8.1.4

2 7

[libvirt] [PATCH V4] Sheepdog: Auto Adding volume and pool refresh.
by joel SIMOES 06 Feb '14

06 Feb '14

From: Joel SIMOES <joel.simoes(a)laposte.net> Libvirt lose sheepdogs volumes on pool refresh or restart. When restarting sheepdog pool, all volumes are missing. This patch add automatically all volume from the added pool. With Jan Tomko help --- src/storage/storage_backend_sheepdog.c | 68 ++++++++++++++++++++++++++++++++-- 1 file changed, 64 insertions(+), 4 deletions(-) diff --git a/src/storage/storage_backend_sheepdog.c b/src/storage/storage_backend_sheepdog.c index a6981ce..7a139e7 100644 --- a/src/storage/storage_backend_sheepdog.c +++ b/src/storage/storage_backend_sheepdog.c @@ -109,22 +109,82 @@ virStorageBackendSheepdogAddHostArg(virCommandPtr cmd, virCommandAddArgFormat(cmd, "%d", port); } +static int +virStorageBackendSheepdogRefreshAllVol(virConnectPtr conn ATTRIBUTE_UNUSED, + virStoragePoolObjPtr pool) +{ + int ret = -1; + char *output = NULL; + char** lines = NULL; + char** cells = NULL; + size_t i; + + virCommandPtr cmd = virCommandNewArgList(COLLIE, "vdi", "list", "-r", NULL); + virStorageBackendSheepdogAddHostArg(cmd, pool); + virCommandSetOutputBuffer(cmd, &output); + if (virCommandRun(cmd, NULL) < 0) + goto cleanup; + + lines = virStringSplit(output, "\n", 0); + if (lines == NULL) + goto cleanup; + + for (i = 0; lines[i]; i++) { + char *line = lines[i]; + if (line == NULL) + break; + cells = virStringSplit(line, " ", 0); + if (cells == NULL || virStringListLength(cells) <= 2) + continue; + char *cell = cells[1]; + virStorageVolDefPtr vol = NULL; + if (VIR_ALLOC(vol) < 0) + goto cleanup; + + if (VIR_STRDUP(vol->name, cell) < 0) + goto cleanup; + + vol->type = VIR_STORAGE_VOL_BLOCK; + + if (VIR_EXPAND_N(pool->volumes.objs, pool->volumes.count, 1) < 0) + goto cleanup; + pool->volumes.objs[pool->volumes.count - 1] = vol; + + if (virStorageBackendSheepdogRefreshVol(conn, pool, vol) < 0) + goto cleanup; + vol = NULL; + virStringFreeList(cells); + } + + ret = 0; + +cleanup: + virCommandFree(cmd); + virStringFreeList(lines); + virStringFreeList(cells); + return ret; +} + static int virStorageBackendSheepdogRefreshPool(virConnectPtr conn ATTRIBUTE_UNUSED, virStoragePoolObjPtr pool) { - int ret; + int ret = -1; char *output = NULL; virCommandPtr cmd; cmd = virCommandNewArgList(COLLIE, "node", "info", "-r", NULL); virStorageBackendSheepdogAddHostArg(cmd, pool); virCommandSetOutputBuffer(cmd, &output); - ret = virCommandRun(cmd, NULL); - if (ret == 0) - ret = virStorageBackendSheepdogParseNodeInfo(pool->def, output); + if (virCommandRun(cmd, NULL) < 0) + goto cleanup; + + if (virStorageBackendSheepdogParseNodeInfo(pool->def, output) < 0) + goto cleanup; + ret = virStorageBackendSheepdogRefreshAllVol(conn, pool); +cleanup: virCommandFree(cmd); VIR_FREE(output); return ret; -- 1.8.3.2

2 1

[libvirt] [PATCH 1/2] qemuxml2argvmock: Mock time() on non-linux platforms too
by Michal Privoznik 06 Feb '14

06 Feb '14

The qemuxml2argvtest is run on more platforms than linux. For instance FreeBSD. On these platforms we are, however, not mocking time() which results in current time being fetched from system and hence tests number 32 and 33 failing. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- tests/qemuxml2argvmock.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/tests/qemuxml2argvmock.c b/tests/qemuxml2argvmock.c index ed9fb13..bff3b0f 100644 --- a/tests/qemuxml2argvmock.c +++ b/tests/qemuxml2argvmock.c @@ -20,9 +20,8 @@ #include <config.h> -#ifdef __linux__ -# include "internal.h" -# include <time.h> +#include "internal.h" +#include <time.h> time_t time(time_t *t) { @@ -31,7 +30,3 @@ time_t time(time_t *t) *t = ret; return ret; } - -#else -/* Nothing to override on non-__linux__ platforms */ -#endif -- 1.8.5.3

4 6

[libvirt] Enabling x2apic by default in KvM (was Re: [Qemu-devel] [PATCH] target-i386: enable x2apic by default on more recent CPU models)
by Eduardo Habkost 06 Feb '14

06 Feb '14

On Tue, Feb 04, 2014 at 03:12:43PM +0100, Andreas Färber wrote: > Am 03.02.2014 20:01, schrieb Eduardo Habkost: > > On Tue, Jan 21, 2014 at 05:13:50PM +0100, Paolo Bonzini wrote: > >> Il 21/01/2014 16:51, Andreas Färber ha scritto: > >>>>> We already do that for other bits (e.g. XSAVE/OSXSAVE), > >>> Please point me to the commit, a search for xsave did not come up with a > >>> commit changing such a thing - either it did not go through my queue or > >>> it slipped me through: Bugs are no excuse to produce more bugs. > >> > >> I meant that "-cpu SandyBridge" with TCG produces a CPU that doesn't > >> have XSAVE. > >> > >>>>> and in fact it > >>>>> is the same that we do for KVM: the KVM_GET_SUPPORTED_CPUID result is > >>>>> used to trim the generic feature bits. > >>> Our model definitions are the place to put stuff that real CPUs have. > >>> Either the CPU has it or it doesn't. If it does, then this patch is > >>> fully correct and it's TCG's job to mask things out. If we're adding > >>> artificial flags to the generic model definitions just to make KVM > >>> faster, then it is wrong - we have a choice of post_initialize and > >>> realize hooks for that. > >> > >> It would make TCG faster as well, and there would be no reason > >> really to avoid the "artificial" x2apic on TCG, if TCG implemented > >> x2apic at all. > > > > So, the discussion seem to have stalled. > > > > Andreas, are you still against the patch, after the arguments from Paolo > > and me? > > Yes, I am. I had proposed to discuss solutions at FOSDEM but Paolo was > not there, so no solution yet. > > My main concern still is that if a CPU does not have a certain feature > we should not list it as one of its features but add it to its features > where sensible. Just because TCG filters it out today is not keeping > anyone from implementing it tomorrow, in which case the emulated CPUs > would suddenly gain the feature. So my question still is, what rule can > we apply for enabling x2apic? (something like greater or equal this > family, etc. - then we can put it in your post_initialize hook so that > users can still override it) The rule needs be simple and predictable, so libvirt can know in advance if the results are going to match what was requested by the user. That's the problem with enabling it automatically based on some rule more complex than the existing "it's enabled if it's on the CPU model table or on the command-line" rule. Considering the whole stack, there's a strong reason to enable x2apic by default on all cases when using KVM. But if keeping the CPU model table match real CPUs bit-by-bit is more important for QEMU (which I don't understand why), I believe we will need to ask libvirt to do it (enable x2apic explicitly in the command-line by default). Or we could make the CPU models look different in KVM and TCG mode, but I am trying to avoid that. But I still believe that enabling x2apic by default in TCG _and_ KVM mode is easier and harmless. I don't see the value in this academic exercise of trying to make the CPU model table match real CPUs bit-by-bit. I mean: I simply don't expect any user will come to us complaining because they didn't want x2apic to be enabled by default. And if some user really cares about that, they can simply use "-x2apic" in the command-line. In other words, if it is not going to affect users negatively, why are we spending energy on it? -- Eduardo

1 0

[libvirt] [PATCH v3 1/2] BSD: implement nodeGetCPUStats
by Roman Bogorodskiy 06 Feb '14

06 Feb '14

Implementation obtains CPU usage information using kern.cp_time and kern.cp_times sysctl(8)s and reports CPU utilization. --- include/libvirt/libvirt.h.in | 8 ++++ src/nodeinfo.c | 104 +++++++++++++++++++++++++++++++++++++++++++ tools/virsh-host.c | 11 ++++- 3 files changed, 121 insertions(+), 2 deletions(-) diff --git a/include/libvirt/libvirt.h.in b/include/libvirt/libvirt.h.in index b3ce000..295d551 100644 --- a/include/libvirt/libvirt.h.in +++ b/include/libvirt/libvirt.h.in @@ -692,6 +692,14 @@ typedef enum { #define VIR_NODE_CPU_STATS_IOWAIT "iowait" /** + * VIR_NODE_CPU_STATS_INTR: + * + * The cumulative interrupt CPU time, + * since the node booting up (in nanoseconds). + */ +#define VIR_NODE_CPU_STATS_INTR "intr" + +/** * VIR_NODE_CPU_STATS_UTILIZATION: * * The CPU utilization of a node. diff --git a/src/nodeinfo.c b/src/nodeinfo.c index 6ebfb4b..9a1548e 100644 --- a/src/nodeinfo.c +++ b/src/nodeinfo.c @@ -34,8 +34,10 @@ #include "conf/domain_conf.h" #if defined(__FreeBSD__) || defined(__APPLE__) +# include <sys/time.h> # include <sys/types.h> # include <sys/sysctl.h> +# include <sys/resource.h> #endif #include "c-ctype.h" @@ -99,8 +101,108 @@ appleFreebsdNodeGetMemorySize(unsigned long *memory) #endif /* defined(__FreeBSD__) || defined(__APPLE__) */ #ifdef __FreeBSD__ +# define BSD_CPU_STATS_ALL 4 # define BSD_MEMORY_STATS_ALL 4 +# define TICK_TO_NSEC (1000ull * 1000ull * 1000ull / (stathz ? stathz : hz)) + +static int +freebsdNodeGetCPUStats(int cpuNum, + virNodeCPUStatsPtr params, + int *nparams) +{ + const char *sysctl_name; + long *cpu_times; + struct clockinfo clkinfo; + size_t i, j, cpu_times_size, clkinfo_size; + int cpu_times_num, offset, hz, stathz, ret = -1; + struct field_cpu_map { + const char *field; + int idx[CPUSTATES]; + } cpu_map[] = { + {VIR_NODE_CPU_STATS_KERNEL, {CP_SYS}}, + {VIR_NODE_CPU_STATS_USER, {CP_USER, CP_NICE}}, + {VIR_NODE_CPU_STATS_IDLE, {CP_IDLE}}, + {VIR_NODE_CPU_STATS_INTR, {CP_INTR}}, + {NULL, {0}} + }; + + if ((*nparams) == 0) { + *nparams = BSD_CPU_STATS_ALL; + return 0; + } + + if ((*nparams) != BSD_CPU_STATS_ALL) { + virReportInvalidArg(*nparams, + _("nparams in %s must be equal to %d"), + __FUNCTION__, BSD_CPU_STATS_ALL); + return -1; + } + + clkinfo_size = sizeof(clkinfo); + if (sysctlbyname("kern.clockrate", &clkinfo, &clkinfo_size, NULL, 0) < 0) { + virReportSystemError(errno, + _("sysctl failed for '%s'"), + "kern.clockrate"); + return -1; + } + + stathz = clkinfo.stathz; + hz = clkinfo.hz; + + if (cpuNum == VIR_NODE_CPU_STATS_ALL_CPUS) { + sysctl_name = "kern.cp_time"; + cpu_times_num = 1; + offset = 0; + } else { + sysctl_name = "kern.cp_times"; + cpu_times_num = appleFreebsdNodeGetCPUCount(); + + if (cpuNum >= cpu_times_num) { + virReportInvalidArg(cpuNum, + _("Invalid cpuNum in %s"), + __FUNCTION__); + return -1; + } + + offset = cpu_times_num * CPUSTATES; + } + + cpu_times_size = sizeof(long) * cpu_times_num * CPUSTATES; + + if (VIR_ALLOC_N(cpu_times, cpu_times_num * CPUSTATES) < 0) + goto cleanup; + + if (sysctlbyname(sysctl_name, cpu_times, &cpu_times_size, NULL, 0) < 0) { + virReportSystemError(errno, + _("sysctl failed for '%s'"), + sysctl_name); + goto cleanup; + } + + for (i = 0; cpu_map[i].field != NULL; i++) { + virNodeCPUStatsPtr param = &params[i]; + + if (virStrcpyStatic(param->field, cpu_map[i].field) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field '%s' too long for destination"), + cpu_map[i].field); + goto cleanup; + } + + param->value = 0; + for (j = 0; j < ARRAY_CARDINALITY(cpu_map[i].idx); j++) + param->value += cpu_times[offset + cpu_map[i].idx[j]] * TICK_TO_NSEC; + } + + ret = 0; + +cleanup: + VIR_FREE(cpu_times); + + return ret; +} + static int freebsdNodeGetMemoryStats(virNodeMemoryStatsPtr params, int *nparams) @@ -1045,6 +1147,8 @@ int nodeGetCPUStats(int cpuNum ATTRIBUTE_UNUSED, return ret; } +#elif defined(__FreeBSD__) + return freebsdNodeGetCPUStats(cpuNum, params, nparams); #else virReportError(VIR_ERR_NO_SUPPORT, "%s", _("node CPU stats not implemented on this platform")); diff --git a/tools/virsh-host.c b/tools/virsh-host.c index 3438ae7..f4ca7db 100644 --- a/tools/virsh-host.c +++ b/tools/virsh-host.c @@ -347,9 +347,10 @@ cmdNodeCpuStats(vshControl *ctl, const vshCmd *cmd) unsigned long long sys; unsigned long long idle; unsigned long long iowait; + unsigned long long intr; unsigned long long util; } cpu_stats[2]; - double user_time, sys_time, idle_time, iowait_time, total_time; + double user_time, sys_time, idle_time, iowait_time, intr_time, total_time; double usage; if (vshCommandOptInt(cmd, "cpu", &cpuNum) < 0) { @@ -390,6 +391,8 @@ cmdNodeCpuStats(vshControl *ctl, const vshCmd *cmd) cpu_stats[i].idle = value; } else if (STREQ(params[j].field, VIR_NODE_CPU_STATS_IOWAIT)) { cpu_stats[i].iowait = value; + } else if (STREQ(params[j].field, VIR_NODE_CPU_STATS_INTR)) { + cpu_stats[i].intr = value; } else if (STREQ(params[j].field, VIR_NODE_CPU_STATS_UTILIZATION)) { cpu_stats[i].util = value; flag_utilization = true; @@ -406,6 +409,7 @@ cmdNodeCpuStats(vshControl *ctl, const vshCmd *cmd) vshPrint(ctl, "%-15s %20llu\n", _("system:"), cpu_stats[0].sys); vshPrint(ctl, "%-15s %20llu\n", _("idle:"), cpu_stats[0].idle); vshPrint(ctl, "%-15s %20llu\n", _("iowait:"), cpu_stats[0].iowait); + vshPrint(ctl, "%-15s %20llu\n", _("intr:"), cpu_stats[0].intr); } } else { if (flag_utilization) { @@ -418,7 +422,8 @@ cmdNodeCpuStats(vshControl *ctl, const vshCmd *cmd) sys_time = cpu_stats[1].sys - cpu_stats[0].sys; idle_time = cpu_stats[1].idle - cpu_stats[0].idle; iowait_time = cpu_stats[1].iowait - cpu_stats[0].iowait; - total_time = user_time + sys_time + idle_time + iowait_time; + intr_time = cpu_stats[1].intr - cpu_stats[0].intr; + total_time = user_time + sys_time + idle_time + iowait_time + intr_time; usage = (user_time + sys_time) / total_time * 100; @@ -432,6 +437,8 @@ cmdNodeCpuStats(vshControl *ctl, const vshCmd *cmd) _("idle:"), idle_time / total_time * 100); vshPrint(ctl, "%-15s %5.1lf%%\n", _("iowait:"), iowait_time / total_time * 100); + vshPrint(ctl, "%-15s %5.1lf%%\n", + _("intr:"), intr_time / total_time * 100); } } -- 1.8.4.3

3 8

[libvirt] [PATCH v3] bhyve: add a basic driver
by Roman Bogorodskiy 06 Feb '14

06 Feb '14

Changes from the previos version, based on Daniel's feedback: * Moved autoconf driver detection to m4/ * Added driver sources to STATEFUL_DRIVER_SOURCE_FILES * Dropped unneeded locking from DefineXML * Initial implemtation of dumpxml/dominfo, stub of state * Moved command generation to bhyve_command.c * Attempt to unload VM on errors Roman Bogorodskiy (1): bhyve: add a basic driver configure.ac | 11 + daemon/libvirtd.c | 9 + include/libvirt/virterror.h | 1 + m4/virt-driver-bhyve.m4 | 52 ++++ src/Makefile.am | 38 +++ src/bhyve/bhyve_command.c | 269 +++++++++++++++++++++ src/bhyve/bhyve_command.h | 41 ++++ src/bhyve/bhyve_driver.c | 566 ++++++++++++++++++++++++++++++++++++++++++++ src/bhyve/bhyve_driver.h | 28 +++ src/bhyve/bhyve_process.c | 205 ++++++++++++++++ src/bhyve/bhyve_process.h | 36 +++ src/bhyve/bhyve_utils.h | 48 ++++ src/conf/domain_conf.c | 3 +- src/conf/domain_conf.h | 1 + src/driver.h | 1 + src/libvirt.c | 3 + src/util/virerror.c | 1 + 17 files changed, 1312 insertions(+), 1 deletion(-) create mode 100644 m4/virt-driver-bhyve.m4 create mode 100644 src/bhyve/bhyve_command.c create mode 100644 src/bhyve/bhyve_command.h create mode 100644 src/bhyve/bhyve_driver.c create mode 100644 src/bhyve/bhyve_driver.h create mode 100644 src/bhyve/bhyve_process.c create mode 100644 src/bhyve/bhyve_process.h create mode 100644 src/bhyve/bhyve_utils.h -- 1.8.4.3

2 5

[libvirt] [PATCH v1.1.0-maint] Push nwfilter update locking up to top level
by Laine Stump 06 Feb '14

06 Feb '14

From: "Daniel P. Berrange" <berrange(a)redhat.com> (see notes about conflict resolution at the end. I'm not sending the other 5 patches required, since they were all simple cherry-picks of: 4f2094346d98f4ed6a2de115d204c166cc563496 b77b16ce4166dcc87963ae5d279b77b162ddbb55 ebca369e3fe5ac999c261c2d44e60a1bac3cfe65 999d72fbd59ea712128ae294b69b6a54039d757b c065984b58000a44c90588198d222a314ac532fd ) The NWFilter code has as a deadlock race condition between the virNWFilter{Define,Undefine} APIs and starting of guest VMs due to mis-matched lock ordering. In the virNWFilter{Define,Undefine} codepaths the lock ordering is 1. nwfilter driver lock 2. virt driver lock 3. nwfilter update lock 4. domain object lock In the VM guest startup paths the lock ordering is 1. virt driver lock 2. domain object lock 3. nwfilter update lock As can be seen the domain object and nwfilter update locks are not acquired in a consistent order. The fix used is to push the nwfilter update lock upto the top level resulting in a lock ordering for virNWFilter{Define,Undefine} of 1. nwfilter driver lock 2. nwfilter update lock 3. virt driver lock 4. domain object lock and VM start using 1. nwfilter update lock 2. virt driver lock 3. domain object lock This has the effect of serializing VM startup once again, even if no nwfilters are applied to the guest. There is also the possibility of deadlock due to a call graph loop via virNWFilterInstantiate and virNWFilterInstantiateFilterLate. These two problems mean the lock must be turned into a read/write lock instead of a plain mutex at the same time. The lock is used to serialize changes to the "driver->nwfilters" hash, so the write lock only needs to be held by the define/undefine methods. All other methods can rely on a read lock which allows good concurrency. Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com> (cherry picked from commit 6e5c79a1b5a8b3a23e7df7ffe58fb272aa17fbfb) Conflicts: src/conf/nwfilter_conf.c - virReportOOMError() in context of one hunk. src/lxc/lxc_driver.c - functions renamed, and lxc object locking changed, creating a conflict in the context. --- src/conf/nwfilter_conf.c | 25 ++++++++++++------------- src/conf/nwfilter_conf.h | 3 ++- src/libvirt_private.syms | 3 ++- src/lxc/lxc_driver.c | 8 +++++++- src/nwfilter/nwfilter_driver.c | 10 ++++++---- src/nwfilter/nwfilter_gentech_driver.c | 6 +----- src/qemu/qemu_driver.c | 6 ++++++ src/uml/uml_driver.c | 4 ++++ 8 files changed, 40 insertions(+), 25 deletions(-) diff --git a/src/conf/nwfilter_conf.c b/src/conf/nwfilter_conf.c index 94b4fe9..e943bab 100644 --- a/src/conf/nwfilter_conf.c +++ b/src/conf/nwfilter_conf.c @@ -2,7 +2,7 @@ * nwfilter_conf.c: network filter XML processing * (derived from storage_conf.c) * - * Copyright (C) 2006-2012 Red Hat, Inc. + * Copyright (C) 2006-2012, 2014 Red Hat, Inc. * Copyright (C) 2006-2008 Daniel P. Berrange * * Copyright (C) 2010-2011 IBM Corporation @@ -143,17 +143,22 @@ static const struct int_map chain_priorities[] = { /* * only one filter update allowed */ -static virMutex updateMutex; +static virRWLock updateLock; static bool initialized = false; void -virNWFilterLockFilterUpdates(void) { - virMutexLock(&updateMutex); +virNWFilterReadLockFilterUpdates(void) { + virRWLockRead(&updateLock); +} + +void +virNWFilterWriteLockFilterUpdates(void) { + virRWLockWrite(&updateLock); } void virNWFilterUnlockFilterUpdates(void) { - virMutexUnlock(&updateMutex); + virRWLockUnlock(&updateLock); } @@ -2992,14 +2997,12 @@ virNWFilterObjAssignDef(virNWFilterObjListPtr nwfilters, return NULL; } - virNWFilterLockFilterUpdates(); if ((nwfilter = virNWFilterObjFindByName(nwfilters, def->name))) { if (virNWFilterDefEqual(def, nwfilter->def, false)) { virNWFilterDefFree(nwfilter->def); nwfilter->def = def; - virNWFilterUnlockFilterUpdates(); return nwfilter; } @@ -3007,7 +3010,6 @@ virNWFilterObjAssignDef(virNWFilterObjListPtr nwfilters, /* trigger the update on VMs referencing the filter */ if (virNWFilterTriggerVMFilterRebuild()) { nwfilter->newDef = NULL; - virNWFilterUnlockFilterUpdates(); virNWFilterObjUnlock(nwfilter); return NULL; } @@ -3015,12 +3017,9 @@ virNWFilterObjAssignDef(virNWFilterObjListPtr nwfilters, virNWFilterDefFree(nwfilter->def); nwfilter->def = def; nwfilter->newDef = NULL; - virNWFilterUnlockFilterUpdates(); return nwfilter; } - virNWFilterUnlockFilterUpdates(); - if (VIR_ALLOC(nwfilter) < 0) { virReportOOMError(); return NULL; @@ -3492,7 +3491,7 @@ int virNWFilterConfLayerInit(virDomainObjListIterator domUpdateCB, initialized = true; - if (virMutexInitRecursive(&updateMutex) < 0) + if (virRWLockInit(&updateLock) < 0) return -1; return 0; @@ -3504,7 +3503,7 @@ void virNWFilterConfLayerShutdown(void) if (!initialized) return; - virMutexDestroy(&updateMutex); + virRWLockDestroy(&updateLock); initialized = false; virNWFilterDomainFWUpdateOpaque = NULL; diff --git a/src/conf/nwfilter_conf.h b/src/conf/nwfilter_conf.h index 29906f1..d460a08 100644 --- a/src/conf/nwfilter_conf.h +++ b/src/conf/nwfilter_conf.h @@ -716,7 +716,8 @@ virNWFilterDefPtr virNWFilterDefParseFile(const char *filename); void virNWFilterObjLock(virNWFilterObjPtr obj); void virNWFilterObjUnlock(virNWFilterObjPtr obj); -void virNWFilterLockFilterUpdates(void); +void virNWFilterWriteLockFilterUpdates(void); +void virNWFilterReadLockFilterUpdates(void); void virNWFilterUnlockFilterUpdates(void); int virNWFilterConfLayerInit(virDomainObjListIterator domUpdateCB, void *opaque); diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index a6348bc..753c698 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -551,7 +551,6 @@ virNWFilterDefParseString; virNWFilterInstFiltersOnAllVMs; virNWFilterJumpTargetTypeToString; virNWFilterLoadAllConfigs; -virNWFilterLockFilterUpdates; virNWFilterObjAssignDef; virNWFilterObjDeleteDef; virNWFilterObjFindByName; @@ -563,6 +562,7 @@ virNWFilterObjSaveDef; virNWFilterObjUnlock; virNWFilterPrintStateMatchFlags; virNWFilterPrintTCPFlags; +virNWFilterReadLockFilterUpdates; virNWFilterRegisterCallbackDriver; virNWFilterRuleActionTypeToString; virNWFilterRuleDirectionTypeToString; @@ -570,6 +570,7 @@ virNWFilterRuleProtocolTypeToString; virNWFilterTestUnassignDef; virNWFilterUnlockFilterUpdates; virNWFilterUnRegisterCallbackDriver; +virNWFilterWriteLockFilterUpdates; # conf/nwfilter_ipaddrmap.h diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c index fed3775..d59f0f6 100644 --- a/src/lxc/lxc_driver.c +++ b/src/lxc/lxc_driver.c @@ -1,5 +1,5 @@ /* - * Copyright (C) 2010-2013 Red Hat, Inc. + * Copyright (C) 2010-2014 Red Hat, Inc. * Copyright IBM Corp. 2008 * * lxc_driver.c: linux container driver functions @@ -1102,6 +1102,8 @@ static int lxcDomainCreateWithFlags(virDomainPtr dom, unsigned int flags) virCheckFlags(VIR_DOMAIN_START_AUTODESTROY, -1); + virNWFilterReadLockFilterUpdates(); + lxcDriverLock(driver); vm = virDomainObjListFindByUUID(driver->domains, dom->uuid); if (!vm) { @@ -1146,6 +1148,7 @@ cleanup: if (event) virDomainEventStateQueue(driver->domainEventState, event); lxcDriverUnlock(driver); + virNWFilterUnlockFilterUpdates(); return ret; } @@ -1184,6 +1187,8 @@ lxcDomainCreateXML(virConnectPtr conn, virCheckFlags(VIR_DOMAIN_START_AUTODESTROY, NULL); + virNWFilterReadLockFilterUpdates(); + lxcDriverLock(driver); if (!(def = virDomainDefParseString(xml, driver->caps, driver->xmlopt, 1 << VIR_DOMAIN_VIRT_LXC, @@ -1235,6 +1240,7 @@ cleanup: if (event) virDomainEventStateQueue(driver->domainEventState, event); lxcDriverUnlock(driver); + virNWFilterUnlockFilterUpdates(); return dom; } diff --git a/src/nwfilter/nwfilter_driver.c b/src/nwfilter/nwfilter_driver.c index 630f622..2d5dcaf 100644 --- a/src/nwfilter/nwfilter_driver.c +++ b/src/nwfilter/nwfilter_driver.c @@ -284,12 +284,14 @@ nwfilterStateReload(void) virNWFilterLearnThreadsTerminate(true); nwfilterDriverLock(driverState); + virNWFilterWriteLockFilterUpdates(); virNWFilterCallbackDriversLock(); virNWFilterLoadAllConfigs(&driverState->nwfilters, driverState->configDir); virNWFilterCallbackDriversUnlock(); + virNWFilterUnlockFilterUpdates(); nwfilterDriverUnlock(driverState); virNWFilterInstFiltersOnAllVMs(); @@ -540,6 +542,7 @@ nwfilterDefineXML(virConnectPtr conn, virNWFilterPtr ret = NULL; nwfilterDriverLock(driver); + virNWFilterWriteLockFilterUpdates(); virNWFilterCallbackDriversLock(); if (!(def = virNWFilterDefParseString(xml))) @@ -566,6 +569,7 @@ cleanup: virNWFilterObjUnlock(nwfilter); virNWFilterCallbackDriversUnlock(); + virNWFilterUnlockFilterUpdates(); nwfilterDriverUnlock(driver); return ret; } @@ -578,10 +582,9 @@ nwfilterUndefine(virNWFilterPtr obj) { int ret = -1; nwfilterDriverLock(driver); + virNWFilterWriteLockFilterUpdates(); virNWFilterCallbackDriversLock(); - virNWFilterLockFilterUpdates(); - nwfilter = virNWFilterObjFindByUUID(&driver->nwfilters, obj->uuid); if (!nwfilter) { virReportError(VIR_ERR_NO_NWFILTER, @@ -612,9 +615,8 @@ cleanup: if (nwfilter) virNWFilterObjUnlock(nwfilter); - virNWFilterUnlockFilterUpdates(); - virNWFilterCallbackDriversUnlock(); + virNWFilterUnlockFilterUpdates(); nwfilterDriverUnlock(driver); return ret; } diff --git a/src/nwfilter/nwfilter_gentech_driver.c b/src/nwfilter/nwfilter_gentech_driver.c index 637e647..c9ce514 100644 --- a/src/nwfilter/nwfilter_gentech_driver.c +++ b/src/nwfilter/nwfilter_gentech_driver.c @@ -947,8 +947,6 @@ _virNWFilterInstantiateFilter(virNWFilterDriverStatePtr driver, int ifindex; int rc; - virNWFilterLockFilterUpdates(); - /* after grabbing the filter update lock check for the interface; if it's not there anymore its filters will be or are being removed (while holding the lock) and we don't want to build new ones */ @@ -976,8 +974,6 @@ _virNWFilterInstantiateFilter(virNWFilterDriverStatePtr driver, foundNewFilter); cleanup: - virNWFilterUnlockFilterUpdates(); - return rc; } @@ -996,7 +992,7 @@ virNWFilterInstantiateFilterLate(virNWFilterDriverStatePtr driver, int rc; bool foundNewFilter = false; - virNWFilterLockFilterUpdates(); + virNWFilterReadLockFilterUpdates(); rc = __virNWFilterInstantiateFilter(driver, vmuuid, diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 8ebfadd..b209614 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -1542,6 +1542,8 @@ static virDomainPtr qemuDomainCreateXML(virConnectPtr conn, if (flags & VIR_DOMAIN_START_AUTODESTROY) start_flags |= VIR_QEMU_PROCESS_START_AUTODESTROY; + virNWFilterReadLockFilterUpdates(); + if (!(caps = virQEMUDriverGetCapabilities(driver, false))) goto cleanup; @@ -1619,6 +1621,7 @@ cleanup: } virObjectUnref(caps); virObjectUnref(qemuCaps); + virNWFilterUnlockFilterUpdates(); return dom; } @@ -5846,6 +5849,8 @@ qemuDomainCreateWithFlags(virDomainPtr dom, unsigned int flags) VIR_DOMAIN_START_BYPASS_CACHE | VIR_DOMAIN_START_FORCE_BOOT, -1); + virNWFilterReadLockFilterUpdates(); + if (!(vm = qemuDomObjFromDomain(dom))) return -1; @@ -5873,6 +5878,7 @@ endjob: cleanup: if (vm) virObjectUnlock(vm); + virNWFilterUnlockFilterUpdates(); return ret; } diff --git a/src/uml/uml_driver.c b/src/uml/uml_driver.c index 74aa94a..5947082 100644 --- a/src/uml/uml_driver.c +++ b/src/uml/uml_driver.c @@ -1572,6 +1572,7 @@ static virDomainPtr umlDomainCreateXML(virConnectPtr conn, const char *xml, virCheckFlags(VIR_DOMAIN_START_AUTODESTROY, NULL); + virNWFilterReadLockFilterUpdates(); umlDriverLock(driver); if (!(def = virDomainDefParseString(xml, driver->caps, driver->xmlopt, 1 << VIR_DOMAIN_VIRT_UML, @@ -1611,6 +1612,7 @@ cleanup: if (event) umlDomainEventQueue(driver, event); umlDriverUnlock(driver); + virNWFilterUnlockFilterUpdates(); return dom; } @@ -1993,6 +1995,7 @@ static int umlDomainCreateWithFlags(virDomainPtr dom, unsigned int flags) { virCheckFlags(VIR_DOMAIN_START_AUTODESTROY, -1); + virNWFilterReadLockFilterUpdates(); umlDriverLock(driver); vm = virDomainObjListFindByUUID(driver->domains, dom->uuid); @@ -2019,6 +2022,7 @@ cleanup: if (event) umlDomainEventQueue(driver, event); umlDriverUnlock(driver); + virNWFilterUnlockFilterUpdates(); return ret; } -- 1.8.5.3

2 1

[libvirt] systemd, LXC and user namespaces
by Richard Weinberger 06 Feb '14

06 Feb '14

Hi! I'm trying to get rid of a hack to make systemd (kind of) work in Linux containers on libvirt. The hack can be found in the first mail of [0]. systemd folks told me that systemd needs a name=systemd cgroup [0], which makes perfectly sense to me. I found that libvirt does this already, but uid 0 within the container is not allowed to access it. (Maybe as Kay noted a chmod() is missing) Now I'm wondering whether this is simply not supported in libvirt (I'm on 1.2.1) or am I doing something horrible wrong. This is my domain: ---cut--- <domain type='lxc'> <name>my2ndcontainer</name> <memory>524288</memory> <os> <type>exe</type> <init>/bin/bash</init> </os> <idmap>  <uid start='0' target='100000' count='998'/> <gid start='0' target='100000' count='998'/> <uid start='65533' target='100998' count='2'/> <gid start='65533' target='100998' count='2'/> </idmap> <devices> <console type='pty'/> <filesystem type='mount'> <source dir='/home/container//my2ndcontainer/rootfs'/> <target dir='/'/> </filesystem> <interface type='bridge'> <source bridge='br0'/> <mac address='4a:19:0a:01:01:a4'/> </interface> </devices> </domain> ---cut--- Within my domain: ---cut--- test1:/ # mount /dev/vda2 on / type ext4 (rw,relatime,data=ordered) proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) proc on /proc/sys type proc (ro,relatime) sysfs on /sys type sysfs (ro,relatime) libvirt on /proc/meminfo type fuse (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,relatime,size=64k,mode=755,uid=100000,gid=100000) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu) cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event) cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd) devfs on /dev type tmpfs (rw,nosuid,relatime,size=64k,mode=755) devpts on /dev/pts type devpts (rw,nosuid,relatime,gid=5,mode=620,ptmxmode=666) devpts on /dev/ptmx type devpts (rw,nosuid,relatime,gid=5,mode=620,ptmxmode=666) test1:/ # ls -la /sys/fs/cgroup/systemd total 0 drwxr-xr-x 2 nobody nogroup 0 Feb 6 09:05 . drwxr-xr-x 11 root root 260 Feb 6 09:05 .. -rw-r--r-- 1 nobody nogroup 0 Feb 6 09:05 cgroup.clone_children --w--w--w- 1 nobody nogroup 0 Feb 6 09:05 cgroup.event_control -rw-r--r-- 1 nobody nogroup 0 Feb 6 09:05 cgroup.procs -rw-r--r-- 1 nobody nogroup 0 Feb 6 09:05 notify_on_release -rw-r--r-- 1 nobody nogroup 0 Feb 6 09:05 tasks test1:/ # exec /sbin/init systemd 208 running in system mode. (+PAM +LIBWRAP +AUDIT +SELINUX -IMA +SYSVINIT +LIBCRYPTSETUP +GCRYPT +ACL +XZ) Detected virtualization 'lxc-libvirt'. Welcome to openSUSE 13.1 (Bottle) (x86_64)! Set hostname to <my2ndcontainer>. Failed to install release agent, ignoring: No such file or directory Failed to create root cgroup hierarchy: Permission denied Failed to allocate manager object: Permission denied ---cut--- You can see that systemd stops executing because it was unable to write to /sys/fs/cgroup/systemd. Is this a configuration issue or does libvirt need some changes? [0] http://lists.freedesktop.org/archives/systemd-devel/2014-February/016699.ht… -- Thanks, //richard

2 1

[libvirt] [PATCH 0/2] Allow localtime clock basis
by Michal Privoznik 06 Feb '14

06 Feb '14

With a nice test enhancement ... Michal Privoznik (2): qemuBuildClockArgStr: Allow localtime clock basis qemuxml2argvtest: Test localtime clock basis src/qemu/qemu_command.c | 18 +++++++---- tests/Makefile.am | 7 ++++ ...muxml2argv-clock-localtime-basis-localtime.args | 5 +++ ...emuxml2argv-clock-localtime-basis-localtime.xml | 28 ++++++++++++++++ tests/qemuxml2argvmock.c | 37 ++++++++++++++++++++++ tests/qemuxml2argvtest.c | 3 +- 6 files changed, 90 insertions(+), 8 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-clock-localtime-basis-localtime.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-clock-localtime-basis-localtime.xml create mode 100644 tests/qemuxml2argvmock.c -- 1.8.5.2

4 6

[libvirt] [PATCH v2] qemu: Fix crash in virDomainMemoryStats with old qemu
by Jiri Denemark 06 Feb '14

06 Feb '14

If virDomainMemoryStats was run on a domain with virtio balloon driver running on an old qemu which supports QMP but does not support qom-list QMP command, libvirtd would crash. The reason is we did not check if qemuMonitorJSONGetObjectListPaths failed and moreover we even stored its result in an unsigned integer type. Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- Notes: version 2: - use signed type for i and j to avoid comparison between signed and unsigned types; gcc-- for not complaining about it src/qemu/qemu_monitor.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index a968901..a2769db 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -1019,7 +1019,7 @@ qemuMonitorFindBalloonObjectPath(qemuMonitorPtr mon, virDomainObjPtr vm, const char *curpath) { - size_t i, j, npaths = 0, nprops = 0; + ssize_t i, j, npaths = 0, nprops = 0; int ret = 0; char *nextpath = NULL; qemuMonitorJSONListPathPtr *paths = NULL; @@ -1045,6 +1045,8 @@ qemuMonitorFindBalloonObjectPath(qemuMonitorPtr mon, VIR_DEBUG("Searching for Balloon Object Path starting at %s", curpath); npaths = qemuMonitorJSONGetObjectListPaths(mon, curpath, &paths); + if (npaths < 0) + return -1; for (i = 0; i < npaths && ret == 0; i++) { @@ -1061,6 +1063,11 @@ qemuMonitorFindBalloonObjectPath(qemuMonitorPtr mon, * then this version of qemu/kvm does not support the feature. */ nprops = qemuMonitorJSONGetObjectListPaths(mon, nextpath, &bprops); + if (nprops < 0) { + ret = -1; + goto cleanup; + } + for (j = 0; j < nprops; j++) { if (STREQ(bprops[j]->name, "guest-stats-polling-interval")) { VIR_DEBUG("Found Balloon Object Path %s", nextpath); -- 1.8.5.3

4 5