November 2013 - Devel - libvirt List Archives

[libvirt] [PATCH 0/3] more storage cleanups
by Eric Blake 07 Nov '13

07 Nov '13

I'm still plugging away at getting gluster backing chains to work; here's a couple of independent cleanups that can be applied first. Eric Blake (3): storage: use simpler 'char *' storage: avoid short reads while chasing backing chain storage: reduce number of stat calls src/libvirt_private.syms | 1 + src/storage/storage_backend.c | 62 +++++++++++++++++-------------------- src/storage/storage_backend.h | 10 ++++-- src/storage/storage_backend_fs.c | 5 +-- src/storage/storage_backend_mpath.c | 7 +++-- src/storage/storage_backend_scsi.c | 7 +++-- src/util/virfile.c | 21 +++++++++++++ src/util/virfile.h | 9 ++++-- src/util/virstoragefile.c | 52 ++++++++++++++----------------- 9 files changed, 100 insertions(+), 74 deletions(-) -- 1.8.3.1

2 8

[libvirt] [PATCH] nodeinfo: fix build on non-Linux
by Eric Blake 07 Nov '13

07 Nov '13

Commit b0f8546 broke the build on mingw, by exposing code that had Linux-specific dependencies but which was previously protected by libnuma ifdef guards: make[3]: Entering directory `/home/eblake/libvirt-tmp/build/src' CC libvirt_driver_la-nodeinfo.lo ../../src/nodeinfo.c: In function 'virNodeGetSiblingsList': ../../src/nodeinfo.c:1543:30: error: 'SYSFS_THREAD_SIBLINGS_LIST_LENGTH_MAX' undeclared (first use in this function) if (virFileReadAll(path, SYSFS_THREAD_SIBLINGS_LIST_LENGTH_MAX, &buf) < 0) ^ ../../src/nodeinfo.c:1543:30: note: each undeclared identifier is reported only once for each function it appears in ../../src/nodeinfo.c: In function 'virNodeCapsFillCPUInfo': ../../src/nodeinfo.c:1562:5: error: implicit declaration of function 'virNodeGetCpuValue' [-Werror=implicit-function-declaration] if ((tmp = virNodeGetCpuValue(SYSFS_CPU_PATH, cpu_id, ^ ../../src/nodeinfo.c:1562:5: error: nested extern declaration of 'virNodeGetCpuValue' [-Werror=nested-externs] ../../src/nodeinfo.c:1562:35: error: 'SYSFS_CPU_PATH' undeclared (first use in this function) if ((tmp = virNodeGetCpuValue(SYSFS_CPU_PATH, cpu_id, ^ cc1: all warnings being treated as errors * src/nodeinfo.c (virNodeCapsFillCPUInfo): Make conditional. (virNodeGetSiblingsList): Move into #ifdef linux block. Signed-off-by: Eric Blake <eblake(a)redhat.com> --- Pushing under the build-breaker rule. src/nodeinfo.c | 56 ++++++++++++++++++++++++++++++++------------------------ 1 file changed, 32 insertions(+), 24 deletions(-) diff --git a/src/nodeinfo.c b/src/nodeinfo.c index 6c1fcc7..1838547 100644 --- a/src/nodeinfo.c +++ b/src/nodeinfo.c @@ -867,6 +867,30 @@ error: virBitmapFree(map); return NULL; } + + +static virBitmapPtr +virNodeGetSiblingsList(const char *dir, int cpu_id) +{ + char *path = NULL; + char *buf = NULL; + virBitmapPtr ret = NULL; + + if (virAsprintf(&path, "%s/cpu%u/topology/thread_siblings_list", + dir, cpu_id) < 0) + goto cleanup; + + if (virFileReadAll(path, SYSFS_THREAD_SIBLINGS_LIST_LENGTH_MAX, &buf) < 0) + goto cleanup; + + if (virBitmapParse(buf, 0, &ret, virNumaGetMaxCPUs()) < 0) + goto cleanup; + +cleanup: + VIR_FREE(buf); + VIR_FREE(path); + return ret; +} #endif int nodeGetInfo(virNodeInfoPtr nodeinfo) @@ -1529,33 +1553,12 @@ nodeGetFreeMemoryFake(void) return ret; } -static virBitmapPtr -virNodeGetSiblingsList(const char *dir, int cpu_id) -{ - char *path = NULL; - char *buf = NULL; - virBitmapPtr ret = NULL; - - if (virAsprintf(&path, "%s/cpu%u/topology/thread_siblings_list", - dir, cpu_id) < 0) - goto cleanup; - - if (virFileReadAll(path, SYSFS_THREAD_SIBLINGS_LIST_LENGTH_MAX, &buf) < 0) - goto cleanup; - - if (virBitmapParse(buf, 0, &ret, virNumaGetMaxCPUs()) < 0) - goto cleanup; - -cleanup: - VIR_FREE(buf); - VIR_FREE(path); - return ret; -} - /* returns 1 on success, 0 if the detection failed and -1 on hard error */ static int -virNodeCapsFillCPUInfo(int cpu_id, virCapsHostNUMACellCPUPtr cpu) +virNodeCapsFillCPUInfo(int cpu_id ATTRIBUTE_UNUSED, + virCapsHostNUMACellCPUPtr cpu ATTRIBUTE_UNUSED) { +#ifdef __linux__ int tmp; cpu->id = cpu_id; @@ -1575,6 +1578,11 @@ virNodeCapsFillCPUInfo(int cpu_id, virCapsHostNUMACellCPUPtr cpu) return -1; return 0; +#else + virReportError(VIR_ERR_NO_SUPPORT, "%s", + _("node cpu info not implemented on this platform")); + return -1; +#endif } int -- 1.8.3.1

1 0

[libvirt] [PATCH] add SystemTap to apps using libvirt
by Jonathan Lebon 06 Nov '13

06 Nov '13

Starting from v2.4 (released today!), SystemTap can use libvirt to execute scripts inside virtual machines. Not sure if this matches the spirit of the page (I see that most other apps are virtualization- oriented by nature). In the case it does not, feel free to ignore this patch. --- docs/apps.html.in | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/apps.html.in b/docs/apps.html.in index c7e2f64..3598062 100644 --- a/docs/apps.html.in +++ b/docs/apps.html.in @@ -103,6 +103,13 @@ in a virtual machine. It prints out a list of facts about the virtual machine, derived from heuristics. </dd> + <dt><a href="http://sourceware.org/systemtap/">stap</a></dt> + <dd> + SystemTap is a tool used to gather rich information about a running + system through the use of scripts. Starting from v2.4, the front-end + application stap can use libvirt to gather data within virtual + machines. + </dd> </dl> <h2><a name="configmgmt">Configuration Management</a></h2> -- 1.8.3.1

2 1

[libvirt] [PATCH] test driver: add support for .connectBaselineCPU
by Giuseppe Scrivano 06 Nov '13

06 Nov '13

It uses the same functionalities of the qemu driver. Signed-off-by: Giuseppe Scrivano <gscrivan(a)redhat.com> --- src/test/test_driver.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/src/test/test_driver.c b/src/test/test_driver.c index c2e530e..2678247 100644 --- a/src/test/test_driver.c +++ b/src/test/test_driver.c @@ -1512,6 +1512,21 @@ static int testConnectGetMaxVcpus(virConnectPtr conn ATTRIBUTE_UNUSED, return 32; } +static char * +testConnectBaselineCPU(virConnectPtr conn ATTRIBUTE_UNUSED, + const char **xmlCPUs, + unsigned int ncpus, + unsigned int flags) +{ + char *cpu; + + virCheckFlags(VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES, NULL); + + cpu = cpuBaselineXML(xmlCPUs, ncpus, NULL, 0, flags); + + return cpu; +} + static int testNodeGetInfo(virConnectPtr conn, virNodeInfoPtr info) { @@ -7177,6 +7192,8 @@ static virDriver testDriver = { .domainSnapshotCreateXML = testDomainSnapshotCreateXML, /* 1.1.4 */ .domainRevertToSnapshot = testDomainRevertToSnapshot, /* 1.1.4 */ .domainSnapshotDelete = testDomainSnapshotDelete, /* 1.1.4 */ + + .connectBaselineCPU = testConnectBaselineCPU, /* 1.1.5 */ }; static virNetworkDriver testNetworkDriver = { -- 1.8.3.1

2 1

[libvirt] [PATCH]docs: fix a typo in formatnwfilter
by Chen Hanxiao 06 Nov '13

06 Nov '13

From: Chen Hanxiao <chenhanxiao(a)cn.fujitsu.com> s/fragement/fragment Signed-off-by: Chen Hanxiao <chenhanxiao(a)cn.fujitsu.com> --- docs/formatnwfilter.html.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/formatnwfilter.html.in b/docs/formatnwfilter.html.in index ec20300..c5ab68d 100644 --- a/docs/formatnwfilter.html.in +++ b/docs/formatnwfilter.html.in @@ -1827,7 +1827,7 @@ initiate a connection from TCP port 80 back towards the VM. By default the connection state match that enables connection tracking and then enforcement of directionality of traffic is turned on. <br/> - The following shows an example XML fragement where this feature has been + The following shows an example XML fragment where this feature has been turned off for incoming connections to TCP port 12345. </p> <pre> -- 1.8.2.1

3 2

[libvirt] ANNOUNCE: libvirt 1.1.3.1 maintenance release
by Cole Robinson 06 Nov '13

06 Nov '13

libvirt 1.1.3.1 maintenance release is now available. This is libvirt 1.1.3 with additional bugfixes that have accumulated upstream since the initial release. This release can be downloaded at: http://libvirt.org/sources/stable_updates/libvirt-1.1.3.1.tar.gz Changes in this version: * Push RPM deps down into libvirt-daemon-driver-XXXX sub-RPMs * Fix race condition reconnecting to vms & loading configs * Fix leak of objects when reconnecting to QEMU instances * Don't update dom->persistent without lock held * Block all use of libvirt.so in setuid programs * Remove (nearly) all use of getuid()/getgid() * Add stub getegid impl for platforms lacking it * Don't allow remote driver daemon autostart when running setuid * Only allow the UNIX transport in remote driver when setuid * Block all use of getenv with syntax-check * Remove all direct use of getenv * Make virCommand env handling robust in setuid env * Initialize threading & error layer in LXC controller * Fix flaw in detecting log format * Move virt-login-shell into libvirt-login-shell sub-RPM * Set a sane $PATH for virt-login-shell * remote: fix regression in event deregistration * python: Fix Create*WithFiles filefd passing * build: fix build of virt-login-shell on systems with older gnutls * build: fix linking virt-login-shell * Don't link virt-login-shell against libvirt.so (CVE-2013-4400) * Close all non-stdio FDs in virt-login-shell (CVE-2013-4400) * Only allow 'stderr' log output when running setuid (CVE-2013-4400) * Add helpers for getting env vars in a setuid environment * Fix perms for virConnectDomainXML{To,From}Native (CVE-2013-4401) * build: Add lxc testcase to dist list * Convert uuid to a string before printing it * LXC: Fix handling of RAM filesystem size units * qemuMonitorJSONSendKey: Avoid double free * rpc: fix getsockopt for LOCAL_PEERCRED on Mac OS X * Remove use of virConnectPtr from all remaining nwfilter code * Don't pass virConnectPtr in nwfilter 'struct domUpdateCBStruct' * Remove virConnectPtr arg from virNWFilterDefParse * qemu: cgroup: Fix crash if starting nographics guest * virNetDevBandwidthEqual: Make it more robust * qemu_hotplug: Allow QoS update in qemuDomainChangeNet * Adjust legacy max payload size to account for header information For info about past maintenance releases, see: http://wiki.libvirt.org/page/Maintenance_Releases Thanks, Cole

1 0

[libvirt] ANNOUNCE: libvirt 1.0.5.7 maintenance release
by Cole Robinson 06 Nov '13

06 Nov '13

libvirt 1.0.5.7 maintenance release is now available. This is libvirt 1.0.5 with additional bugfixes that have accumulated upstream since the initial release. This release can be downloaded at: http://libvirt.org/sources/stable_updates/libvirt-1.0.5.7.tar.gz Changes in this version: * qemuSetupMemoryCgroup: Handle hard_limit properly * qemu: Drop qemuDomainMemoryLimit * remote: fix regression in event deregistration * virsh: Fix debugging * Fix URI connect precedence * virDomainDefParseXML: set the argument of virBitmapFree to NULL after calling virBitmapFree * build: Add lxc testcase to dist list * LXC: Fix handling of RAM filesystem size units * qemuMonitorJSONSendKey: Avoid double free * virsh domjobinfo: Do not return 1 if job is NONE * Remove use of virConnectPtr from all remaining nwfilter code * Don't pass virConnectPtr in nwfilter 'struct domUpdateCBStruct' * Remove virConnectPtr arg from virNWFilterDefParse * virNetDevBandwidthEqual: Make it more robust * qemu_hotplug: Allow QoS update in qemuDomainChangeNet * qemu: Use "migratable" XML definition when doing external checkpoints * qemu: Fix checking of ABI stability when restoring external checkpoints * virsh: Fix regression of vol-resize For info about past maintenance releases, see: http://wiki.libvirt.org/page/Maintenance_Releases Thanks, Cole

1 0

[libvirt] [PATCH 0/3] Some qemu monitor crashers fix
by Michal Privoznik 06 Nov '13

06 Nov '13

The patches are pretty independent with each other. Michal Privoznik (3): qemu: Avoid double free of VM qemu: Don't access vm->priv on unlocked domain qemuMonitorDispose: Reset lastError src/qemu/qemu_migration.c | 9 +++++---- src/qemu/qemu_monitor.c | 1 + src/qemu/qemu_process.c | 9 --------- 3 files changed, 6 insertions(+), 13 deletions(-) -- 1.8.1.5

2 6

[libvirt] [PATCH] pci: properly handle out-of-order SRIOV virtual functions
by Laine Stump 06 Nov '13

06 Nov '13

This resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1025397 When libvirt creates the list of an SRIOV Physical Function's (PF) Virtual Functions (VF), it assumes that the order of "virtfn*" links returned by readdir() from the PF's sysfs directory is already in the correct order. Experience has shown that this is not always the case. This results in 1) incorrect assumptions made by consumers of the output of the virt_functions list of virsh nodedev-dumpxml, and 2) setting MAC address and vlan tag on the wrong VF (since libvirt uses netlink to set mac address and vlan tag, netlink requires the VF#, and the function virPCIGetVirtualFunctionIndex() returns the wrong index due to the improperly ordered VF list). See the bugzilla report for an example of this improper behavior. The solution provided by this patch is for virPCIGetVirtualFunctions to first gather all the "virtfn*" names from the PFs sysfs directory, then allocate a virtual_functions array large enough to hold all entries, and finally to create a device entry for each virtfn* name and place it into the virtual_functions array at the proper index (based on interpreting the value following "virtfn" in the name). Checks are introduced to ensure that 1) the virtfn list given in sysfs is not sparse (ie, there are no indexes larger than the length of the list), and that no two virtfn* entries decode to the same index. --- I briefly debated solving this problem by changing the virtual_functions array from being just a list of PCI device addresses into being an array of new objects that have both a PCI address and a virtual function index. I decided against it because it would have modified the XML format and required more substantial changes to other parts of the code (and anyway is unnecessary, since the virtual function array is never "sparse", so it can be appropriately represented by implying the virtual function index from a device's position in the list). src/util/virpci.c | 91 ++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 66 insertions(+), 25 deletions(-) diff --git a/src/util/virpci.c b/src/util/virpci.c index 148631f..e70fa3d 100644 --- a/src/util/virpci.c +++ b/src/util/virpci.c @@ -2379,6 +2379,8 @@ virPCIGetPhysicalFunction(const char *vf_sysfs_path, return ret; } +static const char *virtfnPrefix = "virtfn"; + /* * Returns virtual functions of a physical function */ @@ -2393,6 +2395,8 @@ virPCIGetVirtualFunctions(const char *sysfs_path, struct dirent *entry = NULL; char *device_link = NULL; char errbuf[64]; + char **virtfnNames = NULL; + size_t nVirtfnNames = 0; VIR_DEBUG("Attempting to get SR IOV virtual functions for device" "with sysfs path '%s'", sysfs_path); @@ -2411,51 +2415,88 @@ virPCIGetVirtualFunctions(const char *sysfs_path, while ((entry = readdir(dir))) { if (STRPREFIX(entry->d_name, "virtfn")) { - virPCIDeviceAddress *config_addr = NULL; + char *tempName; - if (virBuildPath(&device_link, sysfs_path, entry->d_name) == -1) { - virReportOOMError(); + /* save these to sort into virtual_functions array */ + if (VIR_STRDUP(tempName, entry->d_name) < 0) + goto error; + if (VIR_APPEND_ELEMENT(virtfnNames, nVirtfnNames, tempName) < 0) { + VIR_FREE(tempName); goto error; } + } + } - VIR_DEBUG("Number of virtual functions: %d", - *num_virtual_functions); + /* pre-allocate because we will be filling in out of order */ + if (nVirtfnNames && VIR_ALLOC_N(*virtual_functions, nVirtfnNames) < 0) + goto error; + *num_virtual_functions = nVirtfnNames; - if (virPCIGetDeviceAddressFromSysfsLink(device_link, - &config_addr) != - SRIOV_FOUND) { - VIR_WARN("Failed to get SRIOV function from device " - "link '%s'", device_link); - VIR_FREE(device_link); - continue; - } + for (i = 0; i < nVirtfnNames; i++) { + virPCIDeviceAddress *config_addr = NULL; + unsigned int virtfnIndex; - if (VIR_REALLOC_N(*virtual_functions, - *num_virtual_functions + 1) < 0) { - VIR_FREE(config_addr); - goto error; - } + if (virBuildPath(&device_link, sysfs_path, virtfnNames[i]) < 0) { + virReportOOMError(); + goto error; + } + + if (virStrToLong_ui(virtfnNames[i] + strlen(virtfnPrefix), + 0, 10, &virtfnIndex) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Invalid virtual function link name '%s' " + "in physical function directory '%s'"), + virtfnNames[i], sysfs_path); + goto error; + } + + if (virtfnIndex >= nVirtfnNames) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Virtual function link name '%s' larger than " + "total count of virtual functions %zu " + "in physical function directory '%s'"), + virtfnNames[i], nVirtfnNames, sysfs_path); + goto error; + } - (*virtual_functions)[*num_virtual_functions] = config_addr; - (*num_virtual_functions)++; + if ((*virtual_functions)[virtfnIndex]) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Virtual function link name '%s' " + "generates duplicate index %zu " + "in physical function directory '%s'"), + virtfnNames[i], nVirtfnNames, sysfs_path); + goto error; + } + + if (virPCIGetDeviceAddressFromSysfsLink(device_link, &config_addr) != + SRIOV_FOUND) { + VIR_WARN("Failed to get SRIOV function from device link '%s'", + device_link); VIR_FREE(device_link); + continue; } + + VIR_DEBUG("Found virtual function %d", virtfnIndex); + (*virtual_functions)[virtfnIndex] = config_addr; + VIR_FREE(device_link); } + VIR_DEBUG("Number of virtual functions: %d", *num_virtual_functions); ret = 0; cleanup: + for (i = 0; i < nVirtfnNames; i++) + VIR_FREE(virtfnNames[i]); + VIR_FREE(virtfnNames); VIR_FREE(device_link); if (dir) closedir(dir); return ret; error: - if (*virtual_functions) { - for (i = 0; i < *num_virtual_functions; i++) - VIR_FREE((*virtual_functions)[i]); - VIR_FREE(*virtual_functions); - } + for (i = 0; i < *num_virtual_functions; i++) + VIR_FREE((*virtual_functions)[i]); + VIR_FREE(*virtual_functions); goto cleanup; } -- 1.8.3.1

2 2

[libvirt] Live migration for containers - Was "RE: libvirt support for LXC"
by Christian Benvenuti (benve) 06 Nov '13

06 Nov '13

Hello, based on the 3D below, it seems that the most logical way to add support for container live migration to libvirt is to integrate the latter with CRIU. If I understand it correctly, Daniel's suggestion below would be that of - 1st converting CRIU to a library - 2nd making libvirt use that library to C/R the container/s CRIU has recently announced support for CRIU as a service and the reason why they opted for a service instead of a library [1] seems to be associated with a use case they had: ability for an application to invoke a self-dump C/R In Libvirt's case it would not be the container to ask for a self dump, but it would be libvirt itself to orchestrate it. In light of the new CRIU as a service feature, is libvirt's preference still that of using a library? Would a service be equally good? Is there anyone actively working or looking at this (libvirt+CRIU)? Thanks /Chris [1] CRIU: Time and Space Travel Service for Linux Applications http://www.youtube.com/watch?v=R2Net9eItBc @37'10'' - CRIU as a system service > -----Original Message----- > From: libvir-list-bounces(a)redhat.com [mailto:libvir-list- > bounces(a)redhat.com] On Behalf Of Dmitry Guryanov > Sent: Wednesday, February 13, 2013 2:58 AM > To: Daniel P. Berrange > Cc: libvir-list(a)redhat.com; criu(a)openvz.org; Kunal Kushwaha > Subject: Re: [libvirt] libvirt support for LXC > > On 130207 15:19:08, Daniel P. Berrange wrote: > > On Thu, Feb 07, 2013 at 03:51:52PM +0530, Kunal Kushwaha wrote: > > > Hi, > > > > > > I am new to this community and am not much aware of current status > > > libvirt work for LXC. > > > > > > Currently I am evaluating LXC with libvirt. > > > and I found still many features like checkpoint/restore/live > > > migration , > > > > Checkpoint/restore/migration is something we'd have to delegate to the > > CRIU code (http://criu.org/LXC) I'm not sure how easy that is todo > > though, since CRIU code is not a nicely reusable library, it is a set > > of command line tools. This will need some investigation. I don't > > believe anyone is actively working on it. > > Please, feel free to contact CRIU development team > (https://lists.openvz.org/mailman/listinfo/criu) > > We are open for discussion and will help with integration.

3 2