November 2010 - Devel - libvirt List Archives

[libvirt] [PATCH] remote: Fix TLS transport on Windows
by Matthias Bolte 23 Nov '10

23 Nov '10

gnulib wraps Windows' SOCKET handle based send() and recv() functions into file descriptor based ones that are use in libvirt. By default GnuTLS uses the SOCKET handle based send() and recv() on Windows. This makes gnutls_handshake() fail internally with a WSAENOTSOCK error because libvirt passes a file descriptor; GnuTLS needs the SOCKET handle. To avoid this mismatch make sure that GnuTLS uses gnulib's replacment functions, by setting custom pull() and push() functions for GnuTLS. --- src/remote/remote_driver.c | 21 +++++++++++++++++++++ 1 files changed, 21 insertions(+), 0 deletions(-) diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index f45476a..87977ab 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -1212,6 +1212,20 @@ initialize_gnutls(void) static int verify_certificate (virConnectPtr conn, struct private_data *priv, gnutls_session_t session); +#if HAVE_WINSOCK2_H +static ssize_t +custom_gnutls_push(void *s, const void *buf, size_t len) +{ + return send((int)(long)s, buf, len, 0); +} + +static ssize_t +custom_gnutls_pull(void *s, void *buf, size_t len) +{ + return recv((int)(long)s, buf, len, 0); +} +#endif + static gnutls_session_t negotiate_gnutls_on_connection (virConnectPtr conn, struct private_data *priv, @@ -1266,6 +1280,13 @@ negotiate_gnutls_on_connection (virConnectPtr conn, gnutls_transport_set_ptr (session, (gnutls_transport_ptr_t) (long) priv->sock); +#if HAVE_WINSOCK2_H + /* Make sure GnuTLS uses gnulib's replacment functions for send() and + * recv() on Windows */ + gnutls_transport_set_push_function(session, custom_gnutls_push); + gnutls_transport_set_pull_function(session, custom_gnutls_pull); +#endif + /* Perform the TLS handshake. */ again: err = gnutls_handshake (session); -- 1.7.0.4

4 9

[libvirt] [PATCH] tests: Fix dispatching internal error reports
by Cole Robinson 23 Nov '10

23 Nov '10

Without this fix, the test suite doesn't print error messages when a libvirt function fails. Signed-off-by: Cole Robinson <crobinso(a)redhat.com> --- tests/testutils.c | 12 +++++++++--- 1 files changed, 9 insertions(+), 3 deletions(-) diff --git a/tests/testutils.c b/tests/testutils.c index 96181b2..8a3439d 100644 --- a/tests/testutils.c +++ b/tests/testutils.c @@ -126,12 +126,18 @@ virtTestRun(const char *title, int nloops, int (*body)(const void *data), const if (ts) GETTIMEOFDAY(&before); + virResetLastError(); - if ((ret = body(data)) != 0) - break; + ret = body(data); virErrorPtr err = virGetLastError(); - if (err) + if (err) { virDispatchError(NULL); + } + + if (ret != 0) { + break; + } + if (ts) { GETTIMEOFDAY(&after); ts[i] = DIFF_MSEC(&after, &before); -- 1.7.3.2

2 3

[libvirt] [PATCH 0/6] Fix state driver hotplug semantics
by Cole Robinson 23 Nov '10

23 Nov '10

Currently, domain hotplug operations in the state drivers (QEMU, LXC, ...) have strange semantics: a hotplug config change will stick around in memory until the domain is either redefined or libvirtd is restarted. This series fixes hotplug config changes to be discarded when the VM is stopped, which AFAICT is the expected behavior. A few bugs with setvcpus implementations are also addressed. Thanks, Cole Cole Robinson (6): domain_conf: Add virDomainObjSetDefTransient Make state driver device hotplug/update actually transient conf: domain: Improve vcpus validation reporting qemu: setvcpus: Fix maxvcpus check qemu: setvcpus: Simplify altering the persistent config qemu: setvcpus: Save config changes to disk src/conf/domain_conf.c | 76 +++++++++++++++++++++++++++++++++++++++-- src/conf/domain_conf.h | 5 +++ src/libvirt_private.syms | 2 + src/lxc/lxc_driver.c | 3 ++ src/qemu/qemu_driver.c | 56 ++++++++++++++--------------- src/test/test_driver.c | 86 +++++++++++++++++++++------------------------- src/uml/uml_driver.c | 5 ++- 7 files changed, 153 insertions(+), 80 deletions(-) -- 1.7.3.2

4 16

[libvirt] RFC: CPU counting in qemu driver
by Jiri Denemark 23 Nov '10

23 Nov '10

Hi all, libvirt's qemu driver doesn't follow the semantics of CPU-related counters in nodeinfo structure, which is nodes : the number of NUMA cell, 1 for uniform mem access sockets : number of CPU socket per node cores : number of core per socket threads : number of threads per core Qemu driver ignores the "per node" part of sockets semantics, and only gives total number of sockets found on the host. That actually makes more sense but we have to fix it since it doesn't follow the documented semantics of public API. That is, we would do something like the following at the end of linuxNodeInfoCPUPopulate(): nodeinfo->sockets /= nodeinfo->nodes; The problem is that NUMA topology is independent on CPU topology and there are systems for which nodeinfo->sockets % nodeinfo->nodes != 0. An example being the following NUMA topology of a system with 4 CPU sockets: node0 CPUs: 0-5 total memory: 8252920 node1 CPUs: 6-11 total memory: 16547840 node2 CPUs: 12-17 total memory: 8273920 node3 CPUs: 18-23 total memory: 16547840 node4 CPUs: 24-29 total memory: 8273920 node5 CPUs: 30-35 total memory: 16547840 node6 CPUs: 36-41 total memory: 8273920 node7 CPUs: 42-47 total memory: 16547840 which shows that the cores are actually mapped via the AMD intra-socket interconnects. Note that this funky topology was verified to be correct so it's not just a kernel bug which would result in wrong topology being reported. So the suggested calculation wouldn't work on such systems and we cannot really follow the API semantics since it doesn't work in this case. My suggestion is to use the following code in linuxNodeInfoCPUPopulate(): if (nodeinfo->sockets % nodeinfo->nodes == 0) nodeinfo->sockets /= nodeinfo->nodes; else nodeinfo->nodes = 1; That is we would lie about number of NUMA nodes on funky systems. If nodeinfo->nodes is greater than 1, then applications can rely on it being correct. If it's 1, applications that care about NUMA topology should consult /capabilities/host/topology/cells of capabilities XML to check the number of NUMA nodes in a reliable way, which I guess such applications would do anyway. However, if you have a better idea of fixing the issue while staying more compatible with current semantics, don't hesitate to share it. Note, that we have VIR_NODEINFO_MAXCPUS macro in libvirt.h which computes maximum number of CPUs as (nodes * sockets * cores * threads) and we need to keep this working. Jirka

2 6

[libvirt] [PATCH 0/4] Support the QED disk image format (V2)
by Adam Litke 23 Nov '10

23 Nov '10

Changes since V1: - Fix virStorageFileMatchesVersion() for formats without version info - Allow backingStore format probing for QED images since it is safe Qemu is about to gain support for a new disk image format: QED. Details on this format (including specification) can be found here: http://wiki.qemu.org/Features/QED. This short series of patches allows QED images to be used with libvirt. Adam Litke (4): Allow probing of image formats without version information QED: Basic support for QED images storage_file: Add a new flag to mark backing files that are safe to probe Support for probing qed image metadata src/conf/domain_conf.c | 4 ++ src/util/storage_file.c | 89 +++++++++++++++++++++++++++++++++++++++++++++-- src/util/storage_file.h | 2 + 3 files changed, 92 insertions(+), 3 deletions(-) -- 1.7.3.2.164.g6f10c

4 15

[libvirt] [PATCH] Log all errors at level INFO to stop polluting syslog
by Daniel P. Berrange 23 Nov '10

23 Nov '10

Everytime a public API returns an error, libvirtd pollutes syslog with that error message. Reduce the error logging level to INFO so these don't appear by default. * src/util/virterror.c: Log all errors at INFO --- src/util/virterror.c | 14 +------------- 1 files changed, 1 insertions(+), 13 deletions(-) diff --git a/src/util/virterror.c b/src/util/virterror.c index d524d04..ecd9fc9 100644 --- a/src/util/virterror.c +++ b/src/util/virterror.c @@ -64,18 +64,6 @@ void *virUserData = NULL; /* associated data */ }} \ } -static virLogPriority virErrorLevelPriority(virErrorLevel level) { - switch (level) { - case VIR_ERR_NONE: - return(VIR_LOG_INFO); - case VIR_ERR_WARNING: - return(VIR_LOG_WARN); - case VIR_ERR_ERROR: - return(VIR_LOG_ERROR); - } - return(VIR_LOG_ERROR); -} - static const char *virErrorDomainName(virErrorDomain domain) { const char *dom = "unknown"; switch (domain) { @@ -719,7 +707,7 @@ virRaiseErrorFull(virConnectPtr conn ATTRIBUTE_UNUSED, * Hook up the error or warning to the logging facility * XXXX should we include filename as 'category' instead of domain name ? */ - virLogMessage(virErrorDomainName(domain), virErrorLevelPriority(level), + virLogMessage(virErrorDomainName(domain), VIR_LOG_INFO, funcname, linenr, 1, "%s", str); /* -- 1.7.2.3

2 5

[libvirt] [PATCH 00/10] virCommand
by Eric Blake 23 Nov '10

23 Nov '10

This revives a couple of patch series from the past, and rebases them on top of the current tree. I've tested that the daemon hook on SIGHUP works with this rewrite, so I'm fairly confident that it is in decent shape, but there's still a lot more to do for both converting VIR_REALLOC_N to newer VIR_EXPAND_N or VIR_RESIZE_N, as well as converting virExec uses to virCommand. I wanted to get this posted for first reviews while I continue porting more instances. Daniel P. Berrange (5): Fix bug in setting up child stderr/out with /dev/null Remove bogus includes Introduce new APIs for spawning processes virCommand: docs for usage of new command APIs Port hooks and iptables code to new command execution APIs Eric Blake (5): memory: make it safer to expand arrays memory: make it easier to avoid quadratic scaling of arrays daemon: use safer memory growth macros capabilities, cpu: use new array API maint: tighten strncmp syntax check .x-sc_avoid_write | 1 + .x-sc_prohibit_strcmp_and_strncmp | 9 - .x-sc_prohibit_strncmp | 1 + HACKING | 59 ++- Makefile.am | 2 +- cfg.mk | 15 +- daemon/event.c | 44 +- daemon/libvirtd.c | 8 +- daemon/libvirtd.h | 11 +- docs/Makefile.am | 11 +- docs/hacking.html.in | 64 ++- docs/internals.html.in | 9 + docs/internals/command.html.in | 491 +++++++++++++++++++ docs/sitemap.html.in | 4 + docs/subsite.xsl | 25 + po/POTFILES.in | 1 + src/Makefile.am | 1 + src/conf/capabilities.c | 37 +- src/conf/capabilities.h | 20 +- src/conf/cpu_conf.c | 7 +- src/conf/cpu_conf.h | 5 +- src/conf/domain_conf.c | 1 - src/libvirt_private.syms | 32 ++ src/util/command.c | 960 +++++++++++++++++++++++++++++++++++++ src/util/command.h | 213 ++++++++ src/util/hooks.c | 217 +-------- src/util/iptables.c | 73 +--- src/util/memory.c | 94 ++++- src/util/memory.h | 77 +++- src/util/util.c | 2 +- tests/.gitignore | 4 + tests/Makefile.am | 18 +- tests/commanddata/test10.log | 14 + tests/commanddata/test11.log | 14 + tests/commanddata/test12.log | 12 + tests/commanddata/test13.log | 12 + tests/commanddata/test14.log | 12 + tests/commanddata/test15.log | 12 + tests/commanddata/test2.log | 12 + tests/commanddata/test3.log | 14 + tests/commanddata/test4.log | 12 + tests/commanddata/test5.log | 10 + tests/commanddata/test6.log | 6 + tests/commanddata/test7.log | 11 + tests/commanddata/test8.log | 7 + tests/commanddata/test9.log | 18 + tests/commandhelper.c | 136 ++++++ tests/commandtest.c | 572 ++++++++++++++++++++++ tests/testutilsqemu.c | 1 + 49 files changed, 3009 insertions(+), 382 deletions(-) delete mode 100644 .x-sc_prohibit_strcmp_and_strncmp create mode 100644 .x-sc_prohibit_strncmp create mode 100644 docs/internals/command.html.in create mode 100644 docs/subsite.xsl create mode 100644 src/util/command.c create mode 100644 src/util/command.h create mode 100644 tests/commanddata/test10.log create mode 100644 tests/commanddata/test11.log create mode 100644 tests/commanddata/test12.log create mode 100644 tests/commanddata/test13.log create mode 100644 tests/commanddata/test14.log create mode 100644 tests/commanddata/test15.log create mode 100644 tests/commanddata/test2.log create mode 100644 tests/commanddata/test3.log create mode 100644 tests/commanddata/test4.log create mode 100644 tests/commanddata/test5.log create mode 100644 tests/commanddata/test6.log create mode 100644 tests/commanddata/test7.log create mode 100644 tests/commanddata/test8.log create mode 100644 tests/commanddata/test9.log create mode 100644 tests/commandhelper.c create mode 100644 tests/commandtest.c -- 1.7.3.2

5 32

[libvirt] [PATCH] Fix 32-bit int truncation in QED header check
by Daniel P. Berrange 23 Nov '10

23 Nov '10

* src/util/memory.c: Avoid 32-bit truncation extracting a 64bit int --- src/util/storage_file.c | 16 ++++++++-------- 1 files changed, 8 insertions(+), 8 deletions(-) diff --git a/src/util/storage_file.c b/src/util/storage_file.c index aa117e7..37d3ecf 100644 --- a/src/util/storage_file.c +++ b/src/util/storage_file.c @@ -437,14 +437,14 @@ qedGetHeaderUL(const unsigned char *loc) static unsigned long long qedGetHeaderULL(const unsigned char *loc) { - return ( ((unsigned long)loc[7] << 56) - | ((unsigned long)loc[6] << 48) - | ((unsigned long)loc[5] << 40) - | ((unsigned long)loc[4] << 32) - | ((unsigned long)loc[3] << 24) - | ((unsigned long)loc[2] << 16) - | ((unsigned long)loc[1] << 8) - | ((unsigned long)loc[0] << 0)); + return ( ((unsigned long long)loc[7] << 56) + | ((unsigned long long)loc[6] << 48) + | ((unsigned long long)loc[5] << 40) + | ((unsigned long long)loc[4] << 32) + | ((unsigned long long)loc[3] << 24) + | ((unsigned long long)loc[2] << 16) + | ((unsigned long long)loc[1] << 8) + | ((unsigned long long)loc[0] << 0)); } static int -- 1.7.2.3

2 1

[libvirt] [PATCH] Rename 'remove' param to 'toremove' to avoid clash with Win32 header
by Daniel P. Berrange 23 Nov '10

23 Nov '10

The win32 headers have a function called 'remove' declared. This clashes with the 'remove' parameter in virShrinkN * src/util/memory.c: Rename 'remove' to 'toremove' --- src/util/memory.c | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/src/util/memory.c b/src/util/memory.c index 96222db..f5061c0 100644 --- a/src/util/memory.c +++ b/src/util/memory.c @@ -237,18 +237,18 @@ int virResizeN(void *ptrptr, size_t size, size_t *allocptr, size_t count, * @ptrptr: pointer to pointer for address of allocated memory * @size: number of bytes per element * @countptr: pointer to number of elements in array - * @remove: number of elements to remove + * @toremove: number of elements to remove * * Resize the block of memory in 'ptrptr' to be an array of - * '*countptr' - 'remove' elements, each 'size' bytes in length. + * '*countptr' - 'toremove' elements, each 'size' bytes in length. * Update 'ptrptr' and 'countptr' with the details of the newly - * allocated memory. If 'remove' is larger than 'countptr', free + * allocated memory. If 'toremove' is larger than 'countptr', free * the entire array. */ -void virShrinkN(void *ptrptr, size_t size, size_t *countptr, size_t remove) +void virShrinkN(void *ptrptr, size_t size, size_t *countptr, size_t toremove) { - if (remove < *countptr) - ignore_value(virReallocN(ptrptr, size, *countptr -= remove)); + if (toremove < *countptr) + ignore_value(virReallocN(ptrptr, size, *countptr -= toremove)); else { virFree(ptrptr); *countptr = 0; -- 1.7.2.3

2 1

[libvirt] [PATCH] Ensure virExec preserves logging environment
by Daniel P. Berrange 23 Nov '10

23 Nov '10

The virFork call resets all logging handlers that may have been set. Re-enable them after fork in virExec, so that env variables fir LIBVIRT_LOG_OUTPUTS and LIBVIRT_LOG_FILTERS take effect until the execve() * src/util/util.c: Preserve logging in child in virExec --- src/util/util.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/src/util/util.c b/src/util/util.c index f2fe58a..a0849e1 100644 --- a/src/util/util.c +++ b/src/util/util.c @@ -602,6 +602,9 @@ __virExec(const char *const*argv, childout = -1; } + /* Initialize full logging for a while */ + virLogSetFromEnv(); + /* Daemonize as late as possible, so the parent process can detect * the above errors with wait* */ if (flags & VIR_EXEC_DAEMON) { @@ -650,6 +653,9 @@ __virExec(const char *const*argv, virClearCapabilities() < 0) goto fork_error; + /* Close logging again to ensure no FDs leak to child */ + virLogReset(); + if (envp) execve(argv[0], (char **) argv, (char**)envp); else -- 1.7.2.3

2 3