- Devel - libvirt List Archives

[libvirt] RFC: CPU counting in qemu driver
by Jiri Denemark 24 Nov '10

24 Nov '10

Hi all, libvirt's qemu driver doesn't follow the semantics of CPU-related counters in nodeinfo structure, which is nodes : the number of NUMA cell, 1 for uniform mem access sockets : number of CPU socket per node cores : number of core per socket threads : number of threads per core Qemu driver ignores the "per node" part of sockets semantics, and only gives total number of sockets found on the host. That actually makes more sense but we have to fix it since it doesn't follow the documented semantics of public API. That is, we would do something like the following at the end of linuxNodeInfoCPUPopulate(): nodeinfo->sockets /= nodeinfo->nodes; The problem is that NUMA topology is independent on CPU topology and there are systems for which nodeinfo->sockets % nodeinfo->nodes != 0. An example being the following NUMA topology of a system with 4 CPU sockets: node0 CPUs: 0-5 total memory: 8252920 node1 CPUs: 6-11 total memory: 16547840 node2 CPUs: 12-17 total memory: 8273920 node3 CPUs: 18-23 total memory: 16547840 node4 CPUs: 24-29 total memory: 8273920 node5 CPUs: 30-35 total memory: 16547840 node6 CPUs: 36-41 total memory: 8273920 node7 CPUs: 42-47 total memory: 16547840 which shows that the cores are actually mapped via the AMD intra-socket interconnects. Note that this funky topology was verified to be correct so it's not just a kernel bug which would result in wrong topology being reported. So the suggested calculation wouldn't work on such systems and we cannot really follow the API semantics since it doesn't work in this case. My suggestion is to use the following code in linuxNodeInfoCPUPopulate(): if (nodeinfo->sockets % nodeinfo->nodes == 0) nodeinfo->sockets /= nodeinfo->nodes; else nodeinfo->nodes = 1; That is we would lie about number of NUMA nodes on funky systems. If nodeinfo->nodes is greater than 1, then applications can rely on it being correct. If it's 1, applications that care about NUMA topology should consult /capabilities/host/topology/cells of capabilities XML to check the number of NUMA nodes in a reliable way, which I guess such applications would do anyway. However, if you have a better idea of fixing the issue while staying more compatible with current semantics, don't hesitate to share it. Note, that we have VIR_NODEINFO_MAXCPUS macro in libvirt.h which computes maximum number of CPUs as (nodes * sockets * cores * threads) and we need to keep this working. Jirka

2 6

[libvirt] [PATCH 0/4] Support the QED disk image format (V2)
by Adam Litke 24 Nov '10

24 Nov '10

Changes since V1: - Fix virStorageFileMatchesVersion() for formats without version info - Allow backingStore format probing for QED images since it is safe Qemu is about to gain support for a new disk image format: QED. Details on this format (including specification) can be found here: http://wiki.qemu.org/Features/QED. This short series of patches allows QED images to be used with libvirt. Adam Litke (4): Allow probing of image formats without version information QED: Basic support for QED images storage_file: Add a new flag to mark backing files that are safe to probe Support for probing qed image metadata src/conf/domain_conf.c | 4 ++ src/util/storage_file.c | 89 +++++++++++++++++++++++++++++++++++++++++++++-- src/util/storage_file.h | 2 + 3 files changed, 92 insertions(+), 3 deletions(-) -- 1.7.3.2.164.g6f10c

4 15

[libvirt] [PATCH] Log all errors at level INFO to stop polluting syslog
by Daniel P. Berrange 24 Nov '10

24 Nov '10

Everytime a public API returns an error, libvirtd pollutes syslog with that error message. Reduce the error logging level to INFO so these don't appear by default. * src/util/virterror.c: Log all errors at INFO --- src/util/virterror.c | 14 +------------- 1 files changed, 1 insertions(+), 13 deletions(-) diff --git a/src/util/virterror.c b/src/util/virterror.c index d524d04..ecd9fc9 100644 --- a/src/util/virterror.c +++ b/src/util/virterror.c @@ -64,18 +64,6 @@ void *virUserData = NULL; /* associated data */ }} \ } -static virLogPriority virErrorLevelPriority(virErrorLevel level) { - switch (level) { - case VIR_ERR_NONE: - return(VIR_LOG_INFO); - case VIR_ERR_WARNING: - return(VIR_LOG_WARN); - case VIR_ERR_ERROR: - return(VIR_LOG_ERROR); - } - return(VIR_LOG_ERROR); -} - static const char *virErrorDomainName(virErrorDomain domain) { const char *dom = "unknown"; switch (domain) { @@ -719,7 +707,7 @@ virRaiseErrorFull(virConnectPtr conn ATTRIBUTE_UNUSED, * Hook up the error or warning to the logging facility * XXXX should we include filename as 'category' instead of domain name ? */ - virLogMessage(virErrorDomainName(domain), virErrorLevelPriority(level), + virLogMessage(virErrorDomainName(domain), VIR_LOG_INFO, funcname, linenr, 1, "%s", str); /* -- 1.7.2.3

2 5

[libvirt] [PATCH 00/10] virCommand
by Eric Blake 23 Nov '10

23 Nov '10

This revives a couple of patch series from the past, and rebases them on top of the current tree. I've tested that the daemon hook on SIGHUP works with this rewrite, so I'm fairly confident that it is in decent shape, but there's still a lot more to do for both converting VIR_REALLOC_N to newer VIR_EXPAND_N or VIR_RESIZE_N, as well as converting virExec uses to virCommand. I wanted to get this posted for first reviews while I continue porting more instances. Daniel P. Berrange (5): Fix bug in setting up child stderr/out with /dev/null Remove bogus includes Introduce new APIs for spawning processes virCommand: docs for usage of new command APIs Port hooks and iptables code to new command execution APIs Eric Blake (5): memory: make it safer to expand arrays memory: make it easier to avoid quadratic scaling of arrays daemon: use safer memory growth macros capabilities, cpu: use new array API maint: tighten strncmp syntax check .x-sc_avoid_write | 1 + .x-sc_prohibit_strcmp_and_strncmp | 9 - .x-sc_prohibit_strncmp | 1 + HACKING | 59 ++- Makefile.am | 2 +- cfg.mk | 15 +- daemon/event.c | 44 +- daemon/libvirtd.c | 8 +- daemon/libvirtd.h | 11 +- docs/Makefile.am | 11 +- docs/hacking.html.in | 64 ++- docs/internals.html.in | 9 + docs/internals/command.html.in | 491 +++++++++++++++++++ docs/sitemap.html.in | 4 + docs/subsite.xsl | 25 + po/POTFILES.in | 1 + src/Makefile.am | 1 + src/conf/capabilities.c | 37 +- src/conf/capabilities.h | 20 +- src/conf/cpu_conf.c | 7 +- src/conf/cpu_conf.h | 5 +- src/conf/domain_conf.c | 1 - src/libvirt_private.syms | 32 ++ src/util/command.c | 960 +++++++++++++++++++++++++++++++++++++ src/util/command.h | 213 ++++++++ src/util/hooks.c | 217 +-------- src/util/iptables.c | 73 +--- src/util/memory.c | 94 ++++- src/util/memory.h | 77 +++- src/util/util.c | 2 +- tests/.gitignore | 4 + tests/Makefile.am | 18 +- tests/commanddata/test10.log | 14 + tests/commanddata/test11.log | 14 + tests/commanddata/test12.log | 12 + tests/commanddata/test13.log | 12 + tests/commanddata/test14.log | 12 + tests/commanddata/test15.log | 12 + tests/commanddata/test2.log | 12 + tests/commanddata/test3.log | 14 + tests/commanddata/test4.log | 12 + tests/commanddata/test5.log | 10 + tests/commanddata/test6.log | 6 + tests/commanddata/test7.log | 11 + tests/commanddata/test8.log | 7 + tests/commanddata/test9.log | 18 + tests/commandhelper.c | 136 ++++++ tests/commandtest.c | 572 ++++++++++++++++++++++ tests/testutilsqemu.c | 1 + 49 files changed, 3009 insertions(+), 382 deletions(-) delete mode 100644 .x-sc_prohibit_strcmp_and_strncmp create mode 100644 .x-sc_prohibit_strncmp create mode 100644 docs/internals/command.html.in create mode 100644 docs/subsite.xsl create mode 100644 src/util/command.c create mode 100644 src/util/command.h create mode 100644 tests/commanddata/test10.log create mode 100644 tests/commanddata/test11.log create mode 100644 tests/commanddata/test12.log create mode 100644 tests/commanddata/test13.log create mode 100644 tests/commanddata/test14.log create mode 100644 tests/commanddata/test15.log create mode 100644 tests/commanddata/test2.log create mode 100644 tests/commanddata/test3.log create mode 100644 tests/commanddata/test4.log create mode 100644 tests/commanddata/test5.log create mode 100644 tests/commanddata/test6.log create mode 100644 tests/commanddata/test7.log create mode 100644 tests/commanddata/test8.log create mode 100644 tests/commanddata/test9.log create mode 100644 tests/commandhelper.c create mode 100644 tests/commandtest.c -- 1.7.3.2

5 32

[libvirt] [PATCH] Fix 32-bit int truncation in QED header check
by Daniel P. Berrange 23 Nov '10

23 Nov '10

* src/util/memory.c: Avoid 32-bit truncation extracting a 64bit int --- src/util/storage_file.c | 16 ++++++++-------- 1 files changed, 8 insertions(+), 8 deletions(-) diff --git a/src/util/storage_file.c b/src/util/storage_file.c index aa117e7..37d3ecf 100644 --- a/src/util/storage_file.c +++ b/src/util/storage_file.c @@ -437,14 +437,14 @@ qedGetHeaderUL(const unsigned char *loc) static unsigned long long qedGetHeaderULL(const unsigned char *loc) { - return ( ((unsigned long)loc[7] << 56) - | ((unsigned long)loc[6] << 48) - | ((unsigned long)loc[5] << 40) - | ((unsigned long)loc[4] << 32) - | ((unsigned long)loc[3] << 24) - | ((unsigned long)loc[2] << 16) - | ((unsigned long)loc[1] << 8) - | ((unsigned long)loc[0] << 0)); + return ( ((unsigned long long)loc[7] << 56) + | ((unsigned long long)loc[6] << 48) + | ((unsigned long long)loc[5] << 40) + | ((unsigned long long)loc[4] << 32) + | ((unsigned long long)loc[3] << 24) + | ((unsigned long long)loc[2] << 16) + | ((unsigned long long)loc[1] << 8) + | ((unsigned long long)loc[0] << 0)); } static int -- 1.7.2.3

2 1

[libvirt] [PATCH] Rename 'remove' param to 'toremove' to avoid clash with Win32 header
by Daniel P. Berrange 23 Nov '10

23 Nov '10

The win32 headers have a function called 'remove' declared. This clashes with the 'remove' parameter in virShrinkN * src/util/memory.c: Rename 'remove' to 'toremove' --- src/util/memory.c | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/src/util/memory.c b/src/util/memory.c index 96222db..f5061c0 100644 --- a/src/util/memory.c +++ b/src/util/memory.c @@ -237,18 +237,18 @@ int virResizeN(void *ptrptr, size_t size, size_t *allocptr, size_t count, * @ptrptr: pointer to pointer for address of allocated memory * @size: number of bytes per element * @countptr: pointer to number of elements in array - * @remove: number of elements to remove + * @toremove: number of elements to remove * * Resize the block of memory in 'ptrptr' to be an array of - * '*countptr' - 'remove' elements, each 'size' bytes in length. + * '*countptr' - 'toremove' elements, each 'size' bytes in length. * Update 'ptrptr' and 'countptr' with the details of the newly - * allocated memory. If 'remove' is larger than 'countptr', free + * allocated memory. If 'toremove' is larger than 'countptr', free * the entire array. */ -void virShrinkN(void *ptrptr, size_t size, size_t *countptr, size_t remove) +void virShrinkN(void *ptrptr, size_t size, size_t *countptr, size_t toremove) { - if (remove < *countptr) - ignore_value(virReallocN(ptrptr, size, *countptr -= remove)); + if (toremove < *countptr) + ignore_value(virReallocN(ptrptr, size, *countptr -= toremove)); else { virFree(ptrptr); *countptr = 0; -- 1.7.2.3

2 1

[libvirt] [PATCH] Ensure virExec preserves logging environment
by Daniel P. Berrange 23 Nov '10

23 Nov '10

The virFork call resets all logging handlers that may have been set. Re-enable them after fork in virExec, so that env variables fir LIBVIRT_LOG_OUTPUTS and LIBVIRT_LOG_FILTERS take effect until the execve() * src/util/util.c: Preserve logging in child in virExec --- src/util/util.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/src/util/util.c b/src/util/util.c index f2fe58a..a0849e1 100644 --- a/src/util/util.c +++ b/src/util/util.c @@ -602,6 +602,9 @@ __virExec(const char *const*argv, childout = -1; } + /* Initialize full logging for a while */ + virLogSetFromEnv(); + /* Daemonize as late as possible, so the parent process can detect * the above errors with wait* */ if (flags & VIR_EXEC_DAEMON) { @@ -650,6 +653,9 @@ __virExec(const char *const*argv, virClearCapabilities() < 0) goto fork_error; + /* Close logging again to ensure no FDs leak to child */ + virLogReset(); + if (envp) execve(argv[0], (char **) argv, (char**)envp); else -- 1.7.2.3

2 3

[libvirt] Misc improvements to the storage driver
by Daniel P. Berrange 23 Nov '10

23 Nov '10

This patch series started out as an attempt to fix the seriously painful problem with iSCSI targets automatically logging in whenever the network came online. In the process of fixing this alot of other cruft is cleaned up docs/schemas/storagepool.rng | 5 src/conf/storage_conf.c | 21 + src/conf/storage_conf.h | 1 src/storage/storage_backend.h | 3 src/storage/storage_backend_fs.c | 29 ++ src/storage/storage_backend_iscsi.c | 427 +++++++++++++++++++++++++--------- src/storage/storage_backend_logical.c | 24 + src/storage/storage_backend_mpath.c | 20 + src/storage/storage_backend_scsi.c | 96 +++++++ src/storage/storage_driver.c | 65 +++-- tools/virsh.c | 51 ++-- 11 files changed, 573 insertions(+), 169 deletions(-) Daniel

5 40

[libvirt] New features : cloning
by Clément Valentin 23 Nov '10

23 Nov '10

Hi everyone ! I'm currently developing a big project using libvirt to manage ESXi and Xen hypervisor. I was wondering to know if someday libvirt will propose a "cloning" function ? Thanks for taking time to consider my request. Valentin

1 0

[libvirt] [PATCH] Don't catch SIGCHLD in libvirtd
by Daniel P. Berrange 23 Nov '10

23 Nov '10

libvirtd no longer deals with SIGCHLD in its signal handler since the QEMU driver switched to always daemonize processes. Thus remove the sigaction for it, to avoid warning log messages * daemon/libvirtd.c: Don't catch SIGCHLD --- daemon/libvirtd.c | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/daemon/libvirtd.c b/daemon/libvirtd.c index e544c48..2eb2374 100644 --- a/daemon/libvirtd.c +++ b/daemon/libvirtd.c @@ -2973,7 +2973,6 @@ daemonSetupSignals(struct qemud_server *server) sigaction(SIGINT, &sig_action, NULL); sigaction(SIGQUIT, &sig_action, NULL); sigaction(SIGTERM, &sig_action, NULL); - sigaction(SIGCHLD, &sig_action, NULL); sig_action.sa_handler = SIG_IGN; sigaction(SIGPIPE, &sig_action, NULL); -- 1.7.2.3

2 1