- Devel - libvirt List Archives

[libvirt] [libvirt-perl PATCH 0/2] Add new flags
by John Ferlan 07 Oct '16

07 Oct '16

Patch 1 is related to the on list iotune length/duration parameter Patch 2 is related to the already pushed vcpu hotpluggable flag These would be pushed once I push the blkdeviotune changes - the order was only because I made the blkdeviotune changes, then built and found that the vcpu hotpluggable flag cause test failure. John Ferlan (2): Add blkdeviotune length parameters Add VIR_DOMAIN_VCPU_HOTPLUGGABLE Changes | 3 ++- Virt.xs | 13 +++++++++++++ lib/Sys/Virt/Domain.pm | 29 +++++++++++++++++++++++++++++ 3 files changed, 44 insertions(+), 1 deletion(-) -- 2.7.4

2 5

[libvirt] PCP libvirt Plugin
by Marko Myllynen 07 Oct '16

07 Oct '16

Hi, FYI, I've written a PCP plugin (PMDA in PCP parlance) to support most hypervisor / domain information and metrics available over the libvirt Python API, it's up to date as of libvirt 2.3 (so it already supports the recently added perf event metrics). In case you haven't heard about PCP, here's a brief intro: The Performance Co-Pilot (PCP, http://www.pcp.io/) system is a toolkit for collecting, archiving, and processing performance metrics. It supports system and application metrics from local and remote hosts, archives, and containers. A typical Linux PCP installation offers over 1,000 metrics by default and is easily extensible with its own plugins, or PMDAs ("Performance Metrics Domain Agents"). In addition to very complete /proc based statistics, readily available PCP PMDAs provide support components like 389 Directory Server, Apache, Ceph, GFS2, Gluster, MySQL, NFS, Oracle, Postfix, PostgreSQL, Samba, and Sendmail, among others. PCP also runs on many platforms, including Linux, Mac OS X, FreeBSD, NetBSD, Solaris, and Windows. And here's Quick Guide to get started with PCP: http://www.pcp.io/docs/guide.html (So PCP comes with handy clients like pmrep(1) for custom metrics reporting, sophisticated logging and inferencing tools, and support for Grafana based Web UI.) The libvirt metrics available with PCP libvirt PMDA and their descriptions are listed below, they include all the recently added perf event metrics as well as combined and per-device metrics for each VM. The plugin is available for Fedora in updates-testing and for Ubuntu from https://bintray.com/pcp/ . As this is a recent addition, the plugin or some of the later additions like per-device metrics have not propagated to all distributions yet. For those using PCP already, they may also manually grab the needed files from the below page without the need to upgrade any PCP packages: https://myllynen.fedorapeople.org/pmdalibvirt/ Upstream code lives here: http://oss.sgi.com/cgi-bin/gitweb.cgi?p=pcp/pcp.git;a=tree;f=src/pmdas/libv… I wonder could this be mentioned at https://libvirt.org/apps.html ? Here's the list of the PCP metrics available with the current git code: libvirt.dominfo.container [VM container] libvirt.dominfo.description [VM description] libvirt.dominfo.memory.boot [VM memory, at boot] libvirt.dominfo.memory.current [VM memory, current] libvirt.dominfo.memory.max [VM memory, maximum] libvirt.dominfo.name [VM name] libvirt.dominfo.os.type [VM OS type] libvirt.dominfo.title [VM title] libvirt.dominfo.type [VM type] libvirt.dominfo.uuid [VM UUID] libvirt.dominfo.vcpu.current [VM vCPUs, current] libvirt.dominfo.vcpu.max [VM vCPUs, maximum] libvirt.domstats.balloon.current [VM balloon size, current] libvirt.domstats.balloon.maximum [VM balloon size, maximum] libvirt.domstats.block.all.allocation [VM backing imgs, total allocation] libvirt.domstats.block.all.backingIndex [VM block devs, backing chain imgs] libvirt.domstats.block.all.capacity [VM backing imgs, total capacity] libvirt.domstats.block.all.fl.reqs [VM block devs, total fl reqs] libvirt.domstats.block.all.fl.times [VM block devs, total fl times] libvirt.domstats.block.all.name [VM block devs, all names] libvirt.domstats.block.all.path [VM block devs, all paths] libvirt.domstats.block.all.physical [VM backing imgs, total physical] libvirt.domstats.block.all.rd.bytes [VM block devs, total rd bytes] libvirt.domstats.block.all.rd.reqs [VM block devs, total rd reqs] libvirt.domstats.block.all.rd.times [VM block devs, total rd times] libvirt.domstats.block.all.wr.bytes [VM block devs, total wr bytes] libvirt.domstats.block.all.wr.reqs [VM block devs, total wr reqs] libvirt.domstats.block.all.wr.times [VM block devs, total wr times] libvirt.domstats.block.allocation [VM backing img, allocation] libvirt.domstats.block.backingIndex [VM block dev, backing chain img] libvirt.domstats.block.capacity [VM backing img, capacity] libvirt.domstats.block.count [VM block devs, count] libvirt.domstats.block.fl.reqs [VM block dev, fl reqs] libvirt.domstats.block.fl.times [VM block dev, fl times] libvirt.domstats.block.name [VM block dev, name] libvirt.domstats.block.path [VM block dev, path] libvirt.domstats.block.physical [VM backing img, physical] libvirt.domstats.block.rd.bytes [VM block dev, rd bytes] libvirt.domstats.block.rd.reqs [VM block dev, rd reqs] libvirt.domstats.block.rd.times [VM block dev, rd times] libvirt.domstats.block.wr.bytes [VM block dev, wr bytes] libvirt.domstats.block.wr.reqs [VM block dev, wr reqs] libvirt.domstats.block.wr.times [VM block dev, wr times] libvirt.domstats.cpu.system [VM CPU time, system] libvirt.domstats.cpu.time [VM CPU time, total] libvirt.domstats.cpu.user [VM CPU time, user] libvirt.domstats.mem.actual [VM memory, balloon size] libvirt.domstats.mem.available [VM memory, available] libvirt.domstats.mem.major_fault [VM memory, major faults] libvirt.domstats.mem.minor_fault [VM memory, minor faults] libvirt.domstats.mem.rss [VM memory, proc RSS] libvirt.domstats.mem.swap_in [VM memory, swapped in total] libvirt.domstats.mem.swap_out [VM memory, swapped out total] libvirt.domstats.mem.unused [VM memory, unused] libvirt.domstats.net.all.name [VM NICs, all names] libvirt.domstats.net.all.rx.bytes [VM NICs, total rx bytes] libvirt.domstats.net.all.rx.drop [VM NICs, total rx drop] libvirt.domstats.net.all.rx.errs [VM NICs, total rx errs] libvirt.domstats.net.all.rx.pkts [VM NICs, total rx pkts] libvirt.domstats.net.all.tx.bytes [VM NICs, total tx bytes] libvirt.domstats.net.all.tx.drop [VM NICs, total tx drop] libvirt.domstats.net.all.tx.errs [VM NICs, total tx errs] libvirt.domstats.net.all.tx.pkts [VM NICs, total tx pkts] libvirt.domstats.net.count [VM NICs, count] libvirt.domstats.net.name [VM NIC, name] libvirt.domstats.net.rx.bytes [VM NIC, rx bytes] libvirt.domstats.net.rx.drop [VM NIC, rx drop] libvirt.domstats.net.rx.errs [VM NIC, rx errs] libvirt.domstats.net.rx.pkts [VM NIC, rx pkts] libvirt.domstats.net.tx.bytes [VM NIC, tx bytes] libvirt.domstats.net.tx.drop [VM NIC, tx drop] libvirt.domstats.net.tx.errs [VM NIC, tx errs] libvirt.domstats.net.tx.pkts [VM NIC, tx pkts] libvirt.domstats.perf.cache_misses [VM perf stats, cache misses] libvirt.domstats.perf.cache_references [VM perf stats, cache references] libvirt.domstats.perf.cmt [VM perf stats, cmt] libvirt.domstats.perf.cpu_cycles [VM perf stats, CPU cycles/instr] libvirt.domstats.perf.instructions [VM perf stats, instructions count] libvirt.domstats.perf.mbml [VM perf stats, mbml] libvirt.domstats.perf.mbmt [VM perf stats, mbmt] libvirt.domstats.vcpu.all.state [VM vCPUs, total state] libvirt.domstats.vcpu.all.time [VM vCPUs, total time] libvirt.domstats.vcpu.all.wait [VM vCPUs, total wait] libvirt.domstats.vcpu.current [VM vCPUs, current] libvirt.domstats.vcpu.maximum [VM vCPUs, maximum] libvirt.domstats.vcpu.state [VM vCPU, state] libvirt.domstats.vcpu.time [VM vCPU, time] libvirt.domstats.vcpu.wait [VM vCPU, wait] libvirt.hv.devices [Libvirt devices] libvirt.hv.domains.active [Libvirt domains, active] libvirt.hv.domains.inactive [Libvirt domains, inactive] libvirt.hv.driver [Libvirt driver] libvirt.hv.interfaces.active [Libvirt interfaces, active] libvirt.hv.interfaces.inactive [Libvirt interfaces, inactive] libvirt.hv.networkfilters [Libvirt network filters] libvirt.hv.networks.active [Libvirt networks, active] libvirt.hv.networks.inactive [Libvirt networks, inactive] libvirt.hv.secrets [Libvirt secrets] libvirt.hv.storagepools.active [Libvirt storage pools, active] libvirt.hv.storagepools.inactive [Libvirt storage pools, inactive] libvirt.hv.uri [Libvirt URI] libvirt.hv.version [Libvirt version] Thanks, -- Marko Myllynen

3 4

[libvirt] [PATCH] virCgroupDetect: Print pid as long long
by Michal Privoznik 07 Oct '16

07 Oct '16

Pids are signed, report them as such. Before: Detected mount/mapping 2:cpuset at /sys/fs/cgroup/cpuset in /machine/qemu-4-fedora.libvirt-qemu/vcpu0 for pid 18446744073709551615 After: Detected mount/mapping 2:cpuset at /sys/fs/cgroup/cpuset in /machine/qemu-4-fedora.libvirt-qemu/vcpu0 for pid -1 Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/util/vircgroup.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/util/vircgroup.c b/src/util/vircgroup.c index 8b52816..0a06a8a 100644 --- a/src/util/vircgroup.c +++ b/src/util/vircgroup.c @@ -732,11 +732,12 @@ virCgroupDetect(virCgroupPtr group, return -1; } - VIR_DEBUG("Detected mount/mapping %zu:%s at %s in %s for pid %llu", i, + VIR_DEBUG("Detected mount/mapping %zu:%s at %s in %s for pid %lld", + i, virCgroupControllerTypeToString(i), group->controllers[i].mountPoint, group->controllers[i].placement, - (unsigned long long)pid); + (long long) pid); } return 0; -- 2.8.4

3 3

[libvirt] [PATCH] private_syms: add virLogFilterListFree to libvirt_private.syms
by Erik Skultety 07 Oct '16

07 Oct '16

Commit 660468b1 forgot to add it, so let's add it now to prevent future linker issues. Signed-off-by: Erik Skultety <eskultet(a)redhat.com> --- Pushed under trivial rule src/libvirt_private.syms | 1 + 1 file changed, 1 insertion(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index c4af57d..923afd1 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1872,6 +1872,7 @@ virLockSpaceReleaseResourcesForOwner; virLogDefineFilter; virLogDefineOutput; virLogFilterFree; +virLogFilterListFree; virLogGetDefaultPriority; virLogGetFilters; virLogGetNbFilters; -- 2.5.5

1 0

[libvirt] [PATCH] qemu_process: add pid of vm in domain log
by Chen Hanxiao 07 Oct '16

07 Oct '16

From: Chen Hanxiao <chenhanxiao(a)gmail.com> Add pid of VM in domain log. We used to show this info in debug log. For example: If a process send SIGKILL to a qemu process, we could find something in audit logs. Then the pid of VM in domain log will be helpful. Signed-off-by: Chen Hanxiao <chenhanxiao(a)gmail.com> --- src/qemu/qemu_process.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 27d04a4..8510a89 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -5392,11 +5392,14 @@ qemuProcessLaunch(virConnectPtr conn, _("Domain %s didn't show up"), vm->def->name); rv = -1; } - VIR_DEBUG("QEMU vm=%p name=%s running with pid=%llu", - vm, vm->def->name, (unsigned long long)vm->pid); + qemuDomainLogContextWrite(logCtxt, + "QEMU vm=%p name=%s running with pid=%llu\n", + vm, + vm->def->name, + (unsigned long long)vm->pid); } else { - VIR_DEBUG("QEMU vm=%p name=%s failed to spawn", - vm, vm->def->name); + qemuDomainLogContextWrite(logCtxt, "QEMU vm=%p name=%s failed to spawn", + vm, vm->def->name); } VIR_DEBUG("Writing early domain status to disk"); -- 1.8.3.1

3 2

[libvirt] [PATCH] m4: Drop PKG_PROG_PKG_CONFIG compatibility code
by Andrea Bolognani 07 Oct '16

07 Oct '16

This was needed for RHEL 4 vintage distributions, which we haven't supported for a long time now. --- m4/virt-pkgconfig-back-compat.m4 | 23 ----------------------- 1 file changed, 23 deletions(-) delete mode 100644 m4/virt-pkgconfig-back-compat.m4 diff --git a/m4/virt-pkgconfig-back-compat.m4 b/m4/virt-pkgconfig-back-compat.m4 deleted file mode 100644 index 7eab84b..0000000 --- a/m4/virt-pkgconfig-back-compat.m4 +++ /dev/null @@ -1,23 +0,0 @@ -dnl -dnl To support the old pkg-config from RHEL4 vintage, we need -dnl to define the PKG_PROG_PKG_CONFIG macro if its not already -dnl present -m4_ifndef([PKG_PROG_PKG_CONFIG], - [AC_DEFUN([PKG_PROG_PKG_CONFIG], -[m4_pattern_forbid([^_?PKG_[A-Z_]+$]) -m4_pattern_allow([^PKG_CONFIG(_PATH)?$]) -AC_ARG_VAR([PKG_CONFIG], [path to pkg-config utility])dnl -if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then - AC_PATH_TOOL([PKG_CONFIG], [pkg-config]) -fi -if test -n "$PKG_CONFIG"; then - _pkg_min_version=m4_default([$1], [0.9.0]) - AC_MSG_CHECKING([pkg-config is at least version $_pkg_min_version]) - if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then - AC_MSG_RESULT([yes]) - else - AC_MSG_RESULT([no]) - PKG_CONFIG="" - fi -fi[]dnl -])]) -- 2.7.4

2 1

[libvirt] Question about QEMU + hugepages + NUMA memory + migration
by Sam Bobroff 06 Oct '16

06 Oct '16

Hi libvirt people, I've been looking at a (probable) bug and I'm not sure how to progress. The situation is a bit complicated and involves both QEMU and libvirt (and I think it may have been looked at already) so I would really appreciate some advice on how to approach it. I'm using a pretty recent master version of libvirt from git and I'm testing on a ppc64le host with a similar guest but this doesn't seem to be arch-specific. If I create a QEMU guest (e.g. via virt-install) that requests both hugepage backing on the host and NUMA memory placement on the host, the NUMA placement seems to be ignored. If I do: # echo 0 > /proc/sys/vm/nr_hugepages # echo 512 > /sys/devices/system/node/node0/hugepages/hugepages-16384kB/nr_hugepages # virt-install --name tmp --memory=4096 --graphics none --memorybacking hugepages=yes --disk none --import --wait 0 --numatune=8 ... then hugepages are allocated on node 0 and the machine starts successfully, which seems like a bug. I believe it should fail to start due to insufficient memory, and in fact that is what happens if cgroup support isn't detected in the host: there seems to be a fall-back path in libvirt (probably using mbind()) that works as I would expect. Note: the relevant part of the guest XML seems to be this: »·······<memoryBacking> »·······»·······<hugepages/> »·······</memoryBacking> »·······<numatune> »·······»·······<memory mode='strict' nodeset='8'/> »·······</numatune> It seems fairly clear what is happening: although QEMU is capable of allocating hugepages on specific NUMA nodes (using "memory-backend-file") libvirt is not passing those options to QEMU in this situation. I investigated this line of reasoning and if I hack libvirt to pass those options to QEMU it does indeed fix the problem... but it renders the machine state migration-incompatible with unfixed versions. This seems to have been why this hasn't been fixed already :-( So what can we do? I assume it's not acceptible to just break migration with a bugfix, and I can only think of two ways to fix migration: (a) Add a new flag to the XML, and for guests without the flag, maintain the old buggy behaviour (and therefore migration compatability). (b) Hack QEMU so that migration can succeed between un-fixed and fixed versions. (And possibly also in the reverse direction?) I don't like (a) because it's visible in the XML, and would have to be carried forever (or at least a long time?). I don't really like (b) either because it's tricky, and even if it could be made to work reliably, it would add mess and risk to the migration code. I'm not sure how the QEMU community would feel about it either. However, I did hack up some code and it worked at least in some simple cases. Can anyone see a better approach? Is anyone already working on this? Thanks, Sam.

2 5

[libvirt] [PATCH] virt-yajl: Fix detection of yajl requirements
by Martin Kletzander 06 Oct '16

06 Oct '16

Running the output of qemu -help doesn't make any sense. We should be looking for libvirt being mentioned in the output. This worked by accident, let's make it work as expected it to. Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- m4/virt-yajl.m4 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/m4/virt-yajl.m4 b/m4/virt-yajl.m4 index adf2819eeb66..8c452adca653 100644 --- a/m4/virt-yajl.m4 +++ b/m4/virt-yajl.m4 @@ -26,7 +26,7 @@ AC_DEFUN([LIBVIRT_CHECK_YAJL],[ AC_PATH_PROGS([QEMU], [qemu-kvm qemu kvm qemu-system-x86_64], [], [$PATH:/usr/bin:/usr/libexec]) if test -x "$QEMU"; then - if `$QEMU -help | grep libvirt` >/dev/null; then + if $QEMU -help 2>/dev/null | grep -q libvirt; then with_yajl=yes else [qemu_version_sed='s/.*ersion $[0-9.,]*$.*/\1/'] -- 2.10.1

2 1

[libvirt] [PATCH] vsh: Fix warnings in command line completer
by Jiri Denemark 06 Oct '16

06 Oct '16

GCC complained that vsh.c: In function 'vshReadlineOptionsGenerator': vsh.c:2622:29: warning: unused variable 'opt' [-Wunused-variable] const vshCmdOptDef *opt = &cmd->opts[list_index]; ^ vsh.c: In function 'vshReadlineParse': vsh.c:2830:44: warning: 'opt' may be used uninitialized in this function [-Wmaybe-uninitialized] completed_list = opt->completer(autoCompleteOpaque, Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- Notes: I'm not very fond of the second hunk, but the completer is such a horrible piece of code I couldn't believe it was pushed. I just made the easiest fix and ran away from the code screaming. tools/vsh.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tools/vsh.c b/tools/vsh.c index cdd1cba..420eb79 100644 --- a/tools/vsh.c +++ b/tools/vsh.c @@ -2619,7 +2619,6 @@ vshReadlineOptionsGenerator(const char *text, int state, const vshCmdDef *cmd_pa return NULL; while ((name = cmd->opts[list_index].name)) { - const vshCmdOptDef *opt = &cmd->opts[list_index]; char *res; list_index++; @@ -2648,7 +2647,7 @@ vshReadlineParse(const char *text, int state) static vshCommandParser parser, sanitizer; vshCommandToken tk; static const vshCmdDef *cmd; - const vshCmdOptDef *opt; + const vshCmdOptDef *opt = NULL; char *tkdata, *optstr, *const_tkdata, *completed_name; char *res = NULL; static char *ctext, *sanitized_text; -- 2.10.0

4 4

[libvirt] [PATCH 0/3] qemu: report block job errors from qemu to the user
by Nikolay Shirokovskiy 06 Oct '16

06 Oct '16

So that you can see nice report on migration: "error: operation failed: migration of disk sda failed: No space left on device" Nikolay Shirokovskiy (3): qemu: simplify switch case for blockjob events qemu: use blockjob completed event's error field to detect errors qemu: report block job errors from qemu to the user src/qemu/qemu_blockjob.c | 13 +++++++++++-- src/qemu/qemu_blockjob.h | 3 ++- src/qemu/qemu_domain.h | 1 + src/qemu/qemu_driver.c | 4 ++-- src/qemu/qemu_migration.c | 34 ++++++++++++++++++++++------------ src/qemu/qemu_monitor.c | 5 +++-- src/qemu/qemu_monitor.h | 4 +++- src/qemu/qemu_monitor_json.c | 12 +++++------- src/qemu/qemu_process.c | 3 +++ 9 files changed, 52 insertions(+), 27 deletions(-) -- 1.8.3.1

3 9