October 2018 - Devel - libvirt List Archives

[libvirt] [PATCH 00/11] Avoid numerous calls of virQEMUCapsCacheLookup
by Marc Hartmayer 26 Oct '18

26 Oct '18

For a domain definition there are numerous calls of virQEMUCapsCacheLookup (the same applies to the domain start). This slows down the process since virQEMUCapsCacheLookup validates that the QEMU capabilitites are still valid (among other things, a fork is done for this if the user for the QEMU processes is 'qemu'). Therefore let's reduce the number of virQEMUCapsCacheLookup calls whenever possible and reasonable. In addition to the speed up, there is the risk that virQEMUCapsCacheLookup returns different QEMU capabilities during a task if, for example, the QEMU binary has changed during the task. The correct way would be: - get the QEMU capabilities only once per task via virQEMUCapsCacheLookup - do the task with these QEMU capabilities or - abort the task after a cache invalidation Note: With this patch series the behavior is still not (completely) fixed, but the performance has been significantly improved. In a quick test this gave a speed up of factor 4 for a simple define/undefine loop. In general, the more devices a domain has, the more drastic the overhead becomes (because a cache validation is performed for each device). Marc Hartmayer (11): qemu: Use VIR_STEAL_PTR macro qemu: Introduce qemuDomainUpdateQEMUCaps() qemu: Pass QEMUCaps to virDomainDefPostParse conf: Use getParseOpaque() in virDomainObjSetDefTransient conf: Add function description for virDomainDefPostParse conf: Get rid of virDomainDeviceDefPostParseOne conf: Extend virDomainDefValidate(Callback) for parseOpaque conf: Use domainPostParseData(Alloc|Free) in virDomainDefValidate qemu: Use @parseOpaque in qemuDomainDefValidate conf: Extend virDomainDeviceDefValidate(Callback) for parseOpaque qemu: Use @parseOpaque in qemuDomainDeviceDefValidate src/conf/domain_conf.c | 98 ++++++++++++++++++++++++++--------------- src/conf/domain_conf.h | 9 ++-- src/qemu/qemu_domain.c | 82 +++++++++++++++++++--------------- src/qemu/qemu_domain.h | 4 ++ src/qemu/qemu_process.c | 18 +++----- src/vz/vz_driver.c | 6 ++- 6 files changed, 128 insertions(+), 89 deletions(-) -- 2.17.0

5 40

[libvirt] [PATCH v2 0/9] Incremental Backup API additions
by Eric Blake 25 Oct '18

25 Oct '18

The following is the latest version of my API proposal for incremental backups. I have even more work-in-progress patches on top of these: https://repo.or.cz/libvirt/ericb.git which I am slowly improving to be more in line with my thread on the overview of the API usage: https://www.redhat.com/archives/libvir-list/2018-October/msg00217.html But I am fairly satisfied that the API as presented is sufficient for everything I have still been implementing in the qemu driver, and that even when qemu is slightly tweaked (such as dropping the x- prefix on various commands, or maybe adding a new command to make it easier to compute the estimated size of the union of several bitmaps), those changes will be limited to the src/qemu directory rather than affecting the API. Since I will be demonstrating the use of this API at the KVM Forum, I would really like a decision on whether we can commit the API into libvirt now, even if we have to wait for the qemu implementation of the API until qemu stabilizes its interfaces (also, having the libvirt API in place gives qemu an incentive to drop the x- prefix sooner rather than later). That in turn implies that I need code reviews; John Ferlan started reviewing v1, but mostly focused on the documentation grammar: https://www.redhat.com/archives/libvir-list/2018-June/msg01066.html The following gives an idea of how this series has changed since June (in part due to rebases, in part due to documentation improvements): The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively 001/9:[0050] [FC] 'snapshots: Avoid term 'checkpoint' for full system snapshot' 002/9:[down] 'domain_conf: Expose virDomainStorageNetworkParseHost' 003/9:[0342] [FC] 'backup: Document nuances between different state capture APIs' 004/9:[0007] [FC] 'backup: Introduce virDomainCheckpointPtr' 005/9:[0100] [FC] 'backup: Document new XML for backups' 006/9:[0042] [FC] 'backup: Introduce virDomainCheckpoint APIs' 007/9:[0035] [FC] 'backup: Introduce virDomainBackup APIs' 008/9:[----] [--] 'backup: Add new domain:checkpoint access control' 009/9:[0039] [FC] 'backup: Implement backup APIs for remote driver' Eric Blake (9): snapshots: Avoid term 'checkpoint' for full system snapshot domain_conf: Expose virDomainStorageNetworkParseHost backup: Document nuances between different state capture APIs backup: Introduce virDomainCheckpointPtr backup: Document new XML for backups backup: Introduce virDomainCheckpoint APIs backup: Introduce virDomainBackup APIs backup: Add new domain:checkpoint access control backup: Implement backup APIs for remote driver include/libvirt/virterror.h | 6 +- src/util/virerror.c | 21 +- include/libvirt/libvirt-domain-checkpoint.h | 177 ++++ include/libvirt/libvirt-domain-snapshot.h | 2 +- include/libvirt/libvirt-domain.h | 14 +- include/libvirt/libvirt.h | 3 +- src/access/viraccessperm.h | 8 +- src/conf/domain_conf.h | 2 + src/datatypes.h | 31 +- src/driver-hypervisor.h | 74 +- docs/Makefile.am | 3 + docs/apibuild.py | 2 + docs/docs.html.in | 9 +- docs/domainstatecapture.html.in | 314 +++++++ docs/format.html.in | 1 + docs/formatcheckpoint.html.in | 285 ++++++ docs/formatsnapshot.html.in | 33 +- docs/index.html.in | 3 +- docs/schemas/domainbackup.rng | 185 ++++ docs/schemas/domaincheckpoint.rng | 94 ++ libvirt.spec.in | 3 + mingw-libvirt.spec.in | 6 + po/POTFILES | 1 + src/Makefile.am | 2 + src/access/viraccessperm.c | 5 +- src/conf/domain_conf.c | 43 +- src/conf/snapshot_conf.c | 4 +- src/datatypes.c | 62 +- src/libvirt-domain-checkpoint.c | 936 +++++++++++++++++++ src/libvirt-domain-snapshot.c | 7 +- src/libvirt-domain.c | 8 +- src/libvirt_private.syms | 3 + src/libvirt_public.syms | 23 + src/qemu/qemu_driver.c | 12 +- src/remote/remote_daemon_dispatch.c | 15 + src/remote/remote_driver.c | 33 +- src/remote/remote_protocol.x | 240 ++++- src/remote_protocol-structs | 129 +++ src/rpc/gendispatch.pl | 32 +- tests/Makefile.am | 6 +- tests/domainbackupxml2xmlin/backup-pull.xml | 9 + tests/domainbackupxml2xmlin/backup-push.xml | 9 + tests/domainbackupxml2xmlin/empty.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 9 + tests/domainbackupxml2xmlout/backup-push.xml | 9 + tests/domainbackupxml2xmlout/empty.xml | 7 + tests/domaincheckpointxml2xmlin/empty.xml | 1 + tests/domaincheckpointxml2xmlin/sample.xml | 7 + tests/domaincheckpointxml2xmlout/empty.xml | 10 + tests/domaincheckpointxml2xmlout/sample.xml | 16 + tests/virschematest.c | 4 + tools/virsh-domain.c | 3 +- tools/virsh-snapshot.c | 2 +- tools/virsh.pod | 14 +- 54 files changed, 2846 insertions(+), 92 deletions(-) create mode 100644 include/libvirt/libvirt-domain-checkpoint.h create mode 100644 docs/domainstatecapture.html.in create mode 100644 docs/formatcheckpoint.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 docs/schemas/domaincheckpoint.rng create mode 100644 src/libvirt-domain-checkpoint.c create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml create mode 100644 tests/domainbackupxml2xmlin/empty.xml create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml create mode 100644 tests/domainbackupxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml -- 2.17.1

3 19

[libvirt] Release of libvirt-snmp-0.0.4
by Michal Prívozník 25 Oct '18

25 Oct '18

It's out. After 6 years we have a new libvirt-snmp release. Yay. You can obtain it from the usual sources: https://libvirt.org/sources/snmp/ I've pushed it into git and tagged. Thanks everybody who participated in any form. Changelog: - Replace placeholder org OID with libvirt OID - Fix object OIDs for SNMP traps - Send sysUptime in traps - Modernize spec file - Cleaned up some old code and rewritten it to use new Libvirt APIs - Fixed build with newer net-snmp package - Some bug fixes Michal

1 0

[libvirt] [snmp PATCH] configure: Fix MIB_DIR's prefix
by Michal Privoznik 25 Oct '18

25 Oct '18

So far we are taking whatever is `net-snmp-config --default-mibdirs` returns and using that as MIB_DIR to install our MIB. This has a drawback that it ignores whatever prefix user specifies to configure script (which is used by `make distcheck` too). The fix consists of querying for net-snmp-config --prefix and replacing that in MIB_DIR var. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- This is similar to what we do for wireshark in libvirt. configure.ac | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/configure.ac b/configure.ac index baac0e1..f729ac1 100644 --- a/configure.ac +++ b/configure.ac @@ -73,6 +73,14 @@ if test "x$with_mibdir" = "x" ; then MIB_DIR=$(echo $MIB_DIR | awk -F : '{print $NF}') AC_MSG_RESULT([found $MIB_DIR]) + + dnl Replace net-snmp's exec_prefix with our own. + dnl Note that ${exec_prefix} is kept verbatim at this point in time, + dnl and will only be expanded later, when make is called: this makes + dnl it possible to override such prefix at compilation or installation + dnl time + MIB_PREFIX="`$SNMP_CONFIG --prefix`" + MIB_DIR='${exec_prefix}'"${MIB_DIR#$MIB_PREFIX}" else AC_MSG_CHECKING([accessibility of $with_mibdir]) if test ! -d "$with_mibdir" ; then -- 2.18.1

2 1

[libvirt] [PATCH 00/11] Allow modification of IOThread polling values (redux)
by John Ferlan 23 Oct '18

23 Oct '18

This series attempts to resurrect the concept of being able to modify the IOThread polling parameters; however, in a slightly different manner than the previous series posted by posted by Pavel Hrdina <phrdina(a)redhat.com>: https://www.redhat.com/archives/libvir-list/2017-February/msg01047.html The work is prompted by continued pleas found in the bz: https://bugzilla.redhat.com/show_bug.cgi?id=1545732 to provide some way to modify the paremters without needing to supply QEMU command line pass through values. It's accepted that the values being changed are fairly or extremely low level type knobs; however, it's also shown that by being able to turn the knob it is possible for certain, specific appliances to be able to gain a performance benefit for the thread at the expense of other competing threads. Unlike the previous series, this series does not attempt to save the polling values in the guest XML. Rather, it only modifies the live guest's IOThread with new values. It also doesn't provide the polling values in a virsh iothread* command, rather it uses the domstats in order to fetch and display the values. The theory being this leaves the onus on the higher level appliance/application to provide the "proper guidance" and "concerns" related to changing the values to the consumer. Not saving the values means whatever values that are chosen do not "live" in perpetuity. Once the guest is shut down or the IOThread removed from guest, the hypervisor default values take over again. Perhaps not a perfect situation in terms of what the bz requests; however, storage of default values that could cause performance issues is not an optimal situation. So this I figured is a "comprimise" of sorts. If it's still felt that no we don't want to do this, then fine, but please in doing so own the bz, state your case, and close it. I'm 50/50 on it, but figured at least I'd present this option and see what the concensus was. John Ferlan (11): qemu: Check for and return IOThread polling values if available qemu: Split qemuDomainGetIOThreadsLive qemu: Implement the ability to return IOThread stats virsh: Add ability to display IOThread stats lib: Introduce virDomainSetIOThreadParams qemu: Add monitor functions to set IOThread params qemu: Alter qemuDomainChgIOThread to take enum instead of bool qemu: Alter qemuDomainChgIOThread to take qemuMonitorIOThreadInfo qemu: Detect whether iothread polling is supported qemu: Introduce qemuDomainSetIOThreadParams tools: Add virsh iothreadset command include/libvirt/libvirt-domain.h | 45 ++ src/driver-hypervisor.h | 8 + src/libvirt-domain.c | 109 +++++ src/libvirt_public.syms | 5 + src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_driver.c | 393 ++++++++++++++++-- src/qemu/qemu_monitor.c | 19 + src/qemu/qemu_monitor.h | 6 + src/qemu/qemu_monitor_json.c | 48 +++ src/qemu/qemu_monitor_json.h | 4 + src/remote/remote_driver.c | 1 + src/remote/remote_protocol.x | 21 +- src/remote_protocol-structs | 10 + .../caps_2.10.0.aarch64.xml | 1 + .../caps_2.10.0.ppc64.xml | 1 + .../caps_2.10.0.s390x.xml | 1 + .../caps_2.10.0.x86_64.xml | 1 + .../caps_2.11.0.s390x.xml | 1 + .../caps_2.11.0.x86_64.xml | 1 + .../caps_2.12.0.aarch64.xml | 1 + .../caps_2.12.0.ppc64.xml | 1 + .../caps_2.12.0.s390x.xml | 1 + .../caps_2.12.0.x86_64.xml | 1 + .../qemucapabilitiesdata/caps_2.9.0.ppc64.xml | 1 + .../qemucapabilitiesdata/caps_2.9.0.s390x.xml | 1 + .../caps_2.9.0.x86_64.xml | 1 + .../qemucapabilitiesdata/caps_3.0.0.ppc64.xml | 1 + .../caps_3.0.0.riscv32.xml | 1 + .../caps_3.0.0.riscv64.xml | 1 + .../qemucapabilitiesdata/caps_3.0.0.s390x.xml | 1 + .../caps_3.0.0.x86_64.xml | 1 + tools/virsh-domain-monitor.c | 7 + tools/virsh-domain.c | 110 +++++ tools/virsh.pod | 47 ++- 35 files changed, 810 insertions(+), 44 deletions(-) -- 2.17.1

4 29

[libvirt] [PATCH] docs: fix repology link for qemu-kvm package
by Pavel Hrdina 23 Oct '18

23 Oct '18

Signed-off-by: Pavel Hrdina <phrdina(a)redhat.com> --- Pushed under trivial rule. docs/platforms.html.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/platforms.html.in b/docs/platforms.html.in index 0489c60aeb..a037074e1f 100644 --- a/docs/platforms.html.in +++ b/docs/platforms.html.in @@ -39,7 +39,7 @@ <ul> <li><a href="https://repology.org/metapackage/libvirt/versions">libvirt</a></li> <li><a href="https://repology.org/metapackage/qemu/versions">qemu</a></li> - <li><a href="https://repology.org/metapackage/qemu/versions">qemu-kvm</a></li> + <li><a href="https://repology.org/metapackage/qemu-kvm/versions">qemu-kvm</a></li> </ul> -- 2.17.2

1 0

[libvirt] [PATCH] docs: Fix minimum supported version
by Martin Kletzander 23 Oct '18

23 Oct '18

We already have that in the code (commit c1bc9c662b411), we just forgot to mention that in the docs. Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- Pushed as trivial docs/drvqemu.html.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/drvqemu.html.in b/docs/drvqemu.html.in index 13adb5c22b22..0d14027646d5 100644 --- a/docs/drvqemu.html.in +++ b/docs/drvqemu.html.in @@ -8,7 +8,7 @@ <p> The libvirt KVM/QEMU driver can manage any QEMU emulator from - version 0.12.0 or later. + version 1.5.0 or later. </p> <h2><a id="project">Project Links</a></h2> -- 2.19.1

1 0

[libvirt] [jenkins-ci PATCH 0/3] Add libvirt-ocaml
by Pino Toscano 22 Oct '18

22 Oct '18

Now that the libvirt-ocaml repository is fixed, let's build it in CI. Pino Toscano (3): guests: add mappings for OCaml components guests: add libvirt-ocaml project projects: add libvirt-ocaml project guests/host_vars/libvirt-centos-7/main.yml | 1 + guests/host_vars/libvirt-debian-9/main.yml | 1 + guests/host_vars/libvirt-debian-sid/main.yml | 1 + guests/host_vars/libvirt-fedora-27/main.yml | 1 + guests/host_vars/libvirt-fedora-28/main.yml | 1 + .../host_vars/libvirt-fedora-rawhide/main.yml | 1 + guests/host_vars/libvirt-freebsd-10/main.yml | 1 + guests/host_vars/libvirt-freebsd-11/main.yml | 1 + .../libvirt-freebsd-current/main.yml | 1 + guests/host_vars/libvirt-ubuntu-16/main.yml | 1 + guests/host_vars/libvirt-ubuntu-18/main.yml | 1 + guests/playbooks/build/jobs/defaults.yml | 2 ++ .../build/projects/libvirt-ocaml.yml | 27 +++++++++++++++++++ guests/vars/mappings.yml | 6 +++++ guests/vars/projects/libvirt-ocaml.yml | 5 ++++ jobs/defaults.yaml | 2 ++ projects/libvirt-ocaml.yaml | 23 ++++++++++++++++ 17 files changed, 76 insertions(+) create mode 100644 guests/playbooks/build/projects/libvirt-ocaml.yml create mode 100644 guests/vars/projects/libvirt-ocaml.yml create mode 100644 projects/libvirt-ocaml.yaml -- 2.17.2

4 8

[libvirt] [PATCH] util: Fix typo vcups -> vcpus
by Martin Kletzander 22 Oct '18

22 Oct '18

Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- Pushed as trivial src/util/virnuma.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/util/virnuma.h b/src/util/virnuma.h index a3ffb6d6c7c9..edc206100f78 100644 --- a/src/util/virnuma.h +++ b/src/util/virnuma.h @@ -27,7 +27,7 @@ # include "virutil.h" -char *virNumaGetAutoPlacementAdvice(unsigned short vcups, +char *virNumaGetAutoPlacementAdvice(unsigned short vcpus, unsigned long long balloon); int virNumaSetupMemoryPolicy(virDomainNumatuneMemMode mode, -- 2.19.1

1 0

[libvirt] BugFix - Libvirt Ruby Binding Memory Stats Val Method
by iletisim＠muhammetdilmac.com.tr 22 Oct '18

22 Oct '18

1 0