[libvirt] [PATCH 00/11] Avoid numerous calls of virQEMUCapsCacheLookup
by Marc Hartmayer
For a domain definition there are numerous calls of
virQEMUCapsCacheLookup (the same applies to the domain start). This
slows down the process since virQEMUCapsCacheLookup validates that the
QEMU capabilitites are still valid (among other things, a fork is done
for this if the user for the QEMU processes is 'qemu'). Therefore
let's reduce the number of virQEMUCapsCacheLookup calls whenever
possible and reasonable.
In addition to the speed up, there is the risk that
virQEMUCapsCacheLookup returns different QEMU capabilities during a
task if, for example, the QEMU binary has changed during the task.
The correct way would be:
- get the QEMU capabilities only once per task via virQEMUCapsCacheLookup
- do the task with these QEMU capabilities
or
- abort the task after a cache invalidation
Note: With this patch series the behavior is still not (completely)
fixed, but the performance has been significantly improved. In a quick
test this gave a speed up of factor 4 for a simple define/undefine
loop.
In general, the more devices a domain has, the more drastic the
overhead becomes (because a cache validation is performed for each
device).
Marc Hartmayer (11):
qemu: Use VIR_STEAL_PTR macro
qemu: Introduce qemuDomainUpdateQEMUCaps()
qemu: Pass QEMUCaps to virDomainDefPostParse
conf: Use getParseOpaque() in virDomainObjSetDefTransient
conf: Add function description for virDomainDefPostParse
conf: Get rid of virDomainDeviceDefPostParseOne
conf: Extend virDomainDefValidate(Callback) for parseOpaque
conf: Use domainPostParseData(Alloc|Free) in virDomainDefValidate
qemu: Use @parseOpaque in qemuDomainDefValidate
conf: Extend virDomainDeviceDefValidate(Callback) for parseOpaque
qemu: Use @parseOpaque in qemuDomainDeviceDefValidate
src/conf/domain_conf.c | 98 ++++++++++++++++++++++++++---------------
src/conf/domain_conf.h | 9 ++--
src/qemu/qemu_domain.c | 82 +++++++++++++++++++---------------
src/qemu/qemu_domain.h | 4 ++
src/qemu/qemu_process.c | 18 +++-----
src/vz/vz_driver.c | 6 ++-
6 files changed, 128 insertions(+), 89 deletions(-)
--
2.17.0
6 years, 5 months
[libvirt] [PATCH v2 0/9] Incremental Backup API additions
by Eric Blake
The following is the latest version of my API proposal for
incremental backups.
I have even more work-in-progress patches on top of these:
https://repo.or.cz/libvirt/ericb.git
which I am slowly improving to be more in line with my thread
on the overview of the API usage:
https://www.redhat.com/archives/libvir-list/2018-October/msg00217.html
But I am fairly satisfied that the API as presented is sufficient for
everything I have still been implementing in the qemu driver, and
that even when qemu is slightly tweaked (such as dropping the x-
prefix on various commands, or maybe adding a new command to make
it easier to compute the estimated size of the union of several
bitmaps), those changes will be limited to the src/qemu directory
rather than affecting the API.
Since I will be demonstrating the use of this API at the KVM Forum,
I would really like a decision on whether we can commit the API
into libvirt now, even if we have to wait for the qemu implementation
of the API until qemu stabilizes its interfaces (also, having the
libvirt API in place gives qemu an incentive to drop the x- prefix
sooner rather than later). That in turn implies that I need code
reviews; John Ferlan started reviewing v1, but mostly focused on
the documentation grammar:
https://www.redhat.com/archives/libvir-list/2018-June/msg01066.html
The following gives an idea of how this series has changed since
June (in part due to rebases, in part due to documentation improvements):
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively
001/9:[0050] [FC] 'snapshots: Avoid term 'checkpoint' for full system snapshot'
002/9:[down] 'domain_conf: Expose virDomainStorageNetworkParseHost'
003/9:[0342] [FC] 'backup: Document nuances between different state capture APIs'
004/9:[0007] [FC] 'backup: Introduce virDomainCheckpointPtr'
005/9:[0100] [FC] 'backup: Document new XML for backups'
006/9:[0042] [FC] 'backup: Introduce virDomainCheckpoint APIs'
007/9:[0035] [FC] 'backup: Introduce virDomainBackup APIs'
008/9:[----] [--] 'backup: Add new domain:checkpoint access control'
009/9:[0039] [FC] 'backup: Implement backup APIs for remote driver'
Eric Blake (9):
snapshots: Avoid term 'checkpoint' for full system snapshot
domain_conf: Expose virDomainStorageNetworkParseHost
backup: Document nuances between different state capture APIs
backup: Introduce virDomainCheckpointPtr
backup: Document new XML for backups
backup: Introduce virDomainCheckpoint APIs
backup: Introduce virDomainBackup APIs
backup: Add new domain:checkpoint access control
backup: Implement backup APIs for remote driver
include/libvirt/virterror.h | 6 +-
src/util/virerror.c | 21 +-
include/libvirt/libvirt-domain-checkpoint.h | 177 ++++
include/libvirt/libvirt-domain-snapshot.h | 2 +-
include/libvirt/libvirt-domain.h | 14 +-
include/libvirt/libvirt.h | 3 +-
src/access/viraccessperm.h | 8 +-
src/conf/domain_conf.h | 2 +
src/datatypes.h | 31 +-
src/driver-hypervisor.h | 74 +-
docs/Makefile.am | 3 +
docs/apibuild.py | 2 +
docs/docs.html.in | 9 +-
docs/domainstatecapture.html.in | 314 +++++++
docs/format.html.in | 1 +
docs/formatcheckpoint.html.in | 285 ++++++
docs/formatsnapshot.html.in | 33 +-
docs/index.html.in | 3 +-
docs/schemas/domainbackup.rng | 185 ++++
docs/schemas/domaincheckpoint.rng | 94 ++
libvirt.spec.in | 3 +
mingw-libvirt.spec.in | 6 +
po/POTFILES | 1 +
src/Makefile.am | 2 +
src/access/viraccessperm.c | 5 +-
src/conf/domain_conf.c | 43 +-
src/conf/snapshot_conf.c | 4 +-
src/datatypes.c | 62 +-
src/libvirt-domain-checkpoint.c | 936 +++++++++++++++++++
src/libvirt-domain-snapshot.c | 7 +-
src/libvirt-domain.c | 8 +-
src/libvirt_private.syms | 3 +
src/libvirt_public.syms | 23 +
src/qemu/qemu_driver.c | 12 +-
src/remote/remote_daemon_dispatch.c | 15 +
src/remote/remote_driver.c | 33 +-
src/remote/remote_protocol.x | 240 ++++-
src/remote_protocol-structs | 129 +++
src/rpc/gendispatch.pl | 32 +-
tests/Makefile.am | 6 +-
tests/domainbackupxml2xmlin/backup-pull.xml | 9 +
tests/domainbackupxml2xmlin/backup-push.xml | 9 +
tests/domainbackupxml2xmlin/empty.xml | 1 +
tests/domainbackupxml2xmlout/backup-pull.xml | 9 +
tests/domainbackupxml2xmlout/backup-push.xml | 9 +
tests/domainbackupxml2xmlout/empty.xml | 7 +
tests/domaincheckpointxml2xmlin/empty.xml | 1 +
tests/domaincheckpointxml2xmlin/sample.xml | 7 +
tests/domaincheckpointxml2xmlout/empty.xml | 10 +
tests/domaincheckpointxml2xmlout/sample.xml | 16 +
tests/virschematest.c | 4 +
tools/virsh-domain.c | 3 +-
tools/virsh-snapshot.c | 2 +-
tools/virsh.pod | 14 +-
54 files changed, 2846 insertions(+), 92 deletions(-)
create mode 100644 include/libvirt/libvirt-domain-checkpoint.h
create mode 100644 docs/domainstatecapture.html.in
create mode 100644 docs/formatcheckpoint.html.in
create mode 100644 docs/schemas/domainbackup.rng
create mode 100644 docs/schemas/domaincheckpoint.rng
create mode 100644 src/libvirt-domain-checkpoint.c
create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml
create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml
create mode 100644 tests/domainbackupxml2xmlin/empty.xml
create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml
create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml
create mode 100644 tests/domainbackupxml2xmlout/empty.xml
create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml
create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml
create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml
create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml
--
2.17.1
6 years, 5 months
[libvirt] Release of libvirt-snmp-0.0.4
by Michal Prívozník
It's out. After 6 years we have a new libvirt-snmp release. Yay. You can
obtain it from the usual sources:
https://libvirt.org/sources/snmp/
I've pushed it into git and tagged. Thanks everybody who participated in
any form.
Changelog:
- Replace placeholder org OID with libvirt OID
- Fix object OIDs for SNMP traps
- Send sysUptime in traps
- Modernize spec file
- Cleaned up some old code and rewritten it to use new Libvirt APIs
- Fixed build with newer net-snmp package
- Some bug fixes
Michal
6 years, 5 months
[libvirt] [snmp PATCH] configure: Fix MIB_DIR's prefix
by Michal Privoznik
So far we are taking whatever is `net-snmp-config
--default-mibdirs` returns and using that as MIB_DIR to install
our MIB. This has a drawback that it ignores whatever prefix user
specifies to configure script (which is used by `make distcheck`
too). The fix consists of querying for net-snmp-config --prefix
and replacing that in MIB_DIR var.
Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com>
---
This is similar to what we do for wireshark in libvirt.
configure.ac | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/configure.ac b/configure.ac
index baac0e1..f729ac1 100644
--- a/configure.ac
+++ b/configure.ac
@@ -73,6 +73,14 @@ if test "x$with_mibdir" = "x" ; then
MIB_DIR=$(echo $MIB_DIR | awk -F : '{print $NF}')
AC_MSG_RESULT([found $MIB_DIR])
+
+ dnl Replace net-snmp's exec_prefix with our own.
+ dnl Note that ${exec_prefix} is kept verbatim at this point in time,
+ dnl and will only be expanded later, when make is called: this makes
+ dnl it possible to override such prefix at compilation or installation
+ dnl time
+ MIB_PREFIX="`$SNMP_CONFIG --prefix`"
+ MIB_DIR='${exec_prefix}'"${MIB_DIR#$MIB_PREFIX}"
else
AC_MSG_CHECKING([accessibility of $with_mibdir])
if test ! -d "$with_mibdir" ; then
--
2.18.1
6 years, 5 months
[libvirt] [PATCH 00/11] Allow modification of IOThread polling values (redux)
by John Ferlan
This series attempts to resurrect the concept of being able to modify
the IOThread polling parameters; however, in a slightly different
manner than the previous series posted by posted by Pavel Hrdina
<phrdina(a)redhat.com>:
https://www.redhat.com/archives/libvir-list/2017-February/msg01047.html
The work is prompted by continued pleas found in the bz:
https://bugzilla.redhat.com/show_bug.cgi?id=1545732
to provide some way to modify the paremters without needing to supply
QEMU command line pass through values.
It's accepted that the values being changed are fairly or extremely
low level type knobs; however, it's also shown that by being able to
turn the knob it is possible for certain, specific appliances to be
able to gain a performance benefit for the thread at the expense of
other competing threads.
Unlike the previous series, this series does not attempt to save the
polling values in the guest XML. Rather, it only modifies the live
guest's IOThread with new values. It also doesn't provide the polling
values in a virsh iothread* command, rather it uses the domstats
in order to fetch and display the values. The theory being this
leaves the onus on the higher level appliance/application to provide
the "proper guidance" and "concerns" related to changing the values
to the consumer. Not saving the values means whatever values that
are chosen do not "live" in perpetuity. Once the guest is shut down
or the IOThread removed from guest, the hypervisor default values
take over again. Perhaps not a perfect situation in terms of what
the bz requests; however, storage of default values that could
cause performance issues is not an optimal situation. So this I
figured is a "comprimise" of sorts.
If it's still felt that no we don't want to do this, then fine,
but please in doing so own the bz, state your case, and close it.
I'm 50/50 on it, but figured at least I'd present this option and
see what the concensus was.
John Ferlan (11):
qemu: Check for and return IOThread polling values if available
qemu: Split qemuDomainGetIOThreadsLive
qemu: Implement the ability to return IOThread stats
virsh: Add ability to display IOThread stats
lib: Introduce virDomainSetIOThreadParams
qemu: Add monitor functions to set IOThread params
qemu: Alter qemuDomainChgIOThread to take enum instead of bool
qemu: Alter qemuDomainChgIOThread to take qemuMonitorIOThreadInfo
qemu: Detect whether iothread polling is supported
qemu: Introduce qemuDomainSetIOThreadParams
tools: Add virsh iothreadset command
include/libvirt/libvirt-domain.h | 45 ++
src/driver-hypervisor.h | 8 +
src/libvirt-domain.c | 109 +++++
src/libvirt_public.syms | 5 +
src/qemu/qemu_capabilities.c | 2 +
src/qemu/qemu_capabilities.h | 1 +
src/qemu/qemu_driver.c | 393 ++++++++++++++++--
src/qemu/qemu_monitor.c | 19 +
src/qemu/qemu_monitor.h | 6 +
src/qemu/qemu_monitor_json.c | 48 +++
src/qemu/qemu_monitor_json.h | 4 +
src/remote/remote_driver.c | 1 +
src/remote/remote_protocol.x | 21 +-
src/remote_protocol-structs | 10 +
.../caps_2.10.0.aarch64.xml | 1 +
.../caps_2.10.0.ppc64.xml | 1 +
.../caps_2.10.0.s390x.xml | 1 +
.../caps_2.10.0.x86_64.xml | 1 +
.../caps_2.11.0.s390x.xml | 1 +
.../caps_2.11.0.x86_64.xml | 1 +
.../caps_2.12.0.aarch64.xml | 1 +
.../caps_2.12.0.ppc64.xml | 1 +
.../caps_2.12.0.s390x.xml | 1 +
.../caps_2.12.0.x86_64.xml | 1 +
.../qemucapabilitiesdata/caps_2.9.0.ppc64.xml | 1 +
.../qemucapabilitiesdata/caps_2.9.0.s390x.xml | 1 +
.../caps_2.9.0.x86_64.xml | 1 +
.../qemucapabilitiesdata/caps_3.0.0.ppc64.xml | 1 +
.../caps_3.0.0.riscv32.xml | 1 +
.../caps_3.0.0.riscv64.xml | 1 +
.../qemucapabilitiesdata/caps_3.0.0.s390x.xml | 1 +
.../caps_3.0.0.x86_64.xml | 1 +
tools/virsh-domain-monitor.c | 7 +
tools/virsh-domain.c | 110 +++++
tools/virsh.pod | 47 ++-
35 files changed, 810 insertions(+), 44 deletions(-)
--
2.17.1
6 years, 5 months
[libvirt] [PATCH] docs: Fix minimum supported version
by Martin Kletzander
We already have that in the code (commit c1bc9c662b411), we just forgot to
mention that in the docs.
Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com>
---
Pushed as trivial
docs/drvqemu.html.in | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs/drvqemu.html.in b/docs/drvqemu.html.in
index 13adb5c22b22..0d14027646d5 100644
--- a/docs/drvqemu.html.in
+++ b/docs/drvqemu.html.in
@@ -8,7 +8,7 @@
<p>
The libvirt KVM/QEMU driver can manage any QEMU emulator from
- version 0.12.0 or later.
+ version 1.5.0 or later.
</p>
<h2><a id="project">Project Links</a></h2>
--
2.19.1
6 years, 5 months
[libvirt] [jenkins-ci PATCH 0/3] Add libvirt-ocaml
by Pino Toscano
Now that the libvirt-ocaml repository is fixed, let's build it in CI.
Pino Toscano (3):
guests: add mappings for OCaml components
guests: add libvirt-ocaml project
projects: add libvirt-ocaml project
guests/host_vars/libvirt-centos-7/main.yml | 1 +
guests/host_vars/libvirt-debian-9/main.yml | 1 +
guests/host_vars/libvirt-debian-sid/main.yml | 1 +
guests/host_vars/libvirt-fedora-27/main.yml | 1 +
guests/host_vars/libvirt-fedora-28/main.yml | 1 +
.../host_vars/libvirt-fedora-rawhide/main.yml | 1 +
guests/host_vars/libvirt-freebsd-10/main.yml | 1 +
guests/host_vars/libvirt-freebsd-11/main.yml | 1 +
.../libvirt-freebsd-current/main.yml | 1 +
guests/host_vars/libvirt-ubuntu-16/main.yml | 1 +
guests/host_vars/libvirt-ubuntu-18/main.yml | 1 +
guests/playbooks/build/jobs/defaults.yml | 2 ++
.../build/projects/libvirt-ocaml.yml | 27 +++++++++++++++++++
guests/vars/mappings.yml | 6 +++++
guests/vars/projects/libvirt-ocaml.yml | 5 ++++
jobs/defaults.yaml | 2 ++
projects/libvirt-ocaml.yaml | 23 ++++++++++++++++
17 files changed, 76 insertions(+)
create mode 100644 guests/playbooks/build/projects/libvirt-ocaml.yml
create mode 100644 guests/vars/projects/libvirt-ocaml.yml
create mode 100644 projects/libvirt-ocaml.yaml
--
2.17.2
6 years, 5 months
[libvirt] [PATCH] util: Fix typo vcups -> vcpus
by Martin Kletzander
Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com>
---
Pushed as trivial
src/util/virnuma.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/util/virnuma.h b/src/util/virnuma.h
index a3ffb6d6c7c9..edc206100f78 100644
--- a/src/util/virnuma.h
+++ b/src/util/virnuma.h
@@ -27,7 +27,7 @@
# include "virutil.h"
-char *virNumaGetAutoPlacementAdvice(unsigned short vcups,
+char *virNumaGetAutoPlacementAdvice(unsigned short vcpus,
unsigned long long balloon);
int virNumaSetupMemoryPolicy(virDomainNumatuneMemMode mode,
--
2.19.1
6 years, 5 months
