- Devel - libvirt List Archives

[libvirt] [PATCH] libvirt: remove link to outdated mailing list post
by Daniel P. Berrangé 08 Jul '19

08 Jul '19

The virConnectGetType() method has an unfortunate signature, returning a static string that must not be freed by the caller. The remote driver, however, gets this string dynamically over an RPC call, which raised a design discussion on the mailing list. Eventually the problem was resolved by having the remote driver cache the returned string internally and free it when the connection was closed. The link to the mailing list is thus talking about a problem that does not actually exist in the final implementation, and at best serves to confuse the reader into thinking there might be a memory leak. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- src/libvirt-host.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/src/libvirt-host.c b/src/libvirt-host.c index 2978825d22..e5c4e5f72a 100644 --- a/src/libvirt-host.c +++ b/src/libvirt-host.c @@ -98,9 +98,6 @@ virConnectSupportsFeature(virConnectPtr conn, int feature) * hypervisor, use virConnectGetCapabilities(). * * Returns NULL in case of error, a static zero terminated string otherwise. - * - * See also: - * http://www.redhat.com/archives/libvir-list/2007-February/msg00096.html */ const char * virConnectGetType(virConnectPtr conn) -- 2.21.0

2 1

[libvirt] [PATCH v8 00/21] Incremental backups
by Eric Blake 07 Jul '19

07 Jul '19

The API is fairly stable, but there are still probably code cleanups worth adding to the qemu driver implementation. I've also seen a desire on the list to support all APIs in the test driver, so I'll try and whip that together for my next round of patches. If we are confident that the API is worth having in 5.3 (especially since the earlier part of this series already has positive reviews), then it's probably worth committing that part of the series now even while I still address review comments on the later patches. I've pushed a tag backup-v8 to both my libvirt.git and libvirt-python.git repos to match: https://repo.or.cz/libvirt/ericb.git/shortlog/refs/tags/backup-v8 https://repo.or.cz/libvirt-python/ericb.git/shortlog/refs/tags/backup-v8 Here's hoping we're happy enough with the API, including the fact that rudimentary operation of pull mode backups work with qemu 4.0-rc1, for this to make it into the 5.2 release. Notable diffs from v7: - address several review comments (although I didn't get to every comment later in the series) - actually support job id of 0 (it was documented but not working in v7) - add support for Unix socket pull mode backups - improve checkpoint XML handling (more tests, separate postparse actions out from initial parse) - rebase to master 001/21:[0044] [FC] 'backup: Document new XML for checkpoints' 002/21:[----] [--] 'backup: Document new XML for backups' 003/21:[0006] [FC] 'backup: Introduce virDomainCheckpoint APIs' 004/21:[0002] [FC] 'backup: Introduce virDomainBackup APIs' 005/21:[----] [--] 'backup: Document nuances between different state capture APIs' 006/21:[0304] [FC] 'backup: Parse and output checkpoint XML' 007/21:[0007] [FC] 'backup: Allow for lists of checkpoint objects' 008/21:[0004] [FC] 'backup: Add new domain:checkpoint access control' 009/21:[0043] [FC] 'backup: Implement backup APIs for remote driver' 010/21:[down] 'conf: Add parameter to virDomainDiskSourceFormat' 011/21:[0019] [FC] 'backup: Parse and output backup XML' 012/21:[0006] [FC] 'backup: Implement virsh support for checkpoints' 013/21:[0002] [FC] 'backup: Implement virsh support for backup' 014/21:[down] 'backup: Prepare for Unix sockets in QMP nbd-server-start' 015/21:[0008] [FC] 'backup: Add new qemu monitor interactions' 016/21:[0022] [FC] 'backup: qemu: Implement metadata tracking for checkpoint APIs' 017/21:[0004] [FC] 'backup: Wire up qemu checkpoint commands over QMP' 018/21:[0019] [FC] 'backup: qemu: Implement framework for backup job APIs' 019/21:[0042] [FC] 'backup: Wire up qemu full pull backup commands over QMP' 020/21:[----] [-C] 'backup: qemu: Wire up qemu full push backup commands over QMP' 021/21:[0003] [FC] 'backup: implement qemu incremental pull backup' Eric Blake (21): backup: Document new XML for checkpoints backup: Document new XML for backups backup: Introduce virDomainCheckpoint APIs backup: Introduce virDomainBackup APIs backup: Document nuances between different state capture APIs backup: Parse and output checkpoint XML backup: Allow for lists of checkpoint objects backup: Add new domain:checkpoint access control backup: Implement backup APIs for remote driver conf: Add parameter to virDomainDiskSourceFormat backup: Parse and output backup XML backup: Implement virsh support for checkpoints backup: Implement virsh support for backup backup: Prepare for Unix sockets in QMP nbd-server-start backup: Add new qemu monitor interactions backup: qemu: Implement metadata tracking for checkpoint APIs backup: Wire up qemu checkpoint commands over QMP backup: qemu: Implement framework for backup job APIs backup: Wire up qemu full pull backup commands over QMP backup: qemu: Wire up qemu full push backup commands over QMP backup: implement qemu incremental pull backup include/libvirt/libvirt-domain-checkpoint.h | 161 ++ include/libvirt/libvirt-domain-snapshot.h | 8 +- include/libvirt/libvirt-domain.h | 47 +- include/libvirt/libvirt.h | 5 +- src/access/viraccessperm.h | 6 + src/conf/backup_conf.h | 97 + src/conf/checkpoint_conf.h | 103 ++ src/conf/domain_conf.h | 3 + src/conf/virconftypes.h | 12 + src/conf/virdomaincheckpointobjlist.h | 74 + src/conf/virdomainmomentobjlist.h | 7 +- src/conf/virdomainobjlist.h | 7 +- src/driver-hypervisor.h | 79 + src/qemu/qemu_block.h | 3 + src/qemu/qemu_blockjob.h | 1 + src/qemu/qemu_capabilities.h | 2 + src/qemu/qemu_conf.h | 2 + src/qemu/qemu_domain.h | 19 + src/qemu/qemu_monitor.h | 27 +- src/qemu/qemu_monitor_json.h | 22 +- tools/virsh-checkpoint.h | 29 + tools/virsh-completer.h | 4 + tools/virsh-util.h | 3 + tools/virsh.h | 1 + docs/Makefile.am | 3 + docs/apibuild.py | 2 + docs/docs.html.in | 10 +- docs/domainstatecapture.html.in | 315 ++++ docs/format.html.in | 2 + docs/formatbackup.html.in | 184 ++ docs/formatcheckpoint.html.in | 204 +++ docs/formatsnapshot.html.in | 2 + docs/index.html.in | 4 +- docs/schemas/domainbackup.rng | 219 +++ docs/schemas/domaincheckpoint.rng | 87 + examples/object-events/event-test.c | 3 + libvirt.spec.in | 3 + mingw-libvirt.spec.in | 6 + po/POTFILES | 3 + src/Makefile.am | 2 + src/access/viraccessperm.c | 3 +- src/conf/Makefile.inc.am | 6 + src/conf/backup_conf.c | 538 ++++++ src/conf/checkpoint_conf.c | 636 +++++++ src/conf/domain_conf.c | 22 +- src/conf/snapshot_conf.c | 3 +- src/conf/virdomaincheckpointobjlist.c | 223 +++ src/conf/virdomainmomentobjlist.c | 40 +- src/conf/virdomainobjlist.c | 11 + src/conf/virdomainsnapshotobjlist.c | 2 +- src/libvirt-domain-checkpoint.c | 750 ++++++++ src/libvirt-domain-snapshot.c | 89 + src/libvirt-domain.c | 237 ++- src/libvirt_private.syms | 32 + src/libvirt_public.syms | 24 + src/qemu/qemu_block.c | 12 + src/qemu/qemu_capabilities.c | 4 + src/qemu/qemu_conf.c | 5 + src/qemu/qemu_domain.c | 221 ++- src/qemu/qemu_driver.c | 1577 +++++++++++++++++ src/qemu/qemu_migration.c | 9 +- src/qemu/qemu_monitor.c | 72 +- src/qemu/qemu_monitor_json.c | 222 ++- src/qemu/qemu_process.c | 9 + src/remote/remote_daemon_dispatch.c | 20 + src/remote/remote_driver.c | 32 +- src/remote/remote_protocol.x | 258 ++- src/remote_protocol-structs | 139 ++ src/rpc/gendispatch.pl | 32 +- tests/Makefile.am | 13 +- tests/domainbackupxml2xmlin/backup-pull.xml | 9 + tests/domainbackupxml2xmlin/backup-push.xml | 9 + tests/domainbackupxml2xmlin/empty.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 9 + tests/domainbackupxml2xmlout/backup-push.xml | 9 + tests/domainbackupxml2xmlout/empty.xml | 7 + tests/domaincheckpointxml2xmlin/empty.xml | 1 + tests/domaincheckpointxml2xmlin/sample.xml | 7 + tests/domaincheckpointxml2xmlin/size.xml | 4 + tests/domaincheckpointxml2xmlout/empty.xml | 7 + .../internal-active-invalid.xml | 53 + .../internal-inactive-invalid.xml | 53 + tests/domaincheckpointxml2xmlout/redefine.xml | 63 + tests/domaincheckpointxml2xmlout/sample.xml | 12 + tests/domaincheckpointxml2xmlout/size.xml | 11 + tests/domaincheckpointxml2xmltest.c | 223 +++ tests/qemublocktest.c | 3 +- .../caps_4.0.0.riscv32.xml | 2 + .../caps_4.0.0.riscv64.xml | 2 + .../caps_4.0.0.x86_64.xml | 2 + tests/qemumonitorjsontest.c | 32 +- tests/virschematest.c | 4 + tests/virstoragetest.c | 2 +- tools/Makefile.am | 1 + tools/virsh-checkpoint.c | 1370 ++++++++++++++ tools/virsh-completer.c | 51 + tools/virsh-domain-monitor.c | 23 + tools/virsh-domain.c | 268 ++- tools/virsh-snapshot.c | 37 +- tools/virsh-util.c | 11 + tools/virsh.c | 2 + tools/virsh.pod | 302 +++- 102 files changed, 9502 insertions(+), 100 deletions(-) create mode 100644 include/libvirt/libvirt-domain-checkpoint.h create mode 100644 src/conf/backup_conf.h create mode 100644 src/conf/checkpoint_conf.h create mode 100644 src/conf/virdomaincheckpointobjlist.h create mode 100644 tools/virsh-checkpoint.h create mode 100644 docs/domainstatecapture.html.in create mode 100644 docs/formatbackup.html.in create mode 100644 docs/formatcheckpoint.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 docs/schemas/domaincheckpoint.rng create mode 100644 src/conf/backup_conf.c create mode 100644 src/conf/checkpoint_conf.c create mode 100644 src/conf/virdomaincheckpointobjlist.c create mode 100644 src/libvirt-domain-checkpoint.c create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml create mode 100644 tests/domainbackupxml2xmlin/empty.xml create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml create mode 100644 tests/domainbackupxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml create mode 100644 tests/domaincheckpointxml2xmlin/size.xml create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlout/internal-active-invalid.xml create mode 100644 tests/domaincheckpointxml2xmlout/internal-inactive-invalid.xml create mode 100644 tests/domaincheckpointxml2xmlout/redefine.xml create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml create mode 100644 tests/domaincheckpointxml2xmlout/size.xml create mode 100644 tests/domaincheckpointxml2xmltest.c create mode 100644 tools/virsh-checkpoint.c -- 2.20.1

2 46

[libvirt] [PATCH] docs: minor updates to to new API doc
by Jonathon Jongsma 05 Jul '19

05 Jul '19

Fix a filename and add a couple missing words. Signed-off-by: Jonathon Jongsma <jjongsma(a)redhat.com> --- docs/api_extension.html.in | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/api_extension.html.in b/docs/api_extension.html.in index d7fbbd6e90..3bbaf4893c 100644 --- a/docs/api_extension.html.in +++ b/docs/api_extension.html.in @@ -229,7 +229,7 @@ <h3><a name='rpcclient'>Implement the RPC client</a></h3> <p> - Implementing the uses the rpcgen generated .h files. The remote + Implementing the RPC client uses the rpcgen generated .h files. The remote method calls go in: </p> @@ -256,7 +256,7 @@ The server side dispatchers are implemented in: </p> - <p><code>src/remote/daemon_dispatch.c</code></p> + <p><code>src/remote/remote_daemon_dispatch.c</code></p> <p>Again, this step uses the .h files generated by make rpcgen.</p> -- 2.20.1

2 1

[libvirt] [PATCH] tests: fix mocking of virFileGetXAttrQuiet on FreeBSD
by Daniel P. Berrangé 05 Jul '19

05 Jul '19

The qemusecuritytest is failing on FreeBSD 11/12, reporting that files are not correctly restored. Debugging code printfs show that the virFileGetXAttrQuiet mock is returning 0, but the virFileGetXAttr function is seeing -1 as the return value. Essentially there appears to be some kind of optimization between the real virFileGetXAttrQuiet and the real virFileGetXAttr, which breaks when we mock virFileGetXAttrQuiet. Rather than trying to figure out how to avoid this, it is simpler to just mock virFileGetXAttr too since it is very short code. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- Pushed as a CI build fix tests/qemusecuritymock.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/tests/qemusecuritymock.c b/tests/qemusecuritymock.c index 2a9095e1bf..a15eef29c9 100644 --- a/tests/qemusecuritymock.c +++ b/tests/qemusecuritymock.c @@ -156,6 +156,34 @@ virFileGetXAttrQuiet(const char *path, } +/* + * This may look redundant but is needed to work around an + * compiler quirk. The call from the real virFileGetXAttr + * to the real virFileGetXAttrQuiet has a quirk where the + * return value from virFileGetXAttrQuiet gets scrambled + * if we mock virFileGetXAttrQuiet, returning -1 instead + * of 0 despite succeeding. This happens on FreeBSD 11/12 + * hosts with CLang, and is suspected to be some kind of + * compiler optimization. By mocking this function too we + * can workaround it. + */ +int +virFileGetXAttr(const char *path, + const char *name, + char **value) +{ + int ret; + + if ((ret = virFileGetXAttrQuiet(path, name, value)) < 0) { + virReportSystemError(errno, + "Unable to get XATTR %s on %s", + name, path); + } + + return ret; +} + + int virFileSetXAttr(const char *path, const char *name, const char *value) -- 2.21.0

1 0

Re: [libvirt] [libvirt-users] UDP broadcasts vs. nat Masquerading issue
by Nikolai Zhubr 05 Jul '19

05 Jul '19

Hi Daniel and Laine, [...] >> -A POSTROUTING -o br0 -j MASQUERADE >> -A POSTROUTING -o enp0s25 -j MASQUERADE >> -A POSTROUTING -o virbr2_nic -j MASQUERADE >> -A POSTROUTING -o vnet0 -j MASQUERADE > > *None* of those rules were added by libvirt (unless your build of [...] > You can verify my "counter-claim" by running "virsh net-destroy" for all > of your libvirt networks, and seeing that the offending rules haven't > been removed. > > In short, you need to look elsewhere for the culprit. Yes, found it. You were both right, essentially. The offending rules were added by a firewall in response to new interfaces created by libvirt dynamically, due to some dubious relict settings left in the firewall. (Silly me.) So this it not an issue of libvirt indeed! Tons of thanks for the quick and precise hit! Regards, Nikolai >> >> Here, virbr2_nic and vnet0 are used by libvirt for arranging network >> configurations for VMs, ok. However, br0 is a main interface of this >> host with primary ip address, with enp0s25 being a physical nic of >> this host, and it is used for all sorts of regular (unrelated to >> virtualization) communications. Also, br0 is used for attaching >> bridged (as opposed to NATed) VMs managed by libvirt. >> >> Clearly, libvirt somehow chooses to set up masquerading for literally >> all existing network interfaces here (except lo), > > It's clear that the rules are there. It's not clear that they were added > by libvirt. > >> but I can't see a real reason for the first two rules in the list >> above. Furthermore, they corrupt UDP broadcats coming from outside and >> reaching this host (through enp0s25/br0) such that source address gets >> replaced by this hosts primary address (as per masquerading). I've >> verified this by arranging a hand-crafted UDP listener and printing >> the respective source addresses as seen by normal userspace. >> >> Now I've discovered that I can "eliminate" the problem by either: >> >> 1. Removing "-A POSTROUTING -o br0 -j MASQUERADE" (manually) >> 2. Inserting "-A POSTROUTING -s 192.168.0.0/24 -d 192.168.0.255/32 -j >> ACCEPT" >> (Of course correcting rules by hand is not a solution, just a test) >> >> So question is, how the correct rules should ideally look like? And, >> is this issue known/fixed in most current libvirt? > > Except for putting the libvirt-added rules in their own private chains > (appearing in libvirt 5.1.0, released on Feb 1, 2019), the iptables > rules added by libvirt to support its virtual networks didn't materially > change in > 10 years. Your email is the first time I've ever seen such > rules attributed to libvirt so, as I said above, I think you need to > take a deeper dive into your host system's config. > > > Good luck! >

2 1

[libvirt] [PATCH 0/3] remote: misc cleanups to client driver
by Daniel P. Berrangé 05 Jul '19

05 Jul '19

A few bits of cleanup identified during work on the daemon splitting patches sent now since they are independent. Daniel P. Berrangé (3): remote: stop declaring variables in the middle of a function remote: use autofree for many string variables remote: conditionally declare tty variable for non-Win32 platforms src/remote/remote_driver.c | 61 ++++++++++++++------------------------ 1 file changed, 23 insertions(+), 38 deletions(-) -- 2.21.0

2 4

[libvirt] [PATCH rust 0/5] ci related plus some small updates
by Sahid Orentino Ferdjaoui 05 Jul '19

05 Jul '19

Hello Daniel, When you have a moment, can you please merge this serie. It fixes CI issue, switch to bionic, and updates the tested versions. Results: https://travis-ci.org/sahid/libvirt-rust/builds/554170698 Thanks, s. Sahid Orentino Ferdjaoui (5): fix bug integration test with rust multithreading make lookup_by_id() test more robust update tested versions from 2.5.0 to 5.5.0 switch to the last ubuntu lts bionic fix code formating in README .travis.yml | 10 +++++----- README.md | 6 +++++- tests/domain.rs | 19 +++++++++---------- tests/integration_qemu.rs | 15 +++++++++++---- tests/libvirtd.sasl | 2 ++ 5 files changed, 32 insertions(+), 20 deletions(-) create mode 100644 tests/libvirtd.sasl -- 2.20.1

2 14

[libvirt] [PATCH v2 0/4] test_driver: implement some DomainGetParametersAPIs
by Ilias Stamatis 04 Jul '19

04 Jul '19

Changes since v1: - undefine the macro introduced in 1/4 when done with it's usage - simplified 4/4 by using the virNetDevBandwidthRate struct Ilias Stamatis (4): test_driver: introduce TEST_SET_PARAM macro test_driver: implement virDomainGetMemoryParameters test_driver: implement virDomainGetNumaParameters test_driver: implement testDomainGetInterfaceParameters src/test/test_driver.c | 154 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 154 insertions(+) -- 2.22.0

2 7

[libvirt] [PATCH] test_driver: avoid using nerrors unitialized in testDomainGetDiskErrors
by Ilias Stamatis 04 Jul '19

04 Jul '19

Right now nerrors can be used unitialized in the last for loop causing a bug. Signed-off-by: Ilias Stamatis <stamatis.iliass(a)gmail.com> --- src/test/test_driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/test/test_driver.c b/src/test/test_driver.c index ce4ff1a582..2371581f51 100644 --- a/src/test/test_driver.c +++ b/src/test/test_driver.c @@ -3275,7 +3275,7 @@ static int testDomainGetDiskErrors(virDomainPtr dom, virDomainObjPtr vm = NULL; int ret = -1; size_t i; - size_t nerrors; + size_t nerrors = 0; virCheckFlags(0, -1); -- 2.22.0

2 1

[libvirt] [PATCH] test: ensure nerrors variable is initialized
by Daniel P. Berrangé 04 Jul '19

04 Jul '19

There is an error path that jumps over the initialization of nerrors, and the jump target reads the variable contents. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- Pushed as a build fix src/test/test_driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/test/test_driver.c b/src/test/test_driver.c index ce4ff1a582..2371581f51 100644 --- a/src/test/test_driver.c +++ b/src/test/test_driver.c @@ -3275,7 +3275,7 @@ static int testDomainGetDiskErrors(virDomainPtr dom, virDomainObjPtr vm = NULL; int ret = -1; size_t i; - size_t nerrors; + size_t nerrors = 0; virCheckFlags(0, -1); -- 2.21.0

1 0