July 2017 - Devel - libvirt List Archives

[libvirt] [PATCH 0/7] Some virObjectRW* adjustments
by John Ferlan 31 Jul '17

31 Jul '17

As indicated in a couple of recent reviews, the new virObjectRWLockable object and API's I believe needed a few adjustments in order to make them better. So I present those thoughts in patch format to be hopefully dicussed and dissected. Essentially this series, will modify the newly added virObjectLockRead to return a status and to have that status managed. Since there were only 9 new callers, this was straightforward. None of the callers actually needed any sort of "divert logic" in order to jump around code that shouldn't do the Unlock now. Although that could be a problem if ever changing other more deeply nested type locks. That's someone else's headache though. Next, introduce and use virObjectLockWrite to be only for the new style RWLocks. This too will return status. Only the vir*Remove caller has a code path that I'm not happy with, but since these type locks have been processing along anyway without technically ensuring they have the lock before doing so, well no harm, no foul. Next, introduce and use virObjectRWUnlock. Rather than overloading the virObjectUnlock, let's just keep each caller doing it's one thing to be sure that the calling code is doing the "right thing". In this case, returning a failure status is just not going to work. We've gone too far down the path in order for a failure to cause some failure in the caller. We'll just treat it like virObjectUnlock and if the resource remains locked, well we'll find out relatively soon as the next time someone goes to read/write the list they'll be waiting for an unlock that will never happen because of some coding mistake. But that will save us from perhaps other dire consequences involving corruption. Finally, revert the virObjectLock and virObjectUnlock code to just manage virObjectLockable locks. No worse than what happened before the RWLock was introduced. Leaving the consumers to handle things as they would before - which probably ends up in some sort of awful state leading to memory corruption and a quick death as opposed to abort()'g in the void() function which has been deemed a don't do that type operation for libvirt. The last 2 patches actually have appeared before in various common object series postings, so they could be a v2 or 3 type logic, but I see them more related to this than that, so I present them here again. John Ferlan (7): util: Alter virObjectLockRead to return status util: Introduce and use virObjectLockWrite util: Only have virObjectLock handle virObjectLockable util: Introduce virObjectGetRWLockableObj util: Introduce and use virObjectRWUnlock util: Create common error path for invalid object util: Add safety net of checks to ensure valid object src/conf/virdomainobjlist.c | 81 +++++++++++++--------- src/libvirt_private.syms | 2 + src/util/virobject.c | 160 +++++++++++++++++++++++++++++++++----------- src/util/virobject.h | 10 ++- 4 files changed, 181 insertions(+), 72 deletions(-) -- 2.9.4

4 26

[libvirt] Entering freeze for libvirt-3.6.0
by Daniel Veillard 31 Jul '17

31 Jul '17

As planned I tagged 3.6.0-rc1 in git and I made the Release Candidate 1 signed tarball and rpms available from the usual place: ftp://libvirt.org/libvirt/ seems to work in my limited testing, https://ci.centos.org/view/libvirt/ is all green (except for mingw maybe there is a portability issue) Please give it a try, check other systems, and if everything goes well I will make RC2 over the week-end and 3.6.0 may land on Tuesday the 1st ! Thanks in advance, Daniel -- Daniel Veillard | Red Hat Developers Tools http://developer.redhat.com/ veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

4 5

[libvirt] [PATCH 0/3] libxl: Add a test suite for libxl_domain_config generator
by Jim Fehlig 30 Jul '17

30 Jul '17

Long ago danpb posted some patches to test libvirt domXML to libxl_domain_config conversion https://www.redhat.com/archives/libvir-list/2014-May/msg01102.html Some of the prerequisite patches were pushed, but we've never managed to push patches actually providing the conversion tests. I sent several follow-ups to Dan's work but never converged on a satisfactory solution for all the Xen versions supported by libvirt. The last attempt was in Sept 2014 https://www.redhat.com/archives/libvir-list/2014-September/msg00698.html I tried to revive the work in Jan 2015, but that also stalled https://www.redhat.com/archives/libvir-list/2015-January/msg00924.html Fast-forward over 2.5 years from the first attempt and libvirt no longer supports older Xen versions 4.2 and 4.3 that were proving to be problematic. Starting with Xen 4.5 libxl added support for libxl_domain_config_from_json, which provides a way to implement the conversion tests that work with all Xen versions >= 4.5 (including latest xen.git master). The previous approaches compared a static json doc with the results of domXML->libxl_domain_config conversion done in the libxl driver. As discussed in those approaches, the json doc returned from libxl_domain_config_to_json can change over Xen releases as the libxl_domain_config object gains new fields, making it difficult to compare the conversion with a static doc. I had some time last week to pick at this thorn and reworked the test to make use of libxl_domain_config_from_json introduced in Xen 4.5. Instead of comparing the conversion results directly to a static json doc, the static doc is first round-tripped through _from_json -> _to_json. Any new fields added to libxl_domain_config object are then included in both docs, allowing comparison across multiple Xen releases. Patch3 provides the conversion tests using this new approach. The tests are not run on Xen 4.4 (oldest version currently supported by libvirt) since it does not provide libxl_domain_config_from_json. Patches 1 and 2 fix some issues found while working on the tests. See their commit messages for details. Jim Fehlig (3): libxl: determine device model version from emulator name libxl: relax checks on <emulator> libxl: Add a test suite for libxl_domain_config generator m4/virt-driver-libxl.m4 | 6 +- src/libxl/libxl_capabilities.c | 37 ++--- src/libxl/libxl_conf.c | 14 -- tests/Makefile.am | 18 ++- tests/libxlxml2domconfigdata/basic-hvm.json | 89 +++++++++++ tests/libxlxml2domconfigdata/basic-hvm.xml | 36 +++++ tests/libxlxml2domconfigdata/basic-pv.json | 65 ++++++++ tests/libxlxml2domconfigdata/basic-pv.xml | 28 ++++ tests/libxlxml2domconfigdata/moredevs-hvm.json | 111 +++++++++++++ tests/libxlxml2domconfigdata/moredevs-hvm.xml | 63 ++++++++ tests/libxlxml2domconfigtest.c | 208 +++++++++++++++++++++++++ tests/virmocklibxl.c | 87 +++++++++++ 12 files changed, 723 insertions(+), 39 deletions(-) create mode 100644 tests/libxlxml2domconfigdata/basic-hvm.json create mode 100644 tests/libxlxml2domconfigdata/basic-hvm.xml create mode 100644 tests/libxlxml2domconfigdata/basic-pv.json create mode 100644 tests/libxlxml2domconfigdata/basic-pv.xml create mode 100644 tests/libxlxml2domconfigdata/moredevs-hvm.json create mode 100644 tests/libxlxml2domconfigdata/moredevs-hvm.xml create mode 100644 tests/libxlxml2domconfigtest.c create mode 100644 tests/virmocklibxl.c -- 2.11.0

2 13

[libvirt] [PATCH] news: Update for 3.6.0 release
by Michal Privoznik 30 Jul '17

30 Jul '17

I was volunteered to prepare the release notes. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- Feel free to propose your stuff too if I forgot to add it. docs/news.xml | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) diff --git a/docs/news.xml b/docs/news.xml index ea21cbc3e..03494d257 100644 --- a/docs/news.xml +++ b/docs/news.xml @@ -75,8 +75,77 @@ configurations. </description> </change> + <change> + <summary> + Require use of GCC 4.4 or CLang compilers + </summary> + <description> + We only ever test libvirt with GCC or CLang which provides a + GCC compatible compilation environment. Between them, these + compilers cover every important operating system platform, + even Windows. + </description> + </change> + <change> + <summary> + qemu: shared disks with directsync cache should be safe for migration + </summary> + <description> + At present shared disks can be migrated with either readonly or + <code>cache=none</code>. But <code>cache=directsync</code> should be + safe for migration, because both <code>cache=directsync</code> and + cache=none don't use the host page cache, and + <code>cache=direct</code> write through qemu block layer cache. + </description> + </change> + <change> + <summary> + Handle hotplug change on VLAN configuration using OVS + </summary> + <description> + Libvirt was taught to handle VLAN change for running OVS interface. + </description> + </change> </section> <section title="Bug fixes"> + <change> + <summary> + qemu: Use vcpu 'node-id' property and pass it back to qemu + </summary> + <description> + Vcpu properties gathered from query-hotpluggable cpus need to be + passed back to qemu. As qemu did not use the node-id property until + now and libvirt forgot to pass it back properly (it was parsed but + not passed around) we did not honor this. + </description> + </change> + <change> + <summary> + Misceleanous stream fixes + </summary> + <description> + After introducing sparse stream features there were still some known + bugs left. Those are fixed in this release. + </description> + </change> + <change> + <summary> + Misceleanous qemu domain NS fixes + </summary> + <description> + Libvirt starts qemu domains in separate Linux namespaces for a while + now. However, there were still some bugs lingering. For instance + libvirt did not know how to handle file based bind mounts. + </description> + </change> + <change> + <summary> + Various CPU driver improvements + </summary> + <description> + There were some minor bugs when using 'host-model' CPU. + </description> + </change> </section> </release> <release version="v3.5.0" date="2017-07-04"> -- 2.13.0

2 1

[libvirt] [PATCH V2 0/2]
by Jim Fehlig 28 Jul '17

28 Jul '17

This is V2 of a long standing series to add unit tests for libvirt domXML to libxl_domain_config conversions. There's a lot more history than a V2 would imply, which can be read in the last posting of the series https://www.redhat.com/archives/libvir-list/2017-February/msg01477.html Unlike attempts of the past, I actually feel good about this version and I hope others feel the same so we can finally get the series pushed! Patch1 changes the checks for <emulator> in the libxl driver to make them mockable. Patch2 provides mocking of the emulator checks and some libxl functions, and of course provides the tests. Jim Fehlig (2): libxl: make emulator checks mockable libxl: Add a test suite for libxl_domain_config generator m4/virt-driver-libxl.m4 | 4 + src/libxl/libxl_capabilities.c | 30 +++- src/libxl/libxl_capabilities.h | 5 +- src/libxl/libxl_conf.c | 17 +- src/libxl/libxl_domain.c | 2 +- tests/Makefile.am | 18 ++- tests/libxlxml2domconfigdata/basic-hvm.json | 89 +++++++++++ tests/libxlxml2domconfigdata/basic-hvm.xml | 36 +++++ tests/libxlxml2domconfigdata/basic-pv.json | 65 ++++++++ tests/libxlxml2domconfigdata/basic-pv.xml | 28 ++++ tests/libxlxml2domconfigdata/moredevs-hvm.json | 111 +++++++++++++ tests/libxlxml2domconfigdata/moredevs-hvm.xml | 63 ++++++++ tests/libxlxml2domconfigtest.c | 209 +++++++++++++++++++++++++ tests/virmocklibxl.c | 106 +++++++++++++ 14 files changed, 764 insertions(+), 19 deletions(-) create mode 100644 tests/libxlxml2domconfigdata/basic-hvm.json create mode 100644 tests/libxlxml2domconfigdata/basic-hvm.xml create mode 100644 tests/libxlxml2domconfigdata/basic-pv.json create mode 100644 tests/libxlxml2domconfigdata/basic-pv.xml create mode 100644 tests/libxlxml2domconfigdata/moredevs-hvm.json create mode 100644 tests/libxlxml2domconfigdata/moredevs-hvm.xml create mode 100644 tests/libxlxml2domconfigtest.c create mode 100644 tests/virmocklibxl.c -- 2.13.1

1 3

[libvirt] [PATCH v2 0/2] Add test case for node name detection
by Peter Krempa 28 Jul '17

28 Jul '17

Patch 2/2 was originallly included in the node name detection refactor. v2 does not contain an ugly hack to make virTestCompareToFile, but rather fixes the function to treat NULL as an empty string. Peter Krempa (2): tests: utils: Allow NULL strings to be equal to empty file in virTestCompareToFile tests: qemumonitorjson: Old and empty test case for node name detection .../qemumonitorjson-nodename-empty-blockstats.json | 2 + ...qemumonitorjson-nodename-empty-named-nodes.json | 2 + .../qemumonitorjson-nodename-empty.result | 0 .../qemumonitorjson-nodename-old-blockstats.json | 160 +++++++++++++++++++++ .../qemumonitorjson-nodename-old-named-nodes.json | 2 + .../qemumonitorjson-nodename-old.result | 0 tests/qemumonitorjsontest.c | 2 + tests/testutils.c | 5 + 8 files changed, 173 insertions(+) create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-nodename-empty-blockstats.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-nodename-empty-named-nodes.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-nodename-empty.result create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-nodename-old-blockstats.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-nodename-old-named-nodes.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-nodename-old.result -- 2.13.2

3 4

[libvirt] [PATCH] qemu: Split shmem preparation as it's supposed to be
by Martin Kletzander 28 Jul '17

28 Jul '17

Since the introduction of shmem, there was a split of preparation code from the formatting code from qemuBuildCommandLine() into qemuProcessPrepareDomain(). Let's fix shmem in this regard, so that we can slowly get to a cleaner codebase. Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- src/qemu/qemu_command.c | 30 +++++------------------------- src/qemu/qemu_process.c | 5 +++++ 2 files changed, 10 insertions(+), 25 deletions(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index d14bb621148d..90471d89eb8c 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -8876,28 +8876,6 @@ qemuBuildShmemDevStr(virDomainDefPtr def, return virBufferContentAndReset(&buf); } -static char * -qemuBuildShmemBackendChrStr(virLogManagerPtr logManager, - virCommandPtr cmd, - virQEMUDriverConfigPtr cfg, - virDomainDefPtr def, - virDomainShmemDefPtr shmem, - virQEMUCapsPtr qemuCaps, - bool chardevStdioLogd) -{ - char *devstr = NULL; - - if (qemuDomainPrepareShmemChardev(shmem) < 0) - return NULL; - - devstr = qemuBuildChrChardevStr(logManager, cmd, cfg, def, - &shmem->server.chr, - shmem->info.alias, qemuCaps, true, - chardevStdioLogd); - - return devstr; -} - virJSONValuePtr qemuBuildShmemBackendMemProps(virDomainShmemDefPtr shmem) @@ -9007,9 +8985,11 @@ qemuBuildShmemCommandLine(virLogManagerPtr logManager, VIR_FREE(devstr); if (shmem->server.enabled) { - if (!(devstr = qemuBuildShmemBackendChrStr(logManager, cmd, cfg, def, - shmem, qemuCaps, - chardevStdioLogd))) + devstr = qemuBuildChrChardevStr(logManager, cmd, cfg, def, + &shmem->server.chr, + shmem->info.alias, qemuCaps, true, + chardevStdioLogd); + if (!devstr) return -1; virCommandAddArgList(cmd, "-chardev", devstr, NULL); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 525521aaf0ca..b2e1314a3887 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -5325,6 +5325,11 @@ qemuProcessPrepareDomain(virConnectPtr conn, if (qemuProcessUpdateGuestCPU(vm->def, priv->qemuCaps, caps, flags) < 0) goto cleanup; + for (i = 0; i < vm->def->nshmems; i++) { + if (qemuDomainPrepareShmemChardev(vm->def->shmems[i]) < 0) + goto cleanup; + } + ret = 0; cleanup: virObjectUnref(caps); -- 2.13.3

2 1

[libvirt] [PATCH v3 00/16] virObject adjustments for common object
by John Ferlan 28 Jul '17

28 Jul '17

v2: https://www.redhat.com/archives/libvir-list/2017-June/msg00070.html Pushed the first two patches from v2 since they were ACK'd Changes in this series... Fixed a couple of nits from former patch 3 & 4, but since they weren't ACK'd they're here again, but now as patch 1 & 2. Patch 3 & 4 are an adjusted version of the former patches 5 & 6. Instead of going with generic @primaryKey and @secondaryKey names, this patch will use @uuid and @name for the field names. Additionally instead of using PoolableHashElement as a name, go with a much shorter LookupKeys. So far the LookupKeys{UUID|Name} API's (patch 4) aren't used and could be dropped if it's felt no future API would need them. Former patches 7 & 8 dealing with the generic @def and @newDef object were tossed away and the rest of the logic I'd be changing for virObject is presented as the remaining (new to reviewers) patches 5 -> 16. The object is named using LookupHash which is a follow-on of the LookupKeys. If someone has better suggestion for names, then please provide suggestions rather than just saying I hate the name! It's not my favorite name, but it does convey what it is and IMO is better than just Element. The patches also include the Interface object changes to illustrate that the changes do work. I've run them through the various "interface" tests from the Avocado VT test suite. I also did something similar for the Secret object in my private branch, but did not include that since there are 8 patches on list waiting to be reviewed first. John Ferlan (16): util: Generate a common internal only error print util: Add safety net of checks to ensure valid object util: Introduce virObjectLookupKeys util: Introduce virObjectLookupKeysGet{UUID|Name} interface: Use virObjectLookupKeys util: Introduce virObjectLookupKeys*Active API's interface: Use virObjectLookupKeys*Active util: Introduce virObjectLookupHash util: Introduce virObjectLoookupHashGet{UUID|Name} util: Introduce virObjectLookupHash{Add|Remove} util: Introduce virObjectLookupHashFind util: Introduce virObjectLookupHashForEach util: Introduce virObjectLookupHashSearch util: Introduce virObjectLookupHashClone interface: Use virObjectLookupHash test: Clean up test driver Interface interactions src/conf/virinterfaceobj.c | 332 ++++++++++++++---------- src/libvirt_private.syms | 16 ++ src/test/test_driver.c | 55 +--- src/util/virobject.c | 613 ++++++++++++++++++++++++++++++++++++++++++++- src/util/virobject.h | 111 ++++++++ 5 files changed, 938 insertions(+), 189 deletions(-) -- 2.9.4

2 22

[libvirt] [PATCH 0/2] tests: fix build without WITH_LIBVIRTD defined
by Peter Krempa 28 Jul '17

28 Jul '17

'virdrivermoduletest' failure was caught by the centos ci. virt-aa-helper-test while testing it on my machine with specifically disabled libvirtd. Peter Krempa (2): tests: Run virt-aa-helper-test only when the helper was compiled tests: Run virdrivermoduletest only when WITH_LIBVIRTD is defined tests/Makefile.am | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) -- 2.13.2

2 3

[libvirt] [PATCH] docs: Format comment timestamp in UTC
by Martin Kletzander 28 Jul '17

28 Jul '17

I just forgot '-u' in a54c9622860a. Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- docs/Makefile.am | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/Makefile.am b/docs/Makefile.am index a5bff7de1c9d..34f84e526daa 100644 --- a/docs/Makefile.am +++ b/docs/Makefile.am @@ -198,9 +198,9 @@ MAINTAINERCLEANFILES = \ timestamp="$(shell if test -n "$$SOURCE_DATE_EPOCH"; \ then \ - date --date="@$$SOURCE_DATE_EPOCH"; \ + date -u --date="@$$SOURCE_DATE_EPOCH"; \ else \ - date; \ + date -u; \ fi)" all-am: web -- 2.13.3

2 1