March 2019 - Devel - libvirt List Archives

[libvirt] [perl][PATCH 0/3] Add a bunch of constants
by Michal Privoznik 29 Mar '19

29 Mar '19

*** BLURB HERE *** Michal Privoznik (3): Register new error code constants Add VIR_FROM_DOMAIN_CHECKPOINT constant Add VIR_MIGRATE_PARALLEL and VIR_MIGRATE_PARAM_PARALLEL_CONNECTIONS constants Changes | 5 +++++ lib/Sys/Virt.xs | 6 ++++++ lib/Sys/Virt/Domain.pm | 10 ++++++++++ lib/Sys/Virt/Error.pm | 16 ++++++++++++++++ 4 files changed, 37 insertions(+) -- 2.19.2

2 6

[libvirt] Time to drop support for Debian 8 (Jessie)?
by Andrea Bolognani 29 Mar '19

29 Mar '19

We currently support Debian 8 (oldstable) along with Debian 9 (stable), but not without some compromises: * the libvirt-dbus, libvirt-ocaml and virt-manager projects do not support the platform at all because it ships outdated versions of some core components; * on the CI side of things, we are forced to drag in the JRE from backports in order to be able to run the Jenkins agent. All things considered, the situation has been fairly manageable up until now, but a couple of recent developments got me thinking that perhaps it's time to let Jessie go: * the distribution has been moved from the regular Debian infrastructure to archive.debian.org[1], a change which has resulted in the daily update run failing and would require investing time to adapt to; * Debian testing has recently entered the full freeze[2], which means the release of Debian 10 can hopefully be expected to happen within the next few month; * even if the Buster freeze period turned out to be exceedingly long, according to our platform support policy[3] we only promise to support a release for the two years after the most recent major release: given that Debian 9 was released in June 2017[4], we would be able to drop Debian 8 support in three months' time regardless of whether or not Debian 10 has been released in the meantime. Based on the above, I suggest we don't invest any time trying to keep Debian 8 chugging along only to drop it in June, and instead declare it as unsupported right now and move on with our lives. Thoughts? [1] https://lists.debian.org/debian-devel-announce/2019/03/msg00006.html [2] https://lists.debian.org/debian-devel-announce/2019/03/msg00003.html [3] https://libvirt.org/platforms.html [4] https://wiki.debian.org/DebianReleases -- Andrea Bolognani / Red Hat / Virtualization

2 8

[libvirt] [PATCH] virsh: Don't infloop on snapshot/storage_vol failure
by Eric Blake 29 Mar '19

29 Mar '19

Most of our completers used the pattern: if ((nITEM = virITEMListAll()) < 0) return NULL; but the virDomainSnapshot and virStorageVolume completers were instead using goto error because of a prior allocation. If the ListAll fails with -1, the cleanup label was running a loop of 'size_t i < int nITEM', which is an extreme waste of CPU cycles. Reported-by: Ján Tomko <jtomko(a)redhat.com> Signed-off-by: Eric Blake <eblake(a)redhat.com> --- This one is a bug fix, so worth having in 5.2. tools/virsh-completer.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/tools/virsh-completer.c b/tools/virsh-completer.c index c4adbb70d0..e9ef9b99f9 100644 --- a/tools/virsh-completer.c +++ b/tools/virsh-completer.c @@ -278,6 +278,7 @@ virshStorageVolNameCompleter(vshControl *ctl, virshControlPtr priv = ctl->privData; virStoragePoolPtr pool = NULL; virStorageVolPtr *vols = NULL; + int rc; int nvols = 0; size_t i = 0; char **ret = NULL; @@ -290,8 +291,9 @@ virshStorageVolNameCompleter(vshControl *ctl, if (!(pool = virshCommandOptPool(ctl, cmd, "pool", NULL))) return NULL; - if ((nvols = virStoragePoolListAllVolumes(pool, &vols, flags)) < 0) + if ((rc = virStoragePoolListAllVolumes(pool, &vols, flags)) < 0) goto error; + nvols = rc; if (VIR_ALLOC_N(ret, nvols + 1) < 0) goto error; @@ -631,6 +633,7 @@ virshSnapshotNameCompleter(vshControl *ctl, virshControlPtr priv = ctl->privData; virDomainPtr dom = NULL; virDomainSnapshotPtr *snapshots = NULL; + int rc; int nsnapshots = 0; size_t i = 0; char **ret = NULL; @@ -643,8 +646,9 @@ virshSnapshotNameCompleter(vshControl *ctl, if (!(dom = virshCommandOptDomain(ctl, cmd, NULL))) return NULL; - if ((nsnapshots = virDomainListAllSnapshots(dom, &snapshots, flags)) < 0) + if ((rc = virDomainListAllSnapshots(dom, &snapshots, flags)) < 0) goto error; + nsnapshots = rc; if (VIR_ALLOC_N(ret, nsnapshots + 1) < 0) goto error; -- 2.20.1

2 1

[libvirt] [jenkins-ci PATCH] jenkins: only send one email on build failure
by Daniel P. Berrangé 29 Mar '19

29 Mar '19

The current config causes jenkins to send a build on every single failure. This is way too noisy so change it to only send emails on the transition from good to bad. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- jenkins/jobs/autotools.yaml | 8 ++++---- jenkins/jobs/generic.yaml | 8 ++++---- jenkins/jobs/go.yaml | 4 ++-- jenkins/jobs/perl-modulebuild.yaml | 6 +++--- jenkins/jobs/python-distutils.yaml | 6 +++--- 5 files changed, 16 insertions(+), 16 deletions(-) diff --git a/jenkins/jobs/autotools.yaml b/jenkins/jobs/autotools.yaml index e365ec4..f04ec17 100644 --- a/jenkins/jobs/autotools.yaml +++ b/jenkins/jobs/autotools.yaml @@ -50,7 +50,7 @@ publishers: - email: recipients: '{obj:spam}' - notify-every-unstable-build: true + notify-every-unstable-build: false send-to-individuals: false - job-template: @@ -89,7 +89,7 @@ publishers: - email: recipients: '{obj:spam}' - notify-every-unstable-build: true + notify-every-unstable-build: false send-to-individuals: false - job-template: @@ -132,7 +132,7 @@ publishers: - email: recipients: '{obj:spam}' - notify-every-unstable-build: true + notify-every-unstable-build: false send-to-individuals: false - job-template: @@ -174,5 +174,5 @@ publishers: - email: recipients: '{obj:spam}' - notify-every-unstable-build: true + notify-every-unstable-build: false send-to-individuals: false diff --git a/jenkins/jobs/generic.yaml b/jenkins/jobs/generic.yaml index 1990930..ca69513 100644 --- a/jenkins/jobs/generic.yaml +++ b/jenkins/jobs/generic.yaml @@ -45,7 +45,7 @@ publishers: - email: recipients: '{obj:spam}' - notify-every-unstable-build: true + notify-every-unstable-build: false send-to-individuals: false - job-template: @@ -83,7 +83,7 @@ publishers: - email: recipients: '{obj:spam}' - notify-every-unstable-build: true + notify-every-unstable-build: false send-to-individuals: false - job-template: @@ -121,7 +121,7 @@ publishers: - email: recipients: '{obj:spam}' - notify-every-unstable-build: true + notify-every-unstable-build: false send-to-individuals: false - job-template: @@ -159,5 +159,5 @@ publishers: - email: recipients: '{obj:spam}' - notify-every-unstable-build: true + notify-every-unstable-build: false send-to-individuals: false diff --git a/jenkins/jobs/go.yaml b/jenkins/jobs/go.yaml index 3c14c35..1bdcb5f 100644 --- a/jenkins/jobs/go.yaml +++ b/jenkins/jobs/go.yaml @@ -45,7 +45,7 @@ publishers: - email: recipients: '{obj:spam}' - notify-every-unstable-build: true + notify-every-unstable-build: false send-to-individuals: false - job-template: @@ -83,5 +83,5 @@ publishers: - email: recipients: '{obj:spam}' - notify-every-unstable-build: true + notify-every-unstable-build: false send-to-individuals: false diff --git a/jenkins/jobs/perl-modulebuild.yaml b/jenkins/jobs/perl-modulebuild.yaml index 933020a..7173493 100644 --- a/jenkins/jobs/perl-modulebuild.yaml +++ b/jenkins/jobs/perl-modulebuild.yaml @@ -47,7 +47,7 @@ publishers: - email: recipients: '{obj:spam}' - notify-every-unstable-build: true + notify-every-unstable-build: false send-to-individuals: false @@ -86,7 +86,7 @@ publishers: - email: recipients: '{obj:spam}' - notify-every-unstable-build: true + notify-every-unstable-build: false send-to-individuals: false - job-template: @@ -127,5 +127,5 @@ publishers: - email: recipients: '{obj:spam}' - notify-every-unstable-build: true + notify-every-unstable-build: false send-to-individuals: false diff --git a/jenkins/jobs/python-distutils.yaml b/jenkins/jobs/python-distutils.yaml index 6b388f2..f56794c 100644 --- a/jenkins/jobs/python-distutils.yaml +++ b/jenkins/jobs/python-distutils.yaml @@ -47,7 +47,7 @@ publishers: - email: recipients: '{obj:spam}' - notify-every-unstable-build: true + notify-every-unstable-build: false send-to-individuals: false @@ -86,7 +86,7 @@ publishers: - email: recipients: '{obj:spam}' - notify-every-unstable-build: true + notify-every-unstable-build: false send-to-individuals: false - job-template: @@ -127,5 +127,5 @@ publishers: - email: recipients: '{obj:spam}' - notify-every-unstable-build: true + notify-every-unstable-build: false send-to-individuals: false -- 2.20.1

2 4

[libvirt] [jenkins-ci PATCH v2 1/2] guests: Update test dependencies for osinfo-db
by Fabiano Fidêncio 29 Mar '19

29 Mar '19

The new dependencies are: - python3 - python3-lxml - python3-pytest - python3-requests xmllint has been removed in favour of a own crafted test using python3-lxml Signed-off-by: Fabiano Fidêncio <fidencio(a)redhat.com> --- guests/vars/projects/osinfo-db.yml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/guests/vars/projects/osinfo-db.yml b/guests/vars/projects/osinfo-db.yml index d9320a4..56bd77a 100644 --- a/guests/vars/projects/osinfo-db.yml +++ b/guests/vars/projects/osinfo-db.yml @@ -1,4 +1,7 @@ --- packages: - intltool - - xmllint + - python3 + - python3-lxml + - python3-pytest + - python3-requests -- 2.20.1

3 4

[libvirt] [PATCH] tests: Disable some tests on 32 bit systems
by Michal Privoznik 28 Mar '19

28 Mar '19

On 32 bit systems there are two tests failing currently: qemufirmwaretest and qemuxml2argvtest (not every test case in them is failing, only some of them). There are several problems: 1) in qemufirmwaretest it's 'QEMU FW precedence test' which is failing. This is because the code tests qemuFirmwareFetchConfigs() which tries to compile a list of some paths from the system and if one of the paths is an empty file it should not appear on the returned list. The code uses virFileLength() to query the file size which uses stat() under the hood. The test uses virfilewrapper to redirect stat() into qemufirmwaredata/ but for reasons beyond me real_stat() returns mangled buffer making the code see a file with a real size even for empty files. I've track this one to a problem in virfilewrapper.c which will call real___xstat() which points to a function in libvirt.so (!) which returns stat with 32bit members even though we're compiling with LARGE_FILE and therefore expect 64bit members in the stat struct. 2) qemuxml2argvtest has two tests failing: 2a) [aarch64-]os-firmware-* - These fail because of the same reason as described in 1) 2b) pseries-hostdevs-* - These fail because they again rely on some stat (lstat to be precise). When building the cmd line for these test cases the code does some search in /sys/bus/pci/... and since we have virpcimock everything works just fine. Except when virfilewrapper.c is linked in because then lstat() is taken from there and when it initializes itself it will lookup real_lstat() which will now point not to virpcimock but to glibc. Again, reasons beyond me. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- Quite frankly, I hate this patch. But the problem is in our mocking, not in the actual code its testing. And I've tried everything I was able to come up with on how to fix this. So if you have any idea, I'm all ears. tests/qemufirmwaretest.c | 7 +++++++ tests/qemuxml2argvtest.c | 6 ++++++ 2 files changed, 13 insertions(+) diff --git a/tests/qemufirmwaretest.c b/tests/qemufirmwaretest.c index 2b5cbf649b..3332bf0a34 100644 --- a/tests/qemufirmwaretest.c +++ b/tests/qemufirmwaretest.c @@ -52,6 +52,8 @@ testParseFormatFW(const void *opaque) } +#if defined(__x86_64__) || defined(__amd64__) || defined(__aarch64__) +/* XXX Dirty hack, but mocking stat on 32bits is above my skills */ static int testFWPrecedence(const void *opaque ATTRIBUTE_UNUSED) { @@ -97,6 +99,7 @@ testFWPrecedence(const void *opaque ATTRIBUTE_UNUSED) return 0; } +#endif static int @@ -124,8 +127,12 @@ mymain(void) DO_PARSE_TEST("usr/share/qemu/firmware/61-ovmf.json"); DO_PARSE_TEST("usr/share/qemu/firmware/70-aavmf.json"); + +#if defined(__x86_64__) || defined(__amd64__) || defined(__aarch64__) + /* XXX Dirty hack, but mocking stat on 32bits is above my skills */ if (virTestRun("QEMU FW precedence test", testFWPrecedence, NULL) < 0) ret = -1; +#endif virFileWrapperClearPrefixes(); diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 0c0dcae197..364792d24d 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -2038,6 +2038,8 @@ mymain(void) DO_TEST("pseries-many-buses-2", QEMU_CAPS_DEVICE_SPAPR_PCI_HOST_BRIDGE, QEMU_CAPS_VIRTIO_SCSI); +# if defined(__x86_64__) || defined(__amd64__) || defined(__aarch64__) + /* XXX Dirty hack, but mocking stat on 32bits is above my skills */ DO_TEST("pseries-hostdevs-1", QEMU_CAPS_DEVICE_SPAPR_PCI_HOST_BRIDGE, QEMU_CAPS_VIRTIO_SCSI, @@ -2050,6 +2052,7 @@ mymain(void) QEMU_CAPS_DEVICE_SPAPR_PCI_HOST_BRIDGE, QEMU_CAPS_VIRTIO_SCSI, QEMU_CAPS_DEVICE_VFIO_PCI); +# endif DO_TEST("pseries-features", QEMU_CAPS_DEVICE_SPAPR_PCI_HOST_BRIDGE, @@ -3157,10 +3160,13 @@ mymain(void) DO_TEST_CAPS_ARCH_LATEST("x86_64-pc-graphics", "x86_64"); DO_TEST_CAPS_ARCH_LATEST("x86_64-q35-graphics", "x86_64"); +# if defined(__x86_64__) || defined(__amd64__) || defined(__aarch64__) + /* XXX Dirty hack, but mocking stat on 32bits is above my skills */ DO_TEST_CAPS_LATEST("os-firmware-bios"); DO_TEST_CAPS_LATEST("os-firmware-efi"); DO_TEST_CAPS_LATEST("os-firmware-efi-secboot"); DO_TEST_CAPS_ARCH_LATEST("aarch64-os-firmware-efi", "aarch64"); +# endif if (getenv("LIBVIRT_SKIP_CLEANUP") == NULL) virFileDeleteTree(fakerootdir); -- 2.19.2

3 2

[libvirt] Planning for next release
by Daniel Veillard 28 Mar '19

28 Mar '19

It's time to get a new release out. Simplest would be to enter freeze tomorrow, then we can plan an RC2 by Friday, and if everything looks good have the release on time next Monday. Hope this works for everybody, thanks, Daniel -- Daniel Veillard | Red Hat Developers Tools http://developer.redhat.com/ veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

3 3

[libvirt] [PATCH 00/16] snapshot refactoring (incremental backup saga)
by Eric Blake 28 Mar '19

28 Mar '19

A couple of these have seen the list before, but most of them are new. The bulk of this series is about refactoring snapshot_conf.c into smaller pieces that I can then reuse for implementing checkpoints, without having to open-code the hierarchy algorithms a second time (one of John's complaints against my v4 incremental backup series). I suspect that src/vz/ builds may break on one or more of these patches; I could not get a working vz build environment. Eric Blake (16): test: Avoid use-after-free on virDomainSnapshotDelete snapshot: Use accessors for virDomainSnapshot members snapshot: Create virDomainMoment base class vbox: Clean up some snapshot usage snapshot: Drop virDomainSnapshotDef.current snapshot: Track current snapshot in virDomainSnapshotObjList snapshot: Add accessors for updating snapshot list relations snapshot: Access snapshot def directly when needed snapshot: Refactor list filtering snapshot: Factor out virDomainMomentDef class snapshot: Switch type of virDomainSnapshotObj.def snapshot: Rename virDomainSnapshotObjPtr snapshot: Rename file for virDomainMomentObj snapshot: Move snapshot list code into generic file snapshot: Tweaks to support new bulk dumpxml/import API backup: Introduce virDomainCheckpointPtr include/libvirt/virterror.h | 6 +- src/util/virerror.c | 12 +- include/libvirt/libvirt.h | 6 +- src/conf/domain_conf.h | 1 - src/conf/moment_conf.h | 41 +++ src/conf/snapshot_conf.h | 23 +- src/conf/virconftypes.h | 12 +- src/conf/virdomainmomentobjlist.h | 95 +++++ src/conf/virdomainsnapshotobj.h | 50 --- src/conf/virdomainsnapshotobjlist.h | 33 +- src/datatypes.h | 83 ++++- src/qemu/qemu_command.h | 2 +- src/qemu/qemu_domain.h | 6 +- src/qemu/qemu_process.h | 4 +- src/conf/Makefile.inc.am | 6 +- src/conf/moment_conf.c | 40 +++ src/conf/snapshot_conf.c | 179 +++++----- src/conf/virdomainmomentobjlist.c | 521 ++++++++++++++++++++++++++++ src/conf/virdomainsnapshotobj.c | 123 ------- src/conf/virdomainsnapshotobjlist.c | 367 +++++++------------- src/datatypes.c | 132 ++++--- src/esx/esx_driver.c | 82 ++--- src/libvirt-domain-snapshot.c | 26 +- src/libvirt_private.syms | 21 +- src/qemu/qemu_command.c | 2 +- src/qemu/qemu_domain.c | 36 +- src/qemu/qemu_driver.c | 290 ++++++++-------- src/qemu/qemu_process.c | 4 +- src/remote/remote_daemon_dispatch.c | 4 +- src/remote/remote_driver.c | 4 +- src/rpc/gendispatch.pl | 2 +- src/test/test_driver.c | 174 ++++------ src/vbox/vbox_common.c | 207 +++++------ src/vz/vz_driver.c | 89 ++--- src/vz/vz_sdk.c | 4 +- tests/domainsnapshotxml2xmltest.c | 5 +- 36 files changed, 1613 insertions(+), 1079 deletions(-) create mode 100644 src/conf/moment_conf.h create mode 100644 src/conf/virdomainmomentobjlist.h delete mode 100644 src/conf/virdomainsnapshotobj.h create mode 100644 src/conf/moment_conf.c create mode 100644 src/conf/virdomainmomentobjlist.c delete mode 100644 src/conf/virdomainsnapshotobj.c -- 2.20.1

7 70

[libvirt] [PATCH] tests: Avoid writing into $HOME during virsh-snapshot
by Eric Blake 28 Mar '19

28 Mar '19

In a constrained CI environment, where it is intentional that attempts to write outside the current directory will fail, virsh-snapshot was failing: @@ -1,2 +1,3 @@ error: invalid argument: parent s3 for snapshot s2 not found error: marker +error: Failed to create '/home/travis/.cache/libvirt/virsh': Permission denied FAIL virsh-snapshot (exit status: 1) But we've already solved the problem in virsh-uriprecedence: tell virsh to use XDG locations pointing to somewhere we can write rather than its default of falling back to $HOME with the test being at risk of breaking due to the user's environment and/or unacceptably altering the user's normal cache. Hoist that solution into test-lib.sh, so that all scripts can use it as needed. Fixes: 280a2b41e Reported-by: Daniel P. Berrangé <berrange(a)redhat.com> Signed-off-by: Eric Blake <eblake(a)redhat.com> --- tests/test-lib.sh | 13 +++++++++++++ tests/virsh-snapshot | 2 ++ tests/virsh-uriprecedence | 12 +----------- 3 files changed, 16 insertions(+), 11 deletions(-) diff --git a/tests/test-lib.sh b/tests/test-lib.sh index 49e8d22095..64f0b0d401 100644 --- a/tests/test-lib.sh +++ b/tests/test-lib.sh @@ -222,6 +222,19 @@ mkfifo_or_skip_() fi } +# Create mock XDG files/directories to avoid permission problems. +# As it points inside $test_dir_, it is automatically cleaned. +mock_xdg_() +{ + export XDG_CONFIG_HOME="$test_dir_/.config" + export XDG_CACHE_HOME="$test_dir_/.cache" + export XDG_RUNTIME_HOME="XDG_CACHE_HOME" + + mkdir -p "$XDG_CONFIG_HOME/libvirt" "$XDG_CONFIG_HOME/virsh" + mkdir -p "$XDG_CACHE_HOME/libvirt" "$XDG_CACHE_HOME/virsh" + mkdir -p "$XDG_RUNTIME_HOME/libvirt" "$XDG_RUNTIME_HOME/virsh" +} + test_dir_=$(pwd) this_test_() { echo "./$0" | sed 's,.*/,,'; } diff --git a/tests/virsh-snapshot b/tests/virsh-snapshot index fb8a99dd43..cb498cf54e 100755 --- a/tests/virsh-snapshot +++ b/tests/virsh-snapshot @@ -26,6 +26,8 @@ fi fail=0 +mock_xdg_ || framework_failure + # The test driver loses states between restarts, so we perform a script # with some convenient markers for later post-processing of output. $abs_top_builddir/tools/virsh --connect test:///default >out 2>err ' diff --git a/tests/virsh-uriprecedence b/tests/virsh-uriprecedence index 564e3dc42c..fd6ce108c0 100755 --- a/tests/virsh-uriprecedence +++ b/tests/virsh-uriprecedence @@ -11,17 +11,7 @@ virsh_cmd="$virsh_bin" counter=0 ret=0 -cleanup_() { rm -rf "$tmphome"; } - -# Create all mock files/directories to avoid permission problems -tmphome="$PWD/tmp_home" -export XDG_CONFIG_HOME="$tmphome/.config" -export XDG_CACHE_HOME="$tmphome/.cache" -export XDG_RUNTIME_HOME="XDG_CACHE_HOME" - -mkdir -p "$XDG_CONFIG_HOME/libvirt" "$XDG_CONFIG_HOME/virsh" -mkdir -p "$XDG_CACHE_HOME/libvirt" "$XDG_CACHE_HOME/virsh" -mkdir -p "$XDG_RUNTIME_HOME/libvirt" "$XDG_RUNTIME_HOME/virsh" +mock_xdg_ || framework_failure is_uri_good() { -- 2.20.1

3 4

[libvirt] [PATCH 0/3] qemu: clean up USB address releasing
by Peter Krempa 28 Mar '19

28 Mar '19

Peter Krempa (3): qemu: hotplug: Don't release USB address twice when removing disk qemu: Always use 'alias' in warning message when removing USB address qemu: address: Stop reporting warning when USB address can't be released src/qemu/qemu_domain_address.c | 9 ++---- src/qemu/qemu_domain_address.h | 3 +- src/qemu/qemu_hotplug.c | 53 +++++++++++++++++----------------- 3 files changed, 29 insertions(+), 36 deletions(-) -- 2.20.1

2 4