- Devel - libvirt List Archives

[libvirt] [PATCH] qemu: blockjob: Fix saving of inactive XML after completed legacy blockjob
by Peter Krempa 17 May '19

17 May '19

Commit 0ba9afc6b28 introduced a logic bug where we will never save the inactive XML after a blockjob as the variable which was determining whether to do so is cleared right before. Thus even if we correctly modify the inactive state it will be rolled back when libvirtd is restarted. Reported-by: Thomas Stein <hello(a)himbee.re> Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- src/qemu/qemu_blockjob.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/qemu/qemu_blockjob.c b/src/qemu/qemu_blockjob.c index fa7e4c8625..f105632a09 100644 --- a/src/qemu/qemu_blockjob.c +++ b/src/qemu/qemu_blockjob.c @@ -363,7 +363,7 @@ qemuBlockJobEventProcessLegacy(virQEMUDriverPtr driver, if (virDomainSaveStatus(driver->xmlopt, cfg->stateDir, vm, driver->caps) < 0) VIR_WARN("Unable to save status on vm %s after block job", vm->def->name); - if (job->newstate == VIR_DOMAIN_BLOCK_JOB_COMPLETED && vm->newDef) { + if (job->state == VIR_DOMAIN_BLOCK_JOB_COMPLETED && vm->newDef) { if (virDomainSaveConfig(cfg->configDir, driver->caps, vm->newDef) < 0) VIR_WARN("Unable to update persistent definition on vm %s " "after block job", vm->def->name); -- 2.21.0

3 2

[libvirt] [PATCH v2] src: don't statically link code that's already in libvirt.so
by Daniel P. Berrangé 17 May '19

17 May '19

Various binaries are statically linking to libvirt_util.la and other intermediate libraries we build. These intermediate libs all get built into the main libvirt.so shared library eventually, so we can dynamically link to that instead and reduce the on disk footprint. In libvirt-daemon RPM: virtlockd: 1.6 MB -> 153 KB virtlogd: 1.6 MB -> 157 KB libvirt_iohelper: 937 KB -> 23 KB In libvirt-daemon-driver-network RPM: libvirt_leaseshelper: 940 KB -> 26 KB In libvirt-daemon-driver-storage-core RPM: libvirt_parthelper: 926 KB -> 21 KB IOW, about 5.6 MB total space saving Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- src/Makefile.am | 2 +- src/libvirt_remote.syms | 1 + src/locking/Makefile.inc.am | 4 +--- src/logging/Makefile.inc.am | 4 +--- src/network/Makefile.inc.am | 2 +- src/security/Makefile.inc.am | 2 -- src/storage/Makefile.inc.am | 2 +- 7 files changed, 6 insertions(+), 11 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index 7d452a9490..0b562dc250 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -851,7 +851,7 @@ libvirt_iohelper_LDFLAGS = \ $(PIE_LDFLAGS) \ $(NULL) libvirt_iohelper_LDADD = \ - libvirt_util.la \ + libvirt.la \ ../gnulib/lib/libgnu.la if WITH_DTRACE_PROBES libvirt_iohelper_LDADD += libvirt_probes.lo diff --git a/src/libvirt_remote.syms b/src/libvirt_remote.syms index 98586d1584..3d68f6ce0a 100644 --- a/src/libvirt_remote.syms +++ b/src/libvirt_remote.syms @@ -176,6 +176,7 @@ virNetServerClientSetAuthLocked; virNetServerClientSetAuthPendingLocked; virNetServerClientSetCloseHook; virNetServerClientSetDispatcher; +virNetServerClientSetQuietEOF; virNetServerClientSetReadonly; virNetServerClientStartKeepAlive; virNetServerClientWantCloseLocked; diff --git a/src/locking/Makefile.inc.am b/src/locking/Makefile.inc.am index da26fab91f..46ab11c2a9 100644 --- a/src/locking/Makefile.inc.am +++ b/src/locking/Makefile.inc.am @@ -145,10 +145,8 @@ virtlockd_LDFLAGS = \ $(NO_UNDEFINED_LDFLAGS) \ $(NULL) virtlockd_LDADD = \ + libvirt.la \ libvirt_driver_admin.la \ - libvirt-net-rpc-server.la \ - libvirt-net-rpc.la \ - libvirt_util.la \ ../gnulib/lib/libgnu.la \ $(CYGWIN_EXTRA_LIBADD) \ $(NULL) diff --git a/src/logging/Makefile.inc.am b/src/logging/Makefile.inc.am index f5eba2a4f7..d57394cbde 100644 --- a/src/logging/Makefile.inc.am +++ b/src/logging/Makefile.inc.am @@ -80,9 +80,7 @@ virtlogd_LDFLAGS = \ $(NULL) virtlogd_LDADD = \ libvirt_driver_admin.la \ - libvirt-net-rpc-server.la \ - libvirt-net-rpc.la \ - libvirt_util.la \ + libvirt.la \ ../gnulib/lib/libgnu.la \ $(CYGWIN_EXTRA_LIBADD) \ $(NULL) diff --git a/src/network/Makefile.inc.am b/src/network/Makefile.inc.am index 3fed59c13b..52270049d5 100644 --- a/src/network/Makefile.inc.am +++ b/src/network/Makefile.inc.am @@ -56,7 +56,7 @@ libvirt_leaseshelper_LDFLAGS = \ $(PIE_LDFLAGS) \ $(NULL) libvirt_leaseshelper_LDADD = \ - libvirt_util.la \ + libvirt.la \ ../gnulib/lib/libgnu.la if WITH_DTRACE_PROBES libvirt_leaseshelper_LDADD += libvirt_probes.lo diff --git a/src/security/Makefile.inc.am b/src/security/Makefile.inc.am index 7fb6d8cc4f..64e0f46857 100644 --- a/src/security/Makefile.inc.am +++ b/src/security/Makefile.inc.am @@ -72,8 +72,6 @@ virt_aa_helper_LDFLAGS = \ $(NULL) virt_aa_helper_LDADD = \ libvirt.la \ - libvirt_conf.la \ - libvirt_util.la \ libvirt_driver_storage_impl.la \ ../gnulib/lib/libgnu.la \ $(NULL) diff --git a/src/storage/Makefile.inc.am b/src/storage/Makefile.inc.am index ea4681f70e..538709256d 100644 --- a/src/storage/Makefile.inc.am +++ b/src/storage/Makefile.inc.am @@ -378,7 +378,7 @@ libvirt_parthelper_LDFLAGS = \ $(NULL) libvirt_parthelper_LDADD = \ $(LIBPARTED_LIBS) \ - libvirt_util.la \ + libvirt.la \ ../gnulib/lib/libgnu.la \ $(NULL) if WITH_DTRACE_PROBES -- 2.21.0

2 1

[libvirt] surprising <backingStore type='file'> setting in domain.xml
by Thomas Stein 17 May '19

17 May '19

Hello all. My currently used versions: libvirt-5.2.0 and qemu-4.0.0. Here is my problem. I'm struggeling since a few weeks with a strange behaviour by either qemu or libvirt. After a reboot of the hardware node the $domain.xml contains suddenly a backingStore setting which was not there before reboot. Something like that: <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/var/lib/libvirt/shinymail/shinymail_weekly.qcow2-2019-05-15'/> <backingStore type='file'> <format type='qcow2'/> <source file='/var/lib/libvirt/images/shinymail.qcow2'/> </backingStore> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </disk> ... This obviousely happens after a backup has been running. The Backup Script looks like this: <snip> virsh snapshot-create-as --domain shinymail weekly --diskspec vda,file=/var/lib/libvirt/shinymail/shinymail_weekly.qcow2-$(date +%Y-%m-%d) --disk-only --atomic --no-metadata cp ... virsh blockcommit shinymail vda --active --verbose --pivot <snip> So after that "dmblklist shinymail" does show the right source file but after a reboot it tries to use the weekly snapshot again which leads to filesystem errors. Someone has an idea what could cause such a behaviour? cheers t.

3 4

[libvirt] [PATCH 0/2] build: bump minimum udev version to 219
by Ján Tomko 16 May '19

16 May '19

Inspired-by: Pavel Hrdina <phardina(a)redhat.com> Ján Tomko (2): build: bump minimum udev version to 219 build: drop check for udev_monitor_set_receive_buffer_size m4/virt-udev.m4 | 14 ++------------ src/node_device/node_device_udev.c | 2 -- 2 files changed, 2 insertions(+), 14 deletions(-) -- 2.19.2

2 3

[libvirt] [PATCH] src: don't statically link code that's already in libvirt.so
by Daniel P. Berrangé 16 May '19

16 May '19

virtlockd and virtlogd statically link to libvirt-net-rpc-server.la libvirt-net-rpc.la and libvirt_util.la. libvirt-iohelper statically links to libvirt_util.la. All this code is all already built into the main libvirt.so, so we should dynamically link all these pieces. This reduces the size of the libvirt-daemon RPM from 5.4 MB down to 1.2 MB which is useful for container installs seeking minimal disk footprint. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- src/Makefile.am | 2 +- src/libvirt_remote.syms | 1 + src/locking/Makefile.inc.am | 4 +--- src/logging/Makefile.inc.am | 4 +--- 4 files changed, 4 insertions(+), 7 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index 7d452a9490..0b562dc250 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -851,7 +851,7 @@ libvirt_iohelper_LDFLAGS = \ $(PIE_LDFLAGS) \ $(NULL) libvirt_iohelper_LDADD = \ - libvirt_util.la \ + libvirt.la \ ../gnulib/lib/libgnu.la if WITH_DTRACE_PROBES libvirt_iohelper_LDADD += libvirt_probes.lo diff --git a/src/libvirt_remote.syms b/src/libvirt_remote.syms index 98586d1584..3d68f6ce0a 100644 --- a/src/libvirt_remote.syms +++ b/src/libvirt_remote.syms @@ -176,6 +176,7 @@ virNetServerClientSetAuthLocked; virNetServerClientSetAuthPendingLocked; virNetServerClientSetCloseHook; virNetServerClientSetDispatcher; +virNetServerClientSetQuietEOF; virNetServerClientSetReadonly; virNetServerClientStartKeepAlive; virNetServerClientWantCloseLocked; diff --git a/src/locking/Makefile.inc.am b/src/locking/Makefile.inc.am index da26fab91f..46ab11c2a9 100644 --- a/src/locking/Makefile.inc.am +++ b/src/locking/Makefile.inc.am @@ -145,10 +145,8 @@ virtlockd_LDFLAGS = \ $(NO_UNDEFINED_LDFLAGS) \ $(NULL) virtlockd_LDADD = \ + libvirt.la \ libvirt_driver_admin.la \ - libvirt-net-rpc-server.la \ - libvirt-net-rpc.la \ - libvirt_util.la \ ../gnulib/lib/libgnu.la \ $(CYGWIN_EXTRA_LIBADD) \ $(NULL) diff --git a/src/logging/Makefile.inc.am b/src/logging/Makefile.inc.am index f5eba2a4f7..d57394cbde 100644 --- a/src/logging/Makefile.inc.am +++ b/src/logging/Makefile.inc.am @@ -80,9 +80,7 @@ virtlogd_LDFLAGS = \ $(NULL) virtlogd_LDADD = \ libvirt_driver_admin.la \ - libvirt-net-rpc-server.la \ - libvirt-net-rpc.la \ - libvirt_util.la \ + libvirt.la \ ../gnulib/lib/libgnu.la \ $(CYGWIN_EXTRA_LIBADD) \ $(NULL) -- 2.21.0

2 2

[libvirt] [PATCH] virt-aa-helper: clean old reference to libvirt-sandbox
by Christian Ehrhardt 16 May '19

16 May '19

It came up that libvirt-sandbox is a failed (and removed) experiment. Let us clean up and remove it from the virt-aa-helper source as well. Signed-off-by: Christian Ehrhardt <christian.ehrhardt(a)canonical.com> --- src/security/virt-aa-helper.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/security/virt-aa-helper.c b/src/security/virt-aa-helper.c index ad9a7dda94..fea4da80dc 100644 --- a/src/security/virt-aa-helper.c +++ b/src/security/virt-aa-helper.c @@ -511,8 +511,7 @@ valid_path(const char *path, const bool readonly) /* override the above with these */ const char * const override[] = { "/sys/devices/pci", /* for hostdev pci devices */ - "/sys/kernel/config/target/vhost", /* for hostdev vhost_scsi devices */ - "/etc/libvirt-sandbox/services/" /* for virt-sandbox service config */ + "/sys/kernel/config/target/vhost" /* for hostdev vhost_scsi devices */ }; const int nropaths = ARRAY_CARDINALITY(restricted); -- 2.21.0

2 2

[libvirt] [PATCH v2] test_driver: implement virDomainGetDiskErrors
by Ilias Stamatis 16 May '19

16 May '19

Return the number of disks present in the configuration of the test domain when called with @errors as NULL and @maxerrors as 0. Otherwise report an error for every second disk, assigning available error codes in a cyclic order. Signed-off-by: Ilias Stamatis <stamatis.iliass(a)gmail.com> --- src/test/test_driver.c | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/src/test/test_driver.c b/src/test/test_driver.c index a06d1fc402..527c2f5d3b 100644 --- a/src/test/test_driver.c +++ b/src/test/test_driver.c @@ -3046,6 +3046,47 @@ static int testDomainSetAutostart(virDomainPtr domain, return 0; } +static int testDomainGetDiskErrors(virDomainPtr dom, + virDomainDiskErrorPtr errors, + unsigned int maxerrors, + unsigned int flags) +{ + virDomainObjPtr vm = NULL; + int ret = -1; + size_t i; + int n = 0; + int codes[] = {VIR_DOMAIN_DISK_ERROR_UNSPEC, VIR_DOMAIN_DISK_ERROR_NO_SPACE}; + size_t ncodes = sizeof(codes) / sizeof(codes[0]); + + virCheckFlags(0, -1); + + if (!(vm = testDomObjFromDomain(dom))) + goto cleanup; + + if (virDomainObjCheckActive(vm) < 0) + goto cleanup; + + if (!errors) { + ret = vm->def->ndisks; + } else { + for (i = 1; i < vm->def->ndisks && n < maxerrors; i += 2) { + if (VIR_STRDUP(errors[n].disk, vm->def->disks[i]->dst) < 0) + goto cleanup; + errors[n].error = codes[n % ncodes]; + n++; + } + ret = n; + } + + cleanup: + virDomainObjEndAPI(&vm); + if (ret < 0) { + for (i = 0; i < n; i++) + VIR_FREE(errors[i].disk); + } + return ret; +} + static char *testDomainGetSchedulerType(virDomainPtr domain ATTRIBUTE_UNUSED, int *nparams) { @@ -6832,6 +6873,7 @@ static virHypervisorDriver testHypervisorDriver = { .domainUndefineFlags = testDomainUndefineFlags, /* 0.9.4 */ .domainGetAutostart = testDomainGetAutostart, /* 0.3.2 */ .domainSetAutostart = testDomainSetAutostart, /* 0.3.2 */ + .domainGetDiskErrors = testDomainGetDiskErrors, /* 5.4.0 */ .domainGetSchedulerType = testDomainGetSchedulerType, /* 0.3.2 */ .domainGetSchedulerParameters = testDomainGetSchedulerParameters, /* 0.3.2 */ .domainGetSchedulerParametersFlags = testDomainGetSchedulerParametersFlags, /* 0.9.2 */ -- 2.21.0

4 11

[libvirt] [PATCH] virt-aa-helper: allow sysfs path used for vhost-scsi
by Christian Ehrhardt 16 May '19

16 May '19

When a vhost scsi device is hotplugged virt-aa-helper is called to add the respective path. For example the config: <hostdev mode='subsystem' type='scsi_host' managed='no'> <source protocol='vhost' wwpn='naa.50014059de6fba4f'/> </hostdev> Will call it to add: /sys/kernel/config/target/vhost//naa.50014059de6fba4f But in general /sys paths are filtered in virt-aa-helper.c:valid_path To allow the path used for vhost-scsi we need to add it to the list of known and accepted overrides. Fixes: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1829223 Signed-off-by: Christian Ehrhardt <christian.ehrhardt(a)canonical.com> --- src/security/virt-aa-helper.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/security/virt-aa-helper.c b/src/security/virt-aa-helper.c index d0fe86cefc..ad9a7dda94 100644 --- a/src/security/virt-aa-helper.c +++ b/src/security/virt-aa-helper.c @@ -510,8 +510,9 @@ valid_path(const char *path, const bool readonly) }; /* override the above with these */ const char * const override[] = { - "/sys/devices/pci", /* for hostdev pci devices */ - "/etc/libvirt-sandbox/services/" /* for virt-sandbox service config */ + "/sys/devices/pci", /* for hostdev pci devices */ + "/sys/kernel/config/target/vhost", /* for hostdev vhost_scsi devices */ + "/etc/libvirt-sandbox/services/" /* for virt-sandbox service config */ }; const int nropaths = ARRAY_CARDINALITY(restricted); -- 2.21.0

2 2

[libvirt] [PATCH] qemu: Do not override config XML in case of snapshot revert
by Maxiwell S. Garcia 15 May '19

15 May '19

Snapshot create operation saves the live XML and uses it to replace the domain definition in case of revert. But the VM config XML is not saved and the revert operation does not address this issue. This commit prevents the config XML from being overridden by snapshot definition. An active domain stores both current and new definitions. The current definition (vm->def) stores the live XML and the new definition (vm->newDef) stores the config XML. In an inactive domain, only the config XML is persistent, and it's saved in vm->def. The revert operation uses the virDomainObjAssignDef() to set the snapshot definition in vm->newDef, if domain is active, or in vm->def otherwise. But before that, it saves the old value to return to caller. This return is used here to restore the config XML after all snapshot startup process finish. Signed-off-by: Maxiwell S. Garcia <maxiwell(a)linux.ibm.com> --- src/qemu/qemu_driver.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index b2ac737d1f..a73122454a 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -16251,6 +16251,7 @@ qemuDomainRevertToSnapshot(virDomainSnapshotPtr snapshot, qemuDomainObjPrivatePtr priv; int rc; virDomainDefPtr config = NULL; + virDomainDefPtr inactiveConfig = NULL; virQEMUDriverConfigPtr cfg = NULL; virCapsPtr caps = NULL; bool was_stopped = false; @@ -16465,7 +16466,7 @@ qemuDomainRevertToSnapshot(virDomainSnapshotPtr snapshot, goto endjob; } if (config) { - virDomainObjAssignDef(vm, config, false, NULL); + virDomainObjAssignDef(vm, config, false, &inactiveConfig); virCPUDefFree(priv->origCPU); VIR_STEAL_PTR(priv->origCPU, origCPU); } @@ -16474,7 +16475,7 @@ qemuDomainRevertToSnapshot(virDomainSnapshotPtr snapshot, load: was_stopped = true; if (config) - virDomainObjAssignDef(vm, config, false, NULL); + virDomainObjAssignDef(vm, config, false, &inactiveConfig); /* No cookie means libvirt which saved the domain was too old to * mess up the CPU definitions. @@ -16533,6 +16534,9 @@ qemuDomainRevertToSnapshot(virDomainSnapshotPtr snapshot, detail); } } + if (inactiveConfig) + VIR_STEAL_PTR(vm->newDef, inactiveConfig); + break; case VIR_DOMAIN_SNAPSHOT_SHUTDOWN: @@ -16560,8 +16564,11 @@ qemuDomainRevertToSnapshot(virDomainSnapshotPtr snapshot, qemuProcessEndJob(driver, vm); goto cleanup; } - if (config) - virDomainObjAssignDef(vm, config, false, NULL); + if (config) { + virDomainObjAssignDef(vm, config, false, &inactiveConfig); + if (inactiveConfig) + VIR_STEAL_PTR(vm->newDef, inactiveConfig); + } if (flags & (VIR_DOMAIN_SNAPSHOT_REVERT_RUNNING | VIR_DOMAIN_SNAPSHOT_REVERT_PAUSED)) { -- 2.20.1

4 9

[libvirt] [PATCH 0/2] introduction of version attribute for VFIO live migration
by Yan Zhao 15 May '19

15 May '19

This patchset introduces a version attribute under sysfs of VFIO Mediated devices. This version attribute is used by user space software like libvirt to determine whether two mdev devices are compatible for live migration before starting live migration. Patch 1 defines version attribute as mandatory for VFIO live migration. It means if version attribute is missing or it returns errno, the corresponding mdev device is regarded as not supporting live migration. samples for vfio-mdev are modified to demonstrate it. Patch 2 uses GVT as an example to show how to expose version attribute and check device compatibility in vendor driver. Yan Zhao (2): vfio/mdev: add version field as mandatory attribute for mdev device drm/i915/gvt: export mdev device version to sysfs for Intel vGPU Documentation/vfio-mediated-device.txt | 36 +++++++++ drivers/gpu/drm/i915/gvt/Makefile | 2 +- drivers/gpu/drm/i915/gvt/device_version.c | 94 +++++++++++++++++++++++ drivers/gpu/drm/i915/gvt/gvt.c | 55 +++++++++++++ drivers/gpu/drm/i915/gvt/gvt.h | 6 ++ samples/vfio-mdev/mbochs.c | 17 ++++ samples/vfio-mdev/mdpy.c | 16 ++++ samples/vfio-mdev/mtty.c | 16 ++++ 8 files changed, 241 insertions(+), 1 deletion(-) create mode 100644 drivers/gpu/drm/i915/gvt/device_version.c -- 2.17.1

8 33

[libvirt] [PATCH 0/2] Avoid issues due to qemu dropping osxsave and ospke
by Christian Ehrhardt 15 May '19

15 May '19

Hi, this series tries to address a drop of commandline options by qemu in regard to osxsave [1] and ospke [2]. This was already discussed in [3] late last year but got forgotten afterwards. The Ubuntu bug is at [4] and an older Fedora bug is at [5]. TL;DR: - osxsave/ospke features were never really configurable - KVM never returned the bits on GET_SUPPORTED_CPUID - very rare to be seen in the wild - avoid issues with newer qemu and old/odd XMLs to be sure Details: I checked various use cases from virt-install to openstack and some in between. The only cases I found that would define osxsave/ospke is virt-install pior to version 2.0 and even there only when used with --cpu=host-model or --cpu=host-copy. If you ever really enabled the feature you'd have got: error: the CPU is incompatible with host CPU: Host CPU does not provide required features: ospke The problem lies in domain XMLs that explicitly disable it. That would be <feature policy='disable' name='osxsave'/> But due to almost (or actually none) no host exposing this the following also triggers: <feature policy='optional' name='ospke'/> This will make libvirt add it to the qemu commandline like: -cpu ...,osxsave=off,ospke=off And that will crash when qemu starts with: error: internal error: process exited while connecting to monitor: 2019-04-25T12:12:01.698646Z qemu-system-x86_64: can't apply global core2duo-x86_64-cpu.osxsave=off: Property '.osxsave' not found There are much more long term discussions about demoting and dropping qemu features and I'd like to avoid those discussions being mixed. The reason to drop it more or less without notice was that it never did anything to begin with. Due to that our solution might in a similar fashion be more trivial - just stop defining those two features to qemu commandline. [1]: https://git.qemu.org/?p=qemu.git;a=commit;h=f1a23522b03a569f13aad49294bb4c4… [2]: https://git.qemu.org/?p=qemu.git;a=commit;h=9ccb9784b57804f5c74434ad6ccb666… [3]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg561877.html [4]: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1825195 [5]: https://bugzilla.redhat.com/show_bug.cgi?id=1644848 Christian Ehrhardt (2): qemu: do not define known no-op features qemuxml2argvtest: add test for remove cpu features src/qemu/qemu_command.c | 23 +++++++++++++++ .../qemuxml2argvdata/cpu-host-model-cmt.args | 2 +- .../cpu-no-removed-features.args | 29 +++++++++++++++++++ .../cpu-no-removed-features.xml | 23 +++++++++++++++ tests/qemuxml2argvdata/cpu-tsc-frequency.args | 4 +-- tests/qemuxml2argvtest.c | 1 + 6 files changed, 79 insertions(+), 3 deletions(-) create mode 100644 tests/qemuxml2argvdata/cpu-no-removed-features.args create mode 100644 tests/qemuxml2argvdata/cpu-no-removed-features.xml -- 2.17.1

2 11

[libvirt] [PATCH 0/2] Mitigation for Microarchitectural Data Sampling CPU flaws
by Jiri Denemark 14 May '19

14 May '19

This series introduces the libvirt side of mitigations for Microarchitectural Data Sampling microprocessor flaws (CVE-2018-12126, CVE-2018-12127, CVE-2018-12130, CVE-2019-11091) which were published earlier today. To protect your system against possible attacks exploiting these flaws updates to the CPU microcode, Linux kernel, and virtualization stack (QEMU, libvirt, and higher management apps) are required. See https://access.redhat.com/security/vulnerabilities/mds for more details and additional links. Both patches have already been pushed. Jiri Denemark (2): cputest: Add data for Intel(R) Xeon(R) CPU E3-1225 v5 cpu_map: Define md-clear CPUID bit src/cpu_map/x86_features.xml | 3 + tests/cputest.c | 1 + .../x86_64-cpuid-Xeon-E3-1225-v5-disabled.xml | 7 + .../x86_64-cpuid-Xeon-E3-1225-v5-enabled.xml | 8 + .../x86_64-cpuid-Xeon-E3-1225-v5-guest.xml | 29 + .../x86_64-cpuid-Xeon-E3-1225-v5-host.xml | 30 + .../x86_64-cpuid-Xeon-E3-1225-v5-json.xml | 12 + .../x86_64-cpuid-Xeon-E3-1225-v5.json | 652 ++++++++++++++++++ .../x86_64-cpuid-Xeon-E3-1225-v5.sig | 4 + .../x86_64-cpuid-Xeon-E3-1225-v5.xml | 47 ++ .../x86_64-cpuid-Xeon-Platinum-8268-guest.xml | 1 + .../x86_64-cpuid-Xeon-Platinum-8268-host.xml | 1 + 12 files changed, 795 insertions(+) create mode 100644 tests/cputestdata/x86_64-cpuid-Xeon-E3-1225-v5-disabled.xml create mode 100644 tests/cputestdata/x86_64-cpuid-Xeon-E3-1225-v5-enabled.xml create mode 100644 tests/cputestdata/x86_64-cpuid-Xeon-E3-1225-v5-guest.xml create mode 100644 tests/cputestdata/x86_64-cpuid-Xeon-E3-1225-v5-host.xml create mode 100644 tests/cputestdata/x86_64-cpuid-Xeon-E3-1225-v5-json.xml create mode 100644 tests/cputestdata/x86_64-cpuid-Xeon-E3-1225-v5.json create mode 100644 tests/cputestdata/x86_64-cpuid-Xeon-E3-1225-v5.sig create mode 100644 tests/cputestdata/x86_64-cpuid-Xeon-E3-1225-v5.xml -- 2.21.0

2 3

[libvirt] [PATCH] qemuDomainSnapshotCreateXML: Don't leak parsed snapshot definition
by Michal Privoznik 14 May '19

14 May '19

This function gets snapshot XML (provided by used) as an argument. It parses it into a local variable @def and then sets some more members (e.g. it creates a copy of live domain XML). Then it proceeds to checking if snapshot XML is valid (e.g. it contains as many disks as currently in the domain). If this fails then the control jumps to endjob label and subsequently return from the function. This is where AUTOFREE function for @def is ran. Well, because the code says to run plain VIR_FREE() we leak some memory because @def is actually an object and therefore it should have been declared as AUTOUNREF. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/qemu/qemu_driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index f01282a037..0a425b82e5 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -15563,7 +15563,7 @@ qemuDomainSnapshotCreateXML(virDomainPtr domain, virCapsPtr caps = NULL; qemuDomainObjPrivatePtr priv; virDomainSnapshotState state; - VIR_AUTOFREE(virDomainSnapshotDefPtr) def = NULL; + VIR_AUTOUNREF(virDomainSnapshotDefPtr) def = NULL; virCheckFlags(VIR_DOMAIN_SNAPSHOT_CREATE_REDEFINE | VIR_DOMAIN_SNAPSHOT_CREATE_CURRENT | -- 2.21.0

3 2

[libvirt] [PATCH 0/4] Couple of allocation fixes
by Michal Privoznik 14 May '19

14 May '19

These stem from me playing with OOM testing. I've discovered more bugs, but they're mostly in tests themselves, e.g. use without check. Michal Prívozník (4): virstorageobj: Don't clear vols if they weren't initialized virNetServerPreExecRestart: Check for retval of virJSONValueNewArray() virCommand: Make virCommandPassFDGetFDIndex fail if passed command is in error state storagepoolxml2argvtest: Avoid double free src/conf/virstorageobj.c | 6 ++++-- src/rpc/virnetserver.c | 8 ++++++-- src/util/vircommand.c | 3 +++ tests/storagepoolxml2argvtest.c | 1 - 4 files changed, 13 insertions(+), 5 deletions(-) -- 2.21.0

3 9

[libvirt] [PATCH 0/2] node device cleanup
by Pavel Hrdina 13 May '19

13 May '19

Pavel Hrdina (2): node_device_udev: remove deprecated logging function src: remove HAL node device driver configure.ac | 3 +- docs/drvnodedev.html.in | 3 +- docs/hvsupport.pl | 2 +- libvirt.spec.in | 1 - m4/virt-hal.m4 | 10 +- m4/virt-udev.m4 | 5 - po/POTFILES | 1 - src/node_device/Makefile.inc.am | 12 - src/node_device/node_device_driver.c | 10 +- src/node_device/node_device_driver.h | 5 - src/node_device/node_device_hal.c | 804 --------------------------- src/node_device/node_device_hal.h | 22 - src/node_device/node_device_udev.c | 41 -- 13 files changed, 9 insertions(+), 910 deletions(-) delete mode 100644 src/node_device/node_device_hal.c delete mode 100644 src/node_device/node_device_hal.h -- 2.21.0

3 4

[libvirt] [PATCH] virsh: Don't leak disk targets in cmdDomBlkError
by Michal Privoznik 13 May '19

13 May '19

The virDomainGetDiskErrors() API copies disk targets into @disks array that we allocate. But we forgot to free it: ==140828== 16 bytes in 4 blocks are definitely lost in loss record 41 of 242 ==140828== at 0x4C2F08F: malloc (vg_replace_malloc.c:299) ==140828== by 0x8C406D9: strdup (in /lib64/libc-2.28.so) ==140828== by 0x5377DD3: virStrdup (virstring.c:966) ==140828== by 0x54C112F: testDomainGetDiskErrors (test_driver.c:3068) ==140828== by 0x55C863D: virDomainGetDiskErrors (libvirt-domain.c:10988) ==140828== by 0x15D1FA: cmdDomBlkError (virsh-domain-monitor.c:1215) ==140828== by 0x17F1A8: vshCommandRun (vsh.c:1335) ==140828== by 0x13489E: main (virsh.c:920) Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- Pushed under trivial rule. tools/virsh-domain-monitor.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/virsh-domain-monitor.c b/tools/virsh-domain-monitor.c index d87475f6f6..d2b7e5a059 100644 --- a/tools/virsh-domain-monitor.c +++ b/tools/virsh-domain-monitor.c @@ -1229,6 +1229,8 @@ cmdDomBlkError(vshControl *ctl, const vshCmd *cmd) ret = true; cleanup: + for (i = 0; i < count; i++) + VIR_FREE(disks[i].disk); VIR_FREE(disks); virshDomainFree(dom); return ret; -- 2.21.0

1 0

[libvirt] [PATCH 0/4] docs: Ship and install a few more files
by Andrea Bolognani 13 May '19

13 May '19

Our handling of documentation is far from perfect, and a few issues in particular are causing me quite a bit of grief on the Debian packaging side of things. This series takes care of the more pressing matters; I have further cleanups planned for later. Andrea Bolognani (4): docs: Introduce $(modules) docs: Introduce $(devhelphtml_generated) docs: Install documentation under $(docdir) examples: Install under $(docdir) configure.ac | 13 ------------- docs/Makefile.am | 38 ++++++++++++++++++++++++-------------- examples/Makefile.am | 11 +++++++++++ libvirt.spec.in | 14 +------------- 4 files changed, 36 insertions(+), 40 deletions(-) -- 2.20.1

3 7

[libvirt] [PATCH] util: fix a bug found in sorting cache monitor results
by Wang Huaqiang 13 May '19

13 May '19

From: Huaqiang <huaqiang.wang(a)intel.com> The qsort element is a pointer of virResctrlMonitorStats, and the comparing function's arguments have a type of pointer of virResctrlMonitorStatsPtr. Signed-off-by: Huaqiang <huaqiang.wang(a)intel.com> --- src/util/virresctrl.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c index b845f36..a190977 100644 --- a/src/util/virresctrl.c +++ b/src/util/virresctrl.c @@ -2659,8 +2659,8 @@ static int virResctrlMonitorStatsSorter(const void *a, const void *b) { - return ((virResctrlMonitorStatsPtr)a)->id - - ((virResctrlMonitorStatsPtr)b)->id; + return (*(virResctrlMonitorStatsPtr *)a)->id + - (*(virResctrlMonitorStatsPtr *)b)->id; } @@ -2758,7 +2758,7 @@ virResctrlMonitorGetStats(virResctrlMonitorPtr monitor, /* Sort in id's ascending order */ if (*nstats) - qsort(*stats, *nstats, sizeof(*stat), virResctrlMonitorStatsSorter); + qsort(*stats, *nstats, sizeof(stat), virResctrlMonitorStatsSorter); ret = 0; cleanup: -- 2.7.4

3 2

[libvirt] [PATCH RFC] network: Delay creating private chains until starting network
by Jim Fehlig 13 May '19

13 May '19

Automated performance tests found that network-centric workloads suffered a 20 percent decrease when the host libvirt was updated from 5.0.0 to 5.1.0. On the test hosts libvirtd is enabled to start at boot and the "default" network is defined, but it is not set to autostart. libvirt 5.1.0 introduced private firewall chains with commit 5f1e6a7d. The chains are created at libvirtd start, which triggers loading the conntrack kernel module. Subsequent tracking and processing of connections resulted in the performance hit. Prior to commit 5f1e6a7d the conntrack module would not be loaded until starting a network, when libvirt added rules to the builtin chains. Restore the behavior of previous libvirt versions by delaying the creation of private chains until the first network is started. Signed-off-by: Jim Fehlig <jfehlig(a)suse.com> --- I briefly discussed this issue with Daniel on IRC and just now finding time to bring it to the list for broader discussion. The adjustment to the test file feels hacky. The whole approach might by hacky, hence the RFC. src/network/bridge_driver_linux.c | 64 +++------- .../nat-default-linux.args | 116 ++++++++++++++++++ 2 files changed, 131 insertions(+), 49 deletions(-) diff --git a/src/network/bridge_driver_linux.c b/src/network/bridge_driver_linux.c index f2827543ca..a3a09caeba 100644 --- a/src/network/bridge_driver_linux.c +++ b/src/network/bridge_driver_linux.c @@ -35,44 +35,10 @@ VIR_LOG_INIT("network.bridge_driver_linux"); #define PROC_NET_ROUTE "/proc/net/route" -static virErrorPtr errInitV4; -static virErrorPtr errInitV6; +static bool pvtChainsCreated; void networkPreReloadFirewallRules(bool startup) { - bool created = false; - int rc; - - /* We create global rules upfront as we don't want - * the perf hit of conditionally figuring out whether - * to create them each time a network is started. - * - * Any errors here are saved to be reported at time - * of starting the network though as that makes them - * more likely to be seen by a human - */ - rc = iptablesSetupPrivateChains(VIR_FIREWALL_LAYER_IPV4); - if (rc < 0) { - errInitV4 = virSaveLastError(); - virResetLastError(); - } else { - virFreeError(errInitV4); - errInitV4 = NULL; - } - if (rc) - created = true; - - rc = iptablesSetupPrivateChains(VIR_FIREWALL_LAYER_IPV6); - if (rc < 0) { - errInitV6 = virSaveLastError(); - virResetLastError(); - } else { - virFreeError(errInitV6); - errInitV6 = NULL; - } - if (rc) - created = true; - /* * If this is initial startup, and we just created the * top level private chains we either @@ -86,8 +52,8 @@ void networkPreReloadFirewallRules(bool startup) * rules will be present. Thus we can safely just tell it * to always delete from the builin chain */ - if (startup && created) - iptablesSetDeletePrivate(false); + if (startup) + iptablesSetDeletePrivate(true); } @@ -701,19 +667,19 @@ int networkAddFirewallRules(virNetworkDefPtr def) virFirewallPtr fw = NULL; int ret = -1; - if (errInitV4 && - (virNetworkDefGetIPByIndex(def, AF_INET, 0) || - virNetworkDefGetRouteByIndex(def, AF_INET, 0))) { - virSetError(errInitV4); - return -1; - } + if (!pvtChainsCreated) { + if (iptablesSetupPrivateChains(VIR_FIREWALL_LAYER_IPV4) < 0 && + (virNetworkDefGetIPByIndex(def, AF_INET, 0) || + virNetworkDefGetRouteByIndex(def, AF_INET, 0))) + return -1; - if (errInitV6 && - (virNetworkDefGetIPByIndex(def, AF_INET6, 0) || - virNetworkDefGetRouteByIndex(def, AF_INET6, 0) || - def->ipv6nogw)) { - virSetError(errInitV6); - return -1; + if (iptablesSetupPrivateChains(VIR_FIREWALL_LAYER_IPV6) < 0 && + (virNetworkDefGetIPByIndex(def, AF_INET6, 0) || + virNetworkDefGetRouteByIndex(def, AF_INET6, 0) || + def->ipv6nogw)) + return -1; + + pvtChainsCreated = true; } if (def->bridgeZone) { diff --git a/tests/networkxml2firewalldata/nat-default-linux.args b/tests/networkxml2firewalldata/nat-default-linux.args index c9d523d043..61d620b592 100644 --- a/tests/networkxml2firewalldata/nat-default-linux.args +++ b/tests/networkxml2firewalldata/nat-default-linux.args @@ -1,5 +1,121 @@ iptables \ --table filter \ +--list-rules +iptables \ +--table nat \ +--list-rules +iptables \ +--table mangle \ +--list-rules +iptables \ +--table filter \ +--new-chain LIBVIRT_INP +iptables \ +--table filter \ +--insert INPUT \ +--jump LIBVIRT_INP +iptables \ +--table filter \ +--new-chain LIBVIRT_OUT +iptables \ +--table filter \ +--insert OUTPUT \ +--jump LIBVIRT_OUT +iptables \ +--table filter \ +--new-chain LIBVIRT_FWO +iptables \ +--table filter \ +--insert FORWARD \ +--jump LIBVIRT_FWO +iptables \ +--table filter \ +--new-chain LIBVIRT_FWI +iptables \ +--table filter \ +--insert FORWARD \ +--jump LIBVIRT_FWI +iptables \ +--table filter \ +--new-chain LIBVIRT_FWX +iptables \ +--table filter \ +--insert FORWARD \ +--jump LIBVIRT_FWX +iptables \ +--table nat \ +--new-chain LIBVIRT_PRT +iptables \ +--table nat \ +--insert POSTROUTING \ +--jump LIBVIRT_PRT +iptables \ +--table mangle \ +--new-chain LIBVIRT_PRT +iptables \ +--table mangle \ +--insert POSTROUTING \ +--jump LIBVIRT_PRT +ip6tables \ +--table filter \ +--list-rules +ip6tables \ +--table nat \ +--list-rules +ip6tables \ +--table mangle \ +--list-rules +ip6tables \ +--table filter \ +--new-chain LIBVIRT_INP +ip6tables \ +--table filter \ +--insert INPUT \ +--jump LIBVIRT_INP +ip6tables \ +--table filter \ +--new-chain LIBVIRT_OUT +ip6tables \ +--table filter \ +--insert OUTPUT \ +--jump LIBVIRT_OUT +ip6tables \ +--table filter \ +--new-chain LIBVIRT_FWO +ip6tables \ +--table filter \ +--insert FORWARD \ +--jump LIBVIRT_FWO +ip6tables \ +--table filter \ +--new-chain LIBVIRT_FWI +ip6tables \ +--table filter \ +--insert FORWARD \ +--jump LIBVIRT_FWI +ip6tables \ +--table filter \ +--new-chain LIBVIRT_FWX +ip6tables \ +--table filter \ +--insert FORWARD \ +--jump LIBVIRT_FWX +ip6tables \ +--table nat \ +--new-chain LIBVIRT_PRT +ip6tables \ +--table nat \ +--insert POSTROUTING \ +--jump LIBVIRT_PRT +ip6tables \ +--table mangle \ +--new-chain LIBVIRT_PRT +ip6tables \ +--table mangle \ +--insert POSTROUTING \ +--jump LIBVIRT_PRT +iptables \ +--table filter \ --insert LIBVIRT_INP \ --in-interface virbr0 \ --protocol tcp \ -- 2.21.0

3 4

[libvirt] [PATCH] qemu: Fix regression with undefine --snapshots-metadata
by Eric Blake 10 May '19

10 May '19

In refactoring the snapshot code to prepare for checkpoints, I changed qemuDomainMomentDiscardAll to take a callback that would handle the cleanup of either a snapshot or a checkpoint, but failed to set the callback on one of the two snapshot callers. As a result, 'virsh undefine $dom --snapshots-metadata' crashed on a NULL function dereference. Fixes: a487890d371b8cc3662c1717dfe07eea3f1ef1c0 Signed-off-by: Eric Blake <eblake(a)redhat.com> --- src/qemu/qemu_domain.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 2af17816c6..3290c5d490 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -8664,7 +8664,8 @@ qemuDomainSnapshotDiscardAllMetadata(virQEMUDriverPtr driver, virQEMUMomentRemove rem = { .driver = driver, .vm = vm, - .metadata_only = true + .metadata_only = true, + .momentDiscard = qemuDomainSnapshotDiscard, }; virDomainSnapshotForEach(vm->snapshots, qemuDomainMomentDiscardAll, &rem); -- 2.20.1

2 1

[libvirt] Running libvirtd out of source directory connection reset error
by Peter P. 10 May '19

10 May '19

Hi all, I'm getting started with hacking around with libvirt and am trying to familiarize myself with launching and running an instance of libvirtd I built from source on Centos 7.6. Following the instructions from https://libvirt.org/compiling.html to launch my built versions of libvirtd and virsh, I get the following error with no other context when trying to start a domain using "virsh start mydomain": error: Cannot recv data: Connection reset by peer Despite this error, I am able to run commands list virsh list. Are there additional parameters needed to launch libvirtd or additional services I need to start up alongside it? Thanks, Peter

2 2

[libvirt] [PATCH RFC] iohelper: Introduces a small sleep do avoid hunging other tasks
by Leonardo Bras 10 May '19

10 May '19

While dumping very large VMs (over 128GB), iohelper seems to cause very intense IO usage on the disk, and it causes some processes (like journald) to hung, and depending on kernel configuration, to panic. This change creates a time window, after every 10GB written, so this processes can write to the disk, and avoid hunging. Signed-off-by: Leonardo Bras <leonardo(a)linux.ibm.com> --- src/util/iohelper.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/util/iohelper.c b/src/util/iohelper.c index ddc338b7c7..164c1e2085 100644 --- a/src/util/iohelper.c +++ b/src/util/iohelper.c @@ -52,6 +52,8 @@ runIO(const char *path, int fd, int oflags) unsigned long long total = 0; bool direct = O_DIRECT && ((oflags & O_DIRECT) != 0); off_t end = 0; + const unsigned long long sleep_step = (long long)10*1024*1024*1024; + unsigned long long next_sleep = sleep_step; #if HAVE_POSIX_MEMALIGN if (posix_memalign(&base, alignMask + 1, buflen)) { @@ -128,6 +130,12 @@ runIO(const char *path, int fd, int oflags) total += got; + /* sleeps for a while to avoid hunging other tasks */ + if (total > next_sleep) { + next_sleep += sleep_step; + usleep(100*1000); + } + /* handle last write size align in direct case */ if (got < buflen && direct && fdout == fd) { ssize_t aligned_got = (got + alignMask) & ~alignMask; -- 2.20.1

3 3

[libvirt] [PATCH v2] Add support for podman in Makefile.ci
by Martin Kletzander 10 May '19

10 May '19

This way more users can run our CI builds locally. Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- Makefile.ci | 91 ++++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 70 insertions(+), 21 deletions(-) diff --git a/Makefile.ci b/Makefile.ci index 12a62167cc67..241c58d2d4e9 100644 --- a/Makefile.ci +++ b/Makefile.ci @@ -17,7 +17,7 @@ CI_GIT_ROOT = $(shell git rev-parse --show-toplevel) CI_HOST_SRCDIR = $(CI_SCRATCHDIR)/src # The directory holding the source inside the -# container. ie where we told Docker to expose +# container, i.e. where we want to expose # the $(CI_HOST_SRCDIR) directory from the host CI_CONT_SRCDIR = /src @@ -46,14 +46,13 @@ CI_CONFIGURE_ARGS = # cloning them CI_SUBMODULES = $(shell git submodule | awk '{ print $$2 }') -# Location of the Docker images we're going to pull +# Location of the container images we're going to pull # Can be useful to overridde to use a locally built # image instead CI_IMAGE_PREFIX = quay.io/libvirt/buildenv- -# Docker defaults to pulling the ':latest' tag but -# if the Docker repo above uses different conventions -# this can override it +# The default tag is ':latest' but if the container +# repo above uses different conventions this can override it CI_IMAGE_TAG = :master # We delete the virtual root after completion, set @@ -71,15 +70,23 @@ CI_REUSE = 0 CI_UID = $(shell id -u) CI_GID = $(shell id -g) -# Docker doesn't require the IDs you run as to exist in +CI_ENGINE = auto +# Container engine we are going to use, can be overridden per make +# invocation, if it is not we try podman and then default to docker. +ifeq ($(CI_ENGINE),auto) + override CI_ENGINE = $(shell podman version >/dev/null 2>&1 && echo podman || echo docker) +endif + +# IDs you run as do not need to exist in # the container's /etc/passwd & /etc/group files, but -# if they do not, then libvirt's 'make check' will fail +# if they do not, then libvirt's 'make check' will fail # many tests. -# + # We do not directly mount /etc/{passwd,group} as Docker # is liable to mess with SELinux labelling which will -# then prevent the host accessing them. Copying them -# first is safer. +# then prevent the host accessing them. And podman cannot +# relabel the files due to it running rootless. So +# copying them first is safer and error-prone. CI_PWDB_MOUNTS = \ --volume $(CI_SCRATCHDIR)/group:/etc/group:ro,z \ --volume $(CI_SCRATCHDIR)/passwd:/etc/passwd:ro,z \ @@ -90,6 +97,46 @@ CI_PWDB_MOUNTS = \ # libvirt very slow at exec'ing programs. CI_ULIMIT_FILES = 1024 +ifeq ($(CI_ENGINE),podman) + # Podman cannot reuse host namespace when running non-root containers. Until + # support for --keep-uid is added we can just create another mapping that will + # do that for us. Beware, that in {uid,git}map=container_id:host_id:range, + # the host_id does actually refer to the uid in the first mapping where 0 + # (root) is mapped to the current user and rest is offset. + + # In order to set up this mapping, we need to keep all the user IDs to prevent + # possible errors as some images might expect UIDs up to 90000 (looking at you + # fedora), so we don't want the overflowuid to be used for them. For mapping + # all the other users properly ther eneeds to be some math done. Don't worry, + # it's just addition and subtraction. + + # 65536 ought to be enough (tm), but for really rare cases the maximums might + # need to be higher, but that only happens when your /etc/sub{u,g}id allow + # users to have more IDs. Unless --keep-uid is supported, let's do this in a + # way that should work for everyone. + CI_MAX_UID = $(shell sed -n "s/^$USER:[^:]\+://p" /etc/subuid) + CI_MAX_GID = $(shell sed -n "s/^$USER:[^:]\+://p" /etc/subgid) + ifeq ($(CI_MAX_UID),) + CI_MAX_UID = 65536 + endif + ifeq ($(CI_MAX_GID),) + CI_MAX_GID = 65536 + endif + CI_UID_OTHER = $(shell echo $$(($(CI_UID)+1))) + CI_GID_OTHER = $(shell echo $$(($(CI_GID)+1))) + CI_UID_OTHER_RANGE = $(shell echo $$(($(CI_MAX_UID)-$(CI_UID)))) + CI_GID_OTHER_RANGE = $(shell echo $$(($(CI_MAX_GID)-$(CI_GID)))) + + CI_PODMAN_ARGS = \ + --uidmap 0:1:$(CI_UID) \ + --uidmap $(CI_UID):0:1 \ + --uidmap $(CI_UID_OTHER):$(CI_UID_OTHER):$(CI_UID_OTHER_RANGE) \ + --gidmap 0:1:$(CI_GID) \ + --gidmap $(CI_GID):0:1 \ + --gidmap $(CI_GID_OTHER):$(CI_GID_OTHER):$(CI_GID_OTHER_RANGE) \ + $(NULL) +endif + # Args to use when cloning a git repo. # -c stop it complaining about checking out a random hash # -q stop it displaying progress info for local clone @@ -100,7 +147,7 @@ CI_GIT_ARGS = \ --local \ $(NULL) -# Args to use when running the Docker env +# Args to use when running the container # --rm stop inactive containers getting left behind # --user we execute as the same user & group account # as dev so that file ownership matches host @@ -110,22 +157,23 @@ CI_GIT_ARGS = \ # --ulimit lower files limit for performance reasons # --interactive # --tty Ensure we have ability to Ctrl-C the build -CI_DOCKER_ARGS = \ +CI_ENGINE_ARGS = \ --rm \ --user $(CI_UID):$(CI_GID) \ --interactive \ --tty \ + $(CI_PODMAN_ARGS) \ $(CI_PWDB_MOUNTS) \ --volume $(CI_HOST_SRCDIR):$(CI_CONT_SRCDIR):z \ --workdir $(CI_CONT_SRCDIR) \ --ulimit nofile=$(CI_ULIMIT_FILES):$(CI_ULIMIT_FILES) \ $(NULL) -ci-check-docker: - @echo -n "Checking if Docker is available and running..." && \ - docker version 1>/dev/null && echo "yes" +ci-check-engine: + @echo -n "Checking if $(CI_ENGINE) is available..." && \ + $(CI_ENGINE) version 1>/dev/null && echo "yes" -ci-prepare-tree: ci-check-docker +ci-prepare-tree: ci-check-engine @test "$(CI_REUSE)" != "1" && rm -rf $(CI_SCRATCHDIR) || : @if ! test -d $(CI_SCRATCHDIR) ; then \ mkdir -p $(CI_SCRATCHDIR); \ @@ -150,7 +198,7 @@ ci-prepare-tree: ci-check-docker # gl_public_submodule_commit= to disable gnulib's submodule check # which breaks due to way we clone the submodules ci-build@%: ci-prepare-tree - docker run $(CI_DOCKER_ARGS) $(CI_IMAGE_PREFIX)$*$(CI_IMAGE_TAG) \ + $(CI_ENGINE) run $(CI_ENGINE_ARGS) $(CI_IMAGE_PREFIX)$*$(CI_IMAGE_TAG) \ /bin/bash -c '\ mkdir -p $(CI_CONT_BUILDDIR) || exit 1 ; \ cd $(CI_CONT_BUILDDIR) ; \ @@ -179,11 +227,11 @@ ci-check@%: $(MAKE) -f $(CI_MAKEFILE) ci-build@$* CI_MAKE_ARGS="check" ci-shell@%: ci-prepare-tree - docker run $(CI_DOCKER_ARGS) $(CI_IMAGE_PREFIX)$*$(CI_IMAGE_TAG) /bin/bash + $(CI_ENGINE) run $(CI_ENGINE_ARGS) $(CI_IMAGE_PREFIX)$*$(CI_IMAGE_TAG) /bin/bash @test "$(CI_CLEAN)" = "1" && rm -rf $(CI_SCRATCHDIR) || : ci-help: - @echo "Build libvirt inside Docker containers used for CI" + @echo "Build libvirt inside containers used for CI" @echo @echo "Available targets:" @echo @@ -215,6 +263,7 @@ ci-help: @echo @echo "Available make variables:" @echo - @echo " CI_CLEAN=0 - do not delete '$(CI_SCRATCHDIR)' after completion" - @echo " CI_REUSE=1 - re-use existing '$(CI_SCRATCHDIR)' content" + @echo " CI_CLEAN=0 - do not delete '$(CI_SCRATCHDIR)' after completion" + @echo " CI_REUSE=1 - re-use existing '$(CI_SCRATCHDIR)' content" + @echo " CI_ENGINE=auto - container engine to use (podman, docker)" @echo -- 2.21.0

2 1

[libvirt] [jenkins-ci PATCH 0/6] Add support for gtk-vnc builds
by Daniel P. Berrangé 10 May '19

10 May '19

Support gtk-vnc and make virt-viewer depend on it Daniel P. Berrangé (6): mappings: add libgcrypt native & mingw packages mappings: add pulseaudio libs for native packages projects: add gtk-vnc project projects: make virt-viewer depend on gtk-vnc jobs mappings: remove gtk-vnc2 native and mingw packages jobs: allow test-suite.log to exist anywhere guests/host_vars/libvirt-centos-7/main.yml | 1 + guests/host_vars/libvirt-debian-9/main.yml | 1 + guests/host_vars/libvirt-debian-sid/main.yml | 1 + guests/host_vars/libvirt-fedora-29/main.yml | 1 + guests/host_vars/libvirt-fedora-30/main.yml | 1 + .../host_vars/libvirt-fedora-rawhide/main.yml | 3 +++ guests/host_vars/libvirt-freebsd-11/main.yml | 1 + guests/host_vars/libvirt-freebsd-12/main.yml | 1 + .../libvirt-freebsd-current/main.yml | 1 + guests/host_vars/libvirt-ubuntu-18/main.yml | 1 + .../build/jobs/autotools-check-job.yml | 2 +- guests/playbooks/build/jobs/defaults.yml | 3 +++ .../build/projects/gtk-vnc+mingw32.yml | 12 +++++++++ .../build/projects/gtk-vnc+mingw64.yml | 12 +++++++++ guests/playbooks/build/projects/gtk-vnc.yml | 12 +++++++++ guests/vars/mappings.yml | 27 ++++++++++--------- guests/vars/projects/gtk-vnc+mingw32.yml | 6 +++++ guests/vars/projects/gtk-vnc+mingw64.yml | 6 +++++ guests/vars/projects/gtk-vnc.yml | 11 ++++++++ guests/vars/projects/virt-viewer+mingw32.yml | 1 - guests/vars/projects/virt-viewer+mingw64.yml | 1 - guests/vars/projects/virt-viewer.yml | 1 - jenkins/jobs/autotools.yaml | 2 +- jenkins/jobs/defaults.yaml | 3 +++ jenkins/projects/gtk-vnc+mingw32.yaml | 11 ++++++++ jenkins/projects/gtk-vnc+mingw64.yaml | 11 ++++++++ jenkins/projects/gtk-vnc.yaml | 15 +++++++++++ jenkins/projects/virt-viewer+mingw32.yaml | 4 ++- jenkins/projects/virt-viewer+mingw64.yaml | 4 ++- jenkins/projects/virt-viewer.yaml | 4 ++- 30 files changed, 140 insertions(+), 20 deletions(-) create mode 100644 guests/playbooks/build/projects/gtk-vnc+mingw32.yml create mode 100644 guests/playbooks/build/projects/gtk-vnc+mingw64.yml create mode 100644 guests/playbooks/build/projects/gtk-vnc.yml create mode 100644 guests/vars/projects/gtk-vnc+mingw32.yml create mode 100644 guests/vars/projects/gtk-vnc+mingw64.yml create mode 100644 guests/vars/projects/gtk-vnc.yml create mode 100644 jenkins/projects/gtk-vnc+mingw32.yaml create mode 100644 jenkins/projects/gtk-vnc+mingw64.yaml create mode 100644 jenkins/projects/gtk-vnc.yaml -- 2.21.0

2 16

[libvirt] [PATCHv2] build: restore support for libyajl 2.0.1
by Ján Tomko 10 May '19

10 May '19

Commit 105756660f944e7db02de3b55b98bb7c11cd03bf was too eager and did not consider SLE 12 which still has 2.0.1 that does not ship a pkg-config file. Similar to how we check for readline, prefer pkg-config if available and fall back to the old detection code if not found. NB: this is not a clean revert because we're not reintroducing support for YAJL 1. Signed-off-by: Ján Tomko <jtomko(a)redhat.com> Reported-by: Olaf Hering <olaf(a)aepfle.de> --- m4/virt-yajl.m4 | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/m4/virt-yajl.m4 b/m4/virt-yajl.m4 index 494e722963..3caf0c5d07 100644 --- a/m4/virt-yajl.m4 +++ b/m4/virt-yajl.m4 @@ -24,7 +24,19 @@ AC_DEFUN([LIBVIRT_ARG_YAJL],[ AC_DEFUN([LIBVIRT_CHECK_YAJL],[ dnl YAJL JSON library http://lloyd.github.com/yajl/ - LIBVIRT_CHECK_PKG([YAJL], [yajl], [2.0.3]) + PKG_CHECK_EXISTS([readline], [use_pkgconfig=1], [use_pkgconfig=0]) + + if test $use_pkgconfig = 1; then + dnl 2.0.3 was the version where the pkg-config file was first added + LIBVIRT_CHECK_PKG([YAJL], [yajl], [2.0.3]) + else + dnl SUSE SLE 12 and OpenSUSE Leap 42.3 still use 2.0.1 + dnl TODO: delete this in July 2020 + LIBVIRT_CHECK_LIB([YAJL], [yajl], + [yajl_tree_parse], [yajl/yajl_common.h]) + + fi + ]) AC_DEFUN([LIBVIRT_RESULT_YAJL],[ -- 2.19.2

3 3