- Devel - libvirt List Archives

[libvirt] [PATCH] spec: Disable gluster on i686 on RHEL-8
by Jiri Denemark 29 Nov '19

29 Nov '19

Gluster 6.0 is not built on i686 for RHEL-8, which prevents libvirt from building. Let's just disable gluster there as all we need are client libraries anyway. Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- libvirt.spec.in | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/libvirt.spec.in b/libvirt.spec.in index a6219da604..c7fac104d9 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -65,7 +65,15 @@ %else %define with_storage_sheepdog 0 %endif + %define with_storage_gluster 0%{!?_without_storage_gluster:1} +%ifnarch %{qemu_kvm_arches} + # gluster is only built where qemu driver is enabled on RHEL 8 + %if 0%{?rhel} >= 8 + %define with_storage_gluster 0 + %endif +%endif + %define with_numactl 0%{!?_without_numactl:1} # F25+ has zfs-fuse -- 2.24.0

2 1

[libvirt] [PATCH] remote_daemon_stream: Hold an extra reference to stream in daemonStreamFilter
by Michal Privoznik 29 Nov '19

29 Nov '19

In v5.9.0-273-g8ecab214de I've tried to fix a lock ordering problem, but introduced a crasher. Problem is that because the client lock is unlocked (in order to honour lock ordering) the stream we are currently checking in daemonStreamFilter() might be freed and thus stream->priv might not even exist when the control get to virMutexLock() call. To resolve this, grab an extra reference to the stream and handle its cleanup should the refcounter reach zero after the deref. If that's the case and we are the only ones holding a reference to the stream, we MUST return a positive value to make virNetServerClientDispatchRead() break its loop where it iterates over filters. The problem is, if we did not do so, then "filter = filter->next" line will read from a memory that was just freed (freeing a stream also unregisters its filter). Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- Reproducing this issue is very easy: 1) put sleep(5) right after virObjectUnlock(client) in the fist hunk, 2) virsh console --force $dom and type something so that the stream has some data to process 3) while 2) is still running, run the same command from another terminal 4) observe libvirtd crash src/remote/remote_daemon_stream.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/src/remote/remote_daemon_stream.c b/src/remote/remote_daemon_stream.c index 82cadb67ac..73e4d7befb 100644 --- a/src/remote/remote_daemon_stream.c +++ b/src/remote/remote_daemon_stream.c @@ -293,10 +293,25 @@ daemonStreamFilter(virNetServerClientPtr client, daemonClientStream *stream = opaque; int ret = 0; + /* We must honour lock ordering here. Client private data lock must + * be acquired before client lock. Bu we are already called with + * client locked. To avoid stream disappearing while we unlock + * everything, let's increase its refcounter. This has some + * implications though. */ + stream->refs++; virObjectUnlock(client); virMutexLock(&stream->priv->lock); virObjectLock(client); + if (stream->refs == 1) { + /* So we are the only ones holding the reference to the stream. + * Return 1 to signal to the caller that we've processed the + * message. And to "process" means free. */ + virNetMessageFree(msg); + ret = 1; + goto cleanup; + } + if (msg->header.type != VIR_NET_STREAM && msg->header.type != VIR_NET_STREAM_HOLE) goto cleanup; @@ -318,6 +333,10 @@ daemonStreamFilter(virNetServerClientPtr client, cleanup: virMutexUnlock(&stream->priv->lock); + /* Don't pass client here, because client is locked here and this + * function might try to lock it again which would result in a + * deadlock. */ + daemonFreeClientStream(NULL, stream); return ret; } -- 2.23.0

3 8

[libvirt] [PATCH 0/3] qemu: fix regression and design deficiencies of VIR_DOMAIN_STATS_IOTHREAD
by Peter Krempa 29 Nov '19

29 Nov '19

Fix regression of numbering of the output in the statistics for iothreads and improve few aspects so that the numbering is usable. Peter Krempa (3): lib: Fix documentation for the count field of VIR_DOMAIN_STATS_IOTHREAD qemu: Fix indexes in statistics of iothreads qemu: Report which iothread IDs are actually returned with VIR_DOMAIN_STATS_IOTHREAD src/libvirt-domain.c | 12 +++++++----- src/qemu/qemu_driver.c | 20 +++++++++++++++++--- 2 files changed, 24 insertions(+), 8 deletions(-) -- 2.23.0

2 7

[libvirt] [PATCH] storage: backend_disk: Implement the deletePool function
by Erik Skultety 29 Nov '19

29 Nov '19

This is a very simple and straightforward implementation of the opposite what buildPool does for the disk backend. The background for this change comes from an existing test case in TCK which does use the delete method for a pool of type disk, but it truly could not have ever worked since the implementation simply wasn't there for the pool of type disk. Signed-off-by: Erik Skultety <eskultet(a)redhat.com> --- src/storage/storage_backend_disk.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/src/storage/storage_backend_disk.c b/src/storage/storage_backend_disk.c index d971530cd8..45d1257f3d 100644 --- a/src/storage/storage_backend_disk.c +++ b/src/storage/storage_backend_disk.c @@ -531,6 +531,25 @@ virStorageBackendDiskBuildPool(virStoragePoolObjPtr pool, } +/** + * Wipe the existing partition table + */ +static int +virStorageBackendDiskDeletePool(virStoragePoolObjPtr pool, + unsigned int flags) +{ + virStoragePoolDefPtr def = virStoragePoolObjGetDef(pool); + + virCheckFlags(0, -1); + + if (virStorageBackendZeroPartitionTable(def->source.devices[0].path, + 1024 * 1024) < 0) + return -1; + + return 0; +} + + struct virStorageVolNumData { int count; }; @@ -954,6 +973,7 @@ virStorageBackend virStorageBackendDisk = { .startPool = virStorageBackendDiskStartPool, .buildPool = virStorageBackendDiskBuildPool, .refreshPool = virStorageBackendDiskRefreshPool, + .deletePool = virStorageBackendDiskDeletePool, .createVol = virStorageBackendDiskCreateVol, .deleteVol = virStorageBackendDiskDeleteVol, -- 2.23.0

2 1

[libvirt] RFC: stop clearing QEMU emulator capabilities
by Daniel P. Berrangé 29 Nov '19

29 Nov '19

We have an RFE from libguestfs to provide a way to run as root *with* capabilities. I looked integrating this into the DAC security manager as a new flag in the security label, but then I started thinking about the whole idea of clearing capabilities https://bugzilla.redhat.com/show_bug.cgi?id=1045039 Pretty much forever we have explicitly cleared QEMU emulator capabilities when starting it. When QEMU uid/gid is set to non-root this is pointless as if we just used a regular setuid/setgid call, the process will have all its capabilities cleared anyway by the kernel. When QEMU uid/gid is set to root, this is almost (always?) never what people actually want. People make QEMU run as root in order to access some privileged resource that libvirt doesn't support yet and this often requires capabilities. As a result they have to go find the qemu.conf param to turn this off. This is not viable for libguestfs - they want to control everything via thue XML security label to request running as root regardless of the qemu.conf settings for user/group. Clearing capabilities was implemented originally because there was a proposal in Fedora to change permissions such that root, with no capabilities would not be able to compromise the system. ie a locked down root account. This never went anywhere though, and as a result clearing capabilities when running as root does not really get us any security benefit AFAICT. The root user can just do something like create a cronjob, which will then faithfully be run with full capabilities, trivially bypassing the restriction we place. IOW, our clearing of capabilities is both useless from a security POV, and breaks valid use cases when people need to run as root. I'm thinking we should just rip out the code which clears capabilities and allow default loggic to run - If uid/gid is non-root, then no capabilities are present - If uid/gid is root, then full capabilities are present Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

4 3

[libvirt] [PATCH] check-symfile: Use pythonesque string formatting instead of perl
by Peter Krempa 29 Nov '19

29 Nov '19

Commit d30a1ad0443 translated the symbol file checker from perl to python by doing a literal translation in most cases. Unfortunately one string formatting operation was not really translated into python leaving users with non-helpful error: 'Symbol $1 is listed twice' Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- scripts/check-symfile.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/check-symfile.py b/scripts/check-symfile.py index 0c02591991..34396b8623 100755 --- a/scripts/check-symfile.py +++ b/scripts/check-symfile.py @@ -52,7 +52,7 @@ with open(symfile, "r") as fh: line = line.strip(";") if line in wantsyms: - print("Symbol $1 is listed twice", file=sys.stderr) + print("Symbol %s is listed twice" % line ,file=sys.stderr) ret = 1 else: wantsyms[line] = True -- 2.23.0

9 14

[libvirt] Entering freeze for libvirt 5.10.0
by Daniel Veillard 28 Nov '19

28 Nov '19

As planned this is now tagged in git head and signed source tarball and rpm are available at the usual place: https://libvirt.org/sources/ there is a few weirdness on CI, some apparently related to freebsd, others to python and go bindings, worth having a look, some are broken for a while now. Otherwise this passes my very minimal testing, but don't rely on this ! I plan on making a RC2 for Thanksgiving, and if everything looks fine push the final release next Monday. Please make sure we have the docs in the news file :-) thanks ! Daniel -- Daniel Veillard | Red Hat Developers Tools http://developer.redhat.com/ veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

2 2

[libvirt] [PATCH] qemu: make 'xz' image compression viable by using -3
by Daniel P. Berrangé 28 Nov '19

28 Nov '19

For managed save we can choose between various compression methods. I randomly tested the 'xz' program on a 8 GB guest and was surprised to have to wait > 50 minutes for it to finish compressing, with 'xz' burning 100% cpu for the entire time. Despite the impressive compression, this is completely useless in the real world as it is far too long to wait to save the VM. The 'xz' binary defaults to '-6' optimization level which aims for high compression, with moderate memory usage, at the expense of speed. This change switches it to use the '-3' optimization level which is documented as being the one that optimizes speed at expense of compression. Even with this, it will still outperform all the other options in terms of compression level. It is a little less than x4 faster than '-6' which means it starts to be a viable choice to use 'xz' for people who really want best compression. The test results on a 1 GB, fairly freshly booted VM are as follows format | save | restore size =======+=======+============= raw | 05s | 1s | 428 MB lzop | 05s | 3s | 160 MB gzip | 29s | 5s | 118 MB bz2 | 54s | 22s | 114 MB xz | 4m37s | 13s | 86 MB xz -3 | 1m20s | 12s | 95 MB Based on this we can say * For moderate compression with no noticable loss in speed => use lzop * For high compression with moderate loss in speed => use gzip * For best compression with significant loss in speed => use xz Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- src/qemu/qemu_driver.c | 46 ++++++++++++++++++++++----------------- src/qemu/qemu_migration.c | 28 ++++++++---------------- src/qemu/qemu_migration.h | 2 +- 3 files changed, 36 insertions(+), 40 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 8c2670d377..ba50701d3a 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -3177,7 +3177,7 @@ qemuDomainSaveMemory(virQEMUDriverPtr driver, virDomainObjPtr vm, const char *path, virQEMUSaveDataPtr data, - const char *compressedpath, + virCommandPtr compressor, unsigned int flags, qemuDomainAsyncJob asyncJob) { @@ -3216,7 +3216,7 @@ qemuDomainSaveMemory(virQEMUDriverPtr driver, goto cleanup; /* Perform the migration */ - if (qemuMigrationSrcToFile(driver, vm, fd, compressedpath, asyncJob) < 0) + if (qemuMigrationSrcToFile(driver, vm, fd, compressor, asyncJob) < 0) goto cleanup; /* Touch up file header to mark image complete. */ @@ -3259,7 +3259,7 @@ qemuDomainSaveMemory(virQEMUDriverPtr driver, static int qemuDomainSaveInternal(virQEMUDriverPtr driver, virDomainObjPtr vm, const char *path, - int compressed, const char *compressedpath, + int compressed, virCommandPtr compressor, const char *xmlin, unsigned int flags) { g_autofree char *xml = NULL; @@ -3345,7 +3345,7 @@ qemuDomainSaveInternal(virQEMUDriverPtr driver, goto endjob; xml = NULL; - ret = qemuDomainSaveMemory(driver, vm, path, data, compressedpath, + ret = qemuDomainSaveMemory(driver, vm, path, data, compressor, flags, QEMU_ASYNC_JOB_SAVE); if (ret < 0) goto endjob; @@ -3406,13 +3406,14 @@ qemuDomainSaveInternal(virQEMUDriverPtr driver, */ static int ATTRIBUTE_NONNULL(2) qemuGetCompressionProgram(const char *imageFormat, - char **compresspath, + virCommandPtr *compressor, const char *styleFormat, bool use_raw_on_fail) { int ret; + const char *prog; - *compresspath = NULL; + *compressor = NULL; if (!imageFormat) return QEMU_SAVE_FORMAT_RAW; @@ -3423,9 +3424,14 @@ qemuGetCompressionProgram(const char *imageFormat, if (ret == QEMU_SAVE_FORMAT_RAW) return QEMU_SAVE_FORMAT_RAW; - if (!(*compresspath = virFindFileInPath(imageFormat))) + if (!(prog = virFindFileInPath(imageFormat))) goto error; + *compressor = virCommandNew(prog); + virCommandAddArg(*compressor, "-c"); + if (ret == QEMU_SAVE_FORMAT_XZ) + virCommandAddArg(*compressor, "-3"); + return ret; error: @@ -3466,7 +3472,7 @@ qemuDomainSaveFlags(virDomainPtr dom, const char *path, const char *dxml, { virQEMUDriverPtr driver = dom->conn->privateData; int compressed; - g_autofree char *compressedpath = NULL; + g_autoptr(virCommand) compressor = NULL; int ret = -1; virDomainObjPtr vm = NULL; g_autoptr(virQEMUDriverConfig) cfg = NULL; @@ -3477,7 +3483,7 @@ qemuDomainSaveFlags(virDomainPtr dom, const char *path, const char *dxml, cfg = virQEMUDriverGetConfig(driver); if ((compressed = qemuGetCompressionProgram(cfg->saveImageFormat, - &compressedpath, + &compressor, "save", false)) < 0) goto cleanup; @@ -3491,7 +3497,7 @@ qemuDomainSaveFlags(virDomainPtr dom, const char *path, const char *dxml, goto cleanup; ret = qemuDomainSaveInternal(driver, vm, path, compressed, - compressedpath, dxml, flags); + compressor, dxml, flags); cleanup: virDomainObjEndAPI(&vm); @@ -3522,7 +3528,7 @@ qemuDomainManagedSave(virDomainPtr dom, unsigned int flags) virQEMUDriverPtr driver = dom->conn->privateData; g_autoptr(virQEMUDriverConfig) cfg = NULL; int compressed; - g_autofree char *compressedpath = NULL; + g_autoptr(virCommand) compressor = NULL; virDomainObjPtr vm; g_autofree char *name = NULL; int ret = -1; @@ -3548,7 +3554,7 @@ qemuDomainManagedSave(virDomainPtr dom, unsigned int flags) cfg = virQEMUDriverGetConfig(driver); if ((compressed = qemuGetCompressionProgram(cfg->saveImageFormat, - &compressedpath, + &compressor, "save", false)) < 0) goto cleanup; @@ -3558,7 +3564,7 @@ qemuDomainManagedSave(virDomainPtr dom, unsigned int flags) VIR_INFO("Saving state of domain '%s' to '%s'", vm->def->name, name); ret = qemuDomainSaveInternal(driver, vm, name, compressed, - compressedpath, NULL, flags); + compressor, NULL, flags); if (ret == 0) vm->hasManagedSave = true; @@ -3754,14 +3760,14 @@ doCoreDump(virQEMUDriverPtr driver, unsigned int flags = VIR_FILE_WRAPPER_NON_BLOCKING; const char *memory_dump_format = NULL; g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); - g_autofree char *compressedpath = NULL; + g_autoptr(virCommand) compressor = NULL; /* We reuse "save" flag for "dump" here. Then, we can support the same * format in "save" and "dump". This path doesn't need the compression * program to exist and can ignore the return value - it only cares to - * get the compressedpath */ + * get the compressor */ ignore_value(qemuGetCompressionProgram(cfg->dumpImageFormat, - &compressedpath, + &compressor, "dump", true)); /* Create an empty file with appropriate ownership. */ @@ -3809,7 +3815,7 @@ doCoreDump(virQEMUDriverPtr driver, if (!qemuMigrationSrcIsAllowed(driver, vm, false, 0)) goto cleanup; - ret = qemuMigrationSrcToFile(driver, vm, fd, compressedpath, + ret = qemuMigrationSrcToFile(driver, vm, fd, compressor, QEMU_ASYNC_JOB_DUMP); } @@ -15586,7 +15592,7 @@ qemuDomainSnapshotCreateActiveExternal(virQEMUDriverPtr driver, int thaw = 0; /* 1 if freeze succeeded, -1 if freeze failed */ bool pmsuspended = false; int compressed; - g_autofree char *compressedpath = NULL; + g_autoptr(virCommand) compressor = NULL; virQEMUSaveDataPtr data = NULL; /* If quiesce was requested, then issue a freeze command, and a @@ -15656,7 +15662,7 @@ qemuDomainSnapshotCreateActiveExternal(virQEMUDriverPtr driver, JOB_MASK(QEMU_JOB_MIGRATION_OP))); if ((compressed = qemuGetCompressionProgram(cfg->snapshotImageFormat, - &compressedpath, + &compressor, "snapshot", false)) < 0) goto cleanup; @@ -15673,7 +15679,7 @@ qemuDomainSnapshotCreateActiveExternal(virQEMUDriverPtr driver, xml = NULL; if ((ret = qemuDomainSaveMemory(driver, vm, snapdef->file, data, - compressedpath, 0, + compressor, 0, QEMU_ASYNC_JOB_SNAPSHOT)) < 0) goto cleanup; diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index c38bf342d7..dabdda2715 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -5170,13 +5170,12 @@ qemuMigrationDstFinish(virQEMUDriverPtr driver, int qemuMigrationSrcToFile(virQEMUDriverPtr driver, virDomainObjPtr vm, int fd, - const char *compressor, + virCommandPtr compressor, qemuDomainAsyncJob asyncJob) { qemuDomainObjPrivatePtr priv = vm->privateData; int rc; int ret = -1; - virCommandPtr cmd = NULL; int pipeFD[2] = { -1, -1 }; unsigned long saveMigBandwidth = priv->migMaxBandwidth; char *errbuf = NULL; @@ -5221,25 +5220,17 @@ qemuMigrationSrcToFile(virQEMUDriverPtr driver, virDomainObjPtr vm, QEMU_MONITOR_MIGRATE_BACKGROUND, fd); } else { - const char *prog = compressor; - const char *args[] = { - prog, - "-c", - NULL - }; - - cmd = virCommandNewArgs(args); - virCommandSetInputFD(cmd, pipeFD[0]); - virCommandSetOutputFD(cmd, &fd); - virCommandSetErrorBuffer(cmd, &errbuf); - virCommandDoAsyncIO(cmd); + virCommandSetInputFD(compressor, pipeFD[0]); + virCommandSetOutputFD(compressor, &fd); + virCommandSetErrorBuffer(compressor, &errbuf); + virCommandDoAsyncIO(compressor); if (virSetCloseExec(pipeFD[1]) < 0) { virReportSystemError(errno, "%s", _("Unable to set cloexec flag")); ignore_value(qemuDomainObjExitMonitor(driver, vm)); goto cleanup; } - if (virCommandRunAsync(cmd, NULL) < 0) { + if (virCommandRunAsync(compressor, NULL) < 0) { ignore_value(qemuDomainObjExitMonitor(driver, vm)); goto cleanup; } @@ -5260,7 +5251,7 @@ qemuMigrationSrcToFile(virQEMUDriverPtr driver, virDomainObjPtr vm, if (rc < 0) { if (rc == -2) { virErrorPreserveLast(&orig_err); - virCommandAbort(cmd); + virCommandAbort(compressor); if (virDomainObjIsActive(vm) && qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) == 0) { qemuMonitorMigrateCancel(priv->mon); @@ -5270,7 +5261,7 @@ qemuMigrationSrcToFile(virQEMUDriverPtr driver, virDomainObjPtr vm, goto cleanup; } - if (cmd && virCommandWait(cmd, NULL) < 0) + if (compressor && virCommandWait(compressor, NULL) < 0) goto cleanup; qemuDomainEventEmitJobCompleted(driver, vm); @@ -5290,10 +5281,9 @@ qemuMigrationSrcToFile(virQEMUDriverPtr driver, virDomainObjPtr vm, VIR_FORCE_CLOSE(pipeFD[0]); VIR_FORCE_CLOSE(pipeFD[1]); - if (cmd) { + if (errbuf) { VIR_DEBUG("Compression binary stderr: %s", NULLSTR(errbuf)); VIR_FREE(errbuf); - virCommandFree(cmd); } virErrorRestore(&orig_err); diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index 2daae65191..d98fe9f80a 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -204,7 +204,7 @@ int qemuMigrationSrcToFile(virQEMUDriverPtr driver, virDomainObjPtr vm, int fd, - const char *compressor, + virCommandPtr compressor, qemuDomainAsyncJob asyncJob) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT; -- 2.21.0

2 1

[libvirt] [PATCH for 5.10.0] nss: Don't fail on empty files
by Michal Privoznik 28 Nov '19

28 Nov '19

Before we rewrote nss plugin so that it doesn't use libvirt's internal functions it used virLeaseReadCustomLeaseFile() to parse .status files. After the rewrite it's using read() + yajl_parse() + yajl_complete_parse(). There's one catch though, virLeaseReadCustomLeaseFile() skipped over empty files. An empty .status file is created when a network is started. This is because we configure dnsmasq to use our leasehelper. So the first thing it does it calls it as follows: DNSMASQ_INTERFACE=virbr0 /usr/libexec/libvirt_leaseshelper init which causes the leasehelper to create empty virbr0.status file. If there is only one libvirt network then that is no problem - there are no other .status files to parse anyway. But if there are two or more networks then the first empty .status file causes whole parsing process and subsequently the whole name lookup process to fail. Reported-by: Pavel Hrdina <phrdina(a)redhat.com> Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- tools/nss/libvirt_nss_leases.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tools/nss/libvirt_nss_leases.c b/tools/nss/libvirt_nss_leases.c index e96e260da8..7c431e4d53 100644 --- a/tools/nss/libvirt_nss_leases.c +++ b/tools/nss/libvirt_nss_leases.c @@ -379,6 +379,7 @@ findLeases(const char *file, }; yajl_handle parser = NULL; char line[1024]; + ssize_t nreadTotal = 0; int rv; if ((fd = open(file, O_RDONLY)) < 0) { @@ -398,6 +399,7 @@ findLeases(const char *file, goto cleanup; if (rv == 0) break; + nreadTotal += rv; if (yajl_parse(parser, (const unsigned char *)line, rv) != yajl_status_ok) { @@ -409,7 +411,8 @@ findLeases(const char *file, } } - if (yajl_complete_parse(parser) != yajl_status_ok) { + if (nreadTotal > 0 && + yajl_complete_parse(parser) != yajl_status_ok) { ERROR("Parse failed %s", yajl_get_error(parser, 1, NULL, 0)); goto cleanup; -- 2.23.0

2 2

[libvirt] [PATCH] qemu: Fix migration without parameters
by Jiri Denemark 28 Nov '19

28 Nov '19

The virTypedParamsFilter function doesn't mind params == NULL if nparams is zero. And there's no need to check for params == NULL && nparams > 0 because this is checked higher in the stack. In fact all the virCheckNonNull* checks in virTypedParamsFilter are useless. https://bugzilla.redhat.com/show_bug.cgi?id=1777094 Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- src/util/virtypedparam.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/src/util/virtypedparam.c b/src/util/virtypedparam.c index 603fcf213a..be3dac1571 100644 --- a/src/util/virtypedparam.c +++ b/src/util/virtypedparam.c @@ -403,10 +403,6 @@ virTypedParamsFilter(virTypedParameterPtr params, { size_t i, n = 0; - virCheckNonNullArgGoto(params, error); - virCheckNonNullArgGoto(name, error); - virCheckNonNullArgGoto(ret, error); - if (VIR_ALLOC_N(*ret, nparams) < 0) goto error; -- 2.24.0

3 2

[libvirt] [PATCH 1/1] qemu: sync blockjob finishing in qemuDomainGetBlockJobInfo
by Nikolay Shirokovskiy 28 Nov '19

28 Nov '19

Due to race qemuDomainGetBlockJobInfo can return there is no block job for disk but later call to spawn new blockjob can fail because libvirt internally still not process blockjob finishing. Thus let's wait for blockjob finishing if we report there is no more blockjob. Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy(a)virtuozzo.com> --- src/qemu/qemu_driver.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 669c12d6ca..b148df3a57 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -17785,12 +17785,26 @@ qemuDomainGetBlockJobInfo(virDomainPtr dom, goto endjob; } + qemuBlockJobSyncBegin(job); + qemuDomainObjEnterMonitor(driver, vm); ret = qemuMonitorGetBlockJobInfo(qemuDomainGetMonitor(vm), job->name, &rawInfo); if (qemuDomainObjExitMonitor(driver, vm) < 0) ret = -1; - if (ret <= 0) + if (ret < 0) + goto endjob; + + if (ret == 0) { + qemuBlockJobUpdate(vm, job, QEMU_ASYNC_JOB_NONE); + while (qemuBlockJobIsRunning(job)) { + if (virDomainObjWait(vm) < 0) { + ret = -1; + goto endjob; + } + qemuBlockJobUpdate(vm, job, QEMU_ASYNC_JOB_NONE); + } goto endjob; + } if (qemuBlockJobInfoTranslate(&rawInfo, info, disk, flags & VIR_DOMAIN_BLOCK_JOB_INFO_BANDWIDTH_BYTES) < 0) { @@ -17799,6 +17813,8 @@ qemuDomainGetBlockJobInfo(virDomainPtr dom, } endjob: + if (job) + qemuBlockJobSyncEnd(vm, job, QEMU_ASYNC_JOB_NONE); qemuDomainObjEndJob(driver, vm); cleanup: -- 2.23.0

2 6

[libvirt] [go PATCH 0/2] add missing API and constants
by Pavel Hrdina 28 Nov '19

28 Nov '19

Pavel Hrdina (2): Add VIR_FROM_BPF error constant Add support for virDomainAgentSetResponseTimeout API domain.go | 23 +++++++++++++++++++++++ domain_compat.h | 14 ++++++++++++++ domain_wrapper.go | 17 +++++++++++++++++ domain_wrapper.h | 6 ++++++ error.go | 3 +++ error_compat.h | 6 ++++++ 6 files changed, 69 insertions(+) -- 2.23.0

2 5

[libvirt] [PATCH] tests: improve build time for qemuxml2argvtest
by Daniel P. Berrangé 28 Nov '19

28 Nov '19

On Fedora 31 with GCC 9.2.1, compiling qemuxml2argvtest takes about 36 seconds since commit 30c6d99209cd36bba8e14795622f658827698522 Author: Jiri Denemark <jdenemar(a)redhat.com> Date: Thu Oct 24 17:51:42 2019 +0200 qemuxml2argvtest: Update host arch for DO_TEST*ARCH* tests The optimizer is hitting some pathelogical performance behaviour due to the high number of branches in the mymain() method. Pushing the branch tests down into the testCompareXMLToArgv method brings the compile time down to 3 seconds. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- tests/qemuxml2argvtest.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index a2791d0460..86b8899921 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -412,6 +412,9 @@ testCompareXMLToArgv(const void *data) size_t i; qemuDomainObjPrivatePtr priv = NULL; + if (info->arch != VIR_ARCH_NONE && info->arch != VIR_ARCH_X86_64) + qemuTestSetHostArch(driver.caps, info->arch); + memset(&monitor_chr, 0, sizeof(monitor_chr)); if (!(conn = virGetConnect())) @@ -580,6 +583,9 @@ testCompareXMLToArgv(const void *data) virSetConnectStorage(NULL); virObjectUnref(conn); VIR_FREE(migrateURI); + if (info->arch != VIR_ARCH_NONE && info->arch != VIR_ARCH_X86_64) + qemuTestSetHostArch(driver.caps, VIR_ARCH_NONE); + return ret; } @@ -699,13 +705,9 @@ mymain(void) __VA_ARGS__, ARG_END) < 0) \ return EXIT_FAILURE; \ testInfoSetPaths(&info, _suffix); \ - if (info.arch != VIR_ARCH_NONE && info.arch != VIR_ARCH_X86_64) \ - qemuTestSetHostArch(driver.caps, info.arch); \ if (virTestRun("QEMU XML-2-ARGV " _name _suffix, \ testCompareXMLToArgv, &info) < 0) \ ret = -1; \ - if (info.arch != VIR_ARCH_NONE && info.arch != VIR_ARCH_X86_64) \ - qemuTestSetHostArch(driver.caps, VIR_ARCH_NONE); \ testQemuInfoClear(&info); \ } while (0) -- 2.23.0

4 4

[libvirt] [PATCH] selinux: Drop needless getfilecon()-s
by Michal Privoznik 28 Nov '19

28 Nov '19

When SELinux support was first introduced the libselinux library wasn't that advanced and setfilecon_raw() or fsetfilecon_raw() could fail even when the target context was set. Looking at the current code [1][2] this is no longer the case. We can drop our workarounds. 1: https://github.com/SELinuxProject/selinux/blob/master/libselinux/src/setfil… 2: https://github.com/SELinuxProject/selinux/blob/master/libselinux/src/fsetfi… Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/security/security_selinux.c | 22 ---------------------- 1 file changed, 22 deletions(-) diff --git a/src/security/security_selinux.c b/src/security/security_selinux.c index e05ef7593e..0bfb6a7fa6 100644 --- a/src/security/security_selinux.c +++ b/src/security/security_selinux.c @@ -1249,8 +1249,6 @@ virSecuritySELinuxSetFileconImpl(const char *path, const char *tcon, bool privileged) { - security_context_t econ; - /* Be aware that this function might run in a separate process. * Therefore, any driver state changes would be thrown away. */ @@ -1259,15 +1257,6 @@ virSecuritySELinuxSetFileconImpl(const char *path, if (setfilecon_raw(path, (const char *)tcon) < 0) { int setfilecon_errno = errno; - if (getfilecon_raw(path, &econ) >= 0) { - if (STREQ(tcon, econ)) { - freecon(econ); - /* It's alright, there's nothing to change anyway. */ - return 1; - } - freecon(econ); - } - /* If the error complaint is related to an image hosted on a (possibly * read-only) NFS mount, or a usbfs/sysfs filesystem not supporting * labelling, then just ignore it & hope for the best. The user @@ -1401,22 +1390,11 @@ virSecuritySELinuxSetFilecon(virSecurityManagerPtr mgr, static int virSecuritySELinuxFSetFilecon(int fd, char *tcon) { - security_context_t econ; - VIR_INFO("Setting SELinux context on fd %d to '%s'", fd, tcon); if (fsetfilecon_raw(fd, tcon) < 0) { int fsetfilecon_errno = errno; - if (fgetfilecon_raw(fd, &econ) >= 0) { - if (STREQ(tcon, econ)) { - freecon(econ); - /* It's alright, there's nothing to change anyway. */ - return 0; - } - freecon(econ); - } - /* if the error complaint is related to an image hosted on * an nfs mount, or a usbfs/sysfs filesystem not supporting * labelling, then just ignore it & hope for the best. -- 2.23.0

2 1

[libvirt] Change virQEMUDriverConfig defaults?
by Michal Privoznik 28 Nov '19

28 Nov '19

Dear list, lately I was confronted with the way how KubeVirt uses libvirt. They run it inside a docker container with UID mapping and some capabilities dropped. See [1] for more info. Problem is that because of the way we initialize our defaults they had to disable some features. Specifically, virQEMUDriverConfigNew() is passed @privileged argument which equals geteuid() == 0. Apparently, due to UID mapping this leads us to create not valid defaults. For instance, rememberOwner will be set to true in this case, but trying to write any XATTR is going to fail because the container and subsequently libvirtd does have CAP_SYS_ADMIN (and writing trusted.* XATTRs is reserve to CAP_SYS_ADMIN). Similarly, dynamicOwnership won't work unless CAP_CHOWN is granted and if SELinux driver is enabled then we will also need CAP_SYS_ADMIN because setfilecon_raw() does nothing more than write XATTR [2]. So instead of using @privileged we could use capabilities to enable/disable features by default. There is one problem with this though - do we know the full set of capabilities we need for given feature? For instance, does rememberOwner needs just CAP_SYS_ADMIN or also some other capability? Users could still override our defaults through qemu.conf, but I can see us throwing a warning if they are trying to enable say rememberOwner while the libvird lacks CAP_SYS_ADMIN. What is your opinion? Michal 1: https://bugzilla.redhat.com/show_bug.cgi?id=1774373#c17 2: https://github.com/SELinuxProject/selinux/blob/master/libselinux/src/setfil…

2 1

[libvirt] [PATCH for 5.10.0] docs: don't run xmllint when generating API HTML files
by Daniel P. Berrangé 28 Nov '19

28 Nov '19

After generating the API HTML files we run xmllint in docs/html/*.html to validate the correctness. Since commit 0aa8536f14b08dfb5188d0bb3aba532680500d38 Author: Daniel P. Berrangé <berrange(a)redhat.com> Date: Wed Nov 20 14:49:26 2019 +0000 docs: generate API reference pages for admin, qemu & lxc libraries we have many rules generating files into docs/html/. The xmllint calls for each rule are picking up files which are part-generated by other parallel build rules resulting in transient errors like: GEN html/index.html GEN html/index-admin.html GEN html/index-qemu.html GEN html/index-lxc.html GEN hvsupport.html.in html/index-lxc.html:1: parser error : Document is empty ^ make[4]: *** [Makefile:2407: html/index-qemu.html] Error 1 The easiest solution is to move the xmllint rules to the 'make check' phase of the build. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- docs/Makefile.am | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/docs/Makefile.am b/docs/Makefile.am index 2a104bc837..0e43f382b1 100644 --- a/docs/Makefile.am +++ b/docs/Makefile.am @@ -311,16 +311,19 @@ html/index.html: libvirt-api.xml newapi.xsl page.xsl $(APIBUILD_STAMP) $(AM_V_GEN)$(XSLTPROC) --nonet -o ./ \ --stringparam builddir '$(abs_top_builddir)' \ --stringparam timestamp $(timestamp) \ - $(srcdir)/newapi.xsl libvirt-api.xml && \ - $(XMLLINT) --nonet --noout html/*.html + $(srcdir)/newapi.xsl libvirt-api.xml html/index-%.html: libvirt-%-api.xml newapi.xsl page.xsl $(APIBUILD_STAMP) $(AM_V_GEN)$(XSLTPROC) --nonet -o ./ \ --stringparam builddir '$(abs_top_builddir)' \ --stringparam timestamp $(timestamp) \ --stringparam indexfile $(@:html/%=%) \ - $(srcdir)/newapi.xsl $< && \ - $(XMLLINT) --nonet --noout html/*.html + $(srcdir)/newapi.xsl $< + +check-html: + $(XMLLINT) --nonet --noout html/*.html + +check-local: check-html python_generated_files = \ html/libvirt-libvirt-lxc.html \ -- 2.23.0

2 1

[libvirt] [python PATCH] generator.py: add mapping for VIR_DOMAIN_QEMU_AGENT_COMMAND_*
by Pavel Hrdina 28 Nov '19

28 Nov '19

Libvirt commit <95f5ac9ae52455e9da47afc95fa31c9456ac27ae> changed the VIR_DOMAIN_QEMU_AGENT_COMMAND_* enum values to use different enum values instead of direct numbers. We need to translate it back. Traceback (most recent call last): File "generator.py", line 2143, in <module> qemuBuildWrappers(sys.argv[1]) File "generator.py", line 2008, in qemuBuildWrappers items.sort(key=lambda i: (int(i[1]), i[0])) File "generator.py", line 2008, in <lambda> items.sort(key=lambda i: (int(i[1]), i[0])) ValueError: invalid literal for int() with base 10: 'VIR_DOMAIN_AGENT_RESPONSE_TIMEOUT_BLOCK' Signed-off-by: Pavel Hrdina <phrdina(a)redhat.com> --- generator.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/generator.py b/generator.py index 913dab8..3352521 100755 --- a/generator.py +++ b/generator.py @@ -261,6 +261,12 @@ def lxc_enum(type, name, value): def qemu_enum(type, name, value): if type not in qemu_enums: qemu_enums[type] = {} + if value == 'VIR_DOMAIN_AGENT_RESPONSE_TIMEOUT_BLOCK': + value = -2 + elif value == 'VIR_DOMAIN_AGENT_RESPONSE_TIMEOUT_DEFAULT': + value = -1 + elif value == 'VIR_DOMAIN_AGENT_RESPONSE_TIMEOUT_NOWAIT': + value = 0 if onlyOverrides and name not in qemu_enums[type]: return qemu_enums[type][name] = value -- 2.23.0

2 3

[libvirt] [PATCH] Fix bug libvirt daemon segfault when new force console vm session break down existed console session. When force console vm command arrived, libvirtd will break down existed console session, and in this procedure, it will call daemonStreamEvent() to release resources , so daemonStreamFilter() need to check if stream filter existed when get client object lock and client->privData's lock, if not existed, just return -1
by LanceLiu 28 Nov '19

28 Nov '19

--- src/libvirt_remote.syms | 1 + src/remote/remote_daemon_stream.c | 10 +++++++++- src/rpc/virnetserverclient.c | 12 ++++++++++++ src/rpc/virnetserverclient.h | 2 ++ 4 files changed, 24 insertions(+), 1 deletion(-) diff --git a/src/libvirt_remote.syms b/src/libvirt_remote.syms index 0493467..c32e234 100644 --- a/src/libvirt_remote.syms +++ b/src/libvirt_remote.syms @@ -173,6 +173,7 @@ virNetServerClientPreExecRestart; virNetServerClientRemoteAddrStringSASL; virNetServerClientRemoteAddrStringURI; virNetServerClientRemoveFilter; +virNetServerClientCheckFilterExist; virNetServerClientSendMessage; virNetServerClientSetAuthLocked; virNetServerClientSetAuthPendingLocked; diff --git a/src/remote/remote_daemon_stream.c b/src/remote/remote_daemon_stream.c index 82cadb6..de0dca3 100644 --- a/src/remote/remote_daemon_stream.c +++ b/src/remote/remote_daemon_stream.c @@ -292,10 +292,18 @@ daemonStreamFilter(virNetServerClientPtr client, { daemonClientStream *stream = opaque; int ret = 0; + daemonClientPrivatePtr priv = NULL; + int filter_id = stream->filterID; virObjectUnlock(client); + priv = virNetServerClientGetPrivateData(client); virMutexLock(&stream->priv->lock); virObjectLock(client); + if (!virNetServerClientCheckFilterExist(client, filter_id)) { + VIR_WARN("this daemon stream filter: %d have been deleted!", filter_id); + ret = -1; + goto cleanup; + } if (msg->header.type != VIR_NET_STREAM && msg->header.type != VIR_NET_STREAM_HOLE) @@ -317,7 +325,7 @@ daemonStreamFilter(virNetServerClientPtr client, ret = 1; cleanup: - virMutexUnlock(&stream->priv->lock); + virMutexUnlock(&priv->lock); return ret; } diff --git a/src/rpc/virnetserverclient.c b/src/rpc/virnetserverclient.c index 67b3bf9..f80f493 100644 --- a/src/rpc/virnetserverclient.c +++ b/src/rpc/virnetserverclient.c @@ -287,6 +287,18 @@ void virNetServerClientRemoveFilter(virNetServerClientPtr client, virObjectUnlock(client); } +int virNetServerClientCheckFilterExist(virNetServerClientPtr client, + int filterID) +{ + virNetServerClientFilterPtr tmp; + + tmp = client->filters; + while(tmp && tmp->id != filterID) { + tmp = tmp->next; + } + + return (tmp != NULL); +} /* Check the client's access. */ static int diff --git a/src/rpc/virnetserverclient.h b/src/rpc/virnetserverclient.h index 7a3061d..85fda39 100644 --- a/src/rpc/virnetserverclient.h +++ b/src/rpc/virnetserverclient.h @@ -93,6 +93,8 @@ int virNetServerClientAddFilter(virNetServerClientPtr client, void virNetServerClientRemoveFilter(virNetServerClientPtr client, int filterID); +int virNetServerClientCheckFilterExist(virNetServerClientPtr client, + int filterID); int virNetServerClientGetAuth(virNetServerClientPtr client); void virNetServerClientSetAuthLocked(virNetServerClientPtr client, int auth); -- 1.8.3.1

3 7

[libvirt] [PATCH] travis: explicitly require xcode 10.3 image for macOS
by Daniel P. Berrangé 28 Nov '19

28 Nov '19

The default macOS image in travis is broken, throwing python exceptions when trying to install glib. Explicitly ask for the newer 10.3 image which works correctly. We now need to also point to the homebrew installed libxml2 rather than the OS distro provided one, since the OS distro one has a pkg-config file present, but no actual header files. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- .travis.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.travis.yml b/.travis.yml index 75f32e53b9..80f3be5490 100644 --- a/.travis.yml +++ b/.travis.yml @@ -62,8 +62,10 @@ matrix: - compiler: clang language: c os: osx + osx_image: xcode10.3 env: - PATH="/usr/local/opt/gettext/bin:/usr/local/opt/ccache/libexec:/usr/local/opt/rpcgen/bin:$PATH" + - PKG_CONFIG_PATH="/usr/local/opt/libxml2/lib/pkgconfig" script: # We can't run 'distcheck' or 'syntax-check' because they fail on # macOS, but doing 'install' and 'dist' gives us some useful coverage -- 2.21.0

2 1

[libvirt] [PATCH for 5.10.0] news: Update for 5.10.0 release
by Michal Privoznik 27 Nov '19

27 Nov '19

Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- docs/news.xml | 124 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 124 insertions(+) diff --git a/docs/news.xml b/docs/news.xml index 72c0edb317..b6782df136 100644 --- a/docs/news.xml +++ b/docs/news.xml @@ -81,8 +81,132 @@ </change> </section> <section title="Improvements"> + <change> + <summary> + Devices CGroup v2 support + </summary> + <description> + Libvirt supported all controllers of CGroup v2 but the devices + controller which is implemented in this release. + </description> + </change> + <change> + <summary> + Cold plug of sound device + </summary> + <description> + The QEMU driver now can handle cold plug of + <code><sound/></code> devices. + </description> + </change> + <change> + <summary> + More GLib integration + </summary> + <description> + More patches were merged that replace our internal functions with + GLib ones. Also some effort was invested in replacing gnulib modules + with GLib functions. + </description> + </change> + <change> + <summary> + Rewrite of Perl scripts into Python + </summary> + <description> + Libvirt used Perl scripts to check for coding style, generate some + code and things like that. To bring the number of languages used + down, these scripts were rewritten into Python. + </description> + </change> + <change> + <summary> + Probe for default CPU types + </summary> + <description> + With QEMU 4.2.0 we can probe for the default CPU model used by QEMU + for a particular machine type and store it in the domain XML. This + way the chosen CPU model is more visible to users and libvirt will + make sure the guest will see the exact same CPU after migration. + </description> + </change> + <change> + <summary> + Adaptation to qemu's blockdev + </summary> + <description> + QEMU introduced a new way of specifying disks on the command line + which enables fine-grained control over the block stack. Libvirt has + adapted to this. + </description> + </change> </section> <section title="Bug fixes"> + <change> + <summary> + Warn verbosely if using old loader:nvram pairs + </summary> + <description> + Some distributions still use <code>--with-loader-nvram</code> or + <code>nvram</code> variable in qemu.conf. This is now discouraged in + favour of FW descriptors. However, instead of silently ignoring user's + config, libvirt warns if outdated config is detected. + </description> + </change> + <change> + <summary> + Drop pconfig from Icelake-Server CPU model + </summary> + <description> + The pconfig feature was enabled in QEMU by accident in 3.1.0. All + other newer versions do not support it and it was removed from the + Icelake-Server CPU model in QEMU. + </description> + </change> + <change> + <summary> + Wait longer for device removal confirmation on PPC64 + </summary> + <description> + After sending device hot unplug request to QEMU, libvirt waits up to + 5 seconds for qemu to confirm the device removal. On some + architectures (like PPC64) this can take longer time and libvirt now + reflects that. + </description> + </change> + <change> + <summary> + Forcibly create nodes in domain's namespace + </summary> + <description> + The QEMU driver starts a domain in a namepsace with private + <code>/dev</code> and creates only those nodes there which the domain + is configured to have. However, it may have happened that if a node + changed its minor number this change wasn't propagated to the + namespace. + </description> + </change> + <change> + <summary> + Various AppArmor bugfixes + </summary> + <description> + The AppArmor driver now knows how to handle + <code><shmem/></code> devices and also snapshotting more disks + at once. + </description> + </change> + <change> + <summary> + Improved video model autoselection + </summary> + <description> + If a graphics device was added to XML that had no video device, + libvirt automatically added a video device which was always of type + 'cirrus' on x86_64, even if the underlying qemu didn't support + cirrus. Libvirt now bases the decision on qemu's capabilities. + </description> + </change> </section> </release> <release version="v5.9.0" date="2019-11-05"> -- 2.23.0

4 7

[libvirt] [PATCH for 5.10] Revert "qemu: Don't repeat virDomainObjEndAPI in qemuDomainBlockPull"
by Peter Krempa 27 Nov '19

27 Nov '19

This reverts commit 421c9550f5446729b513ee50f5c44e6f6969b5a2. qemuDomainBlockPullCommon calls virDomainObjEndAPI internally so the original commit made us shed two references of @vm instead of one getting us into a premature free of @vm. This is not a straight revert as qemuDomainBlockPull was modified meanwhile. I've also added a warning comment that @vm is consumed. https://bugzilla.redhat.com/show_bug.cgi?id=1777230 Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- /me hides in shame src/qemu/qemu_driver.c | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 669c12d6ca..8c2670d377 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -18443,24 +18443,23 @@ qemuDomainBlockPull(virDomainPtr dom, const char *path, unsigned long bandwidth, unsigned int flags) { virDomainObjPtr vm; - int ret = -1; - virCheckFlags(VIR_DOMAIN_BLOCK_PULL_BANDWIDTH_BYTES, -1); if (!(vm = qemuDomainObjFromDomain(dom))) return -1; - if (virDomainBlockPullEnsureACL(dom->conn, vm->def) < 0) - goto cleanup; - - if (qemuDomainSupportsCheckpointsBlockjobs(vm) < 0) - goto cleanup; + if (virDomainBlockPullEnsureACL(dom->conn, vm->def) < 0) { + virDomainObjEndAPI(&vm); + return -1; + } - ret = qemuDomainBlockPullCommon(vm, path, NULL, bandwidth, flags); + if (qemuDomainSupportsCheckpointsBlockjobs(vm) < 0) { + virDomainObjEndAPI(&vm); + return -1; + } - cleanup: - virDomainObjEndAPI(&vm); - return ret; + /* qemuDomainBlockPullCommon consumes the reference on @vm */ + return qemuDomainBlockPullCommon(vm, path, NULL, bandwidth, flags); } -- 2.23.0

2 1

[libvirt] configuraton error while configuring libvirt
by Billa Surendra 27 Nov '19

27 Nov '19

HI all, I am trying to install libvirt-5.4.0 in fedora 31 host pc, but I am > getting error like *configure: error: You must install the gnutls >= > 3.2.0 pkg-config module to compile libvirt*. I have higher version of > gnutls avilable in my host pc already. Any one tell me how to install > libvirt in fedora and how to resolve above error. Is there any perfect > steps for installation of libvirt. > Thanks In Advance

2 1

[libvirt] [PATCH 0/4] replace setenv and unsetenv by Glib functions (kill-a-gnulib-module-a-day initiative)
by Pavel Hrdina 27 Nov '19

27 Nov '19

2 6

[libvirt] [PATCH 00/10] qemu: Fixes of the blockdev job handling (a blockdev-add series)
by Peter Krempa 27 Nov '19

27 Nov '19

While playing with the incremental backup stuff I came across few corner cases in the blockdev job handling. Peter Krempa (10): qemu: monitor: Finish implementation of infrastructure for 'query-jobs' qemu: blockjob: Properly propagate cancellation of blockjobs qemu: process: Move block job refresh after async job recovery qemu: blockjob: Fix deadlock when terminating job with invalid data qemu: blockjob: Log blockjobs which are dropped when untracked by qemu qemu: blockjob: Mark job with broken data but tracked by qemu as reconnected qemu: blockjob: Don't stop processing the finished job early qemu: blockjob: Separate clearing of per-job data qemu: blockjob: Introduce "broken" block job type qemu: blockjob: Finish handling job with broken data src/qemu/qemu_blockjob.c | 70 ++++++++++++------- src/qemu/qemu_blockjob.h | 3 + src/qemu/qemu_domain.c | 13 +++- src/qemu/qemu_driver.c | 1 + src/qemu/qemu_monitor.h | 4 +- src/qemu/qemu_monitor_json.c | 6 ++ src/qemu/qemu_process.c | 6 +- .../blockjob-blockdev-in.xml | 1 + 8 files changed, 73 insertions(+), 31 deletions(-) -- 2.23.0

2 20

[libvirt] [PATCH 0/9] domain job stats handling improvements (incremental backup prequels)
by Peter Krempa 26 Nov '19

26 Nov '19

Improve few aspecs of the domain job stats handling. I felt these make life simpler when using the domain stats specifically when trying to figure out how stuff works. Peter Krempa (9): virsh: domain: Extract the code converting domain job stats to virDomainJobInfo api: Allow keeping completed domain job stats when reading them virsh: Implement VIR_DOMAIN_JOB_STATS_COMPLETED_KEEP for 'domjobinfo' qemu: Implement VIR_DOMAIN_JOB_STATS_COMPLETED_KEEP virsh: domjobinfo: Print also job operation for failed jobs virsh: domjobinfo: Allow printing stats also for failed and other jobs virsh: domjobinfo: Add switch to print raw fields API: Introduce VIR_DOMAIN_JOB_SUCCESS field for virDomainGetJobStats qemu: Always reset @info in qemuDomainGetJobInfo include/libvirt/libvirt-domain.h | 9 +++ src/libvirt-domain.c | 6 +- src/qemu/qemu_driver.c | 9 ++- tools/virsh-domain.c | 124 +++++++++++++++++++++---------- tools/virsh.pod | 16 +++- 5 files changed, 115 insertions(+), 49 deletions(-) -- 2.23.0

2 18

[libvirt] [PATCH] util: whitelist Oracle ACFS as a shared filesystem
by Daniel P. Berrangé 26 Nov '19

26 Nov '19

The magic number is taken from the coreutils stat.c file since there is no constant for it in normal system headers. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- src/util/virfile.c | 8 +++++++- src/util/virfile.h | 1 + 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/src/util/virfile.c b/src/util/virfile.c index fca7ff9d35..3dd2b1a527 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -3426,6 +3426,8 @@ int virFilePrintf(FILE *fp, const char *msg, ...) # define QB_MAGIC 0x51626d6e # endif +# define VIR_ACFS_MAGIC 0x61636673 + # define PROC_MOUNTS "/proc/mounts" static int @@ -3578,6 +3580,9 @@ virFileIsSharedFSType(const char *path, if ((fstypes & VIR_FILE_SHFS_QB) && (f_type == QB_MAGIC)) return 1; + if ((fstypes & VIR_FILE_SHFS_ACFS) && + (f_type == VIR_ACFS_MAGIC)) + return 1; return 0; } @@ -3765,7 +3770,8 @@ int virFileIsSharedFS(const char *path) VIR_FILE_SHFS_CIFS | VIR_FILE_SHFS_CEPH | VIR_FILE_SHFS_GPFS| - VIR_FILE_SHFS_QB); + VIR_FILE_SHFS_QB | + VIR_FILE_SHFS_ACFS); } diff --git a/src/util/virfile.h b/src/util/virfile.h index 9a8709b52c..bcae40ee06 100644 --- a/src/util/virfile.h +++ b/src/util/virfile.h @@ -209,6 +209,7 @@ enum { VIR_FILE_SHFS_CEPH = (1 << 6), VIR_FILE_SHFS_GPFS = (1 << 7), VIR_FILE_SHFS_QB = (1 << 8), + VIR_FILE_SHFS_ACFS = (1 << 9), }; int virFileIsSharedFSType(const char *path, int fstypes) ATTRIBUTE_NONNULL(1); -- 2.21.0

2 1

[libvirt] [PATCH] libxl: fix typo in error message
by Daniel P. Berrangé 26 Nov '19

26 Nov '19

Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- src/libxl/libxl_conf.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libxl/libxl_conf.c b/src/libxl/libxl_conf.c index e6c6f5f6ad..1e9d5dc875 100644 --- a/src/libxl/libxl_conf.c +++ b/src/libxl/libxl_conf.c @@ -2384,7 +2384,7 @@ libxlDriverNodeGetInfo(libxlDriverPrivatePtr driver, virNodeInfoPtr info) if (virStrcpyStatic(info->model, virArchToString(hostarch)) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, - _("machine type %s too big for destination"), + _("host arch %s too big for destination"), virArchToString(hostarch)); goto cleanup; } -- 2.23.0

3 2

[libvirt] [PATCH] all: don't wait for driver lock during startup
by Michal Privoznik 26 Nov '19

26 Nov '19

There are two daemons that wait for acquiring their pid files: virtnetworkd and virtstoraged. This is undesirable as the idea is to quit early if unable to acquire the pid file. This was missed in v5.6.0-rc1~207. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/network/bridge_driver.c | 2 +- src/storage/storage_driver.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/network/bridge_driver.c b/src/network/bridge_driver.c index 07dba8cfe4..e360645969 100644 --- a/src/network/bridge_driver.c +++ b/src/network/bridge_driver.c @@ -761,7 +761,7 @@ networkStateInitialize(bool privileged, if ((network_driver->lockFD = virPidFileAcquire(network_driver->stateDir, "driver", - true, getpid())) < 0) + false, getpid())) < 0) goto error; /* if this fails now, it will be retried later with dnsmasqCapsRefresh() */ diff --git a/src/storage/storage_driver.c b/src/storage/storage_driver.c index d8355d3c3c..580a5e6f15 100644 --- a/src/storage/storage_driver.c +++ b/src/storage/storage_driver.c @@ -296,7 +296,7 @@ storageStateInitialize(bool privileged, if ((driver->lockFD = virPidFileAcquire(driver->stateDir, "driver", - true, getpid())) < 0) + false, getpid())) < 0) goto error; if (virStoragePoolObjLoadAllState(driver->pools, -- 2.23.0

3 2

[libvirt] ANNOUNCE: libvirt-glib release 3.0.0
by Daniel P. Berrangé 26 Nov '19

26 Nov '19

I am pleased to announce that a new release of the libvirt-glib package, version 3.0.0, is now available from https://libvirt.org/sources/glib/ The packages are GPG signed with Key fingerprint: DAF3 A6FD B26B 6291 2D0E 8E3F BE86 EBB4 1510 4FDF (4096R) Changes in this release: * Add support for bochs video device * Add API to query firmware config * Improve testing coverage * Validate min/max glib API versions in use * Remove deprecated G_PARAM_PRIVATE * Fix docs build linking problems * Convert python demos to be python 3 compatible & use modern best practice for pyobject introspection bindings * Add API to query domain capaibilities * Refresh translations * Simplify build process for handling translations * Fix some memory leaks * Add API for setting storage volume features Thanks to everyone who contributed to this new release through testing, patches, bug reports, translations and more. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

1 0

[libvirt] [PATCH 0/2] driver: change scope of virDomainGetHostname to
by jcfaracco＠gmail.com 26 Nov '19

26 Nov '19

From: Julio Faracco <jcfaracco(a)gmail.com> The first patch add a new parameter into virDomainGetHostname to specify which source an user would like to fetch hostname. QEMU supports agent and lease from dnsmasq. Drivers that are not supporting any of these, are included into default section of switch-case structure. So, they are not affected by missing source argument. The second patch, introduce this missing command for LXC. LXC could retrieve hostname from 'lxc.uts.name' setting. But an user can change it during container execution. It could be get from lease information. Julio Faracco (2): driver: Include source parameter to virDomainGetHostname lxc: Introduce lxcDomainGetHostname include/libvirt/libvirt-domain.h | 10 +++++ src/driver-hypervisor.h | 1 + src/libvirt-domain.c | 7 +++- src/lxc/lxc_driver.c | 71 ++++++++++++++++++++++++++++++++ src/openvz/openvz_driver.c | 30 ++++++++++---- src/qemu/qemu_driver.c | 67 +++++++++++++++++++++++++----- src/remote/remote_protocol.x | 1 + src/remote_protocol-structs | 1 + src/test/test_driver.c | 1 + tools/virsh-domain.c | 25 ++++++++++- 10 files changed, 191 insertions(+), 23 deletions(-) -- 2.20.1

5 7

[libvirt] [jenkins-ci] guests: Add support for openSUSE Leap 15.1
by Jim Fehlig 26 Nov '19

26 Nov '19

Signed-off-by: Jim Fehlig <jfehlig(a)suse.com> --- guests/configs/autoinst.xml | 86 +++++++++++++++++++ .../libvirt-opensuse-15.1/docker.yml | 2 + .../libvirt-opensuse-15.1/install.yml | 2 + .../host_vars/libvirt-opensuse-15.1/main.yml | 22 +++++ guests/inventory | 1 + guests/lcitool | 2 + guests/vars/mappings.yml | 38 +++++++- 7 files changed, 151 insertions(+), 2 deletions(-) diff --git a/guests/configs/autoinst.xml b/guests/configs/autoinst.xml new file mode 100644 index 0000000..9b93a29 --- /dev/null +++ b/guests/configs/autoinst.xml @@ -0,0 +1,86 @@ +<?xml version="1.0"?> +<!DOCTYPE profile> +<profile + xmlns="http://www.suse.com/1.0/yast2ns" + xmlns:config="http://www.suse.com/1.0/configns"> + <general> + <mode> + <confirm config:type="boolean">false</confirm> + </mode> + </general> + <partitioning config:type="list"> + <drive> + <device>/dev/vda</device> + <use>all</use> + <partitions config:type="list"> + <partition> + <filesystem config:type="symbol">swap</filesystem> + <size>500M</size> + <mount>swap</mount> + </partition> + <partition> + <filesystem config:type="symbol">ext4</filesystem> + <mount>/</mount> + <size>max</size> + </partition> + </partitions> + </drive> + </partitioning> + <timezone> + <hwclock>UTC</hwclock> + <timezone>UTC</timezone> + </timezone> + <add-on> + <add_on_products config:type="list"> + <listentry> + <media_url>http://download.opensuse.org/distribution/leap/15.1/repo/oss/</media_url> + <name>repo-oss</name> + </listentry> + <listentry> + <media_url>http://download.opensuse.org/update/leap/15.1/oss</media_url> + <name>repo-update</name> + </listentry> + <listentry> + <media_url>http://download.opensuse.org/distribution/leap/15.1/repo/non-oss/</media_url> + <name>repo-non-oss</name> + </listentry> + <listentry> + <media_url>http://download.opensuse.org/update/leap/15.1/non-oss/</media_url> + <name>repo-update-non-oss</name> + </listentry> + </add_on_products> + </add-on> + <software> + <install_recommended config:type="boolean">false</install_recommended> + <patterns config:type="list"> + <pattern>base</pattern> + <pattern>minimal_base</pattern> + <pattern>yast2_basis</pattern> + </patterns> + </software> + <networking> + <keep_install_network config:type="boolean">true</keep_install_network> + </networking> + <users config:type="list"> + <user> + <username>root</username> + <user_password>root</user_password> + <encrypted config:type="boolean">false</encrypted> + <uid>0</uid> + <gid>0</gid> + <home>/root</home> + <shell>/bin/bash</shell> + </user> + </users> + <services-manager> + <default_target>multi-user</default_target> + <services> + <enable config:type="list"> + <service>sshd</service> + </enable> + </services> + </services-manager> + <firewall> + <enable_firewall>true</enable_firewall> + </firewall> +</profile> diff --git a/guests/host_vars/libvirt-opensuse-15.1/docker.yml b/guests/host_vars/libvirt-opensuse-15.1/docker.yml new file mode 100644 index 0000000..1fcbc38 --- /dev/null +++ b/guests/host_vars/libvirt-opensuse-15.1/docker.yml @@ -0,0 +1,2 @@ +--- +docker_base: openSUSE:15.1 diff --git a/guests/host_vars/libvirt-opensuse-15.1/install.yml b/guests/host_vars/libvirt-opensuse-15.1/install.yml new file mode 100644 index 0000000..d0fdbe5 --- /dev/null +++ b/guests/host_vars/libvirt-opensuse-15.1/install.yml @@ -0,0 +1,2 @@ +--- +install_url: http://download.opensuse.org/distribution/leap/15.1/repo/oss/ diff --git a/guests/host_vars/libvirt-opensuse-15.1/main.yml b/guests/host_vars/libvirt-opensuse-15.1/main.yml new file mode 100644 index 0000000..cdd4509 --- /dev/null +++ b/guests/host_vars/libvirt-opensuse-15.1/main.yml @@ -0,0 +1,22 @@ +--- +projects: + - libosinfo + - libvirt + - libvirt-cim + - libvirt-dbus + - libvirt-glib + - libvirt-perl + - libvirt-python + - libvirt-sandbox + - libvirt-tck + - osinfo-db + - osinfo-db-tools + - virt-manager + - virt-viewer + +package_format: 'rpm' +package_manager: 'zypper' +os_name: 'openSUSE' +os_version: '15.1' + +ansible_python_interpreter: /usr/bin/python3 diff --git a/guests/inventory b/guests/inventory index 3b15513..c86f8cc 100644 --- a/guests/inventory +++ b/guests/inventory @@ -8,5 +8,6 @@ libvirt-fedora-rawhide libvirt-freebsd-11 libvirt-freebsd-12 libvirt-freebsd-current +libvirt-opensuse-15.1 libvirt-ubuntu-16 libvirt-ubuntu-18 diff --git a/guests/lcitool b/guests/lcitool index a630971..39e6853 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -531,6 +531,8 @@ class Application: install_config = "preseed.cfg" elif facts["os_name"] in ["CentOS", "Fedora"]: install_config = "kickstart.cfg" + elif facts["os_name"] == "openSUSE": + install_config = "autoinst.xml" else: raise Exception( "Host {} doesn't support installation".format(host) diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index bd9b161..ca8eb84 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -19,10 +19,10 @@ # - deb, pkg, rpm # # Valid OS names are: -# - CentOS, Debian, Fedora, FreeBSD, Ubuntu +# - CentOS, Debian, Fedora, FreeBSD, openSUSE, Ubuntu # # Valid OS versions are: -# - CentOS7, Debian9, FedoraRawhide, Ubuntu18 and so on +# - CentOS7, Debian9, FedoraRawhide, openSUSE15.1, Ubuntu18 and so on # # The arch specific rules use a prefix "$ARCH-" where $ARCH # is a libvirt arch name. @@ -70,6 +70,7 @@ mappings: apparmor: deb: libapparmor-dev + openSUSE: libapparmor-devel cross-policy-deb: foreign augeas: @@ -117,6 +118,7 @@ mappings: cppi: Fedora: cppi FreeBSD: cppi + openSUSE: cppi cyrus-sasl: deb: libsasl2-dev @@ -127,6 +129,7 @@ mappings: dbus-daemon: default: dbus Fedora: dbus-daemon + openSUSE: dbus-1 device-mapper: deb: libdevmapper-dev @@ -196,6 +199,7 @@ mappings: deb: libglusterfs-dev rpm: glusterfs-api-devel Debian9: glusterfs-common + openSUSE: glusterfs-devel Ubuntu16: glusterfs-common Ubuntu18: glusterfs-common cross-policy-deb: foreign @@ -204,6 +208,7 @@ mappings: deb: libgnutls28-dev pkg: gnutls rpm: gnutls-devel + openSUSE: libgnutls-devel cross-policy-deb: foreign go: @@ -228,6 +233,7 @@ mappings: gtk-update-icon-cache: default: gtk-update-icon-cache + openSUSE: gtk3-tools Ubuntu16: libgtk2.0-bin gtk-vnc2: @@ -248,14 +254,17 @@ mappings: ip: deb: iproute2 rpm: iproute + openSUSE: iproute2 iscsiadm: deb: open-iscsi rpm: iscsi-initiator-utils + openSUSE: open-iscsi isoinfo: default: genisoimage FreeBSD: cdrkit + openSUSE: mkisofs java: deb: openjdk-11-jre-headless @@ -289,6 +298,7 @@ mappings: libaudit: deb: libaudit-dev rpm: audit-libs-devel + openSUSE: audit-devel cross-policy-deb: foreign libblkid: @@ -320,6 +330,7 @@ mappings: deb: libdbus-1-dev pkg: dbus rpm: dbus-devel + openSUSE: dbus-1-devel cross-policy-deb: foreign libgovirt: @@ -347,6 +358,7 @@ mappings: rpm: numactl-devel armv6l-deb: armv7l-deb: + openSUSE: libnuma-devel cross-policy-deb: foreign libparted: @@ -370,6 +382,7 @@ mappings: deb: librbd-dev Fedora: librbd-devel CentOS7: librbd1-devel + openSUSE: librbd-devel cross-policy-deb: foreign libselinux: @@ -436,6 +449,7 @@ mappings: deb: locales Fedora: glibc-langpack-en FreeBSD: + openSUSE: glibc-locale lsof: default: lsof @@ -458,6 +472,7 @@ mappings: ninja: default: ninja-build FreeBSD: ninja + openSUSE: ninja mingw32-curl: Fedora: mingw32-curl @@ -639,6 +654,7 @@ mappings: netcf: deb: libnetcf-dev rpm: netcf-devel + openSUSE: cross-policy-deb: skip net-tools: @@ -678,6 +694,7 @@ mappings: deb: perl pkg: perl5 rpm: perl-Archive-Tar + openSUSE: perl-Archive-Tar-Wrapper perl-CPAN-Changes: deb: libcpan-changes-perl @@ -709,6 +726,7 @@ mappings: deb: libio-compress-perl pkg: p5-IO-Compress rpm: perl-IO-Compress-Bzip2 + openSUSE: perl-Compress-Bzip2 perl-IO-String: deb: libio-string-perl @@ -747,6 +765,7 @@ mappings: deb: libtest-lwp-useragent-perl pkg: p5-Test-LWP-UserAgent Fedora: perl-Test-LWP-UserAgent + openSUSE: perl-Test-LWP-UserAgent perl-Test-Pod: deb: libtest-pod-perl @@ -809,20 +828,24 @@ mappings: deb: python-dev pkg: python2 rpm: python2-devel + openSUSE: python-devel cross-policy-deb: foreign python2-lxml: default: python-lxml Fedora: python2-lxml FreeBSD: py27-lxml + openSUSE: python2-lxml python2-nose: default: python-nose Fedora: python2-nose FreeBSD: py27-nose + openSUSE: python2-nose python2-setuptools: CentOS7: python2-setuptools + openSUSE: python2-setuptools python3: default: python3 @@ -831,6 +854,7 @@ mappings: default: python3-dbus FreeBSD: py36-dbus CentOS7: python36-dbus + openSUSE: python3-dbus-python python3-devel: deb: python3-dev @@ -838,17 +862,20 @@ mappings: Fedora: python3-devel CentOS7: python36-devel cross-policy-deb: foreign + openSUSE: python3-devel python3-gi: deb: python3-gi pkg: py36-gobject3 rpm: python3-gobject CentOS7: python36-gobject + openSUSE: python3-gobject python3-libxml2: default: python3-libxml2 FreeBSD: py36-libxml2 CentOS7: + openSUSE: python3-libxml2-python Ubuntu16: python3-lxml: @@ -864,6 +891,7 @@ mappings: python3-pip: CentOS7: python3-pip Debian9: python3-pip + openSUSE: python3-pip Ubuntu16: python3-pip Ubuntu18: python3-pip @@ -929,6 +957,7 @@ mappings: deb: libspice-client-gtk-3.0-dev pkg: spice-gtk rpm: spice-gtk3-devel + openSUSE: spice-gtk-devel cross-policy-deb: foreign strace: @@ -942,6 +971,7 @@ mappings: deb: iproute2 rpm: iproute-tc CentOS7: iproute + openSUSE: iproute2 unzip: default: unzip @@ -964,10 +994,12 @@ mappings: wireshark: deb: wireshark-dev Fedora: wireshark-devel + openSUSE: wireshark-devel cross-policy-deb: skip xen: Fedora: xen-devel + openSUSE: xen-devel x86_64-deb: libxen-dev armv7l-deb: libxen-dev aarch64-deb: libxen-dev @@ -994,6 +1026,7 @@ mappings: xz-static: deb: liblzma-dev Fedora: xz-static + openSUSE: xz-static-devel cross-policy-deb: foreign yajl: @@ -1015,6 +1048,7 @@ mappings: zlib-static: deb: zlib1g-dev rpm: zlib-static + openSUSE: zlib-devel-static cross-policy-deb: foreign -- 2.23.0

3 18

[libvirt] [PATCH 0/4] qemu: Use host-model CPU on s390 by default
by Jiri Denemark 25 Nov '19

25 Nov '19

On s390 machines host-passthrough and host-model CPUs result in the same guest ABI (with QEMU new enough to be able to tell us what "host" CPU is expanded to, which was implemented around 2.9.0). So instead of using host-passthrough CPU when there's no CPU specified in a domain XML we can safely use host-model and benefit from CPU compatibility checks during migration, snapshot restore and similar operations. This series applies on top of "qemu: Store default CPU in domain XML" which is already acked, but it's waiting for a QEMU patch to be applied to 4.2.0. You can fetch both series at once using git fetch https://gitlab.com/jirkade/libvirt cpu-default-type Jiri Denemark (4): cpu_conf: Fix default value for CPU match attribute cpu_conf: Don't format empty model for host-model CPUs cpu_s390: Don't check match attribute for host-model CPUs qemu: Use host-model CPU on s390 by default src/conf/cpu_conf.c | 25 ++++---------- src/conf/cpu_conf.h | 2 +- src/cpu/cpu_s390.c | 18 +++++----- src/qemu/qemu_domain.c | 33 ++++++++++++------- .../ppc64-host+guest-compat-none.xml | 4 +-- ...t-cpu-kvm-ccw-virtio-4.2.s390x-latest.args | 4 ++- .../cpu-check-default-partial.xml | 4 +-- .../cpu-host-model-features.xml | 1 - ...lt-cpu-kvm-ccw-virtio-4.2.s390x-latest.xml | 2 +- 9 files changed, 45 insertions(+), 48 deletions(-) -- 2.24.0

5 13

[libvirt] [PATCH 0/2] Fix a VM startup failure: QOS must be defined for network 'default'
by Erik Skultety 25 Nov '19

25 Nov '19

Caused by patching a coverity false positive in commit f4db846c32c0a1e99a0f62b340273e48f8a98ed3. Erik Skultety (2): Revert "network: Check for QOS before blindly using it" util: virNetDevBandwidthPlug: Drop ATTRIBUTE_UNUSED(4) src/network/bridge_driver.c | 14 -------------- src/util/virnetdevbandwidth.h | 3 +-- 2 files changed, 1 insertion(+), 16 deletions(-) -- 2.23.0

6 13

[libvirt] [PATCH] qemu_capabilities: Use proper free function for caps->cpuModels
by Michal Privoznik 25 Nov '19

25 Nov '19

The cpuModels member of _virQEMUCapsAccel struct is not a virObject but regular struct with a free function defined: qemuMonitorCPUDefsFree(). Use that when clearing parent structure instead of virObjectUnref() to avoid a memleak: ==212322== 57,275 (48 direct, 57,227 indirect) bytes in 3 blocks are definitely lost in loss record 623 of 627 ==212322== at 0x4838B86: calloc (vg_replace_malloc.c:762) ==212322== by 0x554A158: g_malloc0 (in /usr/lib64/libglib-2.0.so.0.6000.6) ==212322== by 0x17B14BF5: qemuMonitorCPUDefsNew (qemu_monitor.c:3587) ==212322== by 0x17B27BA7: qemuMonitorJSONGetCPUDefinitions (qemu_monitor_json.c:5616) ==212322== by 0x17B14B0B: qemuMonitorGetCPUDefinitions (qemu_monitor.c:3559) ==212322== by 0x17A6AFBB: virQEMUCapsFetchCPUDefinitions (qemu_capabilities.c:2571) ==212322== by 0x17A6B2CC: virQEMUCapsProbeQMPCPUDefinitions (qemu_capabilities.c:2629) ==212322== by 0x17A70C00: virQEMUCapsInitQMPMonitorTCG (qemu_capabilities.c:4769) ==212322== by 0x17A70DDF: virQEMUCapsInitQMPSingle (qemu_capabilities.c:4820) ==212322== by 0x17A70E99: virQEMUCapsInitQMP (qemu_capabilities.c:4848) ==212322== by 0x17A71044: virQEMUCapsNewForBinaryInternal (qemu_capabilities.c:4891) ==212322== by 0x17A7119C: virQEMUCapsNewData (qemu_capabilities.c:4923) Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/qemu/qemu_capabilities.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index e5f19ddcaa..f65af5c228 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -1798,7 +1798,7 @@ virQEMUCapsAccelClear(virQEMUCapsAccelPtr caps) VIR_FREE(caps->machineTypes); virQEMUCapsHostCPUDataClear(&caps->hostCPU); - virObjectUnref(caps->cpuModels); + qemuMonitorCPUDefsFree(caps->cpuModels); } -- 2.23.0

2 1

[libvirt] [PATCH v3 0/4] qemu: fix type of default video device
by Pavel Mores 25 Nov '19

25 Nov '19

This new version mostly integrates Cole's comments about the second version. Refactoring and behaviour change are now separate commits. Tests succeed for every individual patch in the series. Pavel Mores (4): qemu: default video device type selection algoritm moved into its own function qemu: prepare existing test for change of the default video device type qemu: the actual change of default video devide type selection algorithm qemu: added tests of the new default video type selection algorithm src/qemu/qemu_domain.c | 37 ++++++++++------ .../default-video-type-aarch64.xml | 16 +++++++ .../default-video-type-ppc64.xml | 16 +++++++ .../default-video-type-riscv64.xml | 16 +++++++ .../default-video-type-s390x.xml | 16 +++++++ .../default-video-type-x86_64-caps-test-0.xml | 17 ++++++++ .../default-video-type-x86_64-caps-test-1.xml | 17 ++++++++ tests/qemuxml2argvtest.c | 1 + ...ault-video-type-aarch64.aarch64-latest.xml | 42 +++++++++++++++++++ .../default-video-type-ppc64.ppc64-latest.xml | 31 ++++++++++++++ ...ault-video-type-riscv64.riscv64-latest.xml | 39 +++++++++++++++++ .../default-video-type-s390x.s390x-latest.xml | 32 ++++++++++++++ .../default-video-type-x86_64-caps-test-0.xml | 31 ++++++++++++++ .../default-video-type-x86_64-caps-test-1.xml | 31 ++++++++++++++ tests/qemuxml2xmltest.c | 10 ++++- 15 files changed, 339 insertions(+), 13 deletions(-) create mode 100644 tests/qemuxml2argvdata/default-video-type-aarch64.xml create mode 100644 tests/qemuxml2argvdata/default-video-type-ppc64.xml create mode 100644 tests/qemuxml2argvdata/default-video-type-riscv64.xml create mode 100644 tests/qemuxml2argvdata/default-video-type-s390x.xml create mode 100644 tests/qemuxml2argvdata/default-video-type-x86_64-caps-test-0.xml create mode 100644 tests/qemuxml2argvdata/default-video-type-x86_64-caps-test-1.xml create mode 100644 tests/qemuxml2xmloutdata/default-video-type-aarch64.aarch64-latest.xml create mode 100644 tests/qemuxml2xmloutdata/default-video-type-ppc64.ppc64-latest.xml create mode 100644 tests/qemuxml2xmloutdata/default-video-type-riscv64.riscv64-latest.xml create mode 100644 tests/qemuxml2xmloutdata/default-video-type-s390x.s390x-latest.xml create mode 100644 tests/qemuxml2xmloutdata/default-video-type-x86_64-caps-test-0.xml create mode 100644 tests/qemuxml2xmloutdata/default-video-type-x86_64-caps-test-1.xml -- 2.21.0

2 8

[libvirt] [PATCH] Use virNetServerClientImmediateClose() rather than virNetServerClientClose()
by LanceLiu 25 Nov '19

25 Nov '19

--- src/remote/remote_daemon_stream.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/src/remote/remote_daemon_stream.c b/src/remote/remote_daemon_stream.c index de0dca3..d206d12 100644 --- a/src/remote/remote_daemon_stream.c +++ b/src/remote/remote_daemon_stream.c @@ -141,7 +141,7 @@ daemonStreamEvent(virStreamPtr st, int events, void *opaque) (events & VIR_STREAM_EVENT_WRITABLE)) { if (daemonStreamHandleWrite(client, stream) < 0) { daemonRemoveClientStream(client, stream); - virNetServerClientClose(client); + virNetServerClientImmediateClose(client); goto cleanup; } } @@ -151,7 +151,7 @@ daemonStreamEvent(virStreamPtr st, int events, void *opaque) events = events & ~(VIR_STREAM_EVENT_READABLE); if (daemonStreamHandleRead(client, stream) < 0) { daemonRemoveClientStream(client, stream); - virNetServerClientClose(client); + virNetServerClientImmediateClose(client); goto cleanup; } /* If we detected EOF during read processing, @@ -176,7 +176,7 @@ daemonStreamEvent(virStreamPtr st, int events, void *opaque) if (daemonStreamHandleFinish(client, stream, msg) < 0) { virNetMessageFree(msg); daemonRemoveClientStream(client, stream); - virNetServerClientClose(client); + virNetServerClientImmediateClose(client); goto cleanup; } break; @@ -186,7 +186,7 @@ daemonStreamEvent(virStreamPtr st, int events, void *opaque) if (daemonStreamHandleAbort(client, stream, msg) < 0) { virNetMessageFree(msg); daemonRemoveClientStream(client, stream); - virNetServerClientClose(client); + virNetServerClientImmediateClose(client); goto cleanup; } break; @@ -205,7 +205,7 @@ daemonStreamEvent(virStreamPtr st, int events, void *opaque) stream->recvEOF = true; if (!(msg = virNetMessageNew(false))) { daemonRemoveClientStream(client, stream); - virNetServerClientClose(client); + virNetServerClientImmediateClose(client); goto cleanup; } msg->cb = daemonStreamMessageFinished; @@ -219,7 +219,7 @@ daemonStreamEvent(virStreamPtr st, int events, void *opaque) "", 0) < 0) { virNetMessageFree(msg); daemonRemoveClientStream(client, stream); - virNetServerClientClose(client); + virNetServerClientImmediateClose(client); goto cleanup; } } @@ -262,7 +262,7 @@ daemonStreamEvent(virStreamPtr st, int events, void *opaque) } daemonRemoveClientStream(client, stream); if (ret < 0) - virNetServerClientClose(client); + virNetServerClientImmediateClose(client); goto cleanup; } -- 1.8.3.1

3 2

[libvirt] Plans for next release
by Daniel Veillard 25 Nov '19

25 Nov '19

We are getting close to the end of the month, I suggest to enter freeze tomorrow Tuesday, the end of the week will be quiet due to US ThanksGiving, that should give plenty of time to test ! So RC2 on Thursday and if all goes well a release on Monday 2nd ! Hope this works for everybody, thanks, Daniel -- Daniel Veillard | Red Hat Developers Tools http://developer.redhat.com/ veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

1 0

[libvirt] [PATCH 0/2] qemu: cold-plug and cold-unplug of sound
by Jidong Xia 25 Nov '19

25 Nov '19

With this patch users can cold plug and unplug some sound devices. Jidong Xia (2): qemu: cold-plug of sound qemu: cold-unplug of sound src/conf/domain_conf.c | 61 ++++++++++++++++++++++++++++++++++++++++++++++++ src/conf/domain_conf.h | 2 ++ src/libvirt_private.syms | 2 ++ src/qemu/qemu_driver.c | 18 ++++++++++++-- 4 files changed, 81 insertions(+), 2 deletions(-) -- 1.8.3.1

2 4

[libvirt] [PATCH] Fix bug libvirt daemon segfault when new force console vm session break down existed console session. When force console vm command arrived, libvirtd will break down existed console session, and in this procedure, it will call daemonStreamEvent() to release resources , so daemonStreamFilter() need to check if stream filter existed when get client object lock and client->privData's lock, if not existed, just return -1
by LanceLiu 24 Nov '19

24 Nov '19

--- src/remote/remote_daemon_stream.c | 10 +++++++++- src/rpc/virnetserverclient.c | 12 ++++++++++++ src/rpc/virnetserverclient.h | 2 ++ 3 files changed, 23 insertions(+), 1 deletion(-) diff --git a/src/remote/remote_daemon_stream.c b/src/remote/remote_daemon_stream.c index 82cadb6..de0dca3 100644 --- a/src/remote/remote_daemon_stream.c +++ b/src/remote/remote_daemon_stream.c @@ -292,10 +292,18 @@ daemonStreamFilter(virNetServerClientPtr client, { daemonClientStream *stream = opaque; int ret = 0; + daemonClientPrivatePtr priv = NULL; + int filter_id = stream->filterID; virObjectUnlock(client); + priv = virNetServerClientGetPrivateData(client); virMutexLock(&stream->priv->lock); virObjectLock(client); + if (!virNetServerClientCheckFilterExist(client, filter_id)) { + VIR_WARN("this daemon stream filter: %d have been deleted!", filter_id); + ret = -1; + goto cleanup; + } if (msg->header.type != VIR_NET_STREAM && msg->header.type != VIR_NET_STREAM_HOLE) @@ -317,7 +325,7 @@ daemonStreamFilter(virNetServerClientPtr client, ret = 1; cleanup: - virMutexUnlock(&stream->priv->lock); + virMutexUnlock(&priv->lock); return ret; } diff --git a/src/rpc/virnetserverclient.c b/src/rpc/virnetserverclient.c index 67b3bf9..f80f493 100644 --- a/src/rpc/virnetserverclient.c +++ b/src/rpc/virnetserverclient.c @@ -287,6 +287,18 @@ void virNetServerClientRemoveFilter(virNetServerClientPtr client, virObjectUnlock(client); } +int virNetServerClientCheckFilterExist(virNetServerClientPtr client, + int filterID) +{ + virNetServerClientFilterPtr tmp; + + tmp = client->filters; + while(tmp && tmp->id != filterID) { + tmp = tmp->next; + } + + return (tmp != NULL); +} /* Check the client's access. */ static int diff --git a/src/rpc/virnetserverclient.h b/src/rpc/virnetserverclient.h index 7a3061d..85fda39 100644 --- a/src/rpc/virnetserverclient.h +++ b/src/rpc/virnetserverclient.h @@ -93,6 +93,8 @@ int virNetServerClientAddFilter(virNetServerClientPtr client, void virNetServerClientRemoveFilter(virNetServerClientPtr client, int filterID); +int virNetServerClientCheckFilterExist(virNetServerClientPtr client, + int filterID); int virNetServerClientGetAuth(virNetServerClientPtr client); void virNetServerClientSetAuthLocked(virNetServerClientPtr client, int auth); -- 1.8.3.1

1 0

[libvirt] [ruby PATCH] Fix default values for node_cpu_stats() and node_memory_stats()
by Stefano Garzarella 23 Nov '19

23 Nov '19

ruby_libvirt_value_to_int() returns 0 if the optional value is not defined, but in node_cpu_stats() and node_memory_stats() the default value of cpuNum and cellNum is -1. Reported-by: Charlie Smurthwaite <charlie(a)atech.media> Signed-off-by: Stefano Garzarella <sgarzare(a)redhat.com> --- ext/libvirt/connect.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/ext/libvirt/connect.c b/ext/libvirt/connect.c index 5932535..b2d041b 100644 --- a/ext/libvirt/connect.c +++ b/ext/libvirt/connect.c @@ -2079,7 +2079,12 @@ static VALUE libvirt_connect_node_cpu_stats(int argc, VALUE *argv, VALUE c) rb_scan_args(argc, argv, "02", &intparam, &flags); - tmp = ruby_libvirt_value_to_int(intparam); + if (NIL_P(intparam)) { + tmp = -1; + } + else { + tmp = ruby_libvirt_value_to_int(intparam); + } return ruby_libvirt_get_parameters(c, ruby_libvirt_value_to_uint(flags), (void *)&tmp, sizeof(virNodeCPUStats), @@ -2139,7 +2144,12 @@ static VALUE libvirt_connect_node_memory_stats(int argc, VALUE *argv, VALUE c) rb_scan_args(argc, argv, "02", &intparam, &flags); - tmp = ruby_libvirt_value_to_int(intparam); + if (NIL_P(intparam)) { + tmp = -1; + } + else { + tmp = ruby_libvirt_value_to_int(intparam); + } return ruby_libvirt_get_parameters(c, ruby_libvirt_value_to_uint(flags), (void *)&tmp, sizeof(virNodeMemoryStats), -- 2.21.0

3 4

Re: [libvirt] [edk2-discuss] [OVMF] resource assignment fails for passthrough PCI GPU
by Eduardo Habkost 22 Nov '19

22 Nov '19

(+Jiri, +libvir-list) On Fri, Nov 22, 2019 at 04:58:25PM +0000, Dr. David Alan Gilbert wrote: > * Laszlo Ersek (lersek(a)redhat.com) wrote: > > (+Dave, +Eduardo) > > > > On 11/22/19 00:00, dann frazier wrote: > > > On Tue, Nov 19, 2019 at 06:06:15AM +0100, Laszlo Ersek wrote: > > >> On 11/19/19 01:54, dann frazier wrote: > > >>> On Fri, Nov 15, 2019 at 11:51:18PM +0100, Laszlo Ersek wrote: > > >>>> On 11/15/19 19:56, dann frazier wrote: > > >>>>> Hi, > > >>>>> I'm trying to passthrough an Nvidia GPU to a q35 KVM guest, but UEFI > > >>>>> is failing to allocate resources for it. I have no issues if I boot w/ > > >>>>> a legacy BIOS, and it works fine if I tell the linux guest to do the > > >>>>> allocation itself - but I'm looking for a way to make this work w/ > > >>>>> OVMF by default. > > >>>>> > > >>>>> I posted a debug log here: > > >>>>> https://bugs.launchpad.net/ubuntu/+source/edk2/+bug/1849563/+attachment/530… > > >>>>> > > >>>>> Linux guest lspci output is also available for both seabios/OVMF boots here: > > >>>>> https://bugs.launchpad.net/ubuntu/+source/edk2/+bug/1849563 > > >>>> > > >>>> By default, OVMF exposes such a 64-bit MMIO aperture for PCI MMIO BAR > > >>>> allocation that is 32GB in size. The generic PciBusDxe driver collects, > > >>>> orders, and assigns / allocates the MMIO BARs, but it can work only out > > >>>> of the aperture that platform code advertizes. > > >>>> > > >>>> Your GPU's region 1 is itself 32GB in size. Given that there are further > > >>>> PCI devices in the system with further 64-bit MMIO BARs, the default > > >>>> aperture cannot accommodate everything. In such an event, PciBusDxe > > >>>> avoids assigning the largest BARs (to my knowledge), in order to > > >>>> conserve the most aperture possible, for other devices -- hence break > > >>>> the fewest possible PCI devices. > > >>>> > > >>>> You can control the aperture size from the QEMU command line. You can > > >>>> also do it from the libvirt domain XML, technically speaking. The knob > > >>>> is experimental, so no stability or compatibility guarantees are made. > > >>>> (That's also the reason why it's a bit of a hack in the libvirt domain XML.) > > >>>> > > >>>> The QEMU cmdline options is described in the following edk2 commit message: > > >>>> > > >>>> https://github.com/tianocore/edk2/commit/7e5b1b670c38 > > >>> > > >>> Hi Laszlo, > > >>> > > >>> Thanks for taking the time to describe this in detail! The -fw_cfg > > >>> option did avoid the problem for me. > > >> > > >> Good to hear, thanks. > > >> > > >>> I also noticed that the above > > >>> commit message mentions the existence of a 24GB card as a reasoning > > >>> behind choosing the 32GB default aperture. From what you say below, I > > >>> understand that bumping this above 64GB could break hosts w/ <= 37 > > >>> physical address bits. > > >> > > >> Right. > > >> > > >>> What would be the downside of bumping the > > >>> default aperture to, say, 48GB? > > >> > > >> The placement of the aperture is not trivial (please see the code > > >> comments in the linked commit). The base address of the aperture is > > >> chosen so that the largest BAR that can fit in the aperture may be > > >> naturally aligned. (BARs are whole powers of two.) > > >> > > >> The largest BAR that can fit in a 48 GB aperture is 32 GB. Therefore > > >> such an aperture would be aligned at 32 GB -- the lowest base address > > >> (dependent on guest RAM size) would be 32 GB. Meaning that the aperture > > >> would end at 32 + 48 = 80 GB. That still breaches the 36-bit phys > > >> address width. > > >> > > >> 32 GB is the largest aperture size that can work with 36-bit phys > > >> address width; that's the aperture that ends at 64 GB exactly. > > > > > > Thanks, yeah - now that I read the code comments that is clear (as > > > clear as it can be w/ my low level of base knowledge). In the commit you > > > mention Gerd (CC'd) had suggested a heuristic-based approach for > > > sizing the aperture. When you say "PCPU address width" - is that a > > > function of the available physical bits? > > > > "PCPU address width" is not a "function" of the available physical bits > > -- it *is* the available physical bits. "PCPU" simply stands for > > "physical CPU". > > > > > IOW, would that approach > > > allow OVMF to automatically grow the aperture to the max ^2 supported > > > by the host CPU? > > > > Maybe. > > > > The current logic in OVMF works from the guest-physical address space > > size -- as deduced from multiple factors, such as the 64-bit MMIO > > aperture size, and others -- towards the guest-CPU (aka VCPU) address > > width. The VCPU address width is important for a bunch of other purposes > > in the firmware, so OVMF has to calculate it no matter what. > > > > Again, the current logic is to calculate the highest guest-physical > > address, and then deduce the VCPU address width from that (and then > > expose it to the rest of the firmware). > > > > Your suggestion would require passing the PCPU (physical CPU) address > > width from QEMU/KVM into the guest, and reversing the direction of the > > calculation. The PCPU address width would determine the VCPU address > > width directly, and then the 64-bit PCI MMIO aperture would be > > calculated from that. > > > > However, there are two caveats. > > > > (1) The larger your guest-phys address space (as exposed through the > > VCPU address width to the rest of the firmware), the more guest RAM you > > need for page tables. Because, just before entering the DXE phase, the > > firmware builds 1:1 mapping page tables for the entire guest-phys > > address space. This is necessary e.g. so you can access any PCI MMIO BAR. > > > > Now consider that you have a huge beefy virtualization host with say 46 > > phys address bits, and a wimpy guest with say 1.5GB of guest RAM. Do you > > absolutely want tens of *terabytes* for your 64-bit PCI MMIO aperture? > > Do you really want to pay for the necessary page tables with that meager > > guest RAM? > > > > (Such machines do exist BTW, for example: > > > > http://mid.mail-archive.com/9BD73EA91F8E404F851CF3F519B14AA8036C67B5@DGGEMI… > > ) > > > > In other words, you'd need some kind of knob anyway, because otherwise > > your aperture could grow too *large*. > > > > > > (2) Exposing the PCPU address width to the guest may have nasty > > consequences at the QEMU/KVM level, regardless of guest firmware. For > > example, that kind of "guest enlightenment" could interfere with migration. > > > > If you boot a guest let's say with 16GB of RAM, and tell it "hey friend, > > have 40 bits of phys address width!", then you'll have a difficult time > > migrating that guest to a host with a CPU that only has 36-bits wide > > physical addresses -- even if the destination host has plenty of RAM > > otherwise, such as a full 64GB. > > > > There could be other QEMU/KVM / libvirt issues that I m unaware of > > (hence the CC to Dave and Eduardo). > > host physical address width gets messy. There are differences as well > between upstream qemu behaviour, and some downstreams. > I think the story is that: > > a) Qemu default: 40 bits on any host > b) -cpu blah,host-phys-bits=true to follow the host. > c) RHEL has host-phys-bits=true by default > > As you say, the only real problem with host-phys-bits is migration - > between say an E3 and an E5 xeon with different widths. The magic 40's > is generally wrong as well - I think it came from some ancient AMD, > but it's the default on QEMU TCG as well. Yes, and because it affects live migration ability, we have two constraints: 1) It needs to be exposed in the libvirt domain XML; 2) QEMU and libvirt can't choose a value that works for everybody (because neither QEMU or libvirt know where the VM might be migrated later). Which is why the BZ below is important: > > I don't think there's a way to set it in libvirt; > https://bugzilla.redhat.com/show_bug.cgi?id=1578278 is a bz asking for > that. > > IMHO host-phys-bits is actually pretty safe; and makes most sense in a > lot of cases. Yeah, it is mostly safe and makes sense, but messy if you try to migrate to a host with a different size. > > Dave > > > > Thanks, > > Laszlo > > > > > > > > -dann > > > > > >>>> For example, to set a 64GB aperture, pass: > > >>>> > > >>>> -fw_cfg name=opt/ovmf/X-PciMmio64Mb,string=65536 > > >>>> > > >>>> The libvirt domain XML syntax is a bit tricky (and it might "taint" your > > >>>> domain, as it goes outside of the QEMU features that libvirt directly > > >>>> maps to): > > >>>> > > >>>> <domain > > >>>> type='kvm' > > >>>> xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> > > >>>> <qemu:commandline> > > >>>> <qemu:arg value='-fw_cfg'/> > > >>>> <qemu:arg value='opt/ovmf/X-PciMmio64Mb,string=65536'/> > > >>>> </qemu:commandline> > > >>>> </domain> > > >>>> > > >>>> Some notes: > > >>>> > > >>>> (1) The "xmlns:qemu" namespace definition attribute in the <domain> root > > >>>> element is important. You have to add it manually when you add > > >>>> <qemu:commandline> and <qemu:arg> too. Without the namespace > > >>>> definition, the latter elements will make no sense, and libvirt will > > >>>> delete them immediately. > > >>>> > > >>>> (2) The above change will grow your guest's physical address space to > > >>>> more than 64GB. As a consequence, on your *host*, *if* your physical CPU > > >>>> supports nested paging (called "ept" on Intel and "npt" on AMD), *then* > > >>>> the CPU will have to support at least 37 physical address bits too, for > > >>>> the guest to work. Otherwise, the guest will break, hard. > > >>>> > > >>>> Here's how to verify (on the host): > > >>>> > > >>>> (2a) run "egrep -w 'npt|ept' /proc/cpuinfo" --> if this does not produce > > >>>> output, then stop reading here; things should work. Your CPU does not > > >>>> support nested paging, so KVM will use shadow paging, which is slower, > > >>>> but at least you don't have to care about the CPU's phys address width. > > >>>> > > >>>> (2b) otherwise (i.e. when you do have nested paging), run "grep 'bits > > >>>> physical' /proc/cpuinfo" --> if the physical address width is >=37, > > >>>> you're good. > > >>>> > > >>>> (2c) if you have nested paging but exactly 36 phys address bits, then > > >>>> you'll have to forcibly disable nested paging (assuming you want to run > > >>>> a guest with larger than 64GB guest-phys address space, that is). On > > >>>> Intel, issue: > > >>>> > > >>>> rmmod kvm_intel > > >>>> modprobe kvm_intel ept=N > > >>>> > > >>>> On AMD, go with: > > >>>> > > >>>> rmmod kvm_amd > > >>>> modprobe kvm_amd npt=N > > >>>> > > >>>> Hope this helps, > > >>>> Laszlo > > >>>> > > >>> > > >> > > > > > > -- > Dr. David Alan Gilbert / dgilbert(a)redhat.com / Manchester, UK -- Eduardo

1 0

[libvirt] [PATCH v2 0/4] PCI hostdev partial assignment support
by Daniel Henrique Barboza 22 Nov '19

22 Nov '19

Changes from previous version [1], all of them result of feedback from Alex Williamson and Abdulla Bubshait: - use <address type='none'/> instead of creating a new subsys attribute; - expand the change to all PCI hostdevs. Former patch 01 was discarded because we don't need the PCI Multifunction helpers for now; - series changed name to reflect what it's being done - new patch 04: add documentation to formatdomain.html.in To avoid a huge wall of text please refer to [1] for context about the road up to here. Commit msgs of the first 3 patches tells the story as well. [1] https://www.redhat.com/archives/libvir-list/2019-October/msg00298.html What I want to discuss here instead is a caveat that I've found while testing this work, since its first version. This test was done in a Power 8 system with a Broadcom BCM5719 PCIe Multifunction card, with 4 virtual functions. This series enables Libvirt to declare PCI hostdevs that will not be visible to the guest using address type='none'. During the tests I faced a scenario that I expected to fail, but it didn't. This is the relevant XML except: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0001' bus='0x09' slot='0x00' function='0x0'/> </source> <address type='none'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0001' bus='0x09' slot='0x00' function='0x1'/> </source> <address type='none'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0001' bus='0x09' slot='0x00' function='0x2'/> </source> <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x2'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0001' bus='0x09' slot='0x00' function='0x3'/> </source> <address type='none'/> </hostdev> I'm declaring all the BCM5719 functions in the XML, but I am making functions 0, 1 and 3 unassignable by the guest using address type='none'. This test was meant to fail, but it didn't. To my surprise the guest booted and the device is functional: $ lspci 0000:00:01.0 Unclassified device [00ff]: Red Hat, Inc Virtio memory balloon 0000:00:03.0 USB controller: Red Hat, Inc. QEMU XHCI Host Controller (rev 01) 0000:00:04.0 SCSI storage controller: Red Hat, Inc Virtio block device 0001:00:01.2 Ethernet controller: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01) $ I've talked with Michael Roth (QEMU PPC64 developer that worked with the PCI multifunction hotplug/unplug support in the PPC64 machine) about this. He mentioned that this is intended. I'll quote here what he had to say about it: "The current logic is that we only emit the hotplug event when function 0 is attached, but if some other function is attached at boot-time the guest will still see it on the bus, and whether that works or not I think is up to the device/driver" This explains why this test didn't fail as I expected. At least for the PPC64 machine, depending on the device support, this setup is allowed. PPC64 machine uses function 0 hotplug as a signal of 'plug all the queue functions and function 0', but function 0 isn't required at boot time. I would like to hear other opinions in this because I can't say whether this is allowed in x86. I am mentioning all this now because this had a direct impact on the design of this work since the previous version, and I failed to bring it up back then. I am *not* checking for the assignment of function 0 at guest boot time in Libvirt, leaving the user free to decide what to do. I am aware that this will be inconsistent to the logic of the PCI multifunction hotplug/unplug support, where function 0 is required. This also puts a lot of faith in the user, relying that the user is fully aware of the capabilities of the hardware. My question is: should Libvirt force function 0 to be present in boot time as well, regardless of whether the PPC64 guest or some cards are able to boot without it? Thanks, DHB Daniel Henrique Barboza (4): domain_conf: allow address type='none' to unassign PCI hostdevs qemu: handle unassigned PCI hostdevs in command line and alias virhostdev.c: check all IOMMU devs in virHostdevPreparePCIDevices formatdomain.html.in: document <address type='none'/> docs/formatdomain.html.in | 15 +++++ docs/schemas/domaincommon.rng | 5 ++ src/conf/domain_conf.c | 56 ++++++++++++++-- src/conf/domain_conf.h | 3 + src/qemu/qemu_alias.c | 6 ++ src/qemu/qemu_command.c | 4 ++ src/qemu/qemu_domain_address.c | 6 ++ src/util/virhostdev.c | 64 +++++++++++++++++-- .../hostdev-pci-address-none.args | 31 +++++++++ .../hostdev-pci-address-none.xml | 42 ++++++++++++ ...ostdev-pci-multifunction-partial-fail.args | 31 +++++++++ ...hostdev-pci-multifunction-partial-fail.xml | 35 ++++++++++ tests/qemuxml2argvtest.c | 8 +++ .../hostdev-pci-address-none.xml | 58 +++++++++++++++++ tests/qemuxml2xmltest.c | 1 + 15 files changed, 352 insertions(+), 13 deletions(-) create mode 100644 tests/qemuxml2argvdata/hostdev-pci-address-none.args create mode 100644 tests/qemuxml2argvdata/hostdev-pci-address-none.xml create mode 100644 tests/qemuxml2argvdata/hostdev-pci-multifunction-partial-fail.args create mode 100644 tests/qemuxml2argvdata/hostdev-pci-multifunction-partial-fail.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-pci-address-none.xml -- 2.21.0

3 8

[libvirt] [PATCH 0/1] Improve travis CI coverage
by Daniel P. Berrangé 22 Nov '19

22 Nov '19

There is currently a serious hardware outage with the host running our Jenkins CI slaves and we're not sure when we'll get it back up and running. Ths leaves us with Travis CI in the meanwhile. This expands the Travis CI coverage to add latest Fedora stable and rawhide, so that it is covering a representative set of distros overall. Daniel P. Berrangé (1): travis: add fedora-31 & fedora-rawhide to the build images .travis.yml | 14 ++++++++++++++ 1 file changed, 14 insertions(+) -- 2.21.0

2 2

[libvirt] [PATCH v4 00/20] cleanup current build system
by Pavel Hrdina 22 Nov '19

22 Nov '19

As preparation to switch to Meson there are some things that needs be cleaned up to make the conversion easier. The important thing in Meson is that there is a strict separation between source and build directory and the distributed tarball by default contains only files tracked by git with a possibility to write a script which would add some other sources into the tarball. Regardless of the adoption of Meson these patches improve our current build system to fully support VPATH builds. Changes in v2: - some patches from v1 are pushed now - added a patch to mandate build dir != src dir - added a patch to cleanup .gitignore - added patches to fix sc_po_check - improved some patches from v1 Changes in v3: - python-zanata-client is the tool our Makefile uses Pavel Hrdina (20): build: mandate use of a build dir != src dir .gitignore: cleanup old and obsolete ignores syntax-check.mk: fix sc_po_check rule syntax-check.mk: cleanup sc_po_check dependencies syntax-check.mk: cleanup generated_files list for sc_po_check po: generate files into build directory po: rewrite the way how we generate files po: README.md: add a note about which Zanata client is required remote: unify rpc server dispatch generated files src: generate source files into build directory src: access: generate source files into build directory src: admin: generate source files into build directory src: esx: generate source files into build directory src: hyperv: generate source files into build directory src: locking: generate source files into build directory src: logging: generate source files into build directory src: lxc: generate source files into build directory src: remote: generate source files into build directory src: stop distributing generated source files tools: stop distributing generated source files .gitignore | 276 ++------------------- .travis.yml | 3 +- README-hacking | 11 +- README.md | 11 +- bootstrap.conf | 6 + build-aux/syntax-check.mk | 47 ++-- configure.ac | 6 + docs/compiling.html.in | 10 +- docs/windows.html.in | 3 +- libvirt.spec.in | 10 +- po/Makefile.am | 45 ++-- po/POTFILES | 320 ------------------------- po/POTFILES.in | 356 ++++++++++++++++++++++++++++ po/README.md | 3 + src/Makefile.am | 13 +- src/access/Makefile.inc.am | 17 +- src/admin/Makefile.inc.am | 24 +- src/bhyve/Makefile.inc.am | 1 + src/esx/Makefile.inc.am | 9 +- src/esx/esx_vi_generator.py | 11 +- src/hyperv/Makefile.inc.am | 9 +- src/hyperv/hyperv_wmi_generator.py | 11 +- src/interface/Makefile.inc.am | 2 + src/libxl/Makefile.inc.am | 2 + src/locking/Makefile.inc.am | 16 +- src/logging/Makefile.inc.am | 18 +- src/lxc/Makefile.inc.am | 36 ++- src/network/Makefile.inc.am | 2 + src/node_device/Makefile.inc.am | 2 + src/nwfilter/Makefile.inc.am | 2 + src/qemu/Makefile.inc.am | 2 + src/remote/Makefile.inc.am | 45 ++-- src/remote/remote_daemon_dispatch.c | 4 +- src/rpc/Makefile.inc.am | 8 +- src/secret/Makefile.inc.am | 2 + src/storage/Makefile.inc.am | 2 + src/util/Makefile.inc.am | 6 +- src/vbox/Makefile.inc.am | 1 + src/vz/Makefile.inc.am | 1 + tests/Makefile.am | 4 + tools/Makefile.am | 1 - 41 files changed, 636 insertions(+), 722 deletions(-) delete mode 100644 po/POTFILES create mode 100644 po/POTFILES.in -- 2.23.0

3 26

[libvirt] [PATCH 0/2] Simplify hash table data free callbacks
by Daniel P. Berrangé 22 Nov '19

22 Nov '19

2 3

[libvirt] [PATCH 0/8] qemu: Enable -blockdev (blockdev-add saga finale?)
by Peter Krempa 22 Nov '19

22 Nov '19

This patchset depends on: https://www.redhat.com/archives/libvir-list/2019-November/msg00762.html everything together is available at: git fetch https://gitlab.com/pipo.sk/libvirt.git blockdev-enable Looks like finally all pieces are together. Let's enable it. Peter Krempa (8): conf: domcaps: Add 'backingStoreInput' domain capability docs: Document support for obeying <backingStore> of <disk> on input qemu: caps: Base support of 'backingStoreInput' domain feature on QEMU_CAPS_BLOCKDEV tests: qemucapabilities: Refresh data for unreleased qemu-4.2 on x86_64 qemu: caps: Add capability for dynamic 'auto-read-only' support for files qemu: qapi: Add support for command features qemu: capabilities: Add detection of the 'savevm' fix for -blockdev qemu: enable blockdev support docs/formatdomain.html.in | 29 +- docs/formatdomaincaps.html.in | 7 + docs/schemas/domaincaps.rng | 9 + src/conf/domain_capabilities.c | 1 + src/conf/domain_capabilities.h | 1 + src/qemu/qemu_capabilities.c | 17 +- src/qemu/qemu_capabilities.h | 2 + src/qemu/qemu_qapi.c | 15 + tests/domaincapsdata/libxl-xenfv.xml | 1 + tests/domaincapsdata/libxl-xenpv.xml | 1 + .../domaincapsdata/qemu_1.5.3-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_1.5.3-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_1.5.3.x86_64.xml | 1 + .../domaincapsdata/qemu_1.6.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_1.6.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_1.6.0.x86_64.xml | 1 + .../domaincapsdata/qemu_1.7.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_1.7.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_1.7.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.1.1-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.1.1-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.1.1.x86_64.xml | 1 + .../domaincapsdata/qemu_2.10.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.10.0-tcg.x86_64.xml | 1 + .../qemu_2.10.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.10.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.10.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_2.10.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.10.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.11.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.11.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.11.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.11.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.12.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.12.0-tcg.x86_64.xml | 1 + .../qemu_2.12.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.12.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.12.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_2.12.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.12.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.4.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.4.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.4.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.5.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.5.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.5.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.6.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.6.0-tcg.x86_64.xml | 1 + .../qemu_2.6.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.6.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.6.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_2.6.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.7.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.7.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.7.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.7.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.8.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.8.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.8.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.8.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.9.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.9.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.9.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_2.9.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.9.0.x86_64.xml | 1 + .../domaincapsdata/qemu_3.0.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_3.0.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_3.0.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_3.0.0.s390x.xml | 1 + tests/domaincapsdata/qemu_3.0.0.x86_64.xml | 1 + .../domaincapsdata/qemu_3.1.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_3.1.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_3.1.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_3.1.0.x86_64.xml | 1 + .../domaincapsdata/qemu_4.0.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_4.0.0-tcg.x86_64.xml | 1 + .../qemu_4.0.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_4.0.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_4.0.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_4.0.0.s390x.xml | 1 + tests/domaincapsdata/qemu_4.0.0.x86_64.xml | 1 + .../domaincapsdata/qemu_4.1.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_4.1.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_4.1.0.x86_64.xml | 1 + .../domaincapsdata/qemu_4.2.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_4.2.0-tcg.x86_64.xml | 1 + .../qemu_4.2.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_4.2.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_4.2.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_4.2.0.x86_64.xml | 1 + .../caps_4.1.0.x86_64.xml | 1 + .../caps_4.2.0.aarch64.xml | 3 + .../qemucapabilitiesdata/caps_4.2.0.ppc64.xml | 1 + .../caps_4.2.0.x86_64.replies | 3907 +++++++++++------ .../caps_4.2.0.x86_64.xml | 253 +- ...arch64-os-firmware-efi.aarch64-latest.args | 18 +- .../aarch64-virt-graphics.aarch64-latest.args | 26 +- .../aarch64-virt-headless.aarch64-latest.args | 26 +- .../controller-virtio-scsi.x86_64-latest.args | 35 +- .../disk-aio.x86_64-latest.args | 19 +- ...-backing-chains-noindex.x86_64-latest.args | 145 +- .../disk-cache.x86_64-latest.args | 50 +- ...m-empty-network-invalid.x86_64-latest.args | 3 +- .../disk-cdrom-network.x86_64-latest.args | 32 +- .../disk-cdrom-tray.x86_64-latest.args | 24 +- .../disk-cdrom.x86_64-latest.args | 21 +- .../disk-copy_on_read.x86_64-latest.args | 19 +- .../disk-detect-zeroes.x86_64-latest.args | 17 +- .../disk-error-policy.x86_64-latest.args | 30 +- .../disk-floppy-q35-2_11.x86_64-latest.args | 14 +- .../disk-floppy-q35-2_9.x86_64-latest.args | 14 +- .../disk-floppy.x86_64-latest.args | 21 +- .../disk-network-gluster.x86_64-latest.args | 32 +- .../disk-network-iscsi.x86_64-latest.args | 58 +- .../disk-network-nbd.x86_64-latest.args | 41 +- .../disk-network-rbd.x86_64-latest.args | 67 +- .../disk-network-sheepdog.x86_64-latest.args | 16 +- ...isk-network-source-auth.x86_64-latest.args | 30 +- .../disk-network-tlsx509.x86_64-latest.args | 64 +- .../disk-readonly-disk.x86_64-latest.args | 14 +- .../disk-scsi-device-auto.x86_64-latest.args | 14 +- .../disk-scsi.x86_64-latest.args | 35 +- .../disk-shared.x86_64-latest.args | 36 +- ...irtio-scsi-reservations.x86_64-latest.args | 20 +- .../floppy-drive-fat.x86_64-latest.args | 7 +- ...egl-headless-rendernode.x86_64-latest.args | 7 +- .../graphics-egl-headless.x86_64-latest.args | 7 +- ...threads-virtio-scsi-pci.x86_64-latest.args | 25 +- ...y-hotplug-nvdimm-access.x86_64-latest.args | 7 +- ...ry-hotplug-nvdimm-align.x86_64-latest.args | 7 +- ...ry-hotplug-nvdimm-label.x86_64-latest.args | 7 +- ...ory-hotplug-nvdimm-pmem.x86_64-latest.args | 7 +- ...hotplug-nvdimm-readonly.x86_64-latest.args | 7 +- .../memory-hotplug-nvdimm.x86_64-latest.args | 7 +- ...os-firmware-efi-secboot.x86_64-latest.args | 18 +- .../os-firmware-efi.x86_64-latest.args | 18 +- ...host-user-gpu-secondary.x86_64-latest.args | 7 +- .../vhost-user-vga.x86_64-latest.args | 7 +- ...eo-bochs-display-device.x86_64-latest.args | 10 +- ...eo-ramfb-display-device.x86_64-latest.args | 10 +- ...virtio-non-transitional.x86_64-latest.args | 7 +- .../virtio-transitional.x86_64-latest.args | 7 +- .../x86_64-pc-graphics.x86_64-latest.args | 8 +- .../x86_64-pc-headless.x86_64-latest.args | 8 +- .../x86_64-q35-graphics.x86_64-latest.args | 8 +- .../x86_64-q35-headless.x86_64-latest.args | 8 +- 146 files changed, 3790 insertions(+), 1683 deletions(-) -- 2.23.0

3 22

[libvirt] [PATCH 0/2] Deprecate implicit filters
by Vladimir Sementsov-Ogievskiy 22 Nov '19

22 Nov '19

Hi all! Max's series to fix some problems around filters consists of 42 patches. I'm sure that we didn't find all bugs around filters, and that filters would be a constant source of bugs in future, as during developing new feature nobody will consider all possible cases of dealing with filters (OK, somebody will, but it's hard). So, I'm thinking about starting some deprecations which will help us to simplify all the picture at least in not far future. So, I'd really want to deprecate implicit filters, ->file child based filters (move all filters to use backing child), filename based interfaces (use node-names). Most simple thing is implicit filters, so let's start from them. drive-mirror don't support filter-node-name, so I propose to deprecate it at all, together with drive-backup, instead of adding support of filter-node-name, what do you think? Vladimir Sementsov-Ogievskiy (2): qapi: deprecate drive-mirror and drive-backup qapi: deprecate implicit filters qemu-deprecated.texi | 11 +++++++++++ qapi/block-core.json | 10 ++++++++-- qapi/transaction.json | 2 +- include/block/block_int.h | 10 +++++++++- blockdev.c | 20 ++++++++++++++++++++ 5 files changed, 49 insertions(+), 4 deletions(-) -- 2.18.0

9 43

[libvirt] [PATCH 0/3] Speed up syntax-check a little
by Ján Tomko 22 Nov '19

22 Nov '19

This shaves off 4 % of syntax-check runtime on my 8-core system, which is enough of a reason for me. Surely it saves more on our overloaded CI workers. Ján Tomko (3): scripts: speedup prohibit-duplicate-header scripts: check-aclrules: use in instead of find scripts: check-aclrules: use regular expressions less often scripts/check-aclrules.py | 38 ++++++++++++++++++---------- scripts/prohibit-duplicate-header.py | 4 +++ 2 files changed, 29 insertions(+), 13 deletions(-) -- 2.19.2

3 13

[libvirt] [PATCH RFC 00/40] qemu: Add support for incremental backups
by Peter Krempa 22 Nov '19

22 Nov '19

This series is my attempt to clean up and finish incremental backup as it was last posted in v10 here: https://www.redhat.com/archives/libvir-list/2019-August/msg01018.html Out of that series, patch 1 was dropped for now and I'll revisit it later. The patches adding docs, xml parsing and job plumbing were partially modified and stripped of currently unimplemented functionality. The qemu implementation was completely rewritten. Of the new series first 28 patches are cleanups and can be reviewed for pushing. The main incremental backup functionality is not complete yet and should not be pushed right now not even in experimental (disabled [1]) state due to possible API changes: - figure out whether we want our own event for the backup job or want to deal with the domain job event - if we go with the domain job event everything must be rewritten as a single domain job - otherwise plumbing for the event must be added - documentation must reflect the above after that will be sorted out the following needs to be done until we can finally enable it: - more testing. I merely verified that qemu generates files/exports NBD - blockjob support for bitmaps (for commit/pull/copy) - snapshot support for bitmaps - merging of bitmaps accross backing chain entries - figuring out a more elegant way to do the bitmap for the backup than creating two, one for the 'store' (for NBD server) and one for the disk itself. - ... Until then the following XML can be used to enable the functionality for testing without any guarantee [1]: <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> ... <qemu:capabilities> <qemu:add capability='blockdev'/> <qemu:add capability='incremental-backup'/> </qemu:capabilities> </domain> Eric Blake (5): backup: Document new XML for backups backup: Introduce virDomainBackup APIs backup: Implement backup APIs for remote driver backup: Parse and output backup XML backup: Implement virsh support for backup Peter Krempa (35): util: hash: Add possibility to use simpler data free function in virHash util: hash: Add new constructor 'virHashNew' util: hash: Introduce virHashHasEntry qemu: domain: Split out setup of virStorageSource from qemu driver config qemu: domain: Remove pointless return value in qemuDomainPrepareDiskSourceData qemu: domain: clarify sematics of qemuDomainPrepareDiskSourceData qemu: domain: Tolerate NULL @disk in qemuDomainPrepareDiskSourceData conf: Remove virDomainDiskPathByName conf: Introduce virDomainDiskByTarget qemu: Replace use of virDomainDiskFindByBusAndDst with virDomainDiskByTarget conf: Remove unused virDomainDiskFindByBusAndDst Replace virDomainDiskByName by virDomainDiskByTarget in appropriate cases qemu: monitor: Introduce new interface to query-named-block-nodes qemu: block: Don't query monitor in qemuBlockStorageSourceCreateDetectSize qemu: checkpoint: Fix rollback and access to unlocked 'vm' when deleting checkpoints qemu: monitor: Remove non-transaction based dirty bitmap APIs conf: snapshot: Don't clear current snapshot when redefining an existing one conf: snapshot: Remove 'update_current' parameter from virDomainSnapshotRedefinePrep conf: Don't reuse variable for different object in virDomainCheckpointRedefinePrep conf: checkpoint: Don't clear current checkpoint when redefining an existing one conf: checkpoint: Don't clear current checkpoint when redefining qemu: checkpoint: Enforce that 'bitmap' name must match checkpoint name qemu: checkpoint: Split out checkpoint creation code qemu: checkpoint: Extract finalizing steps of checkpoint creation qemu: monitor: Add helper for generating data for block bitmap merging qemu: checkpoint: Use qemuMonitorTransactionBitmapMergeSourceAddBitmap qemu: blockjob: Refactor qemuBlockJobEventProcessConcludedTransition qemu: blockjob: Use 'g_free' in qemuBlockJobDataDispose Add 'backup' block job type qemu: monitor: Add support for blockdev-backup via 'transaction' qemu: domain: Track backup job data in the status XML qemu: blockjob: Track internal data for 'backup' blockjob tests: qemustatusxml2xml: Add test for 'pull' type backup job qemu: Implement backup job APIs and qemu handling qemu: blockjob: Implement concluded blockjob handler for backup blockjobs docs/docs.html.in | 3 +- docs/format.html.in | 1 + docs/formatbackup.html.in | 168 ++++ docs/formatcheckpoint.html.in | 12 +- docs/index.html.in | 3 +- docs/schemas/domainbackup.rng | 219 +++++ examples/c/misc/event-test.c | 3 + include/libvirt/libvirt-domain.h | 29 +- libvirt.spec.in | 1 + mingw-libvirt.spec.in | 2 + src/conf/Makefile.inc.am | 2 + src/conf/backup_conf.c | 518 ++++++++++ src/conf/backup_conf.h | 102 ++ src/conf/checkpoint_conf.c | 17 +- src/conf/domain_addr.c | 4 +- src/conf/domain_conf.c | 50 +- src/conf/domain_conf.h | 8 +- src/conf/snapshot_conf.c | 6 - src/conf/snapshot_conf.h | 1 - src/conf/virconftypes.h | 3 + src/driver-hypervisor.h | 20 + src/libvirt-domain-checkpoint.c | 7 +- src/libvirt-domain.c | 191 ++++ src/libvirt_private.syms | 13 +- src/libvirt_public.syms | 8 + src/libxl/libxl_driver.c | 2 +- src/qemu/Makefile.inc.am | 2 + src/qemu/qemu_backup.c | 895 ++++++++++++++++++ src/qemu/qemu_backup.h | 41 + src/qemu/qemu_block.c | 26 +- src/qemu/qemu_block.h | 5 +- src/qemu/qemu_blockjob.c | 171 ++-- src/qemu/qemu_blockjob.h | 19 + src/qemu/qemu_checkpoint.c | 232 +++-- src/qemu/qemu_checkpoint.h | 15 + src/qemu/qemu_domain.c | 161 +++- src/qemu/qemu_domain.h | 11 +- src/qemu/qemu_driver.c | 116 ++- src/qemu/qemu_hotplug.c | 2 +- src/qemu/qemu_migration.c | 3 +- src/qemu/qemu_monitor.c | 91 +- src/qemu/qemu_monitor.h | 49 +- src/qemu/qemu_monitor_json.c | 242 ++--- src/qemu/qemu_monitor_json.h | 33 +- src/remote/remote_driver.c | 3 + src/remote/remote_protocol.x | 53 +- src/remote_protocol-structs | 28 + src/test/test_driver.c | 2 +- src/util/vircgroup.c | 2 +- src/util/virhash.c | 85 +- src/util/virhash.h | 12 + tests/Makefile.am | 2 + tests/domainbackupxml2xmlin/backup-pull.xml | 9 + tests/domainbackupxml2xmlin/backup-push.xml | 9 + tests/domainbackupxml2xmlin/empty.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 9 + tests/domainbackupxml2xmlout/backup-push.xml | 9 + tests/domainbackupxml2xmlout/empty.xml | 7 + tests/qemublocktest.c | 3 +- tests/qemumonitorjsontest.c | 53 +- .../qemustatusxml2xmldata/backup-pull-in.xml | 607 ++++++++++++ .../qemustatusxml2xmldata/backup-pull-out.xml | 1 + tests/qemuxml2xmltest.c | 2 + tests/virschematest.c | 2 + tools/Makefile.am | 1 + tools/virsh-backup.c | 209 ++++ tools/virsh-backup.h | 21 + tools/virsh-domain.c | 8 +- tools/virsh.c | 2 + tools/virsh.h | 1 + tools/virsh.pod | 37 + 71 files changed, 4133 insertions(+), 552 deletions(-) create mode 100644 docs/formatbackup.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 src/conf/backup_conf.c create mode 100644 src/conf/backup_conf.h create mode 100644 src/qemu/qemu_backup.c create mode 100644 src/qemu/qemu_backup.h create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml create mode 100644 tests/domainbackupxml2xmlin/empty.xml create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml create mode 100644 tests/domainbackupxml2xmlout/empty.xml create mode 100644 tests/qemustatusxml2xmldata/backup-pull-in.xml create mode 120000 tests/qemustatusxml2xmldata/backup-pull-out.xml create mode 100644 tools/virsh-backup.c create mode 100644 tools/virsh-backup.h -- 2.21.0

5 83

[libvirt] [PATCH] fix bug libvirt daemon deadlock when another force console break down an existed console client when this deadlock hanppened, libvirtd backtrace as follow, a typical ABBA deadlock circumstance:
by LanceLiu 22 Nov '19

22 Nov '19

(gdb) thread 23 [Switching to thread 23 (Thread 0x7fbb54810700 (LWP 296966))] (gdb) bt at rpc/virnetserverclient.c:1503 client=client@entry=0x5641d20e20a0, msg=0x7fbb08003ab0, rerr=rerr@entry=0x7fbb5480f7b0, procedure=201, type=type@entry=3, serial=9) at rpc/virnetserverprogram.c:173 msg=<optimized out>, rerr=rerr@entry=0x7fbb5480f7b0, procedure=<optimized out>, serial=<optimized out>) at rpc/virnetserverprogram.c:222 opaque=opaque@entry=0x5641d20e20a0) at stream.c:246 force=force@entry=true) at conf/virchrdev.c:386 at qemu/qemu_driver.c:16188 st=st@entry=0x7fbb08008fc0, flags=1) at libvirt-domain.c:9363 rerr=0x7fbb5480fc10, msg=<optimized out>, client=0x5641d20e4e80) at remote_dispatch.h:8540 rerr=0x7fbb5480fc10, args=0x7fbb08005eb0, ret=<optimized out>) at remote_dispatch.h:8505 server=0x5641d20be420, prog=0x5641d20da580) at rpc/virnetserverprogram.c:437 msg=0x5641d20e9a20) at rpc/virnetserverprogram.c:307 srv=0x5641d20be420) at rpc/virnetserver.c:148 ---Type <return> to continue, or q <return> to quit--- (gdb) thread 26 [Switching to thread 26 (Thread 0x7fbb87a378c0 (LWP 295636))] (gdb) bt at stream.c:286 at rpc/virnetserverclient.c:1247 at rpc/virnetserverclient.c:1457 at util/vireventpoll.c:508 Signed-off-by: LanceLiu <liu.lance.89(a)gmail.com> --- src/remote/remote_daemon_stream.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/remote/remote_daemon_stream.c b/src/remote/remote_daemon_stream.c index 1f6e783..b27348f 100644 --- a/src/remote/remote_daemon_stream.c +++ b/src/remote/remote_daemon_stream.c @@ -284,14 +284,16 @@ daemonStreamEvent(virStreamPtr st, int events, void *opaque) * -1 on fatal client error */ static int -daemonStreamFilter(virNetServerClientPtr client ATTRIBUTE_UNUSED, +daemonStreamFilter(virNetServerClientPtr client, virNetMessagePtr msg, void *opaque) { daemonClientStream *stream = opaque; int ret = 0; + virObjectUnlock(client); virMutexLock(&stream->priv->lock); + virObjectLock(client); if (msg->header.type != VIR_NET_STREAM && msg->header.type != VIR_NET_STREAM_HOLE) -- 1.8.3.1

3 3