- Devel - libvirt List Archives

[libvirt][PATCH v1 0/1] support system default memory policy with numatune
by Luyao Zhong 02 Dec '20

02 Dec '20

Hi libvirt experts, ---------- what does this patch support ---------- This patch seeks the support of system default memory policy when numatune is configured. Before this patch, numatune only has three memory modes: static, interleave and prefered. These memory policies are ultimately set by mbind() system call. Memory policy could be 'hard coded' into the kernel, but none of above policies fit our requirment under this case. mbind() support default memory policy, but it requires a NULL nodemask. So obviously setting allowed memory nodes is cgroups' mission. That means if 'default' mode is specified in numatune, numatune config will be completely cgroups setting, which restricting the memory nodes allowed for each vcpu thread in different cells. <numatune> <memory mode="default" nodeset="1-4,^3" migratable="yes" /> <memnode cellid="0" mode="default" nodeset="1"/> <memnode cellid="2" mode="default" nodeset="2"/> </numatune> Above is current design, we introduce a new "migratable" attribute and a new option "default" for "mode" attribute in numatune. ---------- summary of discussion ---------- But at the beginning in RFC[1], Martin Kletzander and I got a consensus, just introducing a new "migratable" attribute to indicates that we rely on operating system to operate memory policy. <numatune> <memory mode="strict" nodeset="1-4,^3" migratable="yes" /> <memnode cellid="0" mode="strict" nodeset="1"/> <memnode cellid="2" mode="preferred" nodeset="2"/> </numatune> the mode will be ignored, and libvirt will not set those memory policy related configs in qemu command line. Then initial patch got Peter Krempa's comments[2], if I understood it correctly, that means config should not be ignored directly (I just stated in document it will be ignored), it should be rejected in code and output log to lead user correct the config. But when I updated the patch, I found I had to add a new option 'default' for mode attribute since its default value is "strict", if not setting mode it will still be formatted to "strict" at the end. So I sent a RFC[3] again, but unfortunately it sunk into the mail sea. (sorry for the RFC/PATCH title changing) [1] https://www.redhat.com/archives/libvir-list/2020-August/msg00960.html [2] https://www.redhat.com/archives/libvir-list/2020-October/msg00396.html [3] https://www.redhat.com/archives/libvir-list/2020-October/msg01033.html mbind(): https://man7.org/linux/man-pages/man2/mbind.2.html Though we have "migratable" in current implementation, actually we can support this feauture without it since we have "default" mode, so is "migratable" still neccessary? Thanks for your patience to read this. Appreciate if you could give comments. Regards, Luyao Luyao Zhong (1): support system default memory policy with numatune docs/formatdomain.rst | 12 ++++- docs/schemas/domaincommon.rng | 7 +++ include/libvirt/libvirt-domain.h | 1 + src/conf/numa_conf.c | 50 ++++++++++++++++++- src/conf/numa_conf.h | 2 + src/libvirt_private.syms | 1 + src/qemu/qemu_command.c | 7 ++- src/qemu/qemu_process.c | 25 ++++++++++ src/util/virnuma.c | 3 ++ .../numatune-memory-migratable.args | 34 +++++++++++++ ...atune-memory-migratable.x86_64-latest.args | 35 +++++++++++++ .../numatune-memory-migratable.xml | 33 ++++++++++++ tests/qemuxml2argvtest.c | 2 + ...matune-memory-migratable.x86_64-latest.xml | 40 +++++++++++++++ .../numatune-memory-migratable.xml | 39 +++++++++++++++ tests/qemuxml2xmltest.c | 1 + 16 files changed, 288 insertions(+), 4 deletions(-) create mode 100644 tests/qemuxml2argvdata/numatune-memory-migratable.args create mode 100644 tests/qemuxml2argvdata/numatune-memory-migratable.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/numatune-memory-migratable.xml create mode 100644 tests/qemuxml2xmloutdata/numatune-memory-migratable.x86_64-latest.xml create mode 100644 tests/qemuxml2xmloutdata/numatune-memory-migratable.xml -- 2.25.4

2 2

[PATCH v2] src: use singular form instead of plural, for guest disk info
by Daniel P. Berrangé 02 Dec '20

02 Dec '20

Existing practice with the filesystem fields reported for the virDomainGetGuestInfo API is to use the singular form for field names. Ensure the disk info follows this practice. Fixes commit 05a75ca2ce743bc0bb119fb8d532ff84646fafa3 Author: Marc-André Lureau <marcandre.lureau(a)redhat.com> Date: Fri Nov 20 22:09:46 2020 +0400 domain: add disk informations to virDomainGetGuestInfo commit 0cb2d9f05d00497a715352f6ea28cf8fb6921731 Author: Marc-André Lureau <marcandre.lureau(a)redhat.com> Date: Fri Nov 20 22:09:47 2020 +0400 qemu_driver: report guest disk informations commit 172b8304352b1945e328394e61290a24446280dd Author: Marc-André Lureau <marcandre.lureau(a)redhat.com> Date: Fri Nov 20 22:09:48 2020 +0400 virsh: add --disk informations to guestinfo command Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- In v2: also update docs and virsh docs/manpages/virsh.rst | 20 ++++++++++---------- src/libvirt-domain.c | 14 +++++++------- src/qemu/qemu_driver.c | 14 +++++++------- tools/virsh-domain.c | 6 +++--- 4 files changed, 27 insertions(+), 27 deletions(-) diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index 9ef6b68422..aa54bc21ef 100644 --- a/docs/manpages/virsh.rst +++ b/docs/manpages/virsh.rst @@ -2679,7 +2679,7 @@ guestinfo :: guestinfo domain [--user] [--os] [--timezone] [--hostname] [--filesystem] - [--disks] + [--disk] Print information about the guest from the point of view of the guest agent. Note that this command requires a guest agent to be configured and running in @@ -2690,7 +2690,7 @@ are supported by the guest agent. You can limit the types of information that are returned by specifying one or more flags. If a requested information type is not supported, the processes will provide an exit code of 1. Available information types flags are *--user*, *--os*, -*--timezone*, *--hostname*, *--filesystem* and *--disks*. +*--timezone*, *--hostname*, *--filesystem* and *--disk*. Note that depending on the hypervisor type and the version of the guest agent running within the domain, not all of the following information may be @@ -2747,15 +2747,15 @@ returned: * ``fs.<num>.disk.<num>.serial`` - the serial number of disk <num> * ``fs.<num>.disk.<num>.device`` - the device node of disk <num> -*--disks* returns: +*--disk* returns: -* ``disks.count`` - the number of disks defined on this domain -* ``disks.<num>.name`` - device node (Linux) or device UNC (Windows) -* ``disks.<num>.partition`` - whether this is a partition or disk -* ``disks.<num>.dependencies.count`` - the number of device dependencies -* ``disks.<num>.dependencies.<num>.name`` - a dependency name -* ``disks.<num>.alias`` - the device alias of the disk (e.g. sda) -* ``disks.<num>.guest_alias`` - optional alias assigned to the disk +* ``disk.count`` - the number of disks defined on this domain +* ``disk.<num>.name`` - device node (Linux) or device UNC (Windows) +* ``disk.<num>.partition`` - whether this is a partition or disk +* ``disk.<num>.dependency.count`` - the number of device dependencies +* ``disk.<num>.dependency.<num>.name`` - a dependency name +* ``disk.<num>.alias`` - the device alias of the disk (e.g. sda) +* ``disk.<num>.guest_alias`` - optional alias assigned to the disk guestvcpus diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index 35e95e5395..f5cd43ecea 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -12345,17 +12345,17 @@ virDomainSetVcpu(virDomainPtr domain, * Returns information about the disks within the domain. The typed * parameter keys are in this format: * - * "disks.count" - the number of disks defined on this domain + * "disk.count" - the number of disks defined on this domain * as an unsigned int - * "disks.<num>.name" - device node (Linux) or device UNC (Windows) - * "disks.<num>.partition" - whether this is a partition or disk - * "disks.<num>.dependencies.count" - the number of device dependencies + * "disk.<num>.name" - device node (Linux) or device UNC (Windows) + * "disk.<num>.partition" - whether this is a partition or disk + * "disk.<num>.dependency.count" - the number of device dependencies * e.g. for LVs of the LVM this will * hold the list of PVs, for LUKS encrypted volume this will * contain the disk where the volume is placed. (Linux) - * "disks.<num>.dependencies.<num>.name" - a dependency - * "disks.<num>.alias" - the device alias of the disk (e.g. sda) - * "disks.<num>.guest_alias" - optional alias assigned to the disk, on Linux + * "disk.<num>.dependency.<num>.name" - a dependency + * "disk.<num>.alias" - the device alias of the disk (e.g. sda) + * "disk.<num>.guest_alias" - optional alias assigned to the disk, on Linux * this is a name assigned by device mapper * * VIR_DOMAIN_GUEST_INFO_HOSTNAME: diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 8eaa3ce68f..548df6ae68 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -19876,20 +19876,20 @@ qemuAgentDiskInfoFormatParams(qemuAgentDiskInfoPtr *info, size_t i, j, ndeps; if (virTypedParamsAddUInt(params, nparams, maxparams, - "disks.count", ndisks) < 0) + "disk.count", ndisks) < 0) return; for (i = 0; i < ndisks; i++) { char param_name[VIR_TYPED_PARAM_FIELD_LENGTH]; g_snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, - "disks.%zu.name", i); + "disk.%zu.name", i); if (virTypedParamsAddString(params, nparams, maxparams, param_name, info[i]->name) < 0) return; g_snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, - "disks.%zu.partition", i); + "disk.%zu.partition", i); if (virTypedParamsAddBoolean(params, nparams, maxparams, param_name, info[i]->partition) < 0) return; @@ -19897,14 +19897,14 @@ qemuAgentDiskInfoFormatParams(qemuAgentDiskInfoPtr *info, if (info[i]->dependencies) { ndeps = g_strv_length(info[i]->dependencies); g_snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, - "disks.%zu.dependencies.count", i); + "disk.%zu.dependency.count", i); if (ndeps && virTypedParamsAddUInt(params, nparams, maxparams, param_name, ndeps) < 0) return; for (j = 0; j < ndeps; j++) { g_snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, - "disks.%zu.dependencies.%zu.name", i, j); + "disk.%zu.dependency.%zu.name", i, j); if (virTypedParamsAddString(params, nparams, maxparams, param_name, info[i]->dependencies[j]) < 0) return; @@ -19922,7 +19922,7 @@ qemuAgentDiskInfoFormatParams(qemuAgentDiskInfoPtr *info, info[i]->address->unit); if (diskdef) { g_snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, - "disks.%zu.alias", i); + "disk.%zu.alias", i); if (diskdef->dst && virTypedParamsAddString(params, nparams, maxparams, param_name, diskdef->dst) < 0) @@ -19932,7 +19932,7 @@ qemuAgentDiskInfoFormatParams(qemuAgentDiskInfoPtr *info, if (info[i]->alias) { g_snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, - "disks.%zu.guest_alias", i); + "disk.%zu.guest_alias", i); if (virTypedParamsAddString(params, nparams, maxparams, param_name, info[i]->alias) < 0) return; diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 655bac3315..349ea38ec3 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -14209,9 +14209,9 @@ static const vshCmdOptDef opts_guestinfo[] = { .type = VSH_OT_BOOL, .help = N_("report filesystem information"), }, - {.name = "disks", + {.name = "disk", .type = VSH_OT_BOOL, - .help = N_("report disks information"), + .help = N_("report disk information"), }, {.name = NULL} }; @@ -14236,7 +14236,7 @@ cmdGuestInfo(vshControl *ctl, const vshCmd *cmd) types |= VIR_DOMAIN_GUEST_INFO_HOSTNAME; if (vshCommandOptBool(cmd, "filesystem")) types |= VIR_DOMAIN_GUEST_INFO_FILESYSTEM; - if (vshCommandOptBool(cmd, "disks")) + if (vshCommandOptBool(cmd, "disk")) types |= VIR_DOMAIN_GUEST_INFO_DISKS; if (!(dom = virshCommandOptDomain(ctl, cmd, NULL))) -- 2.28.0

2 1

[PATCH v2] qemu: Don't cache NUMA caps
by Michal Privoznik 02 Dec '20

02 Dec '20

In v6.0.0-rc1~439 (and friends) we tried to cache NUMA capabilities because we assumed they are immutable. And to some extent they are (NUMA hotplug is not a thing, is it). However, our capabilities contain also some runtime info that can change, e.g. hugepages pool allocation sizes or total amount of memory per node (host side memory hotplug might change the value). Because of the caching we might not be reporting the correct runtime info in 'virsh capabilities'. The NUMA caps are used in three places: 1) 'virsh capabilities' 2) domain startup, when parsing numad reply 3) parsing domain private data XML In cases 2) and 3) we need NUMA caps to construct list of physical CPUs that belong to NUMA nodes from numad reply. And while this may seem static, it's not really because of possible CPU hotplug on physical host. There are two possible approaches: 1) build a validation mechanism that would invalidate the cached NUMA caps, or 2) drop the caching and construct NUMA caps from scratch on each use. In this commit, the latter approach is implemented, because it's easier. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1819058 Fixes: 1a1d848694f6c2f1d98a371124928375bc3bb4a3 Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- v2 of: https://www.redhat.com/archives/libvir-list/2020-December/msg00023.html diff to v1: - instead of fixing cached capabilities, drop caching completely src/qemu/qemu_conf.c | 23 +---------------------- src/qemu/qemu_conf.h | 6 ------ src/qemu/qemu_domain.c | 7 +++---- src/qemu/qemu_driver.c | 1 - src/qemu/qemu_process.c | 7 +++---- 5 files changed, 7 insertions(+), 37 deletions(-) diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index cbdde0c0dc..0ae86e5468 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -1289,27 +1289,6 @@ virQEMUDriverCreateXMLConf(virQEMUDriverPtr driver, } -virCapsHostNUMAPtr -virQEMUDriverGetHostNUMACaps(virQEMUDriverPtr driver) -{ - virCapsHostNUMAPtr hostnuma; - - qemuDriverLock(driver); - - if (!driver->hostnuma) - driver->hostnuma = virCapabilitiesHostNUMANewHost(); - - hostnuma = driver->hostnuma; - - qemuDriverUnlock(driver); - - if (hostnuma) - virCapabilitiesHostNUMARef(hostnuma); - - return hostnuma; -} - - virCPUDefPtr virQEMUDriverGetHostCPU(virQEMUDriverPtr driver) { @@ -1381,7 +1360,7 @@ virCapsPtr virQEMUDriverCreateCapabilities(virQEMUDriverPtr driver) "DOI \"%s\"", model, doi); } - caps->host.numa = virQEMUDriverGetHostNUMACaps(driver); + caps->host.numa = virCapabilitiesHostNUMANewHost(); caps->host.cpu = virQEMUDriverGetHostCPU(driver); return g_steal_pointer(&caps); } diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index 411d08db36..7025b5222e 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -270,11 +270,6 @@ struct _virQEMUDriver { */ virCapsPtr caps; - /* Lazy initialized on first use, immutable thereafter. - * Require lock to get the pointer & do optional initialization - */ - virCapsHostNUMAPtr hostnuma; - /* Lazy initialized on first use, immutable thereafter. * Require lock to get the pointer & do optional initialization */ @@ -337,7 +332,6 @@ virQEMUDriverConfigSetDefaults(virQEMUDriverConfigPtr cfg); virQEMUDriverConfigPtr virQEMUDriverGetConfig(virQEMUDriverPtr driver); -virCapsHostNUMAPtr virQEMUDriverGetHostNUMACaps(virQEMUDriverPtr driver); virCPUDefPtr virQEMUDriverGetHostCPU(virQEMUDriverPtr driver); virCapsPtr virQEMUDriverCreateCapabilities(virQEMUDriverPtr driver); virCapsPtr virQEMUDriverGetCapabilities(virQEMUDriverPtr driver, diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 663c0af867..8e9c0224e4 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -2498,8 +2498,7 @@ qemuDomainObjPrivateXMLParseVcpu(xmlNodePtr node, static int qemuDomainObjPrivateXMLParseAutomaticPlacement(xmlXPathContextPtr ctxt, - qemuDomainObjPrivatePtr priv, - virQEMUDriverPtr driver) + qemuDomainObjPrivatePtr priv) { g_autoptr(virCapsHostNUMA) caps = NULL; g_autofree char *nodeset = NULL; @@ -2513,7 +2512,7 @@ qemuDomainObjPrivateXMLParseAutomaticPlacement(xmlXPathContextPtr ctxt, if (!nodeset && !cpuset) return 0; - if (!(caps = virQEMUDriverGetHostNUMACaps(driver))) + if (!(caps = virCapabilitiesHostNUMANewHost())) return -1; /* Figure out how big the nodeset bitmap needs to be. @@ -3114,7 +3113,7 @@ qemuDomainObjPrivateXMLParse(xmlXPathContextPtr ctxt, } VIR_FREE(nodes); - if (qemuDomainObjPrivateXMLParseAutomaticPlacement(ctxt, priv, driver) < 0) + if (qemuDomainObjPrivateXMLParseAutomaticPlacement(ctxt, priv) < 0) goto error; if ((tmp = virXPathString("string(./libDir/@path)", ctxt))) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 8eaa3ce68f..d2077f8f16 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -1136,7 +1136,6 @@ qemuStateCleanup(void) virObjectUnref(qemu_driver->qemuCapsCache); virObjectUnref(qemu_driver->xmlopt); virCPUDefFree(qemu_driver->hostcpu); - virCapabilitiesHostNUMAUnref(qemu_driver->hostnuma); virObjectUnref(qemu_driver->caps); ebtablesContextFree(qemu_driver->ebtables); VIR_FREE(qemu_driver->qemuImgBinary); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 579b3c3713..49528f3dab 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -6197,8 +6197,7 @@ qemuProcessUpdateGuestCPU(virDomainDefPtr def, static int -qemuProcessPrepareDomainNUMAPlacement(virQEMUDriverPtr driver, - virDomainObjPtr vm) +qemuProcessPrepareDomainNUMAPlacement(virDomainObjPtr vm) { qemuDomainObjPrivatePtr priv = vm->privateData; g_autofree char *nodeset = NULL; @@ -6226,7 +6225,7 @@ qemuProcessPrepareDomainNUMAPlacement(virQEMUDriverPtr driver, if (virBitmapParse(nodeset, &numadNodeset, VIR_DOMAIN_CPUMASK_LEN) < 0) return -1; - if (!(caps = virQEMUDriverGetHostNUMACaps(driver))) + if (!(caps = virCapabilitiesHostNUMANewHost())) return -1; /* numad may return a nodeset that only contains cpus but cgroups don't play @@ -6428,7 +6427,7 @@ qemuProcessPrepareDomain(virQEMUDriverPtr driver, } virDomainAuditSecurityLabel(vm, true); - if (qemuProcessPrepareDomainNUMAPlacement(driver, vm) < 0) + if (qemuProcessPrepareDomainNUMAPlacement(vm) < 0) return -1; } -- 2.26.2

2 1

[PATCH v2 0/2] qemu: support some missing params on live attaching chardev
by Nikolay Shirokovskiy 02 Dec '20

02 Dec '20

Specifying chardev logfile and file backend append param is ignored yet. Diff from v1: - use existing virJSONValueObjectAdd instead of introducing virJSONValueObjectAppendBooleanTristate. Nikolay Shirokovskiy (2): qemu: support append param on live attaching file chardev qemu: support logfile on live attaching chardev src/qemu/qemu_monitor_json.c | 11 +++++++++++ 1 file changed, 11 insertions(+) -- 1.8.3.1

2 4

[libvirt PATCH] src: use singular form instead of plural, for guest disk info
by Daniel P. Berrangé 02 Dec '20

02 Dec '20

Existing practice with the filesystem fields reported for the virDomainGetGuestInfo API is to use the singular form for field names. Ensure the disk info follows this practice. Fixes commit 05a75ca2ce743bc0bb119fb8d532ff84646fafa3 Author: Marc-André Lureau <marcandre.lureau(a)redhat.com> Date: Fri Nov 20 22:09:46 2020 +0400 domain: add disk informations to virDomainGetGuestInfo commit 0cb2d9f05d00497a715352f6ea28cf8fb6921731 Author: Marc-André Lureau <marcandre.lureau(a)redhat.com> Date: Fri Nov 20 22:09:47 2020 +0400 qemu_driver: report guest disk informations Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- src/libvirt-domain.c | 14 +++++++------- src/qemu/qemu_driver.c | 14 +++++++------- 2 files changed, 14 insertions(+), 14 deletions(-) diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index 35e95e5395..f5cd43ecea 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -12345,17 +12345,17 @@ virDomainSetVcpu(virDomainPtr domain, * Returns information about the disks within the domain. The typed * parameter keys are in this format: * - * "disks.count" - the number of disks defined on this domain + * "disk.count" - the number of disks defined on this domain * as an unsigned int - * "disks.<num>.name" - device node (Linux) or device UNC (Windows) - * "disks.<num>.partition" - whether this is a partition or disk - * "disks.<num>.dependencies.count" - the number of device dependencies + * "disk.<num>.name" - device node (Linux) or device UNC (Windows) + * "disk.<num>.partition" - whether this is a partition or disk + * "disk.<num>.dependency.count" - the number of device dependencies * e.g. for LVs of the LVM this will * hold the list of PVs, for LUKS encrypted volume this will * contain the disk where the volume is placed. (Linux) - * "disks.<num>.dependencies.<num>.name" - a dependency - * "disks.<num>.alias" - the device alias of the disk (e.g. sda) - * "disks.<num>.guest_alias" - optional alias assigned to the disk, on Linux + * "disk.<num>.dependency.<num>.name" - a dependency + * "disk.<num>.alias" - the device alias of the disk (e.g. sda) + * "disk.<num>.guest_alias" - optional alias assigned to the disk, on Linux * this is a name assigned by device mapper * * VIR_DOMAIN_GUEST_INFO_HOSTNAME: diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 8eaa3ce68f..548df6ae68 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -19876,20 +19876,20 @@ qemuAgentDiskInfoFormatParams(qemuAgentDiskInfoPtr *info, size_t i, j, ndeps; if (virTypedParamsAddUInt(params, nparams, maxparams, - "disks.count", ndisks) < 0) + "disk.count", ndisks) < 0) return; for (i = 0; i < ndisks; i++) { char param_name[VIR_TYPED_PARAM_FIELD_LENGTH]; g_snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, - "disks.%zu.name", i); + "disk.%zu.name", i); if (virTypedParamsAddString(params, nparams, maxparams, param_name, info[i]->name) < 0) return; g_snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, - "disks.%zu.partition", i); + "disk.%zu.partition", i); if (virTypedParamsAddBoolean(params, nparams, maxparams, param_name, info[i]->partition) < 0) return; @@ -19897,14 +19897,14 @@ qemuAgentDiskInfoFormatParams(qemuAgentDiskInfoPtr *info, if (info[i]->dependencies) { ndeps = g_strv_length(info[i]->dependencies); g_snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, - "disks.%zu.dependencies.count", i); + "disk.%zu.dependency.count", i); if (ndeps && virTypedParamsAddUInt(params, nparams, maxparams, param_name, ndeps) < 0) return; for (j = 0; j < ndeps; j++) { g_snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, - "disks.%zu.dependencies.%zu.name", i, j); + "disk.%zu.dependency.%zu.name", i, j); if (virTypedParamsAddString(params, nparams, maxparams, param_name, info[i]->dependencies[j]) < 0) return; @@ -19922,7 +19922,7 @@ qemuAgentDiskInfoFormatParams(qemuAgentDiskInfoPtr *info, info[i]->address->unit); if (diskdef) { g_snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, - "disks.%zu.alias", i); + "disk.%zu.alias", i); if (diskdef->dst && virTypedParamsAddString(params, nparams, maxparams, param_name, diskdef->dst) < 0) @@ -19932,7 +19932,7 @@ qemuAgentDiskInfoFormatParams(qemuAgentDiskInfoPtr *info, if (info[i]->alias) { g_snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, - "disks.%zu.guest_alias", i); + "disk.%zu.guest_alias", i); if (virTypedParamsAddString(params, nparams, maxparams, param_name, info[i]->alias) < 0) return; -- 2.28.0

2 1

[libvirt PATCH] meson: add winsock2 library on windows builds
by Daniel P. Berrangé 02 Dec '20

02 Dec '20

If building for windows with curl disabled we get build failures due to missing ws2_32 library needed for winsock2. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- meson.build | 2 ++ 1 file changed, 2 insertions(+) diff --git a/meson.build b/meson.build index 01e7fbd409..b280809a65 100644 --- a/meson.build +++ b/meson.build @@ -1368,10 +1368,12 @@ libutil_dep = cc.find_library('util', required: false) if host_machine.system() == 'windows' ole32_dep = cc.find_library('ole32') oleaut32_dep = cc.find_library('oleaut32') + winsock2_dep = cc.find_library('ws2_32') win32_dep = declare_dependency( dependencies: [ ole32_dep, oleaut32_dep, + winsock2_dep, ], ) if get_option('default_library') == 'static' -- 2.28.0

2 1

[PATCH 0/3] qemu: support some missing params on live attaching chardev
by Nikolay Shirokovskiy 02 Dec '20

02 Dec '20

Specifying chardev logfile and file backend append param is ignored yet. Nikolay Shirokovskiy (3): util: add virJSONValueObjectAppendBooleanTristate qemu: support append param on live attaching file chardev qemu: support logfile on live attaching chardev src/libvirt_private.syms | 1 + src/qemu/qemu_monitor_json.c | 12 ++++++++++++ src/util/virjson.c | 25 +++++++++++++++++++++++++ src/util/virjson.h | 1 + 4 files changed, 39 insertions(+) -- 1.8.3.1

2 4

[PATCH 0/2] Replace VIR_AUTOSTRINGLIST with GStrv
by Michal Privoznik 02 Dec '20

02 Dec '20

2 3

[PATCH v2] qemuDomainGetGuestInfo: Exit early if getting info fails
by Michal Privoznik 02 Dec '20

02 Dec '20

If there is an error getting info from guest agent, then the control on qemuDomainGetGuestInfo() jumps onto 'exitagent' label and subsequently continues on 'endagentjob'. Both labels are hit also in success case too. The control then continues by attempting to match fetched info (e.g. disk addresses) with domain def. But this is needless - the API will return error regardless. To return early from the function move both 'exitagent' and 'endagentjob' labels at the end of the function and jump straight onto 'cleanup' afterwards. This allows us to set 'ret = 0' later - only when we know we succeeded. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- v2 of: https://www.redhat.com/archives/libvir-list/2020-December/msg00020.html src/qemu/qemu_driver.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 8eaa3ce68f..bca1c84630 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -20116,12 +20116,7 @@ qemuDomainGetGuestInfo(virDomainPtr dom, } } - ret = 0; - - exitagent: qemuDomainObjExitAgent(vm, agent); - - endagentjob: qemuDomainObjEndAgentJob(vm); if (nfs > 0 || ndisks > 0) { @@ -20143,6 +20138,8 @@ qemuDomainGetGuestInfo(virDomainPtr dom, qemuDomainObjEndJob(driver, vm); } + ret = 0; + cleanup: for (i = 0; i < nfs; i++) qemuAgentFSInfoFree(agentfsinfo[i]); @@ -20153,6 +20150,13 @@ qemuDomainGetGuestInfo(virDomainPtr dom, virDomainObjEndAPI(&vm); return ret; + + exitagent: + qemuDomainObjExitAgent(vm, agent); + + endagentjob: + qemuDomainObjEndAgentJob(vm); + goto cleanup; } -- 2.26.2

2 1

[Call for Presentations] FOSDEM 2021: Virt & IaaS Devroom
by Kashyap Chamarthy 02 Dec '20

02 Dec '20

[Cross-posting to KVM, QEMU, and libvirt lists] The Call For Papers for FOSDEM's Virt & IaaS Devroom went out yesterday[1]. Here's the text (slightly formatted for readability): ======================================================================= We are excited to announce that the call for proposals is now open for the Virtualization & IaaS devroom at the upcoming FOSDEM 2021, to be hosted virtually on February 6th 2021. This year will mark FOSDEM's 21th anniversary as one of the longest-running free and open source software developer events, attracting thousands of developers and users from all over the world. Due to Covid-19, FOSDEM will be held virtually this year on February 6th & 7th, 2021. Important Dates --------------- * Submission deadline: 20th of December * Acceptance notifications: 25th of December * Final schedule announcement: 31st of December * Recorded presentations upload deadline: 15th of January * Devroom: 6th February 2021 About the Devroom ----------------- The Virtualization & IaaS devroom will feature session topics such as open source hypervisors and virtual machine managers such as Xen Project, KVM, bhyve, and VirtualBox, and Infrastructure-as-a-Service projects such as KubeVirt, Apache CloudStack, Foreman, OpenStack, oVirt, QEMU and OpenNebula. This devroom will host presentations that focus on topics of shared interest, such as KVM; libvirt; shared storage; virtualized networking; cloud security; clustering and high availability; interfacing with multiple hypervisors; hyperconverged deployments; and scaling across hundreds or thousands of servers. Presentations in this devroom will be aimed at users or developers working on these platforms who are looking to collaborate and improve shared infrastructure or solve common problems. We seek topics that encourage dialog between projects and continued work post-FOSDEM. Submit Your Proposal -------------------- All submissions must be made via the Pentabarf event planning site[1]. If you have not used Pentabarf before, you will need to create an account. If you submitted proposals for FOSDEM in previous years, you can use your existing account. After creating the account, select Create Event to start the submission process. Make sure to select Virtualization and IaaS devroom from the Track list. Please fill out all the required fields, and provide a meaningful abstract and description of your proposed session. Submission Guidelines --------------------- We expect more proposals than we can possibly accept, so it is vitally important that you submit your proposal on or before the deadline. Late submissions are unlikely to be considered. All presentation slots are 30 minutes, with 20 minutes planned for presentations, and 10 minutes for Q&A. All presentations will need to be pre-recorded and put into our system at least a couple of weeks before the event. The presentations should be uploaded by 15th of January and made available under Creative Commons licenses. In the Submission notes field, please indicate that you agree that your presentation will be licensed under the CC-By-SA-4.0 or CC-By-4.0 license and that you agree to have your presentation recorded. For example: "If my presentation is accepted for FOSDEM, I hereby agree to license all recordings, slides, and other associated materials under the Creative Commons Attribution Share-Alike 4.0 International License. Sincerely, <NAME>." In the Submission notes field, please also confirm that if your talk is accepted, you will be able to attend the virtual FOSDEM event for the Q&A. We will not consider proposals from prospective speakers who are unsure whether they will be able to attend the FOSDEM virtual event. If you are experiencing problems with Pentabarf, the proposal submission interface, or have other questions, you can email our devroom mailing list[2] and we will try to help you. Code of Conduct --------------- Following the release of the updated code of conduct for FOSDEM, we'd like to remind all speakers and attendees that all of the presentations and discussions in our devroom are held under the guidelines set in the CoC and we expect attendees, speakers, and volunteers to follow the CoC at all times. If you submit a proposal and it is accepted, you will be required to confirm that you accept the FOSDEM CoC. If you have any questions about the CoC or wish to have one of the devroom organizers review your presentation slides or any other content for CoC compliance, please email us and we will do our best to assist you. Call for Volunteers ------------------- We are also looking for volunteers to help run the devroom. We need assistance with helping speakers to record the presentation as well as helping with streaming and chat moderation for the devroom. Please contact devroom mailing list [2] for more information. Questions? ---------- If you have any questions about this devroom, please send your questions to our devroom mailing list. You can also subscribe to the list to receive updates about important dates, session announcements, and to connect with other attendees. See you all at FOSDEM! [1] https://penta.fosdem.org/submission/FOSDEM21 [2] iaas-virt-devroom at lists.fosdem.org ======================================================================= [+] https://lists.fosdem.org/pipermail/fosdem/2020q4/003109.html -- /kashyap

1 0

[PATCH] coding-style: Document 80 chars limit for line length
by Michal Privoznik 02 Dec '20

02 Dec '20

The idea is to have it like a soft limit: if possible then break lines, if not then have a long line instead of some creative approach. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- docs/coding-style.rst | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/docs/coding-style.rst b/docs/coding-style.rst index cfd7b16638..813128bfb6 100644 --- a/docs/coding-style.rst +++ b/docs/coding-style.rst @@ -131,7 +131,7 @@ around operators and keywords: indent-libvirt() { - indent -bad -bap -bbb -bli4 -br -ce -brs -cs -i4 -l75 -lc75 \ + indent -bad -bap -bbb -bli4 -br -ce -brs -cs -i4 -l80 -lc80 \ -sbi4 -psl -saf -sai -saw -sbi4 -ss -sc -cdw -cli4 -npcs -nbc \ --no-tabs "$@" } @@ -141,6 +141,18 @@ further, by piping it through ``expand -i``, since some leading TABs can get through. Usually they're in macro definitions or strings, and should be converted anyhow. +The recommended length for lines is 80 characters, but common sense +should prevail. It may get tricky around some names (because of how +Libvirt constructs names for functions/enums/etc.) + +:: + + foo( + arg1, arg2 + ); // Bad + foo(arg1, + arg2); // Good + Libvirt requires a C99 compiler for various reasons. However, most of the code base prefers to stick to C89 syntax unless there is a compelling reason otherwise. For example, it is preferable to use -- 2.26.2

3 3

regression in meson build, AC_PATH_PROG lost
by Olaf Hering 02 Dec '20

02 Dec '20

autoconf allows to specify the path to a runtime binary that is not required during build via an environment variable: AC_PATH_PROG([PARTED], [parted], [], [$LIBVIRT_SBIN_PATH]) meson lacks such essential feature. As a result the package build environment needs to have more packages than necessary installed. This excessive list also pushes libvirt closer to the end of the build dependency chain. Was this considered while the buildsystem was switched to meson? Since meson does not support environment variables it seems the only way to address this is to introduce an option in meson_options.txt for each runtime executable. Will such change be accepted? Olaf

6 23

[libvirt PATCH] scripts: ignore whitespace in pdwtags output
by Daniel P. Berrangé 02 Dec '20

02 Dec '20

The pdwtags program changed its whitespace formatting for enum values in release 1.19: @@ -145,22 +145,22 @@ u_int flags; }; enum admin_procedure { - ADMIN_PROC_CONNECT_OPEN = 1, - ADMIN_PROC_CONNECT_CLOSE = 2, - ADMIN_PROC_CONNECT_GET_LIB_VERSION = 3, - ADMIN_PROC_CONNECT_LIST_SERVERS = 4, - ADMIN_PROC_CONNECT_LOOKUP_SERVER = 5, + ADMIN_PROC_CONNECT_OPEN = 1, + ADMIN_PROC_CONNECT_CLOSE = 2, + ADMIN_PROC_CONNECT_GET_LIB_VERSION = 3, + ADMIN_PROC_CONNECT_LIST_SERVERS = 4, + ADMIN_PROC_CONNECT_LOOKUP_SERVER = 5, Workaround this by telling diff to ignore whitespace changes. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- scripts/check-remote-protocol.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/check-remote-protocol.py b/scripts/check-remote-protocol.py index cd96e894a7..40c4b481be 100644 --- a/scripts/check-remote-protocol.py +++ b/scripts/check-remote-protocol.py @@ -121,7 +121,7 @@ if n < 1: print("WARNING: install dwarves-1.3 or newer", file=sys.stderr) sys.exit(8) -diff = subprocess.Popen(["diff", "-u", expected, "-"], stdin=subprocess.PIPE) +diff = subprocess.Popen(["diff", "-b", "-u", expected, "-"], stdin=subprocess.PIPE) actualstr = "\n".join(actual) + "\n" # i686 builds on x86_64 host add __attribute__(packed)) to # the structs. This doesn't matter functionally because we -- 2.25.4

2 1

[PATCH for 7.0.0 v1 00/26] Introduce virtio memory support
by Michal Privoznik 02 Dec '20

02 Dec '20

Available also here: https://gitlab.com/MichalPrivoznik/libvirt/-/commits/virtio_mem/ There are new virtio variants of pc-dimm and nvdimm devices. This is the first attempt to impalement support for them in libvirt. Thanks to David Hildenbrand for his valuable input! Michal Prívozník (26): viruuid: Rework virUUIDIsValid() internal.h: Introduce VIR_IS_POW2() docs: Fix nvdimm example wrt to <uuid/> domain_conf: Check NVDIMM UUID in ABI stability qemu_domain_address: Reformat qemuDomainAssignS390Addresses() conf: Require nvdimm path in validate step domain_conf: Fix virDomainMemoryModel type virDomainMemorySourceDefFormat: Utilize virXMLFormatElement() virDomainMemoryTargetDefFormat: Utilize virXMLFormatElement() qemu: Move mem validation into post parse validator conf: Move some of virDomainMemoryDef members into a union conf: Introduce virtio-pmem <memory/> model qemu_capabilities: Introduce QEMU_CAPS_DEVICE_VIRTIO_{P}MEM_PCI qemu_validate: Require virtio-mem device for mem model virtio security: Relabel virtio mem qemu: Allow virtio-pmem in CGroups qemu: Create virtio-pmem in domain namespace qemu_command: Move dimm into qemuBuildDeviceAddressStr() qemu: Build command line for virtio-pmem conf: Introduce virtio-mem <memory/> model qemu: Build command line for virtio-mem qemu: Wire up <memory/> live update qemu: Wire up MEMORY_DEVICE_SIZE_CHANGE event qemu: Refresh the actual size of virtio-mem on monitor reconnect virsh: Simplify @flags handing in cmdSetmem() and cmdSetmaxmem() virsh: Introduce --virtio to setmem docs/formatdomain.rst | 70 +++- docs/schemas/domaincommon.rng | 16 + src/conf/domain_conf.c | 372 ++++++++++++++---- src/conf/domain_conf.h | 38 +- src/internal.h | 10 + src/libvirt_private.syms | 2 + src/qemu/qemu_alias.c | 59 ++- src/qemu/qemu_capabilities.c | 4 + src/qemu/qemu_capabilities.h | 2 + src/qemu/qemu_cgroup.c | 43 +- src/qemu/qemu_command.c | 172 +++++--- src/qemu/qemu_command.h | 5 +- src/qemu/qemu_domain.c | 99 +++-- src/qemu/qemu_domain.h | 1 + src/qemu/qemu_domain_address.c | 98 +++-- src/qemu/qemu_domain_address.h | 3 +- src/qemu/qemu_driver.c | 198 +++++++++- src/qemu/qemu_hotplug.c | 22 +- src/qemu/qemu_hotplug.h | 5 + src/qemu/qemu_monitor.c | 37 ++ src/qemu/qemu_monitor.h | 27 ++ src/qemu/qemu_monitor_json.c | 94 +++-- src/qemu/qemu_monitor_json.h | 5 + src/qemu/qemu_namespace.c | 19 +- src/qemu/qemu_process.c | 52 ++- src/qemu/qemu_validate.c | 78 ++-- src/security/security_apparmor.c | 35 +- src/security/security_dac.c | 48 ++- src/security/security_selinux.c | 48 ++- src/security/virt-aa-helper.c | 22 +- src/util/virrandom.c | 2 +- src/util/viruuid.c | 17 +- src/util/viruuid.h | 2 +- .../caps_4.1.0.x86_64.xml | 1 + .../caps_4.2.0.x86_64.xml | 1 + .../caps_5.0.0.x86_64.xml | 1 + .../caps_5.1.0.x86_64.xml | 2 + .../caps_5.2.0.x86_64.xml | 2 + ...mory-hotplug-virtio-mem.x86_64-latest.args | 53 +++ .../memory-hotplug-virtio-mem.xml | 78 ++++ ...ory-hotplug-virtio-pmem.x86_64-latest.args | 45 +++ .../memory-hotplug-virtio-pmem.xml | 54 +++ tests/qemuxml2argvtest.c | 2 + ...emory-hotplug-virtio-mem.x86_64-latest.xml | 1 + ...mory-hotplug-virtio-pmem.x86_64-latest.xml | 1 + tests/qemuxml2xmltest.c | 3 + tools/virsh-domain.c | 138 ++++++- 47 files changed, 1703 insertions(+), 384 deletions(-) create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.xml create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-pmem.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-pmem.xml create mode 120000 tests/qemuxml2xmloutdata/memory-hotplug-virtio-mem.x86_64-latest.xml create mode 120000 tests/qemuxml2xmloutdata/memory-hotplug-virtio-pmem.x86_64-latest.xml -- 2.26.2

6 38

[PATCH] target/ppc: Remove "compat" property of server class POWER CPUs
by Greg Kurz 02 Dec '20

02 Dec '20

This property has been deprecated since QEMU 5.0 by commit 22062e54bb68. We only kept a legacy hack that internally converts "compat" into the official "max-cpu-compat" property of the pseries machine type. According to our deprecation policy, we could have removed it for QEMU 5.2 already. Do it now ; since ppc_cpu_parse_featurestr() now just calls the generic parent_parse_features handler, drop it as well. Users are supposed to use the "max-cpu-compat" property of the pseries machine type instead. Signed-off-by: Greg Kurz <groug(a)kaod.org> --- docs/system/deprecated.rst | 7 ---- target/ppc/translate_init.c.inc | 59 --------------------------------- 2 files changed, 66 deletions(-) diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst index 565389697e84..09c8f380bc82 100644 --- a/docs/system/deprecated.rst +++ b/docs/system/deprecated.rst @@ -281,13 +281,6 @@ a future version of QEMU. It's unclear whether anybody is still using CPU emulation in QEMU, and there are no test images available to make sure that the code is still working. -``compat`` property of server class POWER CPUs (since 5.0) -'''''''''''''''''''''''''''''''''''''''''''''''''''''''''' - -The ``compat`` property used to set backwards compatibility modes for -the processor has been deprecated. The ``max-cpu-compat`` property of -the ``pseries`` machine type should be used instead. - ``lm32`` CPUs (since 5.2.0) ''''''''''''''''''''''''''' diff --git a/target/ppc/translate_init.c.inc b/target/ppc/translate_init.c.inc index 78cc8f043b92..e4082cfde746 100644 --- a/target/ppc/translate_init.c.inc +++ b/target/ppc/translate_init.c.inc @@ -10470,63 +10470,6 @@ static ObjectClass *ppc_cpu_class_by_name(const char *name) return oc; } -static void ppc_cpu_parse_featurestr(const char *type, char *features, - Error **errp) -{ - Object *machine = qdev_get_machine(); - const PowerPCCPUClass *pcc = POWERPC_CPU_CLASS(object_class_by_name(type)); - - if (!features) { - return; - } - - if (object_property_find(machine, "max-cpu-compat")) { - int i; - char **inpieces; - char *s = features; - Error *local_err = NULL; - char *compat_str = NULL; - - /* - * Backwards compatibility hack: - * - * CPUs had a "compat=" property which didn't make sense for - * anything except pseries. It was replaced by "max-cpu-compat" - * machine option. This supports old command lines like - * -cpu POWER8,compat=power7 - * By stripping the compat option and applying it to the machine - * before passing it on to the cpu level parser. - */ - inpieces = g_strsplit(features, ",", 0); - *s = '\0'; - for (i = 0; inpieces[i]; i++) { - if (g_str_has_prefix(inpieces[i], "compat=")) { - warn_report_once("CPU 'compat' property is deprecated; " - "use max-cpu-compat machine property instead"); - compat_str = inpieces[i]; - continue; - } - if ((i != 0) && (s != features)) { - s = g_stpcpy(s, ","); - } - s = g_stpcpy(s, inpieces[i]); - } - - if (compat_str) { - char *v = compat_str + strlen("compat="); - object_property_set_str(machine, "max-cpu-compat", v, &local_err); - } - g_strfreev(inpieces); - if (local_err) { - error_propagate(errp, local_err); - return; - } - } - - /* do property processing with generic handler */ - pcc->parent_parse_features(type, features, errp); -} - PowerPCCPUClass *ppc_cpu_get_family_class(PowerPCCPUClass *pcc) { ObjectClass *oc = OBJECT_CLASS(pcc); @@ -10905,8 +10848,6 @@ static void ppc_cpu_class_init(ObjectClass *oc, void *data) device_class_set_parent_reset(dc, ppc_cpu_reset, &pcc->parent_reset); cc->class_by_name = ppc_cpu_class_by_name; - pcc->parent_parse_features = cc->parse_features; - cc->parse_features = ppc_cpu_parse_featurestr; cc->has_work = ppc_cpu_has_work; cc->do_interrupt = ppc_cpu_do_interrupt; cc->cpu_exec_interrupt = ppc_cpu_exec_interrupt; -- 2.26.2

3 3

[PATCH] polkit: Allow libvirt group access to libvirtd ro socket
by Jim Fehlig 02 Dec '20

02 Dec '20

As a normal user, 'virsh connect qemu:///system' and 'virsh connect --readonly qemu:///system' will prompt for root password. If the user is added to the libvirt group, only 'virsh connect --readonly qemu:///system' will prompt for root password. The libvirt polkit rules already allow libvirt group members access to the rw socket. Add a rule allowing to access the ro socket. Signed-off-by: Jim Fehlig <jfehlig(a)suse.com> --- src/remote/libvirtd.rules | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/src/remote/libvirtd.rules b/src/remote/libvirtd.rules index 01a15fac2e..d9be94fcc4 100644 --- a/src/remote/libvirtd.rules +++ b/src/remote/libvirtd.rules @@ -1,5 +1,12 @@ -// Allow any user in the 'libvirt' group to connect to system libvirtd -// without entering a password. +// Allow any user in the 'libvirt' group to connect to the system libvirtd +// ro and rw sockets without entering a password. + +polkit.addRule(function(action, subject) { + if (action.id == "org.libvirt.unix.monitor" && + subject.isInGroup("libvirt")) { + return polkit.Result.YES; + } +}); polkit.addRule(function(action, subject) { if (action.id == "org.libvirt.unix.manage" && -- 2.29.2

4 6

[PATCH 00/21] move checks from parse functions to post parse
by Daniel Henrique Barboza 01 Dec '20

01 Dec '20

Hi, This started as a simple NVDIMM change, then I realized there is a Gitlab work item for it [1], so I took the extra mile and did a bit more. I'll copy/paste here the motivation for this kind of change, provided by Cole in [1]: ----- The code that handles domain/VM XML parsing (virDomainDefParseXML in src/domain/domain_conf.c) has various validation checks sprinkled within it. Many of these checks should be moved out of the parse step and into virDomainDefPostParse, so they can be shared by various places in the code that build a domain definition without XML, such as converting from native vmware/virtualbox/xen formats. ----- There are still checks to be moved after this work. If this is accepted I'll update [1] with more details/tips about how to proceed with the remaining checks. Some g_auto* cleanups were done along the way. [1] https://gitlab.com/libvirt/libvirt/-/issues/7 Daniel Henrique Barboza (21): domain_conf.c: move NVDIMM 'labelsize' check to post parse domain_conf.c: use g_autofree in 'dev' in virDomainDefParseBootXML() domain_conf.c: modernize virDomainDefBootOrderPostParse() domain_conf.c: move boot related timeouts check to post parse domain_conf.c: do not leak 'video' in virDomainDefParseXML() domain_conf.c: move primary video check to virDomainDefPostParseVideo() domain_conf.c: use g_autoptr() with virDomainVideoDefPtr domain_conf.c: move QXL attributes check to virDomainVideoDefPostParse() virstorageencryption.h: add AUTOPTR_CLEANUP_FUNC for virStorageEncryptionPtr domain_conf: modernize virDomainDiskDefParseXML() domain_conf.c: move vendor, product and tray checks to post parse domain_conf.c: move smartcard address check to post parse domain_conf.c: modernize virDomainSmartcardDefParseXML domain_conf.c: remove 'error' label in virDomainDefTunablesParse() domain_conf.c: move duplicate blkio path check to post parse domain_conf.c: move virDomainPCIControllerOpts checks to post parse domain_conf.c: move pci-root/pcie-root address check to post parse domain_conf.c: modernize virDomainControllerDefParseXML() domain_conf.c: modernize virDomainDefControllersParse() domain_conf.c: use VIR_ERR_CONFIG_UNSUPPORTED in post parse domain_conf.c: move idmapEntry checks to post parse src/conf/domain_conf.c | 762 ++++++++++-------- src/conf/domain_conf.h | 4 + src/util/virstorageencryption.h | 1 + tests/qemuxml2argvdata/pci-root-address.err | 2 +- .../pseries-default-phb-numa-node.err | 2 +- .../video-multiple-primaries.err | 1 + .../video-multiple-primaries.xml | 32 + tests/qemuxml2argvtest.c | 5 + 8 files changed, 451 insertions(+), 358 deletions(-) create mode 100644 tests/qemuxml2argvdata/video-multiple-primaries.err create mode 100644 tests/qemuxml2argvdata/video-multiple-primaries.xml -- 2.26.2

2 38

[PATCH 0/3] qemu: Couple of qemuDomainGetGuestInfo() related cleanups
by Michal Privoznik 01 Dec '20

01 Dec '20

While reviewing Marc-André's patches I've noticed couple of cleanup possible. Michal Prívozník (3): qemuDomainGetGuestInfo: Exit early if getting info fails virJSONValueObjectGetStringArray: Report error if @key is not an array qemu: Use virJSONValueObjectGetStringArray() more src/qemu/qemu_agent.c | 30 +++---------------- src/qemu/qemu_driver.c | 3 ++ src/qemu/qemu_monitor_json.c | 56 +++++------------------------------- src/util/virjson.c | 6 +++- 4 files changed, 19 insertions(+), 76 deletions(-) -- 2.26.2

2 7

[libvirt PATCH] docs: Update language bindings spotlight
by Andrea Bolognani 01 Dec '20

01 Dec '20

We should highlight the language bindings that are actively maintained, keep up with the core library's development pace, have good API coverage and are relevant to people looking to integrate libvirt into their projects in $currentyear: based on these criteria, it makes sense to highlight the Go binding instead of the Java one. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- docs/index.html.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/index.html.in b/docs/index.html.in index df182c27b6..f44e055174 100644 --- a/docs/index.html.in +++ b/docs/index.html.in @@ -18,7 +18,7 @@ </p> <ul> <li>is a toolkit to manage <a href="platforms.html">virtualization platforms</a></li> - <li>is accessible from C, Python, Perl, Java and more</li> + <li>is accessible from C, Python, Perl, Go and more</li> <li>is licensed under open source licenses</li> <li>supports <a href="drvqemu.html">KVM</a>, <a href="drvqemu.html">QEMU</a>, <a href="drvxen.html">Xen</a>, -- 2.26.2

2 1

[PATCH] qemu: Recreate NUMA caps when creating connect capabilities
by Michal Privoznik 01 Dec '20

01 Dec '20

In v6.0.0-rc1~439 (and friends) we tried to cache NUMA capabilities because we assumed they are immutable. And to some extent they are (NUMA hotplug is not a thing, is it). However, our capabilities contain also some runtime info that can change, e.g. hugepages pool allocation sizes or total amount of memory per node (host side memory hotplug might change the value). When rebuilding virCaps (e.g. for 'virsh capabilities') we have to dropped the cached info and rebuild it from scratch so that the correct runtime info is reported. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1819058 Fixes: 1a1d848694f6c2f1d98a371124928375bc3bb4a3 Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- After this, we are still caching CPU caps, but I'm not sure if that is a problem. Perhaps if CPU was hotplugged/unplugged? I don't know. src/qemu/qemu_conf.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index d6615ca0dd..89a16349bf 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -1380,6 +1380,12 @@ virCapsPtr virQEMUDriverCreateCapabilities(virQEMUDriverPtr driver) "DOI \"%s\"", model, doi); } + /* Forcibly recreate NUMA caps. They are not static + * (e.g. size of hugepages pools can change). */ + qemuDriverLock(driver); + g_clear_pointer(&driver->hostnuma, virCapabilitiesHostNUMAUnref); + qemuDriverUnlock(driver); + caps->host.numa = virQEMUDriverGetHostNUMACaps(driver); caps->host.cpu = virQEMUDriverGetHostCPU(driver); return g_steal_pointer(&caps); -- 2.26.2

2 3

[PATCH v2] lxc: Cleanup after failed startup
by Michal Privoznik 01 Dec '20

01 Dec '20

If starting an container fails, the virLXCProcessStop() is called. But since vm->def->id is not set until libvirt_lxc is spawned (the domain's ID is PID of that process), virLXCProcessStop() returns early as virDomainObjIsActive() returns false. But doing so leaves behind resources reserved for the containers during the startup process. Most notably, hostdevs are not re-attached to the host, the domain's transient XML is not removed, etc. To resolve this, virLXCProcessCleanup() is called in this case. However, it is modified to accept @flags which allows caller to run only specific cleanups (depending how far in container creation the failure occurred). There is plenty of cleanups which don't need this guard because either they detect a NULL pointer or try to release an unique resource. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- v2 of: https://www.redhat.com/archives/libvir-list/2020-November/msg00317.html diff to v1: - Run some cleanups selectively src/lxc/lxc_process.c | 52 ++++++++++++++++++++++++++++++++++--------- 1 file changed, 41 insertions(+), 11 deletions(-) diff --git a/src/lxc/lxc_process.c b/src/lxc/lxc_process.c index c5a710fc3f..5763052e99 100644 --- a/src/lxc/lxc_process.c +++ b/src/lxc/lxc_process.c @@ -145,18 +145,27 @@ lxcProcessRemoveDomainStatus(virLXCDriverConfigPtr cfg, } +typedef enum { + VIR_LXC_PROCESS_CLEANUP_RELEASE_SECLABEL = (1 << 0), + VIR_LXC_PROCESS_CLEANUP_RESTORE_SECLABEL = (1 << 1), + VIR_LXC_PROCESS_CLEANUP_REMOVE_TRANSIENT = (1 << 2), +} virLXCProcessCleanupFlags; + /** * virLXCProcessCleanup: * @driver: pointer to driver structure * @vm: pointer to VM to clean up * @reason: reason for switching the VM to shutoff state + * @flags: allows to run selective cleanups only * - * Cleanout resources associated with the now dead VM - * + * Clean out resources associated with the now dead VM. + * If @flags is zero then whole cleanup process is done, + * otherwise only selected sections are run. */ static void virLXCProcessCleanup(virLXCDriverPtr driver, virDomainObjPtr vm, - virDomainShutoffReason reason) + virDomainShutoffReason reason, + unsigned int flags) { size_t i; virLXCDomainObjPrivatePtr priv = vm->privateData; @@ -164,8 +173,11 @@ static void virLXCProcessCleanup(virLXCDriverPtr driver, virLXCDriverConfigPtr cfg = virLXCDriverGetConfig(driver); virConnectPtr conn = NULL; - VIR_DEBUG("Cleanup VM name=%s pid=%d reason=%d", - vm->def->name, (int)vm->pid, (int)reason); + VIR_DEBUG("Cleanup VM name=%s pid=%d reason=%d flags=0x%x", + vm->def->name, (int)vm->pid, (int)reason, flags); + + if (flags == 0) + flags = ~0; /* now that we know it's stopped call the hook if present */ if (virHookPresent(VIR_HOOK_DRIVER_LXC)) { @@ -177,9 +189,15 @@ static void virLXCProcessCleanup(virLXCDriverPtr driver, NULL, xml, NULL); } - virSecurityManagerRestoreAllLabel(driver->securityManager, - vm->def, false, false); - virSecurityManagerReleaseLabel(driver->securityManager, vm->def); + if (flags & VIR_LXC_PROCESS_CLEANUP_RESTORE_SECLABEL) { + virSecurityManagerRestoreAllLabel(driver->securityManager, + vm->def, false, false); + } + + if (flags & VIR_LXC_PROCESS_CLEANUP_RELEASE_SECLABEL) { + virSecurityManagerReleaseLabel(driver->securityManager, vm->def); + } + /* Clear out dynamically assigned labels */ if (vm->def->nseclabels && vm->def->seclabels[0]->type == VIR_DOMAIN_SECLABEL_DYNAMIC) { @@ -258,7 +276,9 @@ static void virLXCProcessCleanup(virLXCDriverPtr driver, NULL, xml, NULL); } - virDomainObjRemoveTransientDef(vm); + if (flags & VIR_LXC_PROCESS_CLEANUP_REMOVE_TRANSIENT) + virDomainObjRemoveTransientDef(vm); + virObjectUnref(cfg); virObjectUnref(conn); } @@ -904,7 +924,7 @@ int virLXCProcessStop(virLXCDriverPtr driver, } cleanup: - virLXCProcessCleanup(driver, vm, reason); + virLXCProcessCleanup(driver, vm, reason, 0); return 0; } @@ -1198,6 +1218,7 @@ int virLXCProcessStart(virConnectPtr conn, g_autoptr(virCgroup) selfcgroup = NULL; int status; g_autofree char *pidfile = NULL; + unsigned int stopFlags = 0; if (virCgroupNewSelf(&selfcgroup) < 0) return -1; @@ -1265,6 +1286,7 @@ int virLXCProcessStart(virConnectPtr conn, VIR_DEBUG("Setting current domain def as transient"); if (virDomainObjSetDefTransient(driver->xmlopt, vm, NULL) < 0) goto cleanup; + stopFlags |= VIR_LXC_PROCESS_CLEANUP_REMOVE_TRANSIENT; /* Run an early hook to set-up missing devices */ if (virHookPresent(VIR_HOOK_DRIVER_LXC)) { @@ -1312,11 +1334,13 @@ int virLXCProcessStart(virConnectPtr conn, goto cleanup; } virDomainAuditSecurityLabel(vm, true); + stopFlags |= VIR_LXC_PROCESS_CLEANUP_RELEASE_SECLABEL; VIR_DEBUG("Setting domain security labels"); if (virSecurityManagerSetAllLabel(driver->securityManager, vm->def, NULL, false, false) < 0) goto cleanup; + stopFlags |= VIR_LXC_PROCESS_CLEANUP_RESTORE_SECLABEL; VIR_DEBUG("Setting up consoles"); for (i = 0; i < vm->def->nconsoles; i++) { @@ -1525,7 +1549,13 @@ int virLXCProcessStart(virConnectPtr conn, } if (rc != 0) { virErrorPreserveLast(&err); - virLXCProcessStop(driver, vm, VIR_DOMAIN_SHUTOFF_FAILED); + if (virDomainObjIsActive(vm)) { + virLXCProcessStop(driver, vm, VIR_DOMAIN_SHUTOFF_FAILED); + } else { + /* virLXCProcessStop() is NOP if the container is not active. + * If there was a failure whilst creating it, cleanup manually. */ + virLXCProcessCleanup(driver, vm, VIR_DOMAIN_SHUTOFF_FAILED, stopFlags); + } } virCommandFree(cmd); for (i = 0; i < nttyFDs; i++) -- 2.26.2

1 1

[libvirt PATCH 0/9] qemu: report guest disks informations
by marcandre.lureau＠redhat.com 01 Dec '20

01 Dec '20

From: Marc-André Lureau <marcandre.lureau(a)redhat.com> Hi, The following series extends virDomainGetGuestInfo to report disk informations provided by the new QGA "guest-get-disks" command in QEMU 5.2. Please review, Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1899527 Marc-André Lureau (9): qemu_agent: rename qemuAgentDiskInfo->qemuAgentDiskAddress qemu_agent: export qemuAgentDiskAddressFree & add g_auto qemu_agent: factor out qemuAgentGetDiskAddress util: json: add virJSONValueObjectGetStringArray convenience qemu: use virJSONValueObjectGetStringArray qemu_agent: add qemuAgentGetDisks domain: add disk informations to virDomainGetGuestInfo qemu_driver: report guest disk informations virsh: add --disk informations to guestinfo command include/libvirt/libvirt-domain.h | 1 + src/libvirt-domain.c | 17 +++ src/libvirt_private.syms | 1 + src/qemu/qemu_agent.c | 182 +++++++++++++++++++++++-------- src/qemu/qemu_agent.h | 25 ++++- src/qemu/qemu_driver.c | 103 ++++++++++++++++- src/qemu/qemu_monitor_json.c | 34 +----- src/util/virjson.c | 30 +++++ src/util/virjson.h | 1 + tests/qemuagenttest.c | 111 +++++++++++++++++++ tools/virsh-domain.c | 6 + 11 files changed, 430 insertions(+), 81 deletions(-) -- 2.29.0

5 18

[PATCH v2 0/2] gitlab: Add issue templates
by Peter Krempa 01 Dec '20

01 Dec '20

Gitlab allows projects to define templates for filing issues [1]. Libvirt didn't have them yet. We can use them to encourage users to file better bug reports and feature requests. Differences to v1: - dropped bug-qemu.md template - vastly shortened comment blocks - all comments are single line only to prevent users adding info inside of them - more fill-in fields instead of comment guidance You can test-drive the proposed templates in my test repo: https://gitlab.com/pipo.sk/test/-/issues/new Peter Krempa (2): gitlab: Add issue template for reporting a generic bug gitlab: Add issue template for a feature request .gitlab/issue_templates/bug.md | 26 ++++++++++++++++++++++++++ .gitlab/issue_templates/feature.md | 14 ++++++++++++++ 2 files changed, 40 insertions(+) create mode 100644 .gitlab/issue_templates/bug.md create mode 100644 .gitlab/issue_templates/feature.md -- 2.28.0

2 5

Release of libvirt-6.10.0
by Jiri Denemark 01 Dec '20

01 Dec '20

The 6.10.0 release of both libvirt and libvirt-python is tagged and signed tarballs and source RPMs are available at https://libvirt.org/sources/ https://libvirt.org/sources/python/ Thanks everybody who helped with this release by sending patches, reviewing, testing, or providing any other feedback. Your work is greatly appreciated. * Security * qemu: Enable client TLS certificate validation by default for ``chardev``, ``migration``, and ``backup`` servers. The default value if qemu.conf options ``chardev_tls_x509_verify``, ``migrate_tls_x509_verify``, or ``backup_tls_x509_verify`` are not specified explicitly in the config file and also the ``default_tls_x509_verify`` config option is missing are now '1'. This ensures that only legitimate clients access servers, which don't have any additional form of authentication. * New features * qemu: Implement OpenSSH authorized key file management APIs New APIs (``virDomainAuthorizedSSHKeysGet()`` and ``virDomainAuthorizedSSHKeysSet()``) and virsh commands (``get-user-sshkeys`` and ``set-user-sshkeys``) are added to manage authorized_keys SSH file for user. * hyperv: implement new APIs The ``virDomainGetMaxMemory()``, ``virDomainSetMaxMemory()``, ``virDomainGetSchedulerType()``, ``virDomainGetSchedulerParameters()``, ``virDomainGetSchedulerParametersFlags()``, ``virDomainGetVcpus()``, ``virDomainGetVcpusFlags()``, ``virDomainGetMaxVcpus()``, ``virDomainSetVcpus()``, and ``virDomainSetVcpusFlags()`` APIs have been implemented in the Hyper-V driver. * Improvements * virsh: Support network disks in ``virsh attach-disk`` The ``virsh attach-disk`` helper command which simplifies attaching of disks without the need for the user to formulate the disk XML manually now supports network-backed images. Users can specify the protocol and host specification with new command line arguments. Please refer to the man page of virsh for further information. * Bug fixes * remote: fixed performance regression in SSH tunnelling The ``virt-ssh-helper`` binary introduced in 6.8.0 had very poor scalability which impacted libvirt tunnelled migration and storage volume upload/download in particular. It has been updated and now has performance on par with netcat. * Removed features * hyperv: removed support for the Hyper-V V1 WMI API This drops support for Windows Server 2008R2 and 2012. The earliest supported version is now Windows 2012R2. Enjoy. Jirka

1 0

[PATCH 00/24] qemu: Prepare for QAPIfied 'object-add'
by Peter Krempa 30 Nov '20

30 Nov '20

Prepare libvirt for the changes proposed for qemu: https://www.redhat.com/archives/libvir-list/2020-November/msg01580.html Patches 1-12/24 are pure refactors and can be merged right away. Patches 13-15/24 can be merged, but the justification will be weak without the rest. Patches 16-22/24 are RFC until the qemu bits are merged. Patch 23/24 adds capabilities and needs to be udpated later Patch 24/24 is not to be merged! Note that patch 23/24 is intentionally missing from the mailing list posting as it's just a dump of QEMU capabilities from newest qemu. No need to litter the list with it. Also note that after the last patch tests will fail. See that patch for the reason. Fetch the full series at: git fetch https://gitlab.com/pipo.sk/libvirt.git object-add-qapi-3 Peter Krempa (24): qemuMonitorJSONSetMigrationParams: Take double pointer for @params qemuMonitorSetMigrationCapabilities: Take double pointer for @caps qemuMonitorJSONSetMigrationCapabilities: Refactor cleanup testQemuMonitorJSONqemuMonitorJSONGetMigrationCapabilities: refactor cleanup qemuMonitorJSONAddObject: Take double pointer for @props qemuMonitorJSONMakeCommandInternal: Clear @arguments when stolen qemuMonitorAddObject: Fix semantics of @alias qemuMonitorAddObject: Refactor cleanup util: json: Replace virJSONValueObjectSteal by virJSONValueObjectRemoveKey tests: qemuxml2argv: Don't check whether -netdev was QAPIfied repeatedly qemuBuildChrChardevStr: Rename 'flags' to 'cdevflags' testCompareXMLToArgvValidateSchema: Populate autoNodeset qemu: command: Generate commandline of 'masterKey0' secret via JSON qemu: command: Generate commandline of 'sev0' sev-guest object via JSON qemu: command: Generate commandline of iothread objects JSON qemu: capabilities: Introduce QEMU_CAPS_OBJECT_QAPIFIED tests: qemuxml2argv: Validate generation of JSON props for object-add qemu: command: Introduce raw JSON passthrough for '-object' for testing qemu: monitor: Make wrapping of 'props' of 'object-add' optional qemuMonitorCreateObjectPropsWrap: Open-code in qemuBuildMemoryBackendProps qemu: monitor: Don't add 'props' wrapper if qemu has QEMU_CAPS_OBJECT_QAPIFIED qemumonitorjsontest: Remove bomb guarding object-add qemu: capabilities: Add 6.0 capabilities with qapified object-add [DONTMERGE] qemuxml2argvtest: Force QMP validation with latest caps src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 332 +- src/qemu/qemu_migration_params.c | 22 +- src/qemu/qemu_monitor.c | 121 +- src/qemu/qemu_monitor.h | 8 +- src/qemu/qemu_monitor_json.c | 57 +- src/qemu/qemu_monitor_json.h | 6 +- src/util/virjson.c | 29 +- src/util/virqemu.c | 48 +- src/util/virqemu.h | 3 +- .../domaincapsdata/qemu_6.0.0-q35.x86_64.xml | 197 + .../domaincapsdata/qemu_6.0.0-tcg.x86_64.xml | 203 + tests/domaincapsdata/qemu_6.0.0.x86_64.xml | 197 + .../caps_6.0.0.x86_64.replies | 32021 ++++++++++++++++ .../caps_6.0.0.x86_64.xml | 3205 ++ tests/qemumonitorjsontest.c | 40 +- ...v-missing-platform-info.x86_64-2.12.0.args | 2 +- .../launch-security-sev.x86_64-2.12.0.args | 2 +- tests/qemuxml2argvtest.c | 44 +- 20 files changed, 36202 insertions(+), 338 deletions(-) create mode 100644 tests/domaincapsdata/qemu_6.0.0-q35.x86_64.xml create mode 100644 tests/domaincapsdata/qemu_6.0.0-tcg.x86_64.xml create mode 100644 tests/domaincapsdata/qemu_6.0.0.x86_64.xml create mode 100644 tests/qemucapabilitiesdata/caps_6.0.0.x86_64.replies create mode 100644 tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml -- 2.28.0

1 23

[PATCH 0/3] qemu: Remove caching in handler for query-command-line-options
by Peter Krempa 30 Nov '20

30 Nov '20

Shuffle around where we extract the data, so that we don't have to call the monitor multiple times which necessitates caching. Peter Krempa (3): qemu: monitor: Implement new handlers for 'query-command-line-options' virQEMUCapsProbeQMPCommandLine: Rewrite using qemuMonitorGetCommandLineOptions qemuMonitorGetCommandLineOptionParameters: remove the unused function and helpers src/qemu/qemu_capabilities.c | 34 +++++---- src/qemu/qemu_monitor.c | 34 +-------- src/qemu/qemu_monitor.h | 9 +-- src/qemu/qemu_monitor_json.c | 138 +++++++++-------------------------- src/qemu/qemu_monitor_json.h | 6 +- tests/qemumonitorjsontest.c | 117 ----------------------------- 6 files changed, 58 insertions(+), 280 deletions(-) -- 2.28.0

2 4

Re: [PATCH for-6.0 6/6] qapi: Deprecate 'query-kvm'
by Daniel P. Berrangé 30 Nov '20

30 Nov '20

Copying libvir-list for the deprecation warning. On Mon, Nov 16, 2020 at 04:10:11PM +0300, Roman Bolshakov wrote: > 'query-accel' QMP command should be used instead. > > Signed-off-by: Roman Bolshakov <r.bolshakov(a)yadro.com> > --- > qapi/machine.json | 6 +++++- > 1 file changed, 5 insertions(+), 1 deletion(-) docs/system/deprecated.rst needs to be updated for any deprecation to be visible to consumers of QEMU. > > diff --git a/qapi/machine.json b/qapi/machine.json > index 5648d8d24d..130b0dbebc 100644 > --- a/qapi/machine.json > +++ b/qapi/machine.json > @@ -579,6 +579,9 @@ > # > # Returns information about KVM acceleration > # > +# Features: > +# @deprecated: This command is deprecated, use 'query-accel' instead. > +# > # Returns: @AccelInfo > # > # Since: 0.14.0 > @@ -589,7 +592,8 @@ > # <- { "return": { "enabled": true, "present": true } } > # > ## > -{ 'command': 'query-kvm', 'returns': 'AccelInfo' } > +{ 'command': 'query-kvm', 'returns': 'AccelInfo', > + 'features': [ 'deprecated' ] } > > ## > # @query-accel: > -- > 2.29.2 > > Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

5 11

[libvirt PATCH] vircgroupv2: fix virCgroupV2DenyDevice
by Pavel Hrdina 30 Nov '20

30 Nov '20

The original logic is incorrect. We would delete the device entry from eBPF map only if the newval would be same as current val in the map. In case that the device was allowed only as read-only but later we remove all permissions for that device it would remain in the table with empty values. The old code would still deny the device but it's not working as intended. Instead we will update the value in advance. If the updated value is 0 it means that we are removing all permissions so it should be removed from the map, otherwise we will update the value in map. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1810356 Signed-off-by: Pavel Hrdina <phrdina(a)redhat.com> --- src/util/vircgroupv2.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/util/vircgroupv2.c b/src/util/vircgroupv2.c index 2b32f614e4..40cfa20376 100644 --- a/src/util/vircgroupv2.c +++ b/src/util/vircgroupv2.c @@ -1796,7 +1796,9 @@ virCgroupV2DenyDevice(virCgroupPtr group, return 0; } - if (newval == val) { + val = val & ~newval; + + if (val == 0) { if (virBPFDeleteElem(group->unified.devices.mapfd, &key) < 0) { virReportSystemError(errno, "%s", _("failed to remove device from BPF cgroup map")); @@ -1804,7 +1806,6 @@ virCgroupV2DenyDevice(virCgroupPtr group, } group->unified.devices.count--; } else { - val ^= val & newval; if (virBPFUpdateElem(group->unified.devices.mapfd, &key, &val) < 0) { virReportSystemError(errno, "%s", _("failed to update device in BPF cgroup map")); -- 2.28.0

2 1

[PATCH 0/3] gitlab: Add issue templates
by Peter Krempa 30 Nov '20

30 Nov '20

Gitlab allows projects to define templates for filing issues [1]. Libvirt didn't have them yet. We can use them to encourage users to file better bug reports and feature requests. You can test-drive the proposed templates in my test repo: https://gitlab.com/pipo.sk/test/-/issues/new?issue%5Bassignee_id%5D=&issue%… Any suggestions on how to improve them further are very welcome. [1] https://docs.gitlab.com/ee/user/project/description_templates.html Peter Krempa (3): gitlab: Add issue template for reporting a generic bug gitlab: Add issue template for a feature request gitlab: Add issue template for filing a bug in libvirt's qemu driver .gitlab/issue_templates/bug-qemu.md | 56 +++++++++++++++++++++++++++++ .gitlab/issue_templates/bug.md | 55 ++++++++++++++++++++++++++++ .gitlab/issue_templates/feature.md | 26 ++++++++++++++ 3 files changed, 137 insertions(+) create mode 100644 .gitlab/issue_templates/bug-qemu.md create mode 100644 .gitlab/issue_templates/bug.md create mode 100644 .gitlab/issue_templates/feature.md -- 2.28.0

4 10

[PATCH v2 0/6] Final changes to fix test suite on macOS
by Roman Bolshakov 27 Nov '20

27 Nov '20

The set of changes fixes qemuxml2argvtest and closes issue #58. Changes since v1: - Simplified lookup of symbols in main executable (Michal) - Reduced timeout to 90s for qemuxml2argvtest (Andrea) - Two new patches to fix qemucapsprobe Pipeline results: https://gitlab.com/roolebo/libvirt/-/pipelines/220225729 Roman Bolshakov (6): tests: Fix opendir mocks on macOS tests: Fix mock chaining on macOS qemuxml2argvtest: Increase timeout on macOS ci: Run test suite on macOS tests: Delay mock creation qemucapsprobemock: Fix lookup of qemu functions ci/cirrus/build.yml | 3 +-- tests/meson.build | 49 +++++++++++++++++++++++++----------------- tests/virfilewrapper.c | 4 ++++ tests/virmock.h | 13 +++++++++++ tests/virpcimock.c | 5 +++++ 5 files changed, 52 insertions(+), 22 deletions(-) -- 2.29.2

2 14

Entering freeze for libvirt-6.10.0
by Jiri Denemark 27 Nov '20

27 Nov '20

I have just tagged v6.10.0-rc1 in the repository and pushed signed tarballs and source RPMs to https://libvirt.org/sources/ Please give the release candidate some testing and in case you find a serious issue which should have a fix in the upcoming release, feel free to reply to this thread to make sure the issue is more visible. If you have not done so yet, please update NEWS.rst to document any significant change you made since the last release. Thanks, Jirka

1 1

QEMU Advent Calendar 2020 Call for Images
by Eldon Stegall 27 Nov '20

27 Nov '20

Hi, QEMU Advent Calendar 2020 is around the corner and we are looking for volunteers to contribute disk images that showcase something cool, bring back retro computing memories, or simply entertain with a puzzle or game. QEMU Advent Calendar publishes a QEMU disk image each day from December 1-24. Each image is a surprise designed to delight an audience consisting of the QEMU community and beyond. You can see previous years here: https://www.qemu-advent-calendar.org/ You can help us make this year's calendar awesome by: * Sending disk images ( or links to larger images ) * Replying with ideas for disk images (reply off-list to avoid spoilers!) If you have an idea after the start of the advent, go ahead and send it. We may find space to include it, or go ahead and get a jump on 2021! Here are the requirements for disk images: * Content must be freely redistributable (i.e. no proprietary license that prevents distribution). For GPL based software, you need to provide the source code, too. * Provide a name and a short description of the disk image (e.g. with hints on what to try) * Provide a ./run shell script that prints out the name and description/hints and launches QEMU * Provide a 320x240 screenshot/image/logo for the website * Size should be ideally under 100 MB per disk image (but if some few images are bigger, that should be OK, too) Check out this disk image as an example of how to distribute an image: https://www.qemu-advent-calendar.org/2018/download/day24.tar.xz PS: QEMU Advent Calendar is a secular calendar (not religious). The idea is to create a fun experience for the QEMU community which can be shared with everyone. You don't need to celebrate Christmas or another religious festival to participate! Thanks, and best wishes! Eldon

1 0

[libvirt PATCH v4] ci: Switch to meson build system
by Erik Skultety 27 Nov '20

27 Nov '20

Add meson required bits to the ci logic in the repo to be able to run a meson build in a container. This patch also drops several environment variables we don't need with meson anymore. Signed-off-by: Erik Skultety <eskultet(a)redhat.com> --- v3: https://www.redhat.com/archives/libvir-list/2020-November/msg01293.html Since v3: - further review comments - introduced CI_MESON_ARGS variable ci/Makefile | 38 +++++++------------------------------- ci/build.sh | 29 ++++++----------------------- 2 files changed, 13 insertions(+), 54 deletions(-) diff --git a/ci/Makefile b/ci/Makefile index c7c8eb9a45..d9d14f82c2 100644 --- a/ci/Makefile +++ b/ci/Makefile @@ -20,27 +20,6 @@ CI_HOST_SRCDIR = $(CI_SCRATCHDIR)/src # the $(CI_HOST_SRCDIR) directory from the host CI_CONT_SRCDIR = $(CI_USER_HOME)/libvirt -# Relative directory to perform the build in. This -# defaults to using a separate build dir, but can be -# set to empty string for an in-source tree build. -CI_VPATH = build - -# The directory holding the build output inside the -# container. -CI_CONT_BUILDDIR = $(CI_CONT_SRCDIR)/$(CI_VPATH) - -# Can be overridden with mingw{32,64}-configure if desired -CI_CONFIGURE = $(CI_CONT_SRCDIR)/configure - -# Default to using all possible CPUs -CI_SMP = $(shell getconf _NPROCESSORS_ONLN) - -# Any extra arguments to pass to make -CI_MAKE_ARGS = - -# Any extra arguments to pass to configure -CI_CONFIGURE_ARGS = - # Script containing environment preparation steps CI_PREPARE_SCRIPT = $(CI_ROOTDIR)/prepare.sh @@ -220,13 +199,10 @@ ci-run-command@%: ci-prepare-tree --login \ --user="#$(CI_UID)" \ --group="#$(CI_GID)" \ - CONFIGURE_OPTS="$$CONFIGURE_OPTS" \ + MESON_OPTS="$$MESON_OPTS" \ CI_CONT_SRCDIR="$(CI_CONT_SRCDIR)" \ - CI_CONT_BUILDDIR="$(CI_CONT_BUILDDIR)" \ - CI_SMP="$(CI_SMP)" \ - CI_CONFIGURE="$(CI_CONFIGURE)" \ - CI_CONFIGURE_ARGS="$(CI_CONFIGURE_ARGS)" \ - CI_MAKE_ARGS="$(CI_MAKE_ARGS)" \ + CI_MESON_ARGS="$(CI_MESON_ARGS)" \ + CI_NINJA_ARGS="$(CI_NINJA_ARGS)" \ $(CI_COMMAND) || exit 1' @test "$(CI_CLEAN)" = "1" && rm -rf $(CI_SCRATCHDIR) || : @@ -236,8 +212,8 @@ ci-shell@%: ci-build@%: $(MAKE) -C $(CI_ROOTDIR) ci-run-command@$* CI_COMMAND="$(CI_USER_HOME)/build" -ci-check@%: - $(MAKE) -C $(CI_ROOTDIR) ci-build@$* CI_MAKE_ARGS="check" +ci-test@%: + $(MAKE) -C $(CI_ROOTDIR) ci-build@$* CI_NINJA_ARGS+=test ci-list-images: @echo @@ -266,6 +242,6 @@ ci-help: @echo " CI_CLEAN=0 - do not delete '$(CI_SCRATCHDIR)' after completion" @echo " CI_REUSE=1 - re-use existing '$(CI_SCRATCHDIR)' content" @echo " CI_ENGINE=auto - container engine to use (podman, docker)" - @echo " CI_CONFIGURE_ARGS= - extra arguments passed to configure" - @echo " CI_MAKE_ARGS= - extra arguments passed to make, e.g. space delimited list of targets" + @echo " CI_MESON_ARGS= - extra arguments passed to meson" + @echo " CI_NINJA_ARGS= - extra arguments passed to ninja" @echo diff --git a/ci/build.sh b/ci/build.sh index 2da84c080a..740b46a935 100644 --- a/ci/build.sh +++ b/ci/build.sh @@ -7,32 +7,15 @@ # # to make. -mkdir -p "$CI_CONT_BUILDDIR" || exit 1 -cd "$CI_CONT_BUILDDIR" +cd "$CI_CONT_SRCDIR" export VIR_TEST_DEBUG=1 -NOCONFIGURE=1 "$CI_CONT_SRCDIR/autogen.sh" || exit 1 -# $CONFIGURE_OPTS is a env that can optionally be set in the container, +# $MESON_OPTS is an env that can optionally be set in the container, # populated at build time from the Dockerfile. A typical use case would -# be to pass --host/--target args to trigger cross-compilation -# -# This can be augmented by make local args in $CI_CONFIGURE_ARGS -"$CI_CONFIGURE" $CONFIGURE_OPTS $CI_CONFIGURE_ARGS -if test $? != 0; then - test -f config.log && cat config.log - exit 1 -fi -find -name test-suite.log -delete +# be to pass options to trigger cross-compilation -make -j"$CI_SMP" $CI_MAKE_ARGS +meson build --werror $MESON_OPTS $CI_MESON_ARGS || \ +(cat build/meson-logs/meson-log.txt && exit 1) -if test $? != 0; then \ - LOGS=$(find -name test-suite.log) - if test "$LOGS"; then - echo "=== LOG FILE(S) START ===" - cat $LOGS - echo "=== LOG FILE(S) END ===" - fi - exit 1 -fi +ninja -C build $CI_NINJA_ARGS -- 2.26.2

2 4

[PATCH] NEWS: add note about virt-ssh-helper perf improvements
by Daniel P. Berrangé 27 Nov '20

27 Nov '20

Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- NEWS.rst | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/NEWS.rst b/NEWS.rst index aa8a217eb6..8245e34efa 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -44,6 +44,13 @@ v6.10.0 (unreleased) * **Bug fixes** + * remote: fixed performance regression in SSH tunnelling + + The virt-ssh-helper binary introduced in 6.9.0 had very + poor scalability which impacted libvirt tunnelled migration + and storage volume upload/download in particular. It has been + updated and now has performance on a par with netcat. + * **Removed features** * hyperv: removed support for the Hyper-V V1 WMI API -- 2.28.0

2 1

[libvirt][RFC PATCH] add a new 'default' option for attribute mode in numatune
by Luyao Zhong 27 Nov '20

27 Nov '20

Hi Libvirt experts, I would like enhence the numatune snippet configuration. Given a example snippet: <domain> ... <numatune> <memory mode="strict" nodeset="1-4,^3"/> <memnode cellid="0" mode="strict" nodeset="1"/> <memnode cellid="2" mode="preferred" nodeset="2"/> </numatune> ... </domain> Currently, attribute mode is either 'interleave', 'strict', or 'preferred', I propose to add a new 'default' option. I give the reason as following. Presume we are using cgroups v1, Libvirt sets cpuset.mems for all vcpu threads according to 'nodeset' in memory element. And translate the memnode element to qemu config options (--object memory-backend-ram) for per numa cell, which invoking mbind() system call at the end.[1] But what if we want using default memory policy and request each guest numa cell pinned to different host memory nodes? We can't use mbind via qemu config options, because (I quoto here) "For MPOL_DEFAULT, the nodemask and maxnode arguments must be specify the empty set of nodes." [2] So my solution is introducing a new 'default' option for attribute mode. e.g. <domain> ... <numatune> <memory mode="default" nodeset="1-2"/> <memnode cellid="0" mode="default" nodeset="1"/> <memnode cellid="1" mode="default" nodeset="2"/> </numatune> ... </domain> If the mode is 'default', libvirt should avoid generating qemu command line '--object memory-backend-ram', and invokes cgroups to set cpuset.mems for per guest numa combining with numa topology config. Presume the numa topology is : <cpu> ... <numa> <cell id='0' cpus='0-3' memory='512000' unit='KiB' /> <cell id='1' cpus='4-7' memory='512000' unit='KiB' /> </numa> ... </cpu> Then libvirt should set cpuset.mems to '1' for vcpus 0-3, and '2' for vcpus 4-7. Is this reasonable and feasible? Welcome any comments. Regards, Luyao [1]https://github.com/qemu/qemu/blob/f2a1cf9180f63e88bb38ff21c169da97c3f2bad… [2]https://man7.org/linux/man-pages/man2/mbind.2.html -- 2.25.1

4 25

[libvirt][PATCH v1 0/3] introduce 'restrictive' mode in numatune
by Luyao Zhong 27 Nov '20

27 Nov '20

Before this patch set, numatune only has three memory modes: static, interleave and prefered. These memory policies are ultimately set by mbind() system call. Memory policy could be 'hard coded' into the kernel, but none of above policies fit our requirment under this case. mbind() support default memory policy, but it requires a NULL nodemask. So obviously setting allowed memory nodes is cgroups' mission under this case. So we introduce a new option for mode in numatune named 'restrictive'. <numatune> <memory mode="restrictive" nodeset="1-4,^3"/> <memnode cellid="0" mode="restrictive" nodeset="1"/> <memnode cellid="2" mode="restrictive" nodeset="2"/> </numatune> The config above means we only use cgroups to restrict the allowed memory nodes and not setting any specific memory policies explicitly. RFC discussion: https://www.redhat.com/archives/libvir-list/2020-November/msg01256.html Regards, Luyao Luyao Zhong (3): docs: add docs for 'restrictive' option for mode in numatune schema: add 'restrictive' config option for mode in numatune qemu: add parser and formatter for 'restrictive' mode in numatune docs/formatdomain.rst | 7 +++- docs/schemas/domaincommon.rng | 2 + include/libvirt/libvirt-domain.h | 1 + src/conf/numa_conf.c | 9 +++++ src/qemu/qemu_command.c | 6 ++- src/qemu/qemu_process.c | 27 +++++++++++++ src/util/virnuma.c | 3 ++ .../numatune-memnode-invalid-mode.err | 1 + .../numatune-memnode-invalid-mode.xml | 33 +++++++++++++++ ...emnode-restrictive-mode.x86_64-latest.args | 40 +++++++++++++++++++ .../numatune-memnode-restrictive-mode.xml | 33 +++++++++++++++ tests/qemuxml2argvtest.c | 2 + ...memnode-restrictive-mode.x86_64-latest.xml | 40 +++++++++++++++++++ tests/qemuxml2xmltest.c | 1 + 14 files changed, 202 insertions(+), 3 deletions(-) create mode 100644 tests/qemuxml2argvdata/numatune-memnode-invalid-mode.err create mode 100644 tests/qemuxml2argvdata/numatune-memnode-invalid-mode.xml create mode 100644 tests/qemuxml2argvdata/numatune-memnode-restrictive-mode.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/numatune-memnode-restrictive-mode.xml create mode 100644 tests/qemuxml2xmloutdata/numatune-memnode-restrictive-mode.x86_64-latest.xml -- 2.25.4

1 3

[PATCH v2] gitlab-ci: publish test report as an artifact
by Paolo Bonzini 26 Nov '20

26 Nov '20

Since version 0.55, "meson test" produces JUnit XML in the meson-logs directory. The XML can be parsed by GitLab and showed as part of the CI report. However, if the build and tests are performed by "meson dist", the tests are performed in "meson dist"'s own build directory and the logs are not accessible. So switch from "ninja dist" to "meson dist --no-tests" after a separate build step that is shared by the normal and the DIST=skip cases. Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> --- v1->v2: only do it for new-enough distros For an example see https://gitlab.com/bonzini/libvirt/-/pipelines/221545357/test_report. Test durations however are not yet available in upstream Meson. --- .gitlab-ci.yml | 38 ++++++++++++++++++++++++++++++++++---- 1 file changed, 34 insertions(+), 4 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 6792accf8f..c4b54201f8 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -40,6 +40,36 @@ stages: <<: *container_job_definition allow_failure: true +# For new enough distros that have Meson 0.55, include the JUnit XML +# report in the artifacts. In order to preserve it, we cannot use +# "meson dist" and need to do the compilation test manually. +# Fortunately, "meson dist --no-tests" was also added in Meson 0.55. +.native_meson055_build_job_template: &native_meson055_build_job_definition + stage: builds + image: $CI_REGISTRY_IMAGE/ci-$NAME:latest + cache: + paths: + - ccache/ + key: "$CI_JOB_NAME" + before_script: + - *script_variables + script: + - meson build --werror || (cat build/meson-logs/meson-log.txt && exit 1) + - ninja -C build + - ninja -C build test + - DESTDIR=$PWD/install/ ninja -C build install + - meson dist -C build --no-tests + - if test -x /usr/bin/rpmbuild && test "$RPM" != "skip"; + then + rpmbuild --nodeps -ta build/meson-dist/libvirt-*.tar.xz; + fi + artifacts: + when: always + paths: + - build/meson-logs/ + reports: + junit: build/meson-logs/testlog.junit.xml + .native_build_job_template: &native_build_job_definition stage: builds image: $CI_REGISTRY_IMAGE/ci-$NAME:latest @@ -292,7 +322,7 @@ x64-debian-10-clang: CC: clang x64-debian-sid: - <<: *native_build_job_definition + <<: *native_meson055_build_job_definition needs: - x64-debian-sid-container variables: @@ -343,21 +373,21 @@ x64-fedora-31: RPM: skip x64-fedora-32: - <<: *native_build_job_definition + <<: *native_meson055_build_job_definition needs: - x64-fedora-32-container variables: NAME: fedora-32 x64-fedora-rawhide: - <<: *native_build_job_definition + <<: *native_meson055_build_job_definition needs: - x64-fedora-rawhide-container variables: NAME: fedora-rawhide x64-fedora-rawhide-clang: - <<: *native_build_job_definition + <<: *native_meson055_build_job_definition needs: - x64-fedora-rawhide-container variables: -- 2.28.0

1 0

[PATCH] NEWS: Mention network disk support in 'virsh attach-disk'
by Peter Krempa 26 Nov '20

26 Nov '20

Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- NEWS.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/NEWS.rst b/NEWS.rst index aa8a217eb6..8c2b5def0f 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -42,6 +42,14 @@ v6.10.0 (unreleased) * **Improvements** + * virsh: Support network disks in ``virsh attach-disk`` + + The ``virsh attach-disk`` helper command which simplifies attaching of disks + without the need for the user to formulate the disk XML manually now + supports network-backed images. Users can specify the protocol and host + specification with new command line arguments. Please refer to the man + page of virsh for further information. + * **Bug fixes** * **Removed features** -- 2.28.0

2 1

[PATCH] gitlab-ci: publish test report as an artifact
by Paolo Bonzini 26 Nov '20

26 Nov '20

"meson test" produces JUnit XML in the meson-logs directory. The XML can be parsed by GitLab and showed as part of the CI report. However, if the build and tests are performed by "meson dist", the tests are performed in "meson dist"'s own build directory and the logs are not accessible. So switch from "ninja dist" to "meson dist --no-tests" after a separate build step that is shared by the normal and the DIST=skip cases. Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> --- For an example see https://gitlab.com/bonzini/libvirt/-/pipelines/221545357/test_report. Test durations however are not yet available in upstream Meson. --- .gitlab-ci.yml | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 6792accf8f..ce7b60dc6b 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -51,17 +51,23 @@ stages: - *script_variables script: - meson build --werror || (cat build/meson-logs/meson-log.txt && exit 1) + - ninja -C build + - ninja -C build test + - DESTDIR=$PWD/install/ ninja -C build install - if test "$DIST" != "skip"; then - ninja -C build dist; - else - ninja -C build; - ninja -C build test; + meson dist -C build --no-tests; fi - if test -x /usr/bin/rpmbuild && test "$RPM" != "skip"; then rpmbuild --nodeps -ta build/meson-dist/libvirt-*.tar.xz; fi + artifacts: + when: always + paths: + - build/meson-logs/ + reports: + junit: build/meson-logs/testlog.junit.xml # Jobs that we delegate to Cirrus CI because they require an operating # system other than Linux. These jobs will only run if the required -- 2.28.0

1 0

[libvirt PATCH 0/2] fix regression in SSH tunnelling performance
by Daniel P. Berrangé 26 Nov '20

26 Nov '20

In testing the "vol-download" command in virsh, downloading a 1G file takes a ridiculous amount of time (minutes) with the new SSH helper. After the first patch is applied the time gets down to a much more reasonable 5.5 seconds on my test machine. By comparison netcat achieved 4 seconds. After applying the second patch, the time is reduced to 3.5 seconds, so we actually end up beating netcat, as long as we have glib >= 2.64.0 available. Daniel P. Berrangé (2): remote: make ssh-helper massively faster util: avoid glib event loop workaround where possible src/remote/remote_ssh_helper.c | 113 ++++++++++++++++++++------------- src/util/vireventglib.c | 29 ++++++--- 2 files changed, 89 insertions(+), 53 deletions(-) -- 2.25.4

4 8

Migration with "--p2p --tunnelled" hanging in v6.9.0
by Christian Ehrhardt 26 Nov '20

26 Nov '20

Hi, I'm wondering about the best next steps to debug a migration issue. What I found is that with libvirt v6.9.0 a migration hangs if used like: $ virsh migrate --unsafe --live --p2p --tunnelled h-migr-test \ qemu+ssh://testkvm-hirsute-to/system Just "--live --p2p" works fine. Also a bunch of other migration option combinations work with the same build/setup, just p2p+tunnelled fails. Also if either source or target are not on 6.9 the migration works (former version used is v6.6 for me). I looked at alternating qemu versions (5.0 / 5.1), but it had no impact. It only depends on libvirt to be on the new version on source&target to trigger the issue. Unfortunately there is no crash or error to debug into, it just gets stuck with the "virsh migrate" hanging on the source and never leaving the "paused" state on the target. I have compared setups with the least amount of "change": good: Qemu 5.1 + Libvirt 6.9 -> Qemu 5.1 / Libvirt 6.6 bad: Qemu 5.1 + Libvirt 6.9 -> Qemu 5.1 / Libvirt 6.9 [1] has the debug logs of those, beginning with a libvirtd restart that one can likely skip and then into the migration that hangs in the bad case. But I failed to see an obvious reason in the log. In git/news I only found these changes which sounded to be relevant: f51cbe92c0 qemu: Allow migration over UNIX socket c69915ccaf peer2peer migration: allow connecting to local sockets But I'm not using unix: and in the logs the only unix: mentions are for the qemu monitor and qemu-guest-agent. I wanted to ask: - if something related was recently changed that comes to mind? - if someone else sees anything in the linked logs that I missed? - if someone else has seen/reproduced the same? - for best practise to debug a hanging migration Thanks in advance! [1]: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1904584/+attachment/543… -- Christian Ehrhardt Staff Engineer, Ubuntu Server Canonical Ltd

3 12

[libvirt PATCH 0/4] cgroup cpu period and quota fixes
by Pavel Hrdina 25 Nov '20

25 Nov '20

Pavel Hrdina (4): qemu: move cgroup cpu period and quota defines to vircgroup.h vircgroupv1: use defines for cpu period and quota limits vircgroupv2: use defines for cpu period and quota limits vircgroup: fix cpu quota maximum limit src/qemu/qemu_driver.c | 21 ++++++++------------- src/util/vircgroup.h | 7 +++++++ src/util/vircgroupv1.c | 23 ++++++++++++----------- src/util/vircgroupv2.c | 25 +++++++++++++------------ 4 files changed, 40 insertions(+), 36 deletions(-) -- 2.26.2

2 5

[PATCH] qemu: Tweak debug message for qemuMigrationSrcPerformPeer2Peer3
by Martin Kletzander 25 Nov '20

25 Nov '20

Commit 49186372dbe8 forgot to add the new parameter. Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- Pushed as 'trivial'. src/qemu/qemu_migration.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 4be8f3c64c94..fcb33d0364a1 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -4356,12 +4356,12 @@ qemuMigrationSrcPerformPeer2Peer3(virQEMUDriverPtr driver, VIR_DEBUG("driver=%p, sconn=%p, dconn=%p, dconnuri=%s, vm=%p, xmlin=%s, " "dname=%s, uri=%s, graphicsuri=%s, listenAddress=%s, " - "nmigrate_disks=%zu, migrate_disks=%p, nbdPort=%d, " + "nmigrate_disks=%zu, migrate_disks=%p, nbdPort=%d, nbdURI=%s, " "bandwidth=%llu, useParams=%d, flags=0x%lx", driver, sconn, dconn, NULLSTR(dconnuri), vm, NULLSTR(xmlin), NULLSTR(dname), NULLSTR(uri), NULLSTR(graphicsuri), NULLSTR(listenAddress), nmigrate_disks, migrate_disks, nbdPort, - bandwidth, useParams, flags); + NULLSTR(nbdURI), bandwidth, useParams, flags); /* Unlike the virDomainMigrateVersion3 counterpart, we don't need * to worry about auto-setting the VIR_MIGRATE_CHANGE_PROTECTION -- 2.29.2

1 0

[libvirt PATCH 0/2] Add more documentation for migrations over UNIX sockets
by Martin Kletzander 24 Nov '20

24 Nov '20

Few words about SELinux that might not be very clear to some. Martin Kletzander (2): qemu: Disable NBD TLS migration over UNIX socket docs: Document SELinux caveats when migrating over UNIX sockets docs/manpages/virsh.rst | 9 ++++++++- docs/migration.html.in | 9 +++++++++ src/qemu/qemu_migration.c | 10 ++++++++-- 3 files changed, 25 insertions(+), 3 deletions(-) -- 2.29.2

2 4

[PATCH v2 0/5] Hypervisor CPU Baseline Cleanups and Fixes
by Collin Walling 24 Nov '20

24 Nov '20

The following patches provide some TLC to the hypervisor CPU baseline handler within the qemu_driver code. #1 checks for the cpu-model-expansion capability before executing the baseline handler since it is used for feature expansion. #2 fixes a styling where a < 0 condition was missing from one of the if (function()) lines for consistency's sake. #3 will check if the cpu definition(s) are valid and contain a model name. #4 checks the cpu definition(s) model names against the model names known by the hypervisor. This patch must come before #5. #5 will allow the baseline command to be ran with a single cpu definition, whereas before the command would simply fail with an unhelpful error message. A CPU model expansion will be performed in this case, which will produce the same result as if the model were actually baselined. Note: without patch #4, #5 can result in a segfault in the case where a single CPU model is provided and the model is not recognized by the hypervisor. This is because cpu model expansion will return 0 and the result will be NULL. Since the QMP response returns "GenericError" in the case of a missing CPU model, or if the command is not supported (and perhaps for other reasons I am unsure of -- the response does not explicitly detail that the CPU model provided was erroneous), we cannot rely on this response always meaning there was a missing CPU model. So, to be safe-and-sure, the CPU model is checked against the list of CPU models known to the hypervisor prior to baselining / expanding (which were retrieved at some point previously during libvirt init). Collin Walling (5): qemu: check for model-expansion cap before baselining qemu: fix one instance of rc check styling in baseline qemu: report error if missing model name when baselining qemu: check if cpu model is supported before baselining qemu: fix error message when baselining with a single cpu src/qemu/qemu_driver.c | 45 ++++++++++++++++++++++++++++++++++-------- 1 file changed, 37 insertions(+), 8 deletions(-) -- 2.26.2

2 10

[PATCH 0/8] run iptables directly rather than via firewalld
by Laine Stump 24 Nov '20

24 Nov '20

The reasoning for this is explained in Patch 8/8 Laine Stump (8): util: fix typo in VIR_MOCK_WRAP_RET_ARGS() util/tests: enable locking on iptables/ebtables commandlines in unit tests util/tests: enable locking on iptables/ebtables commandlines by default tests: fix iptables test case commandline options in virfirewalltest.c network: be more verbose about the reason for a firewall reload util: always check for ebtables/iptables binaries, even when using firewalld util: synchronize with firewalld before we start calling iptables directly util: call iptables directly rather than via firewalld src/libvirt_private.syms | 2 +- src/network/bridge_driver.c | 10 +- src/util/virfirewall.c | 155 +++--- src/util/virfirewall.h | 2 +- src/util/viriptables.c | 7 + tests/networkxml2firewalldata/base.args | 34 ++ .../nat-default-linux.args | 19 + .../nat-ipv6-linux.args | 30 ++ .../nat-ipv6-masquerade-linux.args | 34 ++ .../nat-many-ips-linux.args | 33 ++ .../nat-no-dhcp-linux.args | 29 ++ .../nat-tftp-linux.args | 21 + .../route-default-linux.args | 14 + tests/networkxml2firewalltest.c | 2 - tests/nwfilterebiptablestest.c | 466 +++++++++--------- .../ah-ipv6-linux.args | 9 + tests/nwfilterxml2firewalldata/ah-linux.args | 9 + .../all-ipv6-linux.args | 9 + tests/nwfilterxml2firewalldata/all-linux.args | 9 + tests/nwfilterxml2firewalldata/arp-linux.args | 5 + .../comment-linux.args | 19 + .../conntrack-linux.args | 7 + .../esp-ipv6-linux.args | 9 + tests/nwfilterxml2firewalldata/esp-linux.args | 9 + .../example-1-linux.args | 12 + .../example-2-linux.args | 10 + .../hex-data-linux.args | 10 + .../icmp-direction-linux.args | 6 + .../icmp-direction2-linux.args | 6 + .../icmp-direction3-linux.args | 6 + .../nwfilterxml2firewalldata/icmp-linux.args | 3 + .../icmpv6-linux.args | 4 + .../nwfilterxml2firewalldata/igmp-linux.args | 9 + tests/nwfilterxml2firewalldata/ip-linux.args | 3 + .../nwfilterxml2firewalldata/ipset-linux.args | 18 + .../ipt-no-macspoof-linux.args | 2 + .../nwfilterxml2firewalldata/ipv6-linux.args | 15 + .../nwfilterxml2firewalldata/iter1-linux.args | 9 + .../nwfilterxml2firewalldata/iter2-linux.args | 171 +++++++ .../nwfilterxml2firewalldata/iter3-linux.args | 15 + tests/nwfilterxml2firewalldata/mac-linux.args | 4 + .../nwfilterxml2firewalldata/rarp-linux.args | 6 + .../sctp-ipv6-linux.args | 9 + .../nwfilterxml2firewalldata/sctp-linux.args | 9 + tests/nwfilterxml2firewalldata/stp-linux.args | 11 + .../target-linux.args | 33 ++ .../target2-linux.args | 12 + .../tcp-ipv6-linux.args | 9 + tests/nwfilterxml2firewalldata/tcp-linux.args | 13 + .../udp-ipv6-linux.args | 9 + tests/nwfilterxml2firewalldata/udp-linux.args | 9 + .../udplite-ipv6-linux.args | 9 + .../udplite-linux.args | 9 + .../nwfilterxml2firewalldata/vlan-linux.args | 7 + tests/nwfilterxml2firewalltest.c | 146 +++--- tests/virfirewalltest.c | 236 ++++----- 56 files changed, 1259 insertions(+), 514 deletions(-) -- 2.28.0

2 12

[PATCH v3 0/2] hyperv: storage volume XML changes
by Matt Coleman 24 Nov '20

24 Nov '20

Changes since v2: * raise an error if the parent device cannot be located * correct the disk count and use constants instead of magic numbers Here's a GitLab MR if you'd prefer to review it there: https://gitlab.com/iammattcoleman/libvirt/-/merge_requests/12/commits This link shows a diff between v2 and v3: https://gitlab.com/iammattcoleman/libvirt/-/merge_requests/12/diffs?diff_id… Matt Coleman (2): hyperv: XML parsing of storage volumes schema: add support for Windows file paths and device names docs/schemas/basictypes.rng | 2 +- docs/schemas/domaincommon.rng | 5 +- src/hyperv/hyperv_driver.c | 425 +++++++++++++++++- src/hyperv/hyperv_driver.h | 5 + src/hyperv/hyperv_private.h | 2 + src/hyperv/hyperv_wmi.c | 45 ++ src/hyperv/hyperv_wmi.h | 8 + src/hyperv/hyperv_wmi_classes.h | 19 + src/hyperv/hyperv_wmi_generator.input | 134 ++++++ .../disk-hyperv-physical.xml | 17 + .../disk-hyperv-virtual.xml | 17 + .../disk-hyperv-physical.xml | 23 + .../disk-hyperv-virtual.xml | 23 + tests/genericxml2xmltest.c | 2 + 14 files changed, 724 insertions(+), 3 deletions(-) create mode 100644 tests/genericxml2xmlindata/disk-hyperv-physical.xml create mode 100644 tests/genericxml2xmlindata/disk-hyperv-virtual.xml create mode 100644 tests/genericxml2xmloutdata/disk-hyperv-physical.xml create mode 100644 tests/genericxml2xmloutdata/disk-hyperv-virtual.xml -- 2.27.0

2 3

[PATCH 0/7] qemu: migration: TLS enforcement and non-shared-storage error improvement
by Peter Krempa 24 Nov '20

24 Nov '20

A collection of fixes for migration. Peter Krempa (7): docs: migration: Fix example for unix socket migration qemu: migration: Remove TODO about implementing NBD for TUNNELLED migration qemu: migration: Aggregate logic depending on tunnelled migration qemu: migration: Forbid tunnelled non-shared storage migration with -blockdev docs: migration: Mention that features may not work with tunnelled migration qemu: conf: Introduce "migrate_tls_force" qemu.conf option docs: migration: Add a mention of VIR_MIGRATE_TLS and it's enforcement for qemu docs/migration.html.in | 21 ++++++++++- src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf | 8 ++++ src/qemu/qemu_conf.c | 2 + src/qemu/qemu_conf.h | 1 + src/qemu/qemu_migration.c | 60 ++++++++++++++++++++++-------- src/qemu/test_libvirtd_qemu.aug.in | 1 + 7 files changed, 78 insertions(+), 16 deletions(-) -- 2.28.0

2 10

[PATCH 0/2] qemu: Fix setting of 'current' checkpoint/snapshot
by Peter Krempa 24 Nov '20

24 Nov '20

See patch 1 Peter Krempa (2): qemu: snapshot: Write metadata of previously-'current' snapshot on update qemu: checkpoint: Write metadata of previously-'current' checkpoint on update src/qemu/qemu_checkpoint.c | 35 +++++++++++++++++++++++++++++++++-- src/qemu/qemu_snapshot.c | 38 +++++++++++++++++++++++++++++++++++--- 2 files changed, 68 insertions(+), 5 deletions(-) -- 2.28.0

2 3

[PATCH 0/5] qemu: Don't abort() when creating internal snapshot with <disk type='volume'
by Peter Krempa 24 Nov '20

24 Nov '20

See patch 5/5 Peter Krempa (5): virCommandAddArg: Don't abort on invalid input virDomainDiskTranslateSourcePool: Don't break error message in half qemuDomainSnapshotForEachQcow2Raw: Avoid a level of indentation qemuDomainSnapshotForEachQcow2Raw: Lock out operation on unsupported storage qemuDomainSnapshotForEachQcow2: Pass in 'def' rather than selecting it internally src/conf/domain_conf.c | 3 +- src/qemu/qemu_domain.c | 102 +++++++++++++++++++++------------------ src/qemu/qemu_domain.h | 2 +- src/qemu/qemu_snapshot.c | 16 ++++-- src/util/vircommand.c | 2 +- 5 files changed, 70 insertions(+), 55 deletions(-) -- 2.28.0

2 6