- Devel - libvirt List Archives

[libvirt] [PATCH 0/9] util/resctrl cleanups and refactors
by Wang Huaqiang 30 May '19

30 May '19

Plan to support Resctrl Control Monitors, which is a feature introduced by kernel 'resctrl' sub-model. Submit some cleanup and refactoring patches for upcoming memory bandwidth resource monitoring (MBM) monitors. Related MBM RFC is https://www.redhat.com/archives/libvir-list/2019-April/msg01409.html. This RFC is not actively discussed since libvirt already implemented similar resctrl cache monitoring (CMT), and lots details have been discussed and implemented during the work of CMT. The cleanups and refactoring includes: 1. Removing some reluctant lines and white spaces that is existing in current code and not meet the libvirt coding style. 2. Replace 'virResctrlAllocIsEmpty' with @n==0 for indicating no resctrl allocation in configuration file. 3. Private API changes, removed 'virResctrlMonitorGetCacheOccupancy' and exported a new API named 'virResctrlMonitorGetStats' with similar functionality, but with capability to be used for retrieving MBM statistical information. 4. Refactoring 'virResctrlMonitorFreeStats' for more reusing in code. 5. Extend data structure 'virResctrlMonitorStats' with the capability to carry multiple statistical information from monitor. Wang Huaqiang (9): conf: code cleanup, remove empty line and one space conf: code cleanup for return error code directly conf: check if vcpus matches with any 'existing' allocaiton conf: Replace 'virResctrlAllocIsEmpty' with @n==0 for indicating no resctrl allocation in configuration file conf: refactor 'virDomainResctrlVcpuMatch' util: Refactor 'virResctrlMonitorFreeStats' util: Refactor 'virResctrlMonitorStats' util: Extend virresctl API to retrieve multiple monitor statistics util: Remove unused virResctrlMonitorGetCacheOccupancy src/conf/domain_conf.c | 89 +++++++++++++++++++++++------------------------- src/libvirt_private.syms | 2 +- src/qemu/qemu_driver.c | 36 +++++++++++++++----- src/util/virresctrl.c | 68 +++++++++++++++--------------------- src/util/virresctrl.h | 19 ++++++++--- 5 files changed, 113 insertions(+), 101 deletions(-) -- 2.7.4

3 25

[libvirt] [PATCH v2 0/2] virSysinfo: Try dmidecode on aarch64
by Michal Privoznik 30 May '19

30 May '19

v2 of: https://www.redhat.com/archives/libvir-list/2019-May/msg00221.html diff to v1: - Pushed 1/3 and 2/3 from v1 as they were ACKed already - New patch 1/2 which renames virSysinfoReadX86 - Patch 2/2 then uses the renamed function Michal Prívozník (2): virsysinfo: Rename virSysinfoReadX86 to virSysinfoReadDMI virSysinfoReadARM: Try reading DMI table src/libvirt_private.syms | 2 +- src/util/virsysinfo.c | 22 ++++++++++++++++++++-- src/util/virsysinfopriv.h | 2 +- tests/sysinfotest.c | 2 +- 4 files changed, 23 insertions(+), 5 deletions(-) -- 2.21.0

3 5

[libvirt] [PATCH v2] qemu: aquire job in qemuDomainDefineXMLFlags
by Nikolay Shirokovskiy 30 May '19

30 May '19

The function updates vm->newDef and frees previous pesistent definition which can be used by in different thread by some API that unlocks domain to communicate to qemu. So we have an access to freed memory in that thread. Let's acquire job condition in case of updating exiting domain. Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy(a)virtuozzo.com> --- diff from v1 [1]: - pay a bit of attention to the problem Reproducer in the Michal's reply to the [1]. [1] https://www.redhat.com/archives/libvir-list/2019-May/msg00804.html src/qemu/qemu_driver.c | 37 +++++++++++++++++++++++++++++++++++-- 1 file changed, 35 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 42b1ce2..2dd4442 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -7620,6 +7620,7 @@ qemuDomainDefineXMLFlags(virConnectPtr conn, virDomainDefPtr def = NULL; virDomainDefPtr oldDef = NULL; virDomainObjPtr vm = NULL; + virDomainObjPtr exvm = NULL; virDomainPtr dom = NULL; virObjectEventPtr event = NULL; virQEMUDriverConfigPtr cfg; @@ -7647,10 +7648,34 @@ qemuDomainDefineXMLFlags(virConnectPtr conn, if (virDomainDefineXMLFlagsEnsureACL(conn, def) < 0) goto cleanup; + if ((exvm = virDomainObjListFindByUUID(driver->domains, def->uuid))) { + if (qemuDomainObjBeginJob(driver, exvm, QEMU_JOB_MODIFY) < 0) { + virDomainObjEndAPI(&exvm); + goto cleanup; + } + virObjectUnlock(exvm); + } + if (!(vm = virDomainObjListAdd(driver->domains, def, driver->xmlopt, - 0, &oldDef))) + 0, &oldDef))) { + if (exvm) + virObjectLock(exvm); goto cleanup; + } + + if (exvm) { + if (vm != exvm) { + virReportError(VIR_ERR_OPERATION_FAILED, + _("domain '%s' is removed"), def->name); + virObjectLock(exvm); + goto cleanup; + } + + /* we don't need extra reference for same domain */ + virObjectUnref(exvm); + } + def = NULL; vm->persistent = 1; @@ -7670,7 +7695,10 @@ qemuDomainDefineXMLFlags(virConnectPtr conn, /* Brand new domain. Remove it */ VIR_INFO("Deleting domain '%s'", vm->def->name); vm->persistent = 0; - qemuDomainRemoveInactiveJob(driver, vm); + if (exvm) + qemuDomainRemoveInactive(driver, vm); + else + qemuDomainRemoveInactiveJob(driver, vm); } goto cleanup; } @@ -7685,8 +7713,13 @@ qemuDomainDefineXMLFlags(virConnectPtr conn, dom = virGetDomain(conn, vm->def->name, vm->def->uuid, vm->def->id); cleanup: + if (exvm) + qemuDomainObjEndJob(driver, exvm); + virDomainDefFree(oldDef); virDomainDefFree(def); + if (exvm && exvm != vm) + virDomainObjEndAPI(&exvm); virDomainObjEndAPI(&vm); virObjectEventStateQueue(driver->domainEventState, event); virObjectUnref(caps); -- 1.8.3.1

2 1

[libvirt] [PATCH v2 0/2] introduction of version attribute for VFIO live migration
by Yan Zhao 29 May '19

29 May '19

This patchset introduces a version attribute under sysfs of VFIO Mediated devices. This version attribute is used to check whether two mdev devices are compatible. user space software can take advantage of this version attribute to determine whether to launch live migration between two mdev devices. Patch 1 defines version attribute in Documentation/vfio-mediated-device.txt Patch 2 uses GVT as an example to show how to expose version attribute and check device compatibility in vendor driver. v2: 1. renamed patched 1 2. made definition of device version string completely private to vendor driver 3. abandoned changes to sample mdev drivers 4. described intent and usage of version attribute more clearly. Yan Zhao (2): vfio/mdev: add version attribute for mdev device drm/i915/gvt: export mdev device version to sysfs for Intel vGPU Documentation/vfio-mediated-device.txt | 135 ++++++++++++++++++++++ drivers/gpu/drm/i915/gvt/Makefile | 2 +- drivers/gpu/drm/i915/gvt/device_version.c | 84 ++++++++++++++ drivers/gpu/drm/i915/gvt/gvt.c | 51 ++++++++ drivers/gpu/drm/i915/gvt/gvt.h | 6 + 5 files changed, 277 insertions(+), 1 deletion(-) create mode 100644 drivers/gpu/drm/i915/gvt/device_version.c -- 2.17.1

7 45

[libvirt] [PATCH v3 0/2] introduction of migration_version attribute for VFIO live migration
by Yan Zhao 29 May '19

29 May '19

This patchset introduces a migration_version attribute under sysfs of VFIO Mediated devices. This migration_version attribute is used to check migration compatibility between two mdev devices of the same mdev type. Patch 1 defines migration_version attribute in Documentation/vfio-mediated-device.txt Patch 2 uses GVT as an example to show how to expose migration_version attribute and check migration compatibility in vendor driver. v3: 1. renamed version to migration_version 2. let errno to be freely defined by vendor driver 3. let checking mdev_type be prerequisite of migration compatibility check 4. reworded most part of patch 1 5. print detailed error log in patch 2 and generate migration_version string at init time v2: 1. renamed patched 1 2. made definition of device version string completely private to vendor driver 3. reverted changes to sample mdev drivers 4. described intent and usage of version attribute more clearly. Yan Zhao (2): vfio/mdev: add migration_version attribute for mdev device drm/i915/gvt: export migration_version to mdev sysfs for Intel vGPU Documentation/vfio-mediated-device.txt | 113 +++++++++++++ drivers/gpu/drm/i915/gvt/Makefile | 2 +- drivers/gpu/drm/i915/gvt/gvt.c | 39 +++++ drivers/gpu/drm/i915/gvt/gvt.h | 5 + drivers/gpu/drm/i915/gvt/migration_version.c | 167 +++++++++++++++++++ drivers/gpu/drm/i915/gvt/vgpu.c | 13 +- 6 files changed, 336 insertions(+), 3 deletions(-) create mode 100644 drivers/gpu/drm/i915/gvt/migration_version.c -- 2.17.1

3 8

[libvirt] [PATCH 00/11] qemu: Introduce SMMUv3 IOMMU support
by Andrea Bolognani 28 May '19

28 May '19

This is the ARM equivalent of Intel IOMMU. Andrea Bolognani (11): qemu: Fix switch() statements for virDomainIOMMUModel qemu: Drop 'ret' from qemuBuildIOMMUCommandLine() qemu: Use VIR_AUTOCLEAN() in qemuBuildIOMMUCommandLine() qemu: Move virBuffer inside switch() statement qemu: Move capability checks inside switch() statements qemu: Introduce QEMU_CAPS_MACHINE_VIRT_IOMMU conf: Parse and format SMMUv3 IOMMU qemu: Add validation for SMMUv3 IOMMU qemu: Move capability checks for IOMMU features qemu: Format SMMUv3 IOMMU news: Update for SMMUv3 IOMMU support docs/formatdomain.html.in | 7 +- docs/news.xml | 8 + docs/schemas/domaincommon.rng | 5 +- src/conf/domain_conf.c | 1 + src/conf/domain_conf.h | 1 + src/qemu/qemu_capabilities.c | 8 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 63 ++++--- src/qemu/qemu_domain.c | 73 +++++--- .../caps_2.12.0.aarch64.replies | 166 +++++++++++++++-- .../caps_3.0.0.riscv32.replies | 131 +++++++++++++- .../caps_3.0.0.riscv64.replies | 131 +++++++++++++- .../caps_4.0.0.aarch64.replies | 171 ++++++++++++++++-- .../caps_4.0.0.aarch64.xml | 1 + .../caps_4.0.0.riscv32.replies | 129 ++++++++++++- .../caps_4.0.0.riscv64.replies | 129 ++++++++++++- .../iommu-smmuv3.aarch64-latest.args | 31 ++++ tests/qemuxml2argvdata/iommu-smmuv3.xml | 15 ++ tests/qemuxml2argvtest.c | 1 + .../iommu-smmuv3.aarch64-latest.xml | 25 +++ tests/qemuxml2xmltest.c | 1 + 21 files changed, 985 insertions(+), 113 deletions(-) create mode 100644 tests/qemuxml2argvdata/iommu-smmuv3.aarch64-latest.args create mode 100644 tests/qemuxml2argvdata/iommu-smmuv3.xml create mode 100644 tests/qemuxml2xmloutdata/iommu-smmuv3.aarch64-latest.xml -- 2.21.0

2 22

[libvirt] [PATCH v3 0/6] numa: deprecate '-numa node, mem' and default memory distribution
by Igor Mammedov 28 May '19

28 May '19

Changes since v2: - taking in account previous review, implement a way for mgmt to intospect if '-numa node,mem' is supported by machine type as suggested by Daniel at https://www.mail-archive.com/qemu-devel@nongnu.org/msg601220.html * ammend "qom-list-properties" to show property values * add "numa-mem-supported" machine property to reflect if '-numa node,mem=SZ' is supported. It culd be used with '-machine none' or at runtime with --preconfig before numa memory mapping are configured * minor fixes to deprecation documentation mentioning "numa-mem-supported" property 1) "I'm considering to deprecating -mem-path/prealloc CLI options and replacing them with a single memdev Machine property to allow interested users to pick used backend for initial RAM (fixes mixed -mem-path+hostmem backends issues) and as a transition step to modeling initial RAM as a Device instead of (ab)using MemoryRegion APIs." (for more details see: https://www.mail-archive.com/qemu-devel@nongnu.org/msg596314.html) However there is a couple of roadblocks on the way (s390x and numa memory handling). I think I finally thought out a way to hack s390x in migration compatible manner, but I don't see any way to do it for -numa node,mem and default RAM assignement to nodes. Considering both numa usecases aren't meaningfully using NUMA (aside guest side testing) and could be replaced with explicitly used memdev parameter, I'd like to propose removing these fake NUMA friends on new machine types, hence this deprecation. And once the last machie type that supported the option is removed we would be able to remove option altogether. As result of removing deprecated options and replacing initial RAM allocation with 'memdev's (1), QEMU will allocate guest RAM in consistent way, fixing mixed use-case and allowing boards to move towards modelling initial RAM as Device(s). Which in its own turn should allow to cleanup NUMA/HMP/memory accounting code more by dropping ad-hoc node_mem tracking and reusing memory device enumeration instead. Reference to previous versions: * [PATCH 0/2] numa: deprecate -numa node, mem and default memory distribution https://www.mail-archive.com/qemu-devel@nongnu.org/msg600706.html * [PATCH] numa: warn if numa 'mem' option or default RAM splitting between nodes is used. https://www.mail-archive.com/qemu-devel@nongnu.org/msg602136.html * [PATCH v2] numa: warn if numa 'mem' option or default RAM splitting between nodes is used. https://www.spinics.net/linux/fedora/libvir/msg180917.html CC: libvir-list(a)redhat.com CC: ehabkost(a)redhat.com CC: pbonzini(a)redhat.com CC: berrange(a)redhat.com CC: armbru(a)redhat.com Igor Mammedov (6): pc: fix possible NULL pointer dereference in pc_machine_get_device_memory_region_size() qmp: make "qom-list-properties" show initial property values qmp: qmp_qom_list_properties(): ignore empty string options numa: introduce "numa-mem-supported" machine property numa: deprecate 'mem' parameter of '-numa node' option numa: deprecate implict memory distribution between nodes include/hw/boards.h | 1 + hw/arm/virt.c | 1 + hw/core/machine.c | 12 ++++++++++++ hw/i386/pc.c | 7 ++++++- hw/ppc/spapr.c | 1 + numa.c | 5 +++++ qapi/misc.json | 5 ++++- qemu-deprecated.texi | 24 ++++++++++++++++++++++++ qmp.c | 15 +++++++++++++++ 9 files changed, 69 insertions(+), 2 deletions(-) -- 2.7.4

3 13

[libvirt] [PATCH] qemu: aquire job in qemuDomainDefineXMLFlags
by Nikolay Shirokovskiy 28 May '19

28 May '19

The function updates vm->newDef and frees previous pesistent definition which can be used by in different thread by some API that unlocks domain to communicate to qemu. So we have an access to freed memory in that thread. Let's acquire job condition when changing persistent xml. Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy(a)virtuozzo.com> --- A bug example (some downstream version of libvirt, nevertheless): (gdb) bt #0 0x00007f3bb77a0533 in virDomainDefHasVcpusOffline (def=def@entry=0x7f3b7802a830) at conf/domain_conf.c:1637 #1 0x00007f3bb77cb32f in virDomainCpuDefFormat (def=0x7f3b7802a830, buf=0x7f3ba7308940) at conf/domain_conf.c:27503 #2 virDomainDefFormatInternal (def=def@entry=0x7f3b7802a830, caps=0x7f3b70000ae0, flags=3, flags@entry=1, buf=buf@entry=0x7f3ba7308940, xmlopt=xmlopt@entry=0x0) at conf/domain_conf.c:27850 #3 0x00007f3bb77ced46 in virDomainDefFormat (def=def@entry=0x7f3b7802a830, caps=<optimized out>, flags=flags@entry=1) at conf/domain_conf.c:28564 #4 0x00007f3bb77d23f9 in virDomainSaveConfig (configDir=0x7f3b981227e0 "/etc/libvirt/qemu", caps=<optimized out>, def=0x7f3b7802a830) at conf/domain_conf.c:28776 #5 0x00007f3ba086b617 in qemuDomainSetMemoryStatsPeriod (dom=<optimized out>, period=30, flags=<optimized out>) at qemu/qemu_driver.c:4839 #6 0x00007f3bb7905613 in virDomainSetMemoryStatsPeriod (domain=domain@entry=0x7f3b74006200, period=30, flags=3) at libvirt-domain.c:2087 (gdb) f 5 #5 0x00007f3ba086b617 in qemuDomainSetMemoryStatsPeriod (dom=<optimized out>, period=30, flags=<optimized out>) at qemu/qemu_driver.c:4839 4839 ret = virDomainSaveConfig(cfg->configDir, driver->caps, persistentDef); (gdb) p vm->def $8 = (virDomainDefPtr) 0x7f3b8400c6f0 (gdb) p vm->newDef $9 = (virDomainDefPtr) 0x7f3b7001de20 (gdb) p persistentDef $10 = (virDomainDefPtr) 0x7f3b7802a830 src/qemu/qemu_driver.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 42b1ce2..683dcaa 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -7626,6 +7626,7 @@ qemuDomainDefineXMLFlags(virConnectPtr conn, virCapsPtr caps = NULL; unsigned int parse_flags = VIR_DOMAIN_DEF_PARSE_INACTIVE | VIR_DOMAIN_DEF_PARSE_ABI_UPDATE; + bool job = false; virCheckFlags(VIR_DOMAIN_DEFINE_VALIDATE, NULL); @@ -7647,6 +7648,10 @@ qemuDomainDefineXMLFlags(virConnectPtr conn, if (virDomainDefineXMLFlagsEnsureACL(conn, def) < 0) goto cleanup; + if (qemuDomainObjBeginJob(driver, vm, QEMU_JOB_MODIFY) < 0) + goto cleanup; + job = true; + if (!(vm = virDomainObjListAdd(driver->domains, def, driver->xmlopt, 0, &oldDef))) @@ -7685,6 +7690,9 @@ qemuDomainDefineXMLFlags(virConnectPtr conn, dom = virGetDomain(conn, vm->def->name, vm->def->uuid, vm->def->id); cleanup: + if (job) + qemuDomainObjEndJob(driver, vm); + virDomainDefFree(oldDef); virDomainDefFree(def); virDomainObjEndAPI(&vm); -- 1.8.3.1

3 3

[libvirt] Plans for next release
by Daniel Veillard 28 May '19

28 May '19

We should enter freeze for 5.4.0 beginning of next week, maybe Monday evening or Tuesday morning, then plan for RC2 around Wed - Thur for a w.e. release. Hope this works for everybody, thanks, Daniel -- Daniel Veillard | Red Hat Developers Tools http://developer.redhat.com/ veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

1 1

[libvirt] [PATCH] vz: fixes: snapshot: s/parent/parent_name/ as prep for virObject
by Nikolay Shirokovskiy 28 May '19

28 May '19

Apply renaming of 36603bc56 for the vz driver. Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy(a)virtuozzo.com> --- Pushed as build breaker/trivial. src/vz/vz_driver.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/vz/vz_driver.c b/src/vz/vz_driver.c index dfd49e7..2286f9a 100644 --- a/src/vz/vz_driver.c +++ b/src/vz/vz_driver.c @@ -2460,14 +2460,14 @@ vzDomainSnapshotGetParent(virDomainSnapshotPtr snapshot, unsigned int flags) if (!(snap = vzSnapObjFromSnapshot(snapshots, snapshot))) goto cleanup; - if (!snap->def->parent) { + if (!snap->def->parent_name) { virReportError(VIR_ERR_NO_DOMAIN_SNAPSHOT, _("snapshot '%s' does not have a parent"), snap->def->name); goto cleanup; } - parent = virGetDomainSnapshot(snapshot->domain, snap->def->parent); + parent = virGetDomainSnapshot(snapshot->domain, snap->def->parent_name); cleanup: virDomainSnapshotObjListFree(snapshots); -- 1.8.3.1

1 0