January 2022 - Devel - libvirt List Archives

[PATCH v2 0/6] support mode option for dirtyrate calculation
by huangy81＠chinatelecom.cn 27 Jan '22

27 Jan '22

From: Hyman Huang(黄勇) <huangy81(a)chinatelecom.cn> v2: Rebase master and fix confilicts with commit "Introduce QEMU_CAPS_DEVICE_VIRTIO_MEM_PCI_PREALLOC" Thanks ! v1: This patchset introduce mode option as the supplement of qemuDomainStartDirtyRateCalc api, add calc_mode for dirtyrate statistics correspondingly. Qemu add mode parameter for calc-dirty-rate command since >= 6.2.0, either of these three mode "page-sampling, dirty-bitmap, dirty-ring" can be specified when calculating dirty page rate. Page sampling is the original mode and used as default mode. Dirty bitmap mode use kvm log sync api to fetch the dirty-bitmap and count the increased 1 bits number during measurement, thus, calculate the dirty page rate. Dirty ring mode use the dirty-ring mechanism implemented in Qemu which can count the increased dirty page on virtual cpu granularity, thus, calculate the per-vcpu dirty page rate. These three calculation mode can be used in different scenarios, and the dirty-bitmap, dirty-ring mode may be more accurate to a certain degree. So maybe it's time to support the mode option for dirtyrate calculation. This series make main modifications as the following: 1. introduce QEMU_CAPS_CALC_DIRTY_RATE capability to probe calc-dirty-rate command in case of failure since it just introduced since >= 5.2.0 2. introduce QEMU_CAPS_DIRTYRATE_MODE capability to probe mode option of calc-dirty-rate command in case of failure, same as 1. 3. implement mode option support for dirtyrate calculation. Please review, thanks ! Best Regards ! Hyman Huang(黄勇) (6): qemu_capabilities: Introduce QEMU_CAPS_CALC_DIRTY_RATE capability qemu_driver: Probe capability before calculating dirty page rate qemu_capabilities: Introduce QEMU_CAPS_DIRTYRATE_MODE capability qemu_driver: Add mode option for qemuDomainStartDirtyRateCalc virsh: Extend domdirtyrate-calc virsh api qemu_driver: Add calc_mode for dirtyrate statistics docs/manpages/virsh.rst | 7 +- include/libvirt/libvirt-domain.h | 13 ++++ src/libvirt-domain.c | 17 ++++- src/qemu/qemu_capabilities.c | 4 ++ src/qemu/qemu_capabilities.h | 2 + src/qemu/qemu_driver.c | 59 ++++++++++++++++- src/qemu/qemu_monitor.c | 5 +- src/qemu/qemu_monitor.h | 20 +++++- src/qemu/qemu_monitor_json.c | 78 +++++++++++++++++++++-- src/qemu/qemu_monitor_json.h | 3 +- tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml | 2 + tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml | 2 + tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml | 2 + tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml | 2 + tools/virsh-domain.c | 28 +++++++- 25 files changed, 239 insertions(+), 15 deletions(-) -- 1.8.3.1

3 12

[PATCH v1 0/6] support mode option for dirtyrate calculation
by huangy81＠chinatelecom.cn 27 Jan '22

27 Jan '22

From: Hyman Huang(黄勇) <huangy81(a)chinatelecom.cn> v1: This patchset introduce mode option as the supplement of qemuDomainStartDirtyRateCalc api, add calc_mode for dirtyrate statistics correspondingly. Qemu add mode parameter for calc-dirty-rate command since >= 6.2.0, either of these three mode "page-sampling, dirty-bitmap, dirty-ring" can be specified when calculating dirty page rate. Page sampling is the original mode and used as default mode. Dirty bitmap mode use kvm log sync api to fetch the dirty-bitmap and count the increased 1 bits number during measurement, thus, calculate the dirty page rate. Dirty ring mode use the dirty-ring mechanism implemented in Qemu which can count the increased dirty page on virtual cpu granularity, thus, calculate the per-vcpu dirty page rate. These three calculation mode can be used in different scenarios, and the dirty-bitmap, dirty-ring mode may be more accurate to a certain degree. So maybe it's time to support the mode option for dirtyrate calculation. This series make main modifications as the following: 1. introduce QEMU_CAPS_CALC_DIRTY_RATE capability to probe calc-dirty-rate command in case of failure since it just introduced since >= 5.2.0 2. introduce QEMU_CAPS_DIRTYRATE_MODE capability to probe mode option of calc-dirty-rate command in case of failure, same as 1. 3. implement mode option support for dirtyrate calculation. Please review, thanks ! Best Regards ! Hyman Huang(黄勇) (6): qemu_capabilities: Introduce QEMU_CAPS_CALC_DIRTY_RATE capability qemu_driver: Probe capability before calculating dirty page rate qemu_capabilities: Introduce QEMU_CAPS_DIRTYRATE_MODE capability qemu_driver: Add mode option for qemuDomainStartDirtyRateCalc virsh: Extend domdirtyrate-calc virsh api qemu_driver: Add calc_mode for dirtyrate statistics docs/manpages/virsh.rst | 7 +- include/libvirt/libvirt-domain.h | 13 ++++ src/libvirt-domain.c | 17 ++++- src/qemu/qemu_capabilities.c | 4 ++ src/qemu/qemu_capabilities.h | 2 + src/qemu/qemu_driver.c | 59 ++++++++++++++++- src/qemu/qemu_monitor.c | 5 +- src/qemu/qemu_monitor.h | 20 +++++- src/qemu/qemu_monitor_json.c | 78 +++++++++++++++++++++-- src/qemu/qemu_monitor_json.h | 3 +- tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml | 2 + tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml | 2 + tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml | 2 + tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml | 2 + tools/virsh-domain.c | 28 +++++++- 25 files changed, 239 insertions(+), 15 deletions(-) -- 1.8.3.1

1 6

[libvirt PATCH] virsh: domsetlaunchsecstate: mark options as mandatory
by Ján Tomko 26 Jan '22

26 Jan '22

We exit if they are not present. Let the virsh option parser do the checking instead of checking it manually. Change the type to OT_DATA (i.e. a mandatory string), mark them as required and remove VSH_OFLAG_REQ_OPT so that the header file and the secret file can be specified without the option names. https://bugzilla.redhat.com/show_bug.cgi?id=2046024 Signed-off-by: Ján Tomko <jtomko(a)redhat.com> --- tools/virsh-domain.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index b56f6a90f5..d279af68b2 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -9587,13 +9587,13 @@ static const vshCmdInfo info_domsetlaunchsecstate[] = { static const vshCmdOptDef opts_domsetlaunchsecstate[] = { VIRSH_COMMON_OPT_DOMAIN_FULL(0), {.name = "secrethdr", - .type = VSH_OT_STRING, - .flags = VSH_OFLAG_REQ_OPT, + .type = VSH_OT_DATA, + .flags = VSH_OFLAG_REQ, .help = N_("path to file containing the secret header"), }, {.name = "secret", - .type = VSH_OT_STRING, - .flags = VSH_OFLAG_REQ_OPT, + .type = VSH_OT_DATA, + .flags = VSH_OFLAG_REQ, .help = N_("path to file containing the secret"), }, {.name = "set-address", @@ -9627,9 +9627,6 @@ cmdDomSetLaunchSecState(vshControl * ctl, const vshCmd * cmd) if (vshCommandOptStringReq(ctl, cmd, "secret", &secfile) < 0) return false; - if (sechdrfile == NULL || secfile == NULL) - return false; - if (virFileReadAll(sechdrfile, 1024*64, &sechdr) < 0) { vshSaveLibvirtError(); return false; -- 2.31.1

4 3

[PATCH 0/2] node_device: Treat NVMe disks as regular disks
by Michal Privoznik 26 Jan '22

26 Jan '22

Apparently udev doesn't think that NVMe disk is really a disk because it doesn't set ID_TYPE attribute to "disk" (it doesn't set the attribute at all). But it does set DEVTYPE=disk so with a little bit of work we can report NVMe disks as devices with VIR_NODE_DEV_CAP_STORAGE capability. Michal Prívozník (2): node_device: Rework udevKludgeStorageType() node_device: Treat NVMe disks as regular disks src/node_device/node_device_udev.c | 43 ++++++++++++++++-------------- 1 file changed, 23 insertions(+), 20 deletions(-) -- 2.34.1

2 4

[PATCH v1] meson: recognize sles when guessing default_qemu_user
by Olaf Hering 26 Jan '22

26 Jan '22

NAME="SLES" VERSION="15-SP3" VERSION_ID="15.3" PRETTY_NAME="SUSE Linux Enterprise Server 15 SP3" ID="sles" ID_LIKE="suse" ANSI_COLOR="0;32" CPE_NAME="cpe:/o:suse:sles:15:sp3" DOCUMENTATION_URL="https://documentation.suse.com/" Signed-off-by: Olaf Hering <olaf(a)aepfle.de> --- meson.build | 1 + 1 file changed, 1 insertion(+) Commit 4c69d64efa3731d074d198f871fd42e74c4a39f6 revealed the bug, /etc/os-release must exist during build. diff --git a/meson.build b/meson.build index b6d1286f3f..b2d9e8ed65 100644 --- a/meson.build +++ b/meson.build @@ -1673,6 +1673,7 @@ if not get_option('driver_qemu').disabled() os_release.contains('fedora') or os_release.contains('gentoo') or os_release.contains('rhel') or + os_release.contains('sles') or os_release.contains('suse')) default_qemu_user = 'qemu' default_qemu_group = 'qemu'

2 9

[libvirt PATCH] spec: Don't clean up *.la and *.a files
by Andrea Bolognani 26 Jan '22

26 Jan '22

autotools used to produce those, but meson doesn't. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- libvirt.spec.in | 14 -------------- 1 file changed, 14 deletions(-) diff --git a/libvirt.spec.in b/libvirt.spec.in index f0ff4c0ece..0e6cd13bb2 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -1195,20 +1195,6 @@ export SOURCE_DATE_EPOCH=$(stat --printf='%Y' %{_specdir}/%{name}.spec) %meson_install -rm -f $RPM_BUILD_ROOT%{_libdir}/*.la -rm -f $RPM_BUILD_ROOT%{_libdir}/*.a -rm -f $RPM_BUILD_ROOT%{_libdir}/libvirt/lock-driver/*.la -rm -f $RPM_BUILD_ROOT%{_libdir}/libvirt/lock-driver/*.a -rm -f $RPM_BUILD_ROOT%{_libdir}/libvirt/connection-driver/*.la -rm -f $RPM_BUILD_ROOT%{_libdir}/libvirt/connection-driver/*.a -rm -f $RPM_BUILD_ROOT%{_libdir}/libvirt/storage-backend/*.la -rm -f $RPM_BUILD_ROOT%{_libdir}/libvirt/storage-backend/*.a -rm -f $RPM_BUILD_ROOT%{_libdir}/libvirt/storage-file/*.la -rm -f $RPM_BUILD_ROOT%{_libdir}/libvirt/storage-file/*.a -%if %{with_wireshark} -rm -f $RPM_BUILD_ROOT%{wireshark_plugindir}/libvirt.la -%endif - # We don't want to install /etc/libvirt/qemu/networks in the main %%files list # because if the admin wants to delete the default network completely, we don't # want to end up re-incarnating it on every RPM upgrade. -- 2.34.1

2 1

[libvirt PATCH 0/3] mingw: Misc cleanups
by Andrea Bolognani 26 Jan '22

26 Jan '22

Andrea Bolognani (3): mingw: Don't ship virt-admin mings: Don't ship virt-(xml|pki)-validate mingw: Don't ship virkey* manual pages mingw-libvirt.spec.in | 14 -------------- 1 file changed, 14 deletions(-) -- 2.34.1

2 4

[PATCH] qemuDomainSetupDisk: Initialize 'targetPaths'
by Peter Krempa 25 Jan '22

25 Jan '22

Compiler isn't able to see that 'virDevMapperGetTargets' in cases e.g. when the devmapper isn't available may not initialize the value in the pointer passed as the second argument. The usage 'qemuDomainSetupDisk' lead to an accidental infinite loop as previous calls apparently doctored the stack to a point where 'g_slist_concat' would end up in an infinite loop trying to find the end of the list. Fixes: 6c49c2ee9fc Closes: https://gitlab.com/libvirt/libvirt/-/issues/268 Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- src/qemu/qemu_namespace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/qemu/qemu_namespace.c b/src/qemu/qemu_namespace.c index 23b1160c5e..94453033f5 100644 --- a/src/qemu/qemu_namespace.c +++ b/src/qemu/qemu_namespace.c @@ -251,7 +251,7 @@ qemuDomainSetupDisk(virStorageSource *src, if (!(tmpPath = virPCIDeviceAddressGetIOMMUGroupDev(&next->nvme->pciAddr))) return -1; } else { - GSList *targetPaths; + GSList *targetPaths = NULL; if (virStorageSourceIsEmpty(next) || !virStorageSourceIsLocalStorage(next)) { -- 2.34.1

2 1

[PATCH] libxl: Add lock process indicator to saved VM state
by Jim Fehlig 25 Jan '22

25 Jan '22

Commit fa58f571ee added a lock processes indicator to the libxlDomainObjPrivate struct to note that a lock process was successfully started for the VM. However, the commit neglected to add the indicator to the VM's saved state file. As a result, the indicator is lost on libvirtd restart, along with the knowledge of whether a lock process was started for the VM. This change adds support for the indicator in the domainObjPrivate data parse and format callbacks, ensuring its value survives libvirtd restarts. Signed-off-by: Jim Fehlig <jfehlig(a)suse.com> --- src/libxl/libxl_domain.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c index feca60f7d2..8962adc60f 100644 --- a/src/libxl/libxl_domain.c +++ b/src/libxl/libxl_domain.c @@ -226,6 +226,7 @@ libxlDomainObjPrivateXMLParse(xmlXPathContextPtr ctxt, libxlDomainObjPrivate *priv = vm->privateData; priv->lockState = virXPathString("string(./lockstate)", ctxt); + priv->lockProcessRunning = virXPathBoolean("count(./lockProcessRunning) > 0", ctxt); return 0; } @@ -239,6 +240,9 @@ libxlDomainObjPrivateXMLFormat(virBuffer *buf, if (priv->lockState) virBufferAsprintf(buf, "<lockstate>%s</lockstate>\n", priv->lockState); + if (priv->lockProcessRunning) + virBufferAddLit(buf, "<lockProcessRunning/>\n"); + return 0; } -- 2.34.1

2 2

[libvirt PATCH v7 00/18] qemu: Introduce hvf domain type for Hypervisor.framework
by Andrea Bolognani 25 Jan '22

25 Jan '22

In order to hopefully address [libvirt#147] at long last, I've picked up Roman's patches from 2018 and attempted to forward-port them. More specifically, I've used the [roolebo/hvf-domain] branch as a starting point, since it seems to contain a few improvements over [v2] and was just easier to pick up. The code is mostly his own, so I've retained the existing authorship information, but I've dropped Reviewed-by tags for commits that have been modified in non-trivial ways. I've applied very minimal style tweaks along the way, but overall I've tried to modify the existing patches as little as possible. The new test cases, such as they are, pass, and no regressions to KVM support appear to have been introduced in the process. I don't currently have access to a machine running macOS, so I can't verify that it's actually possible to start a hardware-accelerated VM by myself, but a couple of users have confirmed that the new feature works. Changes from [v6]: * addressed review comments; * moved virQEMUCapsAccelStr() and friends closer to the existing virQEMUCapsGetAccel() since they are very similar in purpose; * squashed my "fixup!" commits into the original ones now that they have been reviewed. Changes from [v5]: * rebased on top of master, dealing with a couple of straightforward merge conflicts in the process; * moved NEWS entry to the 8.1.0 section. Changes from [v4]: * fixed an issue that prevented machine types from being probed correctly, effectively making the entire thing non functional; * only report HVF support as available when the guest architecture and the host architecture match. Changes from [v3]: * reintroduced the patch that was missing in the initial version of the forward-port; * converted the documentation to reStructuredText and trimmed it significantly; * reworked virQEMUCapsAccelStr() based on Dan's suggestions; * reworked macOS support in the test suite based on Dan's suggestions; * fixed a few minor issues found while doing the above. Changes from [v2]: * rebased on top of master; * added a couple of simple test cases. Useful links: * GitLab: [abologna/hvf] * CI: [pipeline] [libvirt#147] https://gitlab.com/libvirt/libvirt/-/issues/147 [roolebo/hvf-domain] https://github.com/roolebo/libvirt/tree/hvf-domain [abologna/hvf] https://gitlab.com/abologna/libvirt/-/commits/hvf [pipeline] https://gitlab.com/abologna/libvirt/-/pipelines/453678312 [v6] https://listman.redhat.com/archives/libvir-list/2022-January/msg00685.html [v5] https://listman.redhat.com/archives/libvir-list/2022-January/msg00411.html [v4] https://listman.redhat.com/archives/libvir-list/2022-January/msg00280.html [v3] https://listman.redhat.com/archives/libvir-list/2022-January/msg00131.html [v2] https://listman.redhat.com/archives/libvir-list/2018-November/msg00802.html Andrea Bolognani (5): qemu: Only probe KVM on Linux tests: Introduce testQemuHostOS tests: Add macOS support to testutilsqemu tests: Add macOS support to qemuxml2*test tests: Add HVF test cases Roman Bolshakov (13): qemu: Add KVM CPUs into cache only if KVM is present conf: Add hvf domain type qemu: Define hvf capability qemu: Query hvf capability on macOS qemu: Expose hvf domain type if hvf is supported qemu: Introduce virQEMUCapsAccelStr qemu: Introduce virQEMUCapsTypeIsAccelerated qemu: Introduce virQEMUCapsHaveAccel qemu: Correct CPU capabilities probing for hvf docs: Add hvf on QEMU driver page docs: Note hvf support for domain elements docs: Add support page for libvirt on macOS news: Mention hvf domain type NEWS.rst | 5 + docs/docs.html.in | 3 + docs/drvqemu.rst | 48 +++++- docs/formatdomain.rst | 22 +-- docs/index.html.in | 4 +- docs/macos.rst | 44 ++++++ docs/meson.build | 1 + docs/schemas/domaincommon.rng | 1 + src/conf/domain_conf.c | 1 + src/conf/domain_conf.h | 1 + src/qemu/qemu_capabilities.c | 146 ++++++++++++++++-- src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 4 + src/qemu/qemu_process.c | 10 +- .../hvf-aarch64-virt-headless.args | 48 ++++++ .../hvf-aarch64-virt-headless.xml | 45 ++++++ .../hvf-x86_64-q35-headless.args | 47 ++++++ .../hvf-x86_64-q35-headless.x86_64-latest.err | 1 + .../hvf-x86_64-q35-headless.xml | 44 ++++++ tests/qemuxml2argvtest.c | 43 +++++- .../hvf-aarch64-virt-headless.xml | 94 +++++++++++ .../hvf-x86_64-q35-headless.xml | 97 ++++++++++++ tests/qemuxml2xmltest.c | 43 +++++- tests/testutilsqemu.c | 146 ++++++++++++++---- tests/testutilsqemu.h | 10 ++ 25 files changed, 845 insertions(+), 64 deletions(-) create mode 100644 docs/macos.rst create mode 100644 tests/qemuxml2argvdata/hvf-aarch64-virt-headless.args create mode 100644 tests/qemuxml2argvdata/hvf-aarch64-virt-headless.xml create mode 100644 tests/qemuxml2argvdata/hvf-x86_64-q35-headless.args create mode 100644 tests/qemuxml2argvdata/hvf-x86_64-q35-headless.x86_64-latest.err create mode 100644 tests/qemuxml2argvdata/hvf-x86_64-q35-headless.xml create mode 100644 tests/qemuxml2xmloutdata/hvf-aarch64-virt-headless.xml create mode 100644 tests/qemuxml2xmloutdata/hvf-x86_64-q35-headless.xml -- 2.34.1

2 20