August 2024 - Devel - libvirt List Archives

[PATCH v2 0/4] multiple memory backend support for CPR Live Updates
by mgalaxy＠akamai.com 23 Oct '24

23 Oct '24

From: Michael Galaxy <mgalaxy(a)akamai.com> CPR-based support for whole-hypervisor kexec-based live updates is now finally merged into QEMU. In support of this, we need NUMA to be supported in these kinds of environments. To do this we use a technology called PMEM (persistent memory) in Linux, which underpins the ability for CPR Live Updates to work so that QEMU memory can remain in RAM and be recovered after a kexec operationg has completed. Our systems are highly NUMA-aware, and so this patch series enables NUMA awareness for live updates. Further, we make a small change that allows live migrations to work between *non* PMEM-based systems and PMEM-based systems (and vice-versa). This allows for seemless upgrades from non-live-compatible systems to live-update-compatible sytems without any downtime. Michael Galaxy (4): qemu.conf changes to support multiple memory backend Support live migration between file-backed memory and anonymous memory. Update unit test to support multiple memory backends Update documentation to reflect memory_backing_dir change in qemu.conf NEWS.rst | 7 ++ docs/kbase/virtiofs.rst | 2 + src/qemu/qemu.conf.in | 2 + src/qemu/qemu_command.c | 8 ++- src/qemu/qemu_conf.c | 141 +++++++++++++++++++++++++++++++++++----- src/qemu/qemu_conf.h | 14 ++-- src/qemu/qemu_domain.c | 24 +++++-- src/qemu/qemu_driver.c | 29 +++++---- src/qemu/qemu_hotplug.c | 6 +- src/qemu/qemu_process.c | 44 +++++++------ src/qemu/qemu_process.h | 7 +- tests/testutilsqemu.c | 5 +- 12 files changed, 221 insertions(+), 68 deletions(-) -- 2.34.1

3 26

[PATCH V2 0/2] qemu: Clarify purpose of image format settings
by Jim Fehlig 10 Oct '24

10 Oct '24

The current documentation of the various foo_image_format settings in qemu.conf subtly implies they are only used for specifying compression. Patch1 of this small series attempts to clarify and improve the description of the settings. It defines image format as a way to specify the desired layout of guest memory blocks on disk. Patch2 changes the use of 'compressed' with 'format' throughout the code, removing implication that format == compressed. V2: Replace more uses of 'compressed' with 'format' in patch2 Jim Fehlig (2): qemu: conf: Improve the foo_image_format setting descriptions qemu: Use consistent naming for save image format src/qemu/qemu.conf.in | 40 +++++++++++++++++++++++---------------- src/qemu/qemu_driver.c | 30 ++++++++++++++--------------- src/qemu/qemu_saveimage.c | 32 +++++++++++++++---------------- src/qemu/qemu_saveimage.h | 4 ++-- src/qemu/qemu_snapshot.c | 6 +++--- 5 files changed, 60 insertions(+), 52 deletions(-) -- 2.35.3

2 7

[PATCH v2 0/1] Add support for RAPL MSRs feature in QEMU
by Anthony Harivel 07 Oct '24

07 Oct '24

Hi, First of all, kudos to Peter Krempa for his fast review! In this v2, I've addressed the following points: - The socket is *not* mandatory and my code totally confused Peter. Sorry about that! here a snippet of the QEMU code to understand: /* Compute the socket path if necessary */ if (s->msr_energy.socket_path == NULL) { s->msr_energy.socket_path = vmsr_compute_default_paths(); } So I made all the modification to make it not necessary. - Change the socket name to "rapl_helper_socket" - Change the socket to be absFilePath - I did not add anything to honour the _OFF state, because it is not necessary to explicitly disable it. That's about it. Regards, Anthony Anthony Harivel (1): qemu: Add support for RAPL MSRs feature docs/formatdomain.rst | 2 ++ src/conf/domain_conf.c | 18 ++++++++++++++++++ src/conf/domain_conf.h | 2 ++ src/conf/schemas/domaincommon.rng | 10 ++++++++++ src/qemu/qemu_command.c | 11 +++++++++++ tests/qemuxmlconfdata/kvm-features-off.xml | 1 + .../kvm-features.x86_64-latest.args | 2 +- tests/qemuxmlconfdata/kvm-features.xml | 1 + 8 files changed, 46 insertions(+), 1 deletion(-) -- 2.46.0

3 14

[PATCH V2 0/4] Rework qemu internal active snapshots to use QMP
by Nikolai Barybin 03 Oct '24

03 Oct '24

Den, Peter, Daniel thank you for your comments! I'm sending v2 of this patchset. Changes since last revision: - dropped [PATCH 4/4] qemu monitor: reap qemu_monitor_text - added new patch: qemu capabilities: add QEMU_CAPS_SNAPSHOT_SAVE/_DELETE - preserved old-style snapshotting (HMP savevm) in case we have QEMU < 6.0 - enhanced requirements for allowing snapshotting. All writable disks should be qcow2, non-shared. If such disks exist and we have qcow2 NVRAM, add NVRAM device to the list of wrdevs. But never save vmstate to NVRAM - make char** wrdevs list allocation inside qemuSnapshotActiveInternalGetWrdevListHelper() Nikolai Barybin (4): qemu monitor: add snaphot-save/delete QMP commands qemu blockjob: add snapshot-save/delete job types qemu capabilities: add QEMU_CAPS_SNAPSHOT_SAVE/_DELETE qemu snapshot: use QMP snapshot-save/delete for internal snapshots src/qemu/qemu_block.c | 2 + src/qemu/qemu_blockjob.c | 6 +- src/qemu/qemu_blockjob.h | 2 + src/qemu/qemu_capabilities.c | 4 + src/qemu/qemu_capabilities.h | 2 + src/qemu/qemu_domain.c | 4 + src/qemu/qemu_monitor.c | 30 +++ src/qemu/qemu_monitor.h | 13 ++ src/qemu/qemu_monitor_json.c | 66 ++++++ src/qemu/qemu_monitor_json.h | 13 ++ src/qemu/qemu_snapshot.c | 207 ++++++++++++++++-- .../caps_6.0.0_aarch64.xml | 2 + .../qemucapabilitiesdata/caps_6.0.0_s390x.xml | 2 + .../caps_6.0.0_x86_64.xml | 2 + .../caps_6.1.0_x86_64.xml | 2 + .../caps_6.2.0_aarch64.xml | 2 + .../qemucapabilitiesdata/caps_6.2.0_ppc64.xml | 2 + .../caps_6.2.0_x86_64.xml | 2 + .../caps_7.0.0_aarch64+hvf.xml | 2 + .../caps_7.0.0_aarch64.xml | 2 + .../qemucapabilitiesdata/caps_7.0.0_ppc64.xml | 2 + .../caps_7.0.0_x86_64.xml | 2 + .../qemucapabilitiesdata/caps_7.1.0_ppc64.xml | 2 + .../caps_7.1.0_x86_64.xml | 2 + tests/qemucapabilitiesdata/caps_7.2.0_ppc.xml | 2 + .../caps_7.2.0_x86_64+hvf.xml | 2 + .../caps_7.2.0_x86_64.xml | 2 + .../caps_8.0.0_riscv64.xml | 2 + .../caps_8.0.0_x86_64.xml | 2 + .../qemucapabilitiesdata/caps_8.1.0_s390x.xml | 2 + .../caps_8.1.0_x86_64.xml | 2 + .../caps_8.2.0_aarch64.xml | 2 + .../caps_8.2.0_armv7l.xml | 2 + .../caps_8.2.0_loongarch64.xml | 2 + .../qemucapabilitiesdata/caps_8.2.0_s390x.xml | 2 + .../caps_8.2.0_x86_64.xml | 2 + .../qemucapabilitiesdata/caps_9.0.0_sparc.xml | 2 + .../caps_9.0.0_x86_64.xml | 2 + .../caps_9.1.0_x86_64.xml | 2 + 39 files changed, 391 insertions(+), 14 deletions(-) -- 2.43.5

2 14

[PATCH 0/2] add NIC hotplug support to test hypervisor
by John Levon 03 Oct '24

03 Oct '24

These two patches add basic support for NIC hot[un]plug to the test hypervisor, based on the qemu driver; only ethernet and bridge type VNICS are currently supported. John Levon (2): test_driver: provide basic NIC hotplug support test_driver: provide basic NIC hotunplug support src/test/test_driver.c | 305 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 283 insertions(+), 22 deletions(-) -- 2.34.1

2 9

[PATCH v6 00/13] qemu: Introduce shared_filesystems configuration option
by Andrea Bolognani 02 Oct '24

02 Oct '24

The need to have something like this in the first place is driven by KubeVirt (see [1] and [2]). A draft version of this series has been integrated into KubeVirt and it has been confirmed that it was effective in removing the need to use LD_PRELOAD hacks in the storage provider. Changes from [v5]: * make migration of domains with TPM work (patches 12 and 13); * fixed all typos for "remember"; * added R-bs for Peter's patches. Changes from [v4] (v5 was posted by Peter): * added patch 7 cleaning up a helper function (noticed just while reading the code) * added patch 8 properly unrefing security labels in dac/selinux drivers on outgoing migration * patch 11: added handling of the 'nvram' image file (and refactored the function to allow reuse) Changes from [v3] (v4 was posted by Peter): * patch 2/8 was modified to change the docs for the new option. * patches 1-5 will get an R-b by me as I've adopted them. * patches 6, 9-11 are new. * patches 7, 8 were not part of v3 Changes from [v2]: * added canonicalization for user-provided paths; * fixed compilation issues when AppArmor support is enabled. Changes from [v1]: * documented more explicitly that the newly introduced option is intended for very specific scenarios and not general usage; as part of this, the NEWS update has been dropped too; * made a few tweaks and addressed a few oversight based on review feedback; * several preparatory cleanup patches have been pushed. Changes from [v0]: * reworked approach. [v5] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/message/HNF… [v4] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/message/FWR… [v3] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/message/PIS… [v2] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/XPCP… [v1] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/XEIS… [v0] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/MMKV… [1] https://issues.redhat.com/browse/CNV-34322 [2] https://issues.redhat.com/browse/CNV-39370 Andrea Bolognani (7): security: Fix alignment qemu: Introduce shared_filesystems configuration option qemu: Propagate shared_filesystems utils: Use overrides in virFileIsSharedFS() qemu: Always set labels for TPM state security: Always forget labels for TPM state directory qemu: Don't lock TPM state directory for incoming migration Peter Krempa (6): virFileIsSharedFSOverride: Export virParseOwnershipIds: Refactor virSecuritySELinuxRestoreImageLabelInt: Move FD image relabeling after 'migrated' check security_(dac|selinux): Unref remembered security labels on outgoing migration storage_source: Add field for skipping seclabel remembering qemu: migration: Don't remember seclabel for images shared from current host src/conf/storage_source_conf.c | 3 + src/conf/storage_source_conf.h | 9 ++ src/libvirt_private.syms | 1 + src/lxc/lxc_controller.c | 3 +- src/lxc/lxc_driver.c | 2 +- src/lxc/lxc_process.c | 4 +- src/qemu/libvirtd_qemu.aug | 3 + src/qemu/qemu.conf.in | 26 +++++ src/qemu/qemu_conf.c | 31 ++++++ src/qemu/qemu_conf.h | 2 + src/qemu/qemu_domain.c | 7 +- src/qemu/qemu_extdevice.c | 2 +- src/qemu/qemu_migration.c | 86 ++++++++++++++--- src/qemu/qemu_security.c | 95 +++++++++++++----- src/qemu/qemu_security.h | 6 +- src/qemu/qemu_tpm.c | 50 ++++++---- src/qemu/qemu_tpm.h | 10 +- src/qemu/test_libvirtd_qemu.aug.in | 5 + src/security/security_apparmor.c | 8 +- src/security/security_dac.c | 53 +++++++++-- src/security/security_driver.h | 8 +- src/security/security_manager.c | 33 +++++-- src/security/security_manager.h | 9 +- src/security/security_nop.c | 5 + src/security/security_selinux.c | 148 +++++++++++++++++++++++------ src/security/security_stack.c | 32 +++++-- src/util/virfile.c | 63 +++++++++++- src/util/virfile.h | 5 +- src/util/virutil.c | 20 ++-- tests/securityselinuxlabeltest.c | 2 +- tests/virfiletest.c | 2 +- 31 files changed, 594 insertions(+), 139 deletions(-) -- 2.46.0

2 17

[libvirt PATCH 0/6] scripts: group-qemu-caps: improve readability
by Ján Tomko 02 Oct '24

02 Oct '24

Even though it still stays a Perl script at heart. Ján Tomko (6): scripts: group-qemu-caps: use read scripts: group-qemu-caps: remove cryptic bool scripts: group-qemu-caps: remove unecessary regex scripts: group-qemu-caps: separate file loading scripts: group-qemu-caps: regroup_caps: operate on split lines scripts: group-qemu-caps: separate file operations from the check scripts/group-qemu-caps.py | 120 +++++++++++++++++++++---------------- 1 file changed, 68 insertions(+), 52 deletions(-) -- 2.45.2

2 7

[PATCH 0/4] vircommand: Make FD closing more robust on __APPLE__
by Michal Privoznik 13 Sep '24

13 Sep '24

Found these on an old branch. Might as well post them. Michal Prívozník (4): vircommand: Drop unused arguments from virCommandMassCloseGetFDs*() vircommand: Isolate FD dir parsing into a separate function vircommand: Make sysconf(_SC_OPEN_MAX) failure non-fatal vircommand: Parse /dev/fd on *BSD-like systems when looking for opened FDs src/util/vircommand.c | 43 ++++++++++++++++++------------------------- 1 file changed, 18 insertions(+), 25 deletions(-) -- 2.44.2

4 14

[PATCH 00/14] Introduce fuzzing of XML formats
by Rayhan Faizel 04 Sep '24

04 Sep '24

This series introduces multiple fuzzers developed as part of Google Summer of Code 2024. We adopt a structure-aware fuzzing approach to fuzz libvirt XML formats. The fuzzing methodology makes use of libFuzzer and libprotobuf-mutator. The fuzzers work by mutating intermediate protobufs and converting them to XML. The fuzzing method in use requires inclusion of C++ sources. However, C++ compilation will be done only if '-Dfuzz' is enabled. Otherwise, libvirt will compile normally as before. The fuzzing method works only on clang compilers which support libFuzzer. This series introduces a total of six fuzzers: 1. QEMU XML domain 2. QEMU XML hotplug 3. CH XML domain 4. VMX XML domain 5. libXL XML domain 6. NWFilter XML In terms of the number of crashes discovered, QEMU XML domain, QEMU XML hotplug and libXL fuzzers are the most interesting ones. The setup process is documented at the end of the series (patch 14). Rayhan Faizel (14): src: Tweak source code to allow C++ compilation meson: Add support for clang/LLVM coverage instrumentation tests: Export handlers for fake secondary drivers schemas: Refactor relaxNG schema to ease protobuf conversion scripts: Add script to convert relaxNG to protobuf fuzz: Implement base fuzzing setup for XML domain fuzz: Implement QEMU XML domain fuzzer fuzz: Implement QEMU XML hotplug fuzzer ch: Remove unused variables fuzz: Implement CH XML domain fuzzer fuzz: Implement VMX XML domain fuzzer fuzz: Implement libXL XML domain fuzzer fuzz: Implement NWFilter XML fuzzer docs: Document the fuzzers build-aux/syntax-check.mk | 1 + docs/kbase/index.rst | 3 + docs/kbase/internals/meson.build | 1 + docs/kbase/internals/xml-fuzzing.rst | 120 ++++ meson.build | 55 ++ meson_options.txt | 5 +- scripts/meson.build | 1 + scripts/relaxng-to-proto.py | 521 ++++++++++++++++++ src/ch/ch_monitor.c | 2 +- src/ch/ch_monitor.h | 3 + src/ch/ch_process.c | 2 - src/conf/domain_conf.c | 18 +- src/conf/domain_conf.h | 6 +- src/conf/netdev_vport_profile_conf.c | 2 +- src/conf/schemas/basictypes.rng | 20 +- src/conf/schemas/domaincommon.rng | 11 +- src/conf/schemas/networkcommon.rng | 14 +- src/qemu/qemu_hotplug.c | 4 + src/qemu/qemu_monitor.c | 6 +- src/qemu/qemu_monitor.h | 2 +- src/util/virfile.h | 2 +- src/util/virnetdev.h | 12 +- src/util/virnetdevip.h | 2 +- src/util/virnetdevmacvlan.h | 2 +- src/util/virnetdevvportprofile.c | 2 +- src/util/virnetdevvportprofile.h | 2 +- src/util/virnvme.c | 4 +- src/util/virnvme.h | 2 +- src/util/viruuid.h | 2 +- tests/commandhelper.c | 8 +- tests/fuzz/README.rst | 131 +++++ tests/fuzz/ch_xml_domain_fuzz.cc | 157 ++++++ tests/fuzz/libxl_xml_domain_fuzz.cc | 159 ++++++ tests/fuzz/llvm_symbolizer_wrapper.c | 11 + tests/fuzz/meson.build | 183 ++++++ tests/fuzz/proto_custom_datatypes.cc | 234 ++++++++ tests/fuzz/proto_custom_datatypes.h | 30 + tests/fuzz/proto_header_common.h | 51 ++ tests/fuzz/proto_to_xml.cc | 277 ++++++++++ tests/fuzz/proto_to_xml.h | 39 ++ tests/fuzz/protos/meson.build | 46 ++ tests/fuzz/protos/xml_datatypes.proto | 93 ++++ tests/fuzz/protos/xml_domain.proto | 62 +++ tests/fuzz/protos/xml_domain_disk_only.proto | 21 + .../protos/xml_domain_interface_only.proto | 21 + tests/fuzz/protos/xml_hotplug.proto | 38 ++ tests/fuzz/protos/xml_nwfilter.proto | 9 + tests/fuzz/qemu_xml_domain_fuzz.cc | 277 ++++++++++ tests/fuzz/qemu_xml_hotplug_fuzz.cc | 340 ++++++++++++ tests/fuzz/run_fuzz.in | 142 +++++ tests/fuzz/vmx_xml_domain_fuzz.cc | 208 +++++++ tests/fuzz/xml_nwfilter_fuzz.cc | 149 +++++ tests/meson.build | 5 + tests/qemumonitortestutils.c | 48 ++ tests/qemumonitortestutils.h | 6 + tests/qemuxmlconftest.c | 249 --------- tests/testutilsqemu.c | 256 +++++++++ tests/testutilsqemu.h | 57 ++ 58 files changed, 3832 insertions(+), 302 deletions(-) create mode 100644 docs/kbase/internals/xml-fuzzing.rst create mode 100644 scripts/relaxng-to-proto.py create mode 100644 tests/fuzz/README.rst create mode 100644 tests/fuzz/ch_xml_domain_fuzz.cc create mode 100644 tests/fuzz/libxl_xml_domain_fuzz.cc create mode 100644 tests/fuzz/llvm_symbolizer_wrapper.c create mode 100644 tests/fuzz/meson.build create mode 100644 tests/fuzz/proto_custom_datatypes.cc create mode 100644 tests/fuzz/proto_custom_datatypes.h create mode 100644 tests/fuzz/proto_header_common.h create mode 100644 tests/fuzz/proto_to_xml.cc create mode 100644 tests/fuzz/proto_to_xml.h create mode 100644 tests/fuzz/protos/meson.build create mode 100644 tests/fuzz/protos/xml_datatypes.proto create mode 100644 tests/fuzz/protos/xml_domain.proto create mode 100644 tests/fuzz/protos/xml_domain_disk_only.proto create mode 100644 tests/fuzz/protos/xml_domain_interface_only.proto create mode 100644 tests/fuzz/protos/xml_hotplug.proto create mode 100644 tests/fuzz/protos/xml_nwfilter.proto create mode 100644 tests/fuzz/qemu_xml_domain_fuzz.cc create mode 100644 tests/fuzz/qemu_xml_hotplug_fuzz.cc create mode 100644 tests/fuzz/run_fuzz.in create mode 100644 tests/fuzz/vmx_xml_domain_fuzz.cc create mode 100644 tests/fuzz/xml_nwfilter_fuzz.cc -- 2.34.1

3 22

[PATCH] spec: Fix dtrace BuildRequires on fedora 41+
by Cole Robinson 04 Sep '24

04 Sep '24

/usr/bin/dtrace has been split from `systemtap-sdt-devel` into `systemtap-sdt-dtrace` It's forward and backward compatible to require the dtrace binary directly. We still need the latter dep though, for sdt.h in generated libvirt_probes.h Signed-off-by: Cole Robinson <crobinso(a)redhat.com> --- libvirt.spec.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libvirt.spec.in b/libvirt.spec.in index 29101e74fe..86af13a861 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -425,8 +425,8 @@ BuildRequires: libcurl-devel BuildRequires: libwsman-devel >= 2.6.3 %endif BuildRequires: audit-libs-devel -# we need /usr/sbin/dtrace BuildRequires: systemtap-sdt-devel +BuildRequires: /usr/bin/dtrace # For mount/umount in FS driver BuildRequires: util-linux # For showmount in FS driver (netfs discovery) -- 2.46.0

2 1