- Devel - libvirt List Archives

[PATCH v2 0/4] multiple memory backend support for CPR Live Updates
by mgalaxy＠akamai.com 23 Oct '24

23 Oct '24

From: Michael Galaxy <mgalaxy(a)akamai.com> CPR-based support for whole-hypervisor kexec-based live updates is now finally merged into QEMU. In support of this, we need NUMA to be supported in these kinds of environments. To do this we use a technology called PMEM (persistent memory) in Linux, which underpins the ability for CPR Live Updates to work so that QEMU memory can remain in RAM and be recovered after a kexec operationg has completed. Our systems are highly NUMA-aware, and so this patch series enables NUMA awareness for live updates. Further, we make a small change that allows live migrations to work between *non* PMEM-based systems and PMEM-based systems (and vice-versa). This allows for seemless upgrades from non-live-compatible systems to live-update-compatible sytems without any downtime. Michael Galaxy (4): qemu.conf changes to support multiple memory backend Support live migration between file-backed memory and anonymous memory. Update unit test to support multiple memory backends Update documentation to reflect memory_backing_dir change in qemu.conf NEWS.rst | 7 ++ docs/kbase/virtiofs.rst | 2 + src/qemu/qemu.conf.in | 2 + src/qemu/qemu_command.c | 8 ++- src/qemu/qemu_conf.c | 141 +++++++++++++++++++++++++++++++++++----- src/qemu/qemu_conf.h | 14 ++-- src/qemu/qemu_domain.c | 24 +++++-- src/qemu/qemu_driver.c | 29 +++++---- src/qemu/qemu_hotplug.c | 6 +- src/qemu/qemu_process.c | 44 +++++++------ src/qemu/qemu_process.h | 7 +- tests/testutilsqemu.c | 5 +- 12 files changed, 221 insertions(+), 68 deletions(-) -- 2.34.1

3 26

[PATCH v3 00/18] maintainer updates (testing, gdbstub, plugins) pre-PR
by Alex Bennée 23 Oct '24

23 Oct '24

This is an aggregation of three of my maintainer trees which you can also get from their respective branches (testing/next, gdbstub/next and plugins/next). It is now ready for a pull request. Since last post: - added r-b tags - some minor typo fixes - dropped meson plugin contrib build and gdb hook changes The last thing that still needs review: gitlab: make check-[dco|patch] a little more verbose Alex Bennée (9): tests/docker: add NOFETCH env variable for testing MAINTAINERS: mention my testing/next tree scripts/ci: remove architecture checks for build-environment updates accel/tcg: add tracepoints for cpu_loop_exit_atomic gitlab: make check-[dco|patch] a little more verbose MAINTAINERS: mention my gdbstub/next tree config/targets: update aarch64_be-linux-user gdb XML list tests/tcg: enable basic testing for aarch64_be-linux-user MAINTAINERS: mention my plugins/next tree Gustavo Romero (2): tests/tcg/aarch64: Use raw strings for regexes in test-mte.py testing: Enhance gdb probe script Ilya Leoshkevich (2): tests/docker: Fix microblaze atomics tests/tcg/x86_64: Add cross-modifying code test Pierrick Bouvier (5): meson: hide tsan related warnings docs/devel: update tsan build documentation dockerfiles: fix default targets for debian-loongarch-cross contrib/plugins: remove Makefile for contrib/plugins plugins: fix qemu_plugin_reset MAINTAINERS | 3 + docs/devel/testing/main.rst | 26 +++++- configure | 23 ++--- Makefile | 10 --- configs/targets/aarch64_be-linux-user.mak | 2 +- meson.build | 10 ++- accel/tcg/plugin-gen.c | 4 + accel/tcg/user-exec.c | 2 +- tests/tcg/aarch64_be/hello.c | 35 ++++++++ tests/tcg/x86_64/cross-modifying-code.c | 80 +++++++++++++++++ accel/tcg/ldst_atomicity.c.inc | 9 ++ .gitlab-ci.d/check-dco.py | 5 +- .gitlab-ci.d/check-patch.py | 5 +- accel/tcg/trace-events | 12 +++ contrib/plugins/Makefile | 87 ------------------- scripts/ci/setup/ubuntu/build-environment.yml | 2 - scripts/probe-gdb-support.py | 75 ++++++++-------- tests/docker/Makefile.include | 5 +- .../dockerfiles/debian-loongarch-cross.docker | 4 +- .../build-toolchain.sh | 8 ++ .../dockerfiles/debian-toolchain.docker | 7 ++ tests/tcg/Makefile.target | 7 +- tests/tcg/aarch64/gdbstub/test-mte.py | 4 +- tests/tcg/aarch64_be/Makefile.target | 17 ++++ tests/tcg/x86_64/Makefile.target | 4 + 25 files changed, 273 insertions(+), 173 deletions(-) create mode 100644 tests/tcg/aarch64_be/hello.c create mode 100644 tests/tcg/x86_64/cross-modifying-code.c delete mode 100644 contrib/plugins/Makefile create mode 100644 tests/tcg/aarch64_be/Makefile.target -- 2.39.5

2 19

[PATCH 0/9] qemu: Change CPU comparison algorithm for future models
by Jiri Denemark 23 Oct '24

23 Oct '24

When starting a domain we check whether the guest CPU definition is compatible with the host (i.e., when the host supports all features required both explicitly and by the specified CPU model) as long as check == 'partial', which is the default. We are doing so by checking our definition of the CPU model in the CPU map amending it with explicitly mentioned features and comparing it to features QEMU would enabled when started with -cpu host. But since our CPU model definitions often slightly differ from QEMU we may be checking features which are not actually needed and on the other hand not checking something that is part of the CPU model in QEMU. This patch changes the algorithm for CPU models added in the future (changing it for existing models could cause them to suddenly become incompatible with the host and domains using them would fail to start). The new algorithm uses information we probe from QEMU about features that block each model from being directly usable. If all those features are explicitly disabled in the CPU definition we consider the base model compatible with the host. Then we only need to check that all explicitly required features are supported by QEMU on the host to get the result for the whole CPU definition. After this we only use the model definitions (for newly added models) from CPU map for creating a CPU definition for host-model. Jiri Denemark (9): cpu_x86: Introduce <check> element for CPU models cpu_map: Use compat partial check for all x86 CPU models cpu: Introduce virCPUGetCheckMode qemu: Use g_autoptr in qemuConnectCompareHypervisorCPU qemu: Use virCPUCompare in qemuConnectCompareHypervisorCPU directly qemu: Separate partial CPU check into a function cpu: Introduce virCPUCompareUnusable qemu_capabilities: Introduce virQEMUCapsGetCPUBlockers qemu: Change CPU comparison algorithm for future models src/cpu/cpu.c | 104 ++++++++++++++++++ src/cpu/cpu.h | 17 +++ src/cpu/cpu_x86.c | 67 +++++++++++ src/cpu_map/x86_486.xml | 1 + src/cpu_map/x86_Broadwell-IBRS.xml | 1 + src/cpu_map/x86_Broadwell-noTSX-IBRS.xml | 1 + src/cpu_map/x86_Broadwell-noTSX.xml | 1 + src/cpu_map/x86_Broadwell.xml | 1 + src/cpu_map/x86_Cascadelake-Server-noTSX.xml | 1 + src/cpu_map/x86_Cascadelake-Server.xml | 1 + src/cpu_map/x86_Conroe.xml | 1 + src/cpu_map/x86_Cooperlake.xml | 1 + src/cpu_map/x86_Dhyana.xml | 1 + src/cpu_map/x86_EPYC-Genoa.xml | 1 + src/cpu_map/x86_EPYC-IBPB.xml | 1 + src/cpu_map/x86_EPYC-Milan.xml | 1 + src/cpu_map/x86_EPYC-Rome.xml | 1 + src/cpu_map/x86_EPYC.xml | 1 + src/cpu_map/x86_GraniteRapids.xml | 1 + src/cpu_map/x86_Haswell-IBRS.xml | 1 + src/cpu_map/x86_Haswell-noTSX-IBRS.xml | 1 + src/cpu_map/x86_Haswell-noTSX.xml | 1 + src/cpu_map/x86_Haswell.xml | 1 + src/cpu_map/x86_Icelake-Client-noTSX.xml | 1 + src/cpu_map/x86_Icelake-Client.xml | 1 + src/cpu_map/x86_Icelake-Server-noTSX.xml | 1 + src/cpu_map/x86_Icelake-Server.xml | 1 + src/cpu_map/x86_IvyBridge-IBRS.xml | 1 + src/cpu_map/x86_IvyBridge.xml | 1 + src/cpu_map/x86_Nehalem-IBRS.xml | 1 + src/cpu_map/x86_Nehalem.xml | 1 + src/cpu_map/x86_Opteron_G1.xml | 1 + src/cpu_map/x86_Opteron_G2.xml | 1 + src/cpu_map/x86_Opteron_G3.xml | 1 + src/cpu_map/x86_Opteron_G4.xml | 1 + src/cpu_map/x86_Opteron_G5.xml | 1 + src/cpu_map/x86_Penryn.xml | 1 + src/cpu_map/x86_SandyBridge-IBRS.xml | 1 + src/cpu_map/x86_SandyBridge.xml | 1 + src/cpu_map/x86_SapphireRapids.xml | 1 + src/cpu_map/x86_SierraForest.xml | 1 + src/cpu_map/x86_Skylake-Client-IBRS.xml | 1 + src/cpu_map/x86_Skylake-Client-noTSX-IBRS.xml | 1 + src/cpu_map/x86_Skylake-Client.xml | 1 + src/cpu_map/x86_Skylake-Server-IBRS.xml | 1 + src/cpu_map/x86_Skylake-Server-noTSX-IBRS.xml | 1 + src/cpu_map/x86_Skylake-Server.xml | 1 + src/cpu_map/x86_Snowridge.xml | 1 + src/cpu_map/x86_Westmere-IBRS.xml | 1 + src/cpu_map/x86_Westmere.xml | 1 + src/cpu_map/x86_athlon.xml | 1 + src/cpu_map/x86_core2duo.xml | 1 + src/cpu_map/x86_coreduo.xml | 1 + src/cpu_map/x86_cpu64-rhel5.xml | 1 + src/cpu_map/x86_cpu64-rhel6.xml | 1 + src/cpu_map/x86_kvm32.xml | 1 + src/cpu_map/x86_kvm64.xml | 1 + src/cpu_map/x86_n270.xml | 1 + src/cpu_map/x86_pentium.xml | 1 + src/cpu_map/x86_pentium2.xml | 1 + src/cpu_map/x86_pentium3.xml | 1 + src/cpu_map/x86_pentiumpro.xml | 1 + src/cpu_map/x86_phenom.xml | 1 + src/cpu_map/x86_qemu32.xml | 1 + src/cpu_map/x86_qemu64.xml | 1 + src/libvirt_private.syms | 2 + src/qemu/qemu_capabilities.c | 38 +++++++ src/qemu/qemu_capabilities.h | 4 + src/qemu/qemu_domain.c | 65 +++++++++++ src/qemu/qemu_domain.h | 7 ++ src/qemu/qemu_driver.c | 45 ++++---- src/qemu/qemu_process.c | 7 +- 72 files changed, 390 insertions(+), 28 deletions(-) -- 2.47.0

2 10

[PATCH] ci: Move definition of exit codes allowed to fail for cirrus jobs
by Peter Krempa 23 Oct '24

23 Oct '24

Update with latest lcitool. Update the build templates to move the definition of exit codes which are allowed to fail for cirrus jobs for cases when we run out of CI minutes. The previous location was overridden with the per-job 'allow_failure' value and thus didn't apply. Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- Based on: https://gitlab.com/libvirt/libvirt-ci/-/merge_requests/510 example run: https://gitlab.com/libvirt/libvirt/-/pipelines/1508364775 ci/gitlab/build-templates.yml | 2 -- ci/gitlab/builds.yml | 9 ++++++--- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/ci/gitlab/build-templates.yml b/ci/gitlab/build-templates.yml index 31a901e47c..b1e41b0783 100644 --- a/ci/gitlab/build-templates.yml +++ b/ci/gitlab/build-templates.yml @@ -282,8 +282,6 @@ image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:latest interruptible: true needs: [] - allow_failure: - exit_codes: 3 script: - set -o allexport - source ci/cirrus/$NAME.vars diff --git a/ci/gitlab/builds.yml b/ci/gitlab/builds.yml index 771fa67501..c24421378c 100644 --- a/ci/gitlab/builds.yml +++ b/ci/gitlab/builds.yml @@ -599,7 +599,8 @@ mingw64-fedora-rawhide: x86_64-freebsd-13: extends: .cirrus_build_job needs: [] - allow_failure: false + allow_failure: + exit_codes: 3 variables: CIRRUS_VM_IMAGE_NAME: freebsd-13-3 CIRRUS_VM_IMAGE_SELECTOR: image_family @@ -613,7 +614,8 @@ x86_64-freebsd-13: x86_64-freebsd-14: extends: .cirrus_build_job needs: [] - allow_failure: false + allow_failure: + exit_codes: 3 variables: CIRRUS_VM_IMAGE_NAME: freebsd-14-0 CIRRUS_VM_IMAGE_SELECTOR: image_family @@ -627,7 +629,8 @@ x86_64-freebsd-14: aarch64-macos-14: extends: .cirrus_build_job needs: [] - allow_failure: false + allow_failure: + exit_codes: 3 variables: CIRRUS_VM_IMAGE_NAME: ghcr.io/cirruslabs/macos-runner:sonoma CIRRUS_VM_IMAGE_SELECTOR: image -- 2.47.0

2 1

[PATCH v2 0/6] ch: handle events from cloud-hypervisor
by Purna Pavan Chandra Aekkaladevi 23 Oct '24

23 Oct '24

changes from v1->v2: * Rebase on latest master * Use /* */ for comments * Remove fifo file if already exists * Address other comments from Praveen Paladugu cloud-hypervisor raises various events, including VM lifecylce operations such as boot, shutdown, pause, resume, etc. Libvirt will now read these events and take the necessary actions, such as correctly updating the domain state. A FIFO file is passed to `--event-monitor` option of cloud-hypervisor. Libvirt creates a new thread that acts as the reader of the fifo file and continuously monitors for new events. Currently, shutdown events are handled by updating the domain state appropriately. Purna Pavan Chandra Aekkaladevi (6): utils: Implement virFileIsNamedPipe ch: pass --event-monitor option to cloud-hypervisor ch: start a new thread for handling ch events ch: events: Read and parse cloud-hypervisor events ch: events: facilitate lifecycle events handling NEWS: Mention event handling support in ch driver NEWS.rst | 7 + po/POTFILES | 1 + src/ch/ch_events.c | 349 +++++++++++++++++++++++++++++++++++++++ src/ch/ch_events.h | 54 ++++++ src/ch/ch_monitor.c | 52 +++++- src/ch/ch_monitor.h | 11 ++ src/ch/meson.build | 2 + src/libvirt_private.syms | 1 + src/util/virfile.c | 8 + src/util/virfile.h | 1 + 10 files changed, 479 insertions(+), 7 deletions(-) create mode 100644 src/ch/ch_events.c create mode 100644 src/ch/ch_events.h -- 2.34.1

2 10

[PATCH (pushed)] ci: Regenerate with latest lcitool to pick up 'cirrus-ci' out of credit detection
by Peter Krempa 22 Oct '24

22 Oct '24

Allow the cirrus jobs to fail on return code '3' which is emitted if we've run out of credits. Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- Pushed as CI fix. ci/gitlab/build-templates.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ci/gitlab/build-templates.yml b/ci/gitlab/build-templates.yml index b1e41b0783..31a901e47c 100644 --- a/ci/gitlab/build-templates.yml +++ b/ci/gitlab/build-templates.yml @@ -282,6 +282,8 @@ image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:latest interruptible: true needs: [] + allow_failure: + exit_codes: 3 script: - set -o allexport - source ci/cirrus/$NAME.vars -- 2.47.0

1 0

[PATCH v2 0/3] util/xml: make xml parser error reporting less terrible
by Daniel P. Berrangé 22 Oct '24

22 Oct '24

libxml2 has terrible error reporting when failing to open files, which has led to significant wasted time on wild goose chases. This improves it by taking libxml2 out of the business of opening & reading files. Daniel P. Berrangé (3): util/xml: fix extraction of XML context util/xml: don't assume libxml2 has the filename of the document util/xml: open XML files before calling libxml2 src/util/virxml.c | 29 +++++++++++++------ .../broken-xml-invalid.x86_64-latest.err | 4 +-- .../nonexistent-file.x86_64-latest.err | 2 +- 3 files changed, 23 insertions(+), 12 deletions(-) -- 2.46.0

3 7

[PATCH] docs: permit a user's chosen identity with SoB
by Daniel P. Berrangé 22 Oct '24

22 Oct '24

The docs for submitting a patch describe using your "Legal Name" with the Signed-off-by line. In recent times, there's been a general push back[1] against the notion that use of Signed-off-by in a project automatically requires / implies the use of legal ("real") names and greater awareness of the downsides. Full discussion of the problems of such policies is beyond the scope of this commit message, but at a high level they are liable to marginalize, disadvantage, and potentially result in harm, to contributors. TL;DR: there are compelling reasons for a person to choose distinct identities in different contexts & a decision to override that choice should not be taken lightly. A number of key projects have responded to the issues raised by making it clear that a contributor is free to determine the identity used in SoB lines: * Linux has clarified[2] that they merely expect use of the contributor's "known identity", removing the previous explicit rejection of pseudonyms. * CNCF has clarified[3] that the real name is simply the identity the contributor chooses to use in the context of the community and does not have to be a legal name, nor birth name, nor appear on any government ID. Since we have no intention of ever routinely checking any form of ID documents for contributors[4], realistically we have no way of knowing anything about the name they are using, except through chance, or through the contributor volunteering the information. IOW, we almost certainly already have people using pseudonyms for contributions. This proposes to accept that reality and eliminate unnecessary friction, by following Linux & the CNCF in merely asking that a contributors' commonly known identity, of their choosing, be used with the SoB line. [1] Raised in many contexts at many times, but a decent overall summary can be read at https://drewdevault.com/2023/10/31/On-real-names.html [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… [3] https://github.com/cncf/foundation/blob/659fd32c86dc/dco-guidelines.md [4] Excluding the rare GPG key signing parties for regular maintainers Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- docs/governance.rst | 6 +++--- docs/hacking.rst | 9 ++++++--- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/docs/governance.rst b/docs/governance.rst index 44dd54d4a0..04bb739237 100644 --- a/docs/governance.rst +++ b/docs/governance.rst @@ -112,9 +112,9 @@ There are no special requirements to becoming a contributor other than having the interest and ability to provide a contribution. The libvirt project **does not require** any *"Contributor License Agreement"* to be signed prior to engagement with the community. However for contributing patches, providing a -'Signed-off-by' line with the author's legal name and e-mail address to -demonstrate agreement and compliance with the `Developer Certificate of -Origin <https://developercertificate.org/>`__ is required. +'Signed-off-by' line with the author's chosen name and e-mail address to +demonstrate agreement and compliance with the `Developer Certificate +of Origin <hacking.html#developer-certificate-of-origin>`__ is required. In making a non-patch contribution to the project, the community member is implicitly stating that they accept the terms of the license under which the diff --git a/docs/hacking.rst b/docs/hacking.rst index c2e79e4501..825b8b83d2 100644 --- a/docs/hacking.rst +++ b/docs/hacking.rst @@ -83,9 +83,12 @@ Contributors to libvirt projects **must** assert that they are in compliance with the `Developer Certificate of Origin 1.1 <https://developercertificate.org/>`__. This is achieved by adding a "Signed-off-by" line containing the contributor's name -and e-mail to every commit message. The presence of this line -attests that the contributor has read the above lined DCO and -agrees with its statements. +and e-mail to every commit message. The name should be the identity +the contributor has chosen to be known as in the context of the +community. It does not need to be a legal name, nor match any +formal ID documents, but should not be anonymous, nor misrepresent +who you are. The presence of this line attests that the contributor +has read the above linked DCO and agrees with its statements. Further reading =============== -- 2.46.0

2 1

[PATCH v3 00/10] swtpm: Add support for profiles
by Stefan Berger 22 Oct '24

22 Oct '24

Upcoming libtpms v0.10 and swtpm v0.10 will have TPM profile support that allows to restrict a TPM's provided set of crypto algorithms and commands and through which backwards compatibility and migration from newer versions of libtpms to older ones (up to libtpms v0.9) is supported. For the latter to work it is necessary that the user chooses the right ('null') profile. This series adds support for passing a profile choice to swtpm_setup by setting it in the domain XML using the <profile/> XML node. An optional attribute 'remove_disabled' can be set in this node and accepts two values: "check": test a few crypto algorithms (tdes, camellia, unpadded encryption, and others) for whether they are currently disabled due to FIPS mode on the host and remove these algorithms in the 'custom' profile if they are disabled; "fips-host": do not test but remove all the possibly disabled crypto algorithms (from list above) Also extend the documentation but point the user to swtpm and libtpms documentation for further details. Follow Deniel's suggestions there's now a PR for swtpm_setup to support searching for profiles though a configurable local directory, distro directory and if no profile could be found there (with appended ".json" suffix) it will fall back to try to use a built-in profile by the provided name: https://github.com/stefanberger/swtpm/pull/918 Stefan v3: - 2/10: Adjustments to due rebase - Applied Marc-André's R-b tags - 10/10: Read back profile name from swtpm and adjust it in emulator defs Stefan Berger (10): conf: Move TPM emulator parameters into own struct qemu: Pass virQEMUDriverConfig rather than some of its fields util: Add parsing support for swtpm_setup's cmdarg-profile capability conf: Define enum virDomainTPMProfileRemoveDisabled schema: Extend schema for TPM emulator profile node conf: Add support for profile parameter on TPM emulator in domain XML docs: Add documentation for the TPM backend profile node qemu: Extend swtpm_setup command line to set a profile by its name qemu: Move adding of keys to swtpm command line into own function qemu: Read the profile name after creation of TPM instance docs/formatdomain.rst | 30 +++ src/conf/domain_conf.c | 43 +++++ src/conf/domain_conf.h | 35 ++-- src/conf/domain_validate.c | 7 + src/conf/schemas/domaincommon.rng | 25 +++ src/conf/virconftypes.h | 2 + src/qemu/qemu_extdevice.c | 5 +- src/qemu/qemu_tpm.c | 310 ++++++++++++++++++++---------- src/qemu/qemu_tpm.h | 3 +- src/util/virtpm.c | 2 + src/util/virtpm.h | 2 + tests/testutilsqemu.c | 1 + 12 files changed, 355 insertions(+), 110 deletions(-) -- 2.47.0

3 13

Help with setting up libvirt for development.
by validsudoku＠gmail.com 22 Oct '24

22 Oct '24

Hi, pardon my ignorance. I'm trying to add some changes to the libvirt codebase (https://gitlab.com/libvirt/libvirt) I've successfully built it with `meson setup /build` and a `compile_commands.json` was generated in the build folder. But vscode keeps showing that there are errors from trying to find certain files and identifiers. I have also copied the `compile_commands.json` to the root of the project with no success. I'm new to working on a large C codebase, as i'm mainly into Go. I have also installed the `meson.build` extension as prescribed in https://mesonbuild.com/IDE-integration.html, is there anything i'm missing?

2 1