- Devel - libvirt List Archives

[libvirt PATCH v2 00/10] docs: Expand documentation on the CI/testing topic
by Erik Skultety 14 Jul '22

14 Jul '22

This series was inspired by QEMU's documentation on CI and focuses on the following: - adds documentation on how to add a custom runner to the libvirt project or its fork, mainly in context of running functional testing - replaces the old 'testsuites' article with a more detailed landing page describing different types of testing available in libvirt - updates the TCK documentation with details on how TCK should be installed and used nowadays Since v1: - described the --force argument with TCK usage - moved the article about adding new platforms to libvirt-ci and linked it from our docs -> corresponding MR here: https://gitlab.com/libvirt/libvirt-ci/-/merge_requests/295 -> ^the MR must merged **FIRST**! - moved Podman prerequistes for local container builds before Docker Here's a fresh website pipeline with artifacts: https://gitlab.com/eskultety/libvirt/-/jobs/2722840726 Erik Skultety (10): docs: Move the CI dashboard to its own RST module docs: Provide more information about the TCK test suite docs: Provide an article on how to add a custom runner to the project docs: Provide an article on testing docs: Replace testsuites article with the new 'testing' article docs: Drop the testsuites article docs: Change the CI headline from "CI Testing" to "CI" docs: ci: Add info about the two major types of jobs our CI runs docs: ci: Add a section on how to add a new platform to libvirt CI docs: ci: Add a brief section on how to run the CI workload locally docs/ci-dashboard.rst | 216 ++++++++++++++++++++++++++++++++++++++ docs/ci-runners.rst | 86 +++++++++++++++ docs/ci.rst | 237 ++++++------------------------------------ docs/docs.rst | 10 +- docs/meson.build | 4 +- docs/testing.rst | 172 ++++++++++++++++++++++++++++++ docs/testsuites.rst | 37 ------- docs/testtck.rst | 99 ++++++++++++++++-- 8 files changed, 602 insertions(+), 259 deletions(-) create mode 100644 docs/ci-dashboard.rst create mode 100644 docs/ci-runners.rst create mode 100644 docs/testing.rst delete mode 100644 docs/testsuites.rst -- 2.36.1

2 14

[PATCH 0/3] qemu: add an API for "query-stats" QMP command
by Amneesh Singh 14 Jul '22

14 Jul '22

QEMU is gaining introspectable statistics which can be queried via the "query-stats" QMP command. This patchset aims to add an API for the same. The returned JSON for "query-stats" is an array of objects containing their own respective array of statistics. Patch 1 adds the API which returns the deserialized JSON in the form of a GPtrArray of GHashTables. Patch 2 adds the "query-stats" to QEMU capabilities. Patch 3 uses the API to query the halt poll success time and the halt poll failure time. v2 -> v3 ======== Sorry for the late patchset, I was under the impression I had sent it on Monday but apparently, I did not. [1/3]: - use a single enum for all the statistics. - use QEMU_MONITOR_QUERY_STATS_NAME_LAST as the sentinel value for the provider constructor. - relevant changes to enum values. [2/3]: - fix comment spacing [3/3] - better checks v1 -> v2 ======== I have been tinkering with the v1 patchset and have rewrote the v2 patches a couple of times. I believe the current patchset is still not perfect and would appreciate some reviews. I have another patch or two written but they do not make any significant changes to the current patchset. [1/3]: - use virBitmap instead of an array of strings for statistics. - add enums for the stat names and add qemuMonitorQueryStatsNameTypeToString to switch between the "ToString" functions based on the target type. - change qemuMonitorQueryStatsProviderNew to a variadic function that takes stat enum values with the sentinel value being -1. [2/3]: - No changes [3/3]: - Add relevant monitor related checks to check if the domain is active. - Acquire and release qemuMonitorObj lock before and after calling qemuMonitorQueryStats respectively. - Add the check for privileged access. Relevant QEMU patches can be found here: https://lore.kernel.org/all/20220530150714.756954-1-pbonzini@redhat.com/ This patchset is part of the 2022 GSOC contributor project. Amneesh Singh (3): qemu_monitor: add qemuMonitorQueryStats qemu_capabilities: add "query-stats" QMP command to the QEMU capabilities qemu_driver: use qemuMonitorQueryStats to extract halt poll time src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_driver.c | 69 ++++++++++++++++--- src/qemu/qemu_monitor.c | 70 +++++++++++++++++++ src/qemu/qemu_monitor.h | 45 ++++++++++++ src/qemu/qemu_monitor_json.c | 130 +++++++++++++++++++++++++++++++++++ src/qemu/qemu_monitor_json.h | 6 ++ 7 files changed, 315 insertions(+), 8 deletions(-) -- 2.36.1

1 0

[PATCH v2 0/6] conf: Introduce @guestReset to hostdev's <source/>
by Michal Privoznik 13 Jul '22

13 Jul '22

v2 of: https://listman.redhat.com/archives/libvir-list/2022-July/232756.html diff to v2: - Fixed capabilities files per Jano's review Michal Prívozník (6): tests: Update hostdev-usb-* hostdev-pci-* xml2xml and xml2argv tests tests: Fix qemu replies wrt usb-host properties conf: Introduce @guestReset to hostdev's <source/> qemu_capabilities: Track usb-host.guest-resets-all capability qemu_validate: Validate that QEMU's capable of guestReset qemu: Generate cmd line for guestReset docs/formatdomain.rst | 15 +++++++- src/conf/domain_conf.c | 18 +++++++++ src/conf/domain_conf.h | 13 +++++++ src/conf/schemas/domaincommon.rng | 9 +++++ src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 21 ++++++++++ src/qemu/qemu_validate.c | 8 ++++ .../caps_3.1.0.ppc64.replies | 15 -------- .../caps_3.1.0.x86_64.replies | 15 -------- .../caps_4.0.0.aarch64.replies | 10 ----- .../caps_4.0.0.ppc64.replies | 10 ----- .../caps_4.0.0.riscv32.replies | 10 ----- .../caps_4.0.0.riscv64.replies | 10 ----- .../caps_4.0.0.s390x.replies | 10 ----- .../caps_4.0.0.x86_64.replies | 10 ----- .../caps_4.1.0.x86_64.replies | 10 ----- .../caps_4.2.0.aarch64.replies | 5 --- .../caps_4.2.0.aarch64.xml | 1 + .../caps_4.2.0.ppc64.replies | 5 --- .../qemucapabilitiesdata/caps_4.2.0.ppc64.xml | 1 + .../caps_4.2.0.s390x.replies | 5 --- .../qemucapabilitiesdata/caps_4.2.0.s390x.xml | 1 + .../caps_4.2.0.x86_64.replies | 5 --- .../caps_4.2.0.x86_64.xml | 1 + .../caps_5.0.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_5.0.0.ppc64.xml | 1 + .../caps_5.0.0.riscv64.xml | 1 + .../caps_5.0.0.x86_64.xml | 1 + .../caps_5.1.0.x86_64.xml | 1 + .../caps_5.2.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_5.2.0.ppc64.xml | 1 + .../caps_5.2.0.riscv64.xml | 1 + .../caps_5.2.0.x86_64.xml | 1 + .../caps_6.0.0.aarch64.xml | 1 + .../caps_6.0.0.x86_64.xml | 1 + .../caps_6.1.0.x86_64.xml | 1 + .../caps_6.2.0.aarch64.xml | 1 + .../caps_6.2.0.x86_64.xml | 1 + .../caps_7.0.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_7.0.0.ppc64.xml | 1 + .../caps_7.0.0.x86_64.xml | 1 + .../caps_7.1.0.x86_64.xml | 1 + .../hostdev-pci-address-device.args | 34 ----------------- ...tdev-pci-address-device.x86_64-latest.args | 38 +++++++++++++++++++ .../hostdev-pci-address-device.xml | 2 +- .../qemuxml2argvdata/hostdev-pci-address.args | 33 ---------------- .../hostdev-pci-address.x86_64-latest.args | 37 ++++++++++++++++++ .../qemuxml2argvdata/hostdev-pci-address.xml | 2 +- .../hostdev-usb-address-device-boot.args | 34 ----------------- ...usb-address-device-boot.x86_64-latest.args | 38 +++++++++++++++++++ .../hostdev-usb-address-device-boot.xml | 2 +- .../hostdev-usb-address-device.args | 34 ----------------- ...tdev-usb-address-device.x86_64-latest.args | 38 +++++++++++++++++++ .../hostdev-usb-address-device.xml | 2 +- .../qemuxml2argvdata/hostdev-usb-address.args | 33 ---------------- .../hostdev-usb-address.x86_64-latest.args | 37 ++++++++++++++++++ .../qemuxml2argvdata/hostdev-usb-address.xml | 4 +- tests/qemuxml2argvtest.c | 10 ++--- ... => hostdev-pci-address.x86_64-latest.xml} | 7 +++- ... => hostdev-usb-address.x86_64-latest.xml} | 9 +++-- tests/qemuxml2xmltest.c | 4 +- 62 files changed, 320 insertions(+), 307 deletions(-) delete mode 100644 tests/qemuxml2argvdata/hostdev-pci-address-device.args create mode 100644 tests/qemuxml2argvdata/hostdev-pci-address-device.x86_64-latest.args delete mode 100644 tests/qemuxml2argvdata/hostdev-pci-address.args create mode 100644 tests/qemuxml2argvdata/hostdev-pci-address.x86_64-latest.args delete mode 100644 tests/qemuxml2argvdata/hostdev-usb-address-device-boot.args create mode 100644 tests/qemuxml2argvdata/hostdev-usb-address-device-boot.x86_64-latest.args delete mode 100644 tests/qemuxml2argvdata/hostdev-usb-address-device.args create mode 100644 tests/qemuxml2argvdata/hostdev-usb-address-device.x86_64-latest.args delete mode 100644 tests/qemuxml2argvdata/hostdev-usb-address.args create mode 100644 tests/qemuxml2argvdata/hostdev-usb-address.x86_64-latest.args rename tests/qemuxml2xmloutdata/{hostdev-pci-address.xml => hostdev-pci-address.x86_64-latest.xml} (86%) rename tests/qemuxml2xmloutdata/{hostdev-usb-address.xml => hostdev-usb-address.x86_64-latest.xml} (83%) -- 2.35.1

2 8

[libvirt PATCH] docs: replace ARRAY_CARDINALITY with G_N_ELEMENTS
by Daniel P. Berrangé 13 Jul '22

13 Jul '22

Fix docs illustrating call with an obsolete macro. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- docs/kbase/internals/locking.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/kbase/internals/locking.rst b/docs/kbase/internals/locking.rst index 50d2b92d57..7a863ab4b2 100644 --- a/docs/kbase/internals/locking.rst +++ b/docs/kbase/internals/locking.rst @@ -127,7 +127,7 @@ locks, this will be done from the libvirtd process. }; mgr = virLockManagerNew(lockPlugin, VIR_LOCK_MANAGER_TYPE_DOMAIN, - ARRAY_CARDINALITY(params), + G_N_ELEMENTS(params), params, 0))); @@ -171,7 +171,7 @@ however, a process may voluntarily give up the lock by running }; mgr = virLockManagerNew(lockPlugin, VIR_LOCK_MANAGER_TYPE_DOMAIN, - ARRAY_CARDINALITY(params), + G_N_ELEMENTS(params), params, 0))); -- 2.36.1

2 1

[libvirt PATCH 00/10] docs: Expand documentation on the CI/testing topic
by Erik Skultety 13 Jul '22

13 Jul '22

This series was inspired by QEMU's documentation on CI and focuses on the following: - adds documentation on how to add a custom runner to the libvirt project or its fork, mainly in context of running functional testing - replaces the old 'testsuites' article with a more detailed landing page describing different types of testing available in libvirt - updates the TCK documentation with details on how TCK should be installed and used nowadays For sure we can add lots more content related to testing in libvirt but I think this series provides a solid foundation that can be easily expanded on in the future. Here's the website build artifact to see what it'll look like eventually: https://gitlab.com/eskultety/libvirt/-/jobs/2709823650 Erik Skultety (10): docs: Move the CI dashboard to its own RST module docs: Provide more information about the TCK test suite docs: Provide an article on how to add a custom runner to the project docs: Provide an article on testing docs: Replace testsuites article with the new 'testing' article docs: Drop the testsuites article docs: Change the CI headline from "CI Testing" to "CI" docs: ci: Add info about the two major types of jobs our CI runs docs: ci: Add a section on how to add a new platform to libvirt CI docs: ci: Add a brief section on how to run the CI workload locally docs/ci-dashboard.rst | 216 +++++++++++++++++++++++++++++++++++ docs/ci-runners.rst | 86 ++++++++++++++ docs/ci.rst | 255 +++++++++++------------------------------- docs/docs.rst | 10 +- docs/meson.build | 4 +- docs/testing.rst | 176 +++++++++++++++++++++++++++++ docs/testsuites.rst | 37 ------ docs/testtck.rst | 92 +++++++++++++-- 8 files changed, 631 insertions(+), 245 deletions(-) create mode 100644 docs/ci-dashboard.rst create mode 100644 docs/ci-runners.rst create mode 100644 docs/testing.rst delete mode 100644 docs/testsuites.rst -- 2.36.1

2 20

[PATCH 0/5] conf: Introduce @guestReset to hostdev's <source/>
by Michal Privoznik 13 Jul '22

13 Jul '22

There are couple of examples where this knob would be useful: https://bugs.launchpad.net/qemu/+bug/1846451 (look for those cmd line passthrough hacks), and one pretty recent request: https://listman.redhat.com/archives/libvir-list/2022-June/232611.html The last patch uses info from Kraxel's patch (which he posted after our private discussion): https://lists.gnu.org/archive/html/qemu-devel/2022-07/msg01733.html Michal Prívozník (5): tests: Update hostdev-usb-* hostdev-pci-* xml2xml and xml2argv tests conf: Introduce @guestReset to hostdev's <source/> qemu_capabilities: Track usb-host.guest-resets-all capability qemu_validate: Validate that QEMU's capable of guestReset qemu: Generate cmd line for guestReset docs/formatdomain.rst | 15 +++++++- src/conf/domain_conf.c | 18 +++++++++ src/conf/domain_conf.h | 13 +++++++ src/conf/schemas/domaincommon.rng | 9 +++++ src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 21 ++++++++++ src/qemu/qemu_validate.c | 8 ++++ .../qemucapabilitiesdata/caps_3.1.0.ppc64.xml | 1 + .../caps_3.1.0.x86_64.xml | 1 + .../caps_4.0.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_4.0.0.ppc64.xml | 1 + .../caps_4.0.0.riscv32.xml | 1 + .../caps_4.0.0.riscv64.xml | 1 + .../qemucapabilitiesdata/caps_4.0.0.s390x.xml | 1 + .../caps_4.0.0.x86_64.xml | 1 + .../caps_4.1.0.x86_64.xml | 1 + .../caps_4.2.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_4.2.0.ppc64.xml | 1 + .../qemucapabilitiesdata/caps_4.2.0.s390x.xml | 1 + .../caps_4.2.0.x86_64.xml | 1 + .../caps_5.0.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_5.0.0.ppc64.xml | 1 + .../caps_5.0.0.riscv64.xml | 1 + .../caps_5.0.0.x86_64.xml | 1 + .../caps_5.1.0.x86_64.xml | 1 + .../caps_5.2.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_5.2.0.ppc64.xml | 1 + .../caps_5.2.0.riscv64.xml | 1 + .../caps_5.2.0.x86_64.xml | 1 + .../caps_6.0.0.aarch64.xml | 1 + .../caps_6.0.0.x86_64.xml | 1 + .../caps_6.1.0.x86_64.xml | 1 + .../caps_6.2.0.aarch64.xml | 1 + .../caps_6.2.0.x86_64.xml | 1 + .../caps_7.0.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_7.0.0.ppc64.xml | 1 + .../caps_7.0.0.x86_64.xml | 1 + .../caps_7.1.0.x86_64.xml | 1 + .../hostdev-pci-address-device.args | 34 ----------------- ...tdev-pci-address-device.x86_64-latest.args | 38 +++++++++++++++++++ .../hostdev-pci-address-device.xml | 2 +- .../qemuxml2argvdata/hostdev-pci-address.args | 33 ---------------- .../hostdev-pci-address.x86_64-latest.args | 37 ++++++++++++++++++ .../qemuxml2argvdata/hostdev-pci-address.xml | 2 +- .../hostdev-usb-address-device-boot.args | 34 ----------------- ...usb-address-device-boot.x86_64-latest.args | 38 +++++++++++++++++++ .../hostdev-usb-address-device-boot.xml | 2 +- .../hostdev-usb-address-device.args | 34 ----------------- ...tdev-usb-address-device.x86_64-latest.args | 38 +++++++++++++++++++ .../hostdev-usb-address-device.xml | 2 +- .../qemuxml2argvdata/hostdev-usb-address.args | 33 ---------------- .../hostdev-usb-address.x86_64-latest.args | 37 ++++++++++++++++++ .../qemuxml2argvdata/hostdev-usb-address.xml | 4 +- tests/qemuxml2argvtest.c | 10 ++--- ... => hostdev-pci-address.x86_64-latest.xml} | 7 +++- ... => hostdev-usb-address.x86_64-latest.xml} | 9 +++-- tests/qemuxml2xmltest.c | 4 +- 58 files changed, 329 insertions(+), 187 deletions(-) delete mode 100644 tests/qemuxml2argvdata/hostdev-pci-address-device.args create mode 100644 tests/qemuxml2argvdata/hostdev-pci-address-device.x86_64-latest.args delete mode 100644 tests/qemuxml2argvdata/hostdev-pci-address.args create mode 100644 tests/qemuxml2argvdata/hostdev-pci-address.x86_64-latest.args delete mode 100644 tests/qemuxml2argvdata/hostdev-usb-address-device-boot.args create mode 100644 tests/qemuxml2argvdata/hostdev-usb-address-device-boot.x86_64-latest.args delete mode 100644 tests/qemuxml2argvdata/hostdev-usb-address-device.args create mode 100644 tests/qemuxml2argvdata/hostdev-usb-address-device.x86_64-latest.args delete mode 100644 tests/qemuxml2argvdata/hostdev-usb-address.args create mode 100644 tests/qemuxml2argvdata/hostdev-usb-address.x86_64-latest.args rename tests/qemuxml2xmloutdata/{hostdev-pci-address.xml => hostdev-pci-address.x86_64-latest.xml} (86%) rename tests/qemuxml2xmloutdata/{hostdev-usb-address.xml => hostdev-usb-address.x86_64-latest.xml} (83%) -- 2.35.1

3 11

[PATCH 0/5] Follow up fixes for IOThreads
by Michal Privoznik 13 Jul '22

13 Jul '22

Couple of issues were found with my recent work on IOThread and its pool size limits. Here are fixes. Michal Prívozník (5): domain_conf: Format <defaultiothread/> more often domain_conf: Format iothread IDs more often qemu: Make IOThread changing more robust qemuDomainSetIOThreadParams: Accept VIR_DOMAIN_AFFECT_CONFIG flag virsh: Implement --config for iothreadset src/conf/domain_conf.c | 50 ++++++++++---------- src/qemu/qemu_driver.c | 101 +++++++++++++++++++++++------------------ tools/virsh-domain.c | 8 ++++ 3 files changed, 90 insertions(+), 69 deletions(-) -- 2.35.1

2 6

[PATCH 0/4] qemu: Report supported TPM version in domcaps
by Michal Privoznik 13 Jul '22

13 Jul '22

*** BLURB HERE *** Michal Prívozník (4): docs: Document TPM portion of domcaps virtpm: Introduce TPM-1.2 and TPM-2.0 capabilieis domcaps: Introduce TPM backendVersion qemu: Report supported TPM version in domcaps docs/formatdomaincaps.rst | 36 ++++++++++++++++++++++++++++++++++ src/conf/domain_capabilities.c | 1 + src/conf/domain_capabilities.h | 1 + src/qemu/qemu_capabilities.c | 15 +++++++++++--- src/util/virtpm.c | 2 ++ src/util/virtpm.h | 2 ++ 6 files changed, 54 insertions(+), 3 deletions(-) -- 2.35.1

2 8

[libvirt PATCH v2] docs: add info about factors affecting CPU compatibility
by Daniel P. Berrangé 13 Jul '22

13 Jul '22

While libvirt solves most of the problem of ensuring compatibility, when there is incompatibility it can be hard for users to track down the cause. Everything knows to check the physical CPU model, but there are a surprisingly large number of other factors influencing compatibility. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- In v2: - Also mention nested virt docs/drvqemu.rst | 83 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) diff --git a/docs/drvqemu.rst b/docs/drvqemu.rst index c174bdb4cf..c9f2432268 100644 --- a/docs/drvqemu.rst +++ b/docs/drvqemu.rst @@ -416,6 +416,89 @@ following command should be run as root, prior to starting libvirtd libvirt will then place each virtual machine in a cgroup at ``/dev/cgroup/libvirt/qemu/$VMNAME/`` + +Live migration compatibility +---------------------------- + +Many factors can affect the ability to live migrate a guest between a pair +of hosts. It is critical that when QEMU is started on the destination host, +the exposed guest machine ABI matches what was exposed by the existing QEMU +process on the source host. To facilitate this, when libvirt receives a +guest configuration document, it will attempt to expand any features that +were not specified, to ensure a stable guest machine ABI. Mostly this involves +adding address information to all devices, and adding controllers to attach +the devices to. + +Certain features that affect the guest ABI, however, may only be known at the +time the guest is started and can be influenced by features of the host OS +and its hardware. This means that even if the guest XML configuration is the +same, it may still be impossible to migrate the guest between two hosts. + +Migration CPU model compatibility +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The most common problems with migration compatibility surround the use of +the guest CPU ``host-model`` or ``host-passthrough`` modes. Both of these +modes attempt to expose the full host CPU featureset to the guest. The +``host-model`` mode attempts to expose as many features as possible while +retaining the ability to accurately check compatibility between hosts prior +to migration running. The ``host-passthrough`` mode attempts to expose the +host CPU as precisely as possible, but with the cost that it is not possible +for libvirt to check compatibility prior to migration. + +If using ``host-model`` the target host hardware and software deployment +must expose a superset of the features of the source host CPU. If using +``host-passthrough`` the target host CPU and software deployment must +always expose a superset of the fetures, however, it is further strongly +recommended that the source and destination hosts be identical in every +way. + +In both cases, there are a number of factors that will influence the CPU +features available to the guest + +* **Physical CPU model** - the core constraint on what features are available. + Check ``/proc/cpuinfo`` for CPU model name. +* **Firmware revision (BIOS/UEFI/etc)** - firmware updates may bundle microcode + updates which arbitrarily add or remove CPU features, typically in response + to new hardware vulnerabilities. Check ``dmidecode`` for details on ``x86`` + and ``aarch64`` platforms for firmware version, and ``/proc/cpuinfo`` for + associated microcode version (if not updated by the OS). +* **Firmware settings** - certain firmware settings can affect accessibility of + features. For example, turning on/off SMT/HT not only affects the number + of logical CPUs available to the OS, but can indirectly influence other + factors such as the number of performance counters available for use. Check + the firmware specific configuration interface. +* **Host kernel version** - the host kernel software version may have a + need to block certain physical CPU features from use in the guest. It can + also emulate certain features that may not exist in the silicon, for example, + x2apic. Check ``uname -r`` output for kernel version. +* **Host kernel settings** - the kernel command line options can be used to + block certain physical CPU features from use in the guest, for example, + ``tsx=off``, ``l1tf=...`` or ``nosmt``. Check ``/proc/cmdline`` and + ``/etc/modprobe.d/*.conf``. +* **microcode update version** - while the firmware will load the initial + microcode in to the CPU, the OS may ship packages providing newer microcode + updates since these can be deployed on a more timely manner than firmware + updates. These updates can arbitrarily load add or remove CPU features. + Check ``/proc/cpuinfo`` for microcode version. +* **QEMU version** - even when the kernel supports exposing a CPU feature to + the guest, an update in the QEMU emulator version will be required to unlock + its usage with a guest, except with ``host-passthrough``. Check the output + of ``$QEMU -version``. +* **libvirt version** - even when the kernel and QEMU support exposing a CPU + feature to the guest, an update in the libvirt version will be required to + unlock its usage with a guest, except with ``host-passthrough``. Check + ``virsh version``. +* **Nested virtualization** - due to the limitations of nested virtualization, + a L1 nested host may not be able to expose the same featureset as a bare + metal host, even if everything else is the same. + +The ``virsh capabilities`` output will provide information on the high level +CPU model, its features, microcode version. Most of the time this will provide +enough information to know whether the CPUs of two hosts will be compatible. +If there are unexpected differences though, checking the above list of +influencing factors can reveal where the difference arises from. + Import and export of libvirt domain XML configs ----------------------------------------------- -- 2.36.1

2 1

[libvirt PATCH] docs: add info about factors affecting CPU compatibility
by Daniel P. Berrangé 12 Jul '22

12 Jul '22

While libvirt solves most of the problem of ensuring compatibility, when there is incompatibility it can be hard for users to track down the cause. Everything knows to check the physical CPU model, but there are a surprisingly large number of other factors influencing compatibility. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- docs/drvqemu.rst | 80 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) diff --git a/docs/drvqemu.rst b/docs/drvqemu.rst index c174bdb4cf..3344df4fb6 100644 --- a/docs/drvqemu.rst +++ b/docs/drvqemu.rst @@ -416,6 +416,86 @@ following command should be run as root, prior to starting libvirtd libvirt will then place each virtual machine in a cgroup at ``/dev/cgroup/libvirt/qemu/$VMNAME/`` + +Live migration compatibility +---------------------------- + +Many factors can affect the ability to live migrate a guest between a pair +of hosts. It is critical that when QEMU is started on the destination host, +the exposed guest machine ABI matches what was exposed by the existing QEMU +process on the source host. To facilitate this, when libvirt receives a +guest configuration document, it will attempt to expand any features that +were not specified, to ensure a stable guest machine ABI. Mostly this involves +adding address information to all devices, and adding controllers to attach +the devices to. + +Certain features that affect the guest ABI, however, may only be known at the +time the guest is started and can be influenced by features of the host OS +and its hardware. This means that even if the guest XML configuration is the +same, it may still be impossible to migrate the guest between two hosts. + +Migration CPU model compatibility +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The most common problems with migration compatibility surround the use of +the guest CPU ``host-model`` or ``host-passthrough`` modes. Both of these +modes attempt to expose the full host CPU featureset to the guest. The +``host-model`` mode attempts to expose as many features as possible while +retaining the ability to accurately check compatibility between hosts prior +to migration running. The ``host-passthrough`` mode attempts to expose the +host CPU as precisely as possible, but with the cost that it is not possible +for libvirt to check compatibility prior to migration. + +If using ``host-model`` the target host hardware and software deployment +must expose a superset of the features of the source host CPU. If using +``host-passthrough`` the target host CPU and software deployment must +always expose a superset of the fetures, however, it is further strongly +recommended that the source and destination hosts be identical in every +way. + +In both cases, there are a number of factors that will influence the CPU +features available to the guest + +* **Physical CPU model** - the core constraint on what features are available. + Check ``/proc/cpuinfo`` for CPU model name. +* **Firmware revision (BIOS/UEFI/etc)** - firmware updates may bundle microcode + updates which arbitrarily add or remove CPU features, typically in response + to new hardware vulnerabilities. Check ``dmidecode`` for details on ``x86`` + and ``aarch64`` platforms for firmware version, and ``/proc/cpuinfo`` for + associated microcode version (if not updated by the OS). +* **Firmware settings** - certain firmware settings can affect accessibility of + features. For example, turning on/off SMT/HT not only affects the number + of logical CPUs available to the OS, but can indirectly influence other + factors such as the number of performance counters available for use. Check + the firmware specific configuration interface. +* **Host kernel version** - the host kernel software version may have a + need to block certain physical CPU features from use in the guest. It can + also emulate certain features that may not exist in the silicon, for example, + x2apic. Check ``uname -r`` output for kernel version. +* **Host kernel settings** - the kernel command line options can be used to + block certain physical CPU features from use in the guest, for example, + ``tsx=off``, ``l1tf=...`` or ``nosmt``. Check ``/proc/cmdline`` and + ``/etc/modprobe.d/*.conf``. +* **microcode update version** - while the firmware will load the initial + microcode in to the CPU, the OS may ship packages providing newer microcode + updates since these can be deployed on a more timely manner than firmware + updates. These updates can arbitrarily load add or remove CPU features. + Check ``/proc/cpuinfo`` for microcode version. +* **QEMU version** - even when the kernel supports exposing a CPU feature to + the guest, an update in the QEMU emulator version will be required to unlock + its usage with a guest, except with ``host-passthrough``. Check the output + of ``$QEMU -version``. +* **libvirt version** - even when the kernel and QEMU support exposing a CPU + feature to the guest, an update in the libvirt version will be required to + unlock its usage with a guest, except with ``host-passthrough``. Check + ``virsh version``. + +The ``virsh capabilities`` output will provide information on the high level +CPU model, its features, microcode version. Most of the time this will provide +enough information to know whether the CPUs of two hosts will be compatible. +If there are unexpected differences though, checking the above list of +influencing factors can reveal where the difference arises from. + Import and export of libvirt domain XML configs ----------------------------------------------- -- 2.36.1

1 0