- Devel - libvirt List Archives

Release of libvirt-10.3.0
by Jiri Denemark 02 May '24

02 May '24

Somehow I forgot to make rc2 on Tuesday, but the only commits after rc1 at that time were translations so I guess it's not a big deal. The 10.3.0 release of both libvirt and libvirt-python is tagged and signed tarballs are available at https://download.libvirt.org/ https://download.libvirt.org/python/ Thanks everybody who helped with this release by sending patches, reviewing, testing, or providing feedback. Your work is greatly appreciated. * New features * qemu: Proper support for USB network device USB address is now automatically assigned to USB network devices thus they can be used without manual configuration. * conf: Introduce memReserve attribute to <controller/> Some PCI devices have large non-prefetchable memory. This can be a problem in case when such device needs to be hotplugged as the firmware can't foresee such situation. The user thus can override the value calculated at start to accomodate for such devices. * Improvements * Improve validation of USB devices Certain USB device types ('sound', 'fs', 'chr', 'ccid' and 'net') were not properly handled in the check whether the VM config supports USB and thus would result in poor error messages. * virsh: Fix behaviour of ``--name`` and ``--parent`` used together when listing checkpoint and snapshots The ``checkpoint-list`` and ``snapshot-list`` commands would ignore the ``--name`` option to print only the name when used with ``--parent``. * Extend libvirt-guests to shutdown only persistent VMs Users can now choose to shutdown only persistent VMs when the host is being shut down. * Bug fixes * qemu: Fix migration with custom XML Libvirt 10.2.0 would sometimes complain about incompatible CPU definition when trying to migrate or save a domain and passing a custom XML even though such XML was properly generated as migratable. Hitting this bug depends on the guest CPU definition and the host on which a particular domain was running. * qemu: Fix TLS hostname verification failure in certain non-shared storage migration scenarios In certain scenarios (parallel migration, newly also post-copy migration) libvirt would wrongly pass an empty hostname to QEMU to be used for TLS certificate hostname validation, which would result into failure of the non-shared storage migration step:: error: internal error: unable to execute QEMU command 'blockdev-add': Certificate does not match the hostname * Create OVS ports as transient Libvirt now creates OVS ports as transient which prevents them from reappearing or going stale on sudden reboots. * Clear OVS QoS settings when domain shuts down Libvirt now clears QoS settings on domain shutdown, so they no longer pile up in OVS database. Enjoy. Jirka

1 0

[PATCH RESEND] NEWS: Document my contributions for upcoming release
by Michal Privoznik 02 May '24

02 May '24

Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- Rebased version of the patch sent earlier, because the file was changed meanwhile. NEWS.rst | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/NEWS.rst b/NEWS.rst index 7ced82e3da..554ca559a2 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -22,6 +22,13 @@ v10.3.0 (unreleased) USB address is now automatically assigned to USB network devices thus they can be used without manual configuration. + * conf: Introduce memReserve to <controller/> + + Some PCI devices have large non-prefetchable memory. This is not a problem + for coldplug because firmware sets up such devices properly. But it may be + a problem for hotplug devices. To resolve this, new ``memReserve`` + attribute is introduced which allows overriding value computed by firmware. + * **Improvements** * Improve validation of USB devices @@ -35,6 +42,11 @@ v10.3.0 (unreleased) The ``checkpoint-list`` and ``snapshot-list`` commands would ignore the ``--name`` option to print only the name when used with ``--parent``. + * Extend libvirt-guests to shutdown only persistent VMs + + Users can now chose to shutdown only persistent VMs when the host is being + shut down. + * **Bug fixes** * qemu: Fix migration with custom XML @@ -54,6 +66,16 @@ v10.3.0 (unreleased) error: internal error: unable to execute QEMU command 'blockdev-add': Certificate does not match the hostname + * Create OVS ports as transient + + Libvirt now creates OVS ports as transient which prevents them from + reappearing or going stale on sudden reboots. + + * Clear OVS QoS settings when domain shuts down + + Libvirt now clears QoS settings on domain shutdown, so they no longer pile + up in OVS database. + v10.2.0 (2024-04-02) ==================== -- 2.43.2

4 4

Revisiting parallel save/restore
by Jim Fehlig 02 May '24

02 May '24

Hi All, While Fabiano has been working on improving save/restore performance in qemu, I've been tinkering with the same in libvirt. The end goal is to introduce a new VIR_DOMAIN_SAVE_PARALLEL flag for save/restore, along with a VIR_DOMAIN_SAVE_PARAM_PARALLEL_CONNECTIONS parameter to specify the number of concurrent channels used for the save/restore. Recall Claudio previously posted a patch series implementing parallel save/restore completely in libvirt, using qemu's multifd functionality [1]. A good starting point on this journey is supporting the new mapped-ram capability in qemu 9.0 [2]. Since mapped-ram is a new on-disk format, I assume we'll need a new QEMU_SAVE_VERSION 3 when using it? Otherwise I'm not sure how to detect if a saved image is in mapped-ram format vs the existing, sequential stream format. IIUC, mapped-ram cannot be used with the exiting 'fd:' migration URI and instead must use 'file:'. Does qemu advertise support for that? I couldn't find it. If not, 'file:' (available in qemu 8.2) predates mapped-ram, so in theory we could live without the advertisement. It's also not clear when we want to enable the mapped-ram capability. Should it always be enabled if supported by the underlying qemu? One motivation for creating the mapped-ram was to support direct-io of the migration stream in qemu, in which case it could be tied to VIR_DOMAIN_SAVE_BYPASS_CACHE. E.g. the mapped-ram capability is enabled when user specifies VIR_DOMAIN_SAVE_BYPASS_CACHE && user-provided path results in a seekable fd && qemu supports mapped-ram? Looking ahead, should the mapped-ram capability be required for supporting the VIR_DOMAIN_SAVE_PARALLEL flag? As I understand, parallel save/restore was another motivation for creating the mapped-ram feature. It allows multifd threads to write exclusively to the offsets provided by mapped-ram. Can multiple multifd threads concurrently write to an fd without mapped-ram? Regards, Jim [1] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/3Y5G… [2] https://gitlab.com/qemu-project/qemu/-/blob/master/docs/devel/migration/map…

4 11

[PATCH] docs/about: Automatically deprecate versioned machine types older than 6 years
by Thomas Huth 30 Apr '24

30 Apr '24

Old machine types often have bugs or work-arounds that affect our possibilities to move forward with the QEMU code base (see for example https://gitlab.com/qemu-project/qemu/-/issues/2213 for a bug that likely cannot be fixed without breaking live migration with old machine types, or https://lists.gnu.org/archive/html/qemu-devel/2018-12/msg04516.html or commit ea985d235b86). So instead of going through the process of manually deprecating old machine types again and again, let's rather add an entry that can stay, which declares that machine types older than 6 years are considered as deprecated automatically. Six years should be sufficient to support the release cycles of most Linux distributions. Signed-off-by: Thomas Huth <thuth(a)redhat.com> --- docs/about/deprecated.rst | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 6d595de3b6..fe69e2d44c 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -220,6 +220,17 @@ is a chance the code will bitrot without anyone noticing. System emulator machines ------------------------ +Versioned machine types older than 6 years +'''''''''''''''''''''''''''''''''''''''''' + +Starting with the release of QEMU 10.0, versioned machine types older than +6 years will automatically be considered as deprecated and might be due to +removal without furthor notice. For example, this affects machine types like +pc-i440fx-X.Y, pc-q35-X.Y, pseries-X.Y, s390-ccw-virtio-X.Y or virt-X.Y where +X is the major number and Y is the minor number of the old QEMU version. +If you are still using machine types from QEMU versions older than 6 years, +please update your setting to use a newer versioned machine type instead. + Arm ``virt`` machine ``dtb-kaslr-seed`` property (since 7.1) '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' -- 2.44.0

5 10

[PATCH] NEWS: Document my contributions for upcoming release
by Michal Privoznik 30 Apr '24

30 Apr '24

Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- NEWS.rst | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/NEWS.rst b/NEWS.rst index 852dadf532..4958326875 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -17,8 +17,20 @@ v10.3.0 (unreleased) * **New features** + * conf: Introduce memReserve to <controller/> + + Some PCI devices have large non-prefetchable memory. This is not a problem + for coldplug because firmware sets up such devices properly. But it may be + a problem for hotplug devices. To resolve this, new ``memReserve`` + attribute is introduced which allows overriding value computed by firmware. + * **Improvements** + * Extend libvirt-guests to shutdown only persistent VMs + + Users can now chose to shutdown only persistent VMs when the host is being + shut down. + * **Bug fixes** * qemu: Fix migration with custom XML @@ -29,6 +41,16 @@ v10.3.0 (unreleased) depends on the guest CPU definition and the host on which a particular domain was running. + * Create OVS ports as transient + + Libvirt now creates OVS ports as transient which prevents them from + reappearing or going stale on sudden reboots. + + * Clear OVS QoS settings when domain shuts down + + Libvirt now clears QoS settings on domain shutdown, so they no longer pile + up in OVS database. + v10.2.0 (2024-04-02) ==================== -- 2.43.2

1 0

[PATCH 0/2] NEWS fix/update
by Peter Krempa 30 Apr '24

30 Apr '24

Peter Krempa (2): NEWS: Fix marking of 'path' as inline literal NEWS: Document TLS migration bug, usb-net support and two fixes NEWS.rst | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) -- 2.44.0

2 3

[PATCH v2 0/6] migration removals & deprecations
by Fabiano Rosas 29 Apr '24

29 Apr '24

[respinning because master moved and there were conflicts] Hi everyone, Here's some cleaning up of deprecated code. It removes the old block migration and compression code. Both have suitable replacements in the form of the blockdev-mirror driver and multifd compression, respectively. There's also a deprecation for fd: + file to cope with the fact that the new MigrationAddress API defines transports instead of protocols (loose terms) like the old string API did. So we cannot map 1:1 from fd: to any transport because fd: allows *both* file migration and socket migration. v1: https://lore.kernel.org/r/20240425150939.19268-1-farosas@suse.de Fabiano Rosas (6): migration: Remove 'skipped' field from MigrationStats migration: Remove 'inc' option from migrate command migration: Remove 'blk/-b' option from migrate commands migration: Remove block migration migration: Remove non-multifd compression migration: Deprecate fd: for file migration .gitlab-ci.d/buildtest.yml | 2 +- MAINTAINERS | 1 - docs/about/deprecated.rst | 51 +- docs/about/removed-features.rst | 103 +++ docs/devel/migration/main.rst | 2 +- hw/core/machine.c | 1 - include/migration/misc.h | 6 - meson.build | 2 - meson_options.txt | 2 - migration/block.c | 1019 ------------------------------ migration/block.h | 52 -- migration/colo.c | 1 - migration/meson.build | 4 - migration/migration-hmp-cmds.c | 97 +-- migration/migration.c | 70 +- migration/migration.h | 7 - migration/options.c | 229 ------- migration/ram-compress.c | 564 ----------------- migration/ram.c | 166 +---- migration/savevm.c | 5 - qapi/migration.json | 205 +----- scripts/meson-buildoptions.sh | 4 - tests/qemu-iotests/183 | 147 ----- tests/qemu-iotests/183.out | 66 -- tests/qemu-iotests/common.filter | 7 - tests/qtest/migration-test.c | 139 ---- 26 files changed, 130 insertions(+), 2822 deletions(-) delete mode 100644 migration/block.c delete mode 100644 migration/block.h delete mode 100644 migration/ram-compress.c delete mode 100755 tests/qemu-iotests/183 delete mode 100644 tests/qemu-iotests/183.out base-commit: a118c4aff4087eafb68f7132b233ad548cf16376 -- 2.35.3

2 17

[PATCH 0/4] implement 'ras' feature support
by Kristina Hanicova 29 Apr '24

29 Apr '24

*** BLURB HERE *** Kristina Hanicova (4): Introduce QEMU_CAPS_MACHINE_VIRT_RAS capability conf: parse and format machine virt ras feature qemu: validate machine virt ras feature qemu: format machine virt ras feature and test it docs/formatdomain.rst | 5 +++ src/conf/domain_conf.c | 6 +++- src/conf/domain_conf.h | 1 + src/conf/schemas/domaincommon.rng | 5 +++ src/qemu/qemu_capabilities.c | 2 ++ src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 5 +++ src/qemu/qemu_validate.c | 16 ++++++++++ .../caps_5.2.0_aarch64.xml | 1 + .../caps_6.0.0_aarch64.xml | 1 + .../caps_6.2.0_aarch64.xml | 1 + .../caps_7.0.0_aarch64+hvf.xml | 1 + .../caps_7.0.0_aarch64.xml | 1 + .../caps_8.2.0_aarch64.xml | 1 + .../caps_8.2.0_armv7l.xml | 1 + .../aarch64-features-ras.aarch64-latest.args | 31 +++++++++++++++++++ .../aarch64-features-ras.aarch64-latest.xml | 1 + .../qemuxmlconfdata/aarch64-features-ras.xml | 26 ++++++++++++++++ tests/qemuxmlconftest.c | 2 ++ 19 files changed, 107 insertions(+), 1 deletion(-) create mode 100644 tests/qemuxmlconfdata/aarch64-features-ras.aarch64-latest.args create mode 120000 tests/qemuxmlconfdata/aarch64-features-ras.aarch64-latest.xml create mode 100644 tests/qemuxmlconfdata/aarch64-features-ras.xml -- 2.42.0

4 10

introduce virDomainJobWait for dump plus testing
by Thanos Makatos 26 Apr '24

26 Apr '24

This patch series introduces funcionality for waiting for a job from a different context, for now only for dump jobs, plus testing. This can be useful in situations where the original requester of the job crashes/restarts and then needs to continue waiting for that potentially ongoing job. To avoid waiting for the wrong job type, the caller must specify the required job type. For now I've only implemented this for dump jobs; if the logic is correct I can extended it for all other job types. The first patch introduces the core functionality. I see that there is a function specifically for waiting for a dump job: qemuDumpWaitForCompletion. Ideally we should only use qemuDomainObjWait, which would work for any job type, however I'm not sure it's correct to use it instead of qemuDumpWaitForCompletion because the latter checks dumpCompleted. It's not obvious to me what is the purpose of dumpCompleted, is it because qemuDomainObjWait can return because some other job, running in parallel with the dump job, has completed, so it's effectively a false alarm? The second patch is mainly for testing the wait functionality via virsh. The remaining patches extend the test hypervisor's functionality on dump testing.

1 6

[PATCH v3 00/27] [PATCH v3 00/27] native support for nftables in virtual network driver
by Laine Stump 26 Apr '24

26 Apr '24

V2: https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/5RTZ… This patch series enables libvirt to use nftables rules rather than iptables *when setting up virtual networks* (it does *not* add nftables support to the nwfilter driver). I've added the Reviewed-by's from Daniel where given in V2 (as long as I haven't made any non-trivial changes). That includes patches 1-9, 11-13, 16, 20, and 22. Changes from V2 - mainly I've addressed the issues that Dan pointed out in his reviews (details in each patch). Functionally the main changes are: 1) You can now choose whether iptables or nftables should be the default backend with the new meson option "firewall_backend" (which is set to "nftables" by default). 2) rpm spec now requires iptables or nftables (rather than recommending both) 3) The <firewall> element in the network status XML now has a "name='fwRemoval'" attribute, just in case we ever add another <firewall> element to keep track of all the commands that were run to create the firewall as well as the commands needed to remove it. 4) Failure to find the binary needed for any firewall backend now results in an error log and termination of the daemon. Laine Stump (27): util/network: move viriptables.[ch] from util to network directory network: move all functions manipulating iptables rules into network_iptables.c network: make all iptables functions used only in network_iptables.c static util: #define the names used for private packet filter chains util: change name of virFirewallRule to virFirewallCmd util: rename virNetFilterAction to iptablesAction, and add VIR_ENUM_DECL/IMPL util: check for 0 args when applying iptables rule util: add -w/--concurrent when applying a FirewallCmd rather than when building it util: determine ignoreErrors value when creating virFirewallCmd, not when applying util/network: new virFirewallBackend enum network: add (empty) network.conf file to distribution files network: support setting firewallBackend from network.conf network: framework to call backend-specific function to init private filter chains util: new functions to support adding individual firewall rollback commands util: implement rollback rule autocreation for iptables commands network: turn on auto-rollback for the rules added for virtual networks util: add name attribute to virFirewall util: new function virFirewallNewFromRollback() util: new functions virFirewallParseXML() and virFirewallFormat() conf: add a virFirewall object to virNetworkObj network: use previously saved list of firewall removal commands network: save network status when firewall rules are reloaded meson: stop looking for iptables/ip6tables/ebtables at build time network: add an nftables backend for network driver's firewall construction tests: test cases for nftables backend network: prefer the nftables backend over iptables spec: require either iptables or nftables if network driver is installed libvirt.spec.in | 7 +- meson.build | 10 +- meson_options.txt | 1 + po/POTFILES | 3 +- src/conf/virnetworkobj.c | 47 + src/conf/virnetworkobj.h | 11 + src/libvirt_private.syms | 59 +- src/network/bridge_driver.c | 35 +- src/network/bridge_driver_conf.c | 64 + src/network/bridge_driver_conf.h | 3 + src/network/bridge_driver_linux.c | 630 +------ src/network/bridge_driver_nop.c | 6 +- src/network/bridge_driver_platform.h | 6 +- src/network/libvirtd_network.aug | 39 + src/network/meson.build | 36 + src/network/network.conf.in | 28 + src/network/network_iptables.c | 1677 +++++++++++++++++ src/network/network_iptables.h | 30 + src/network/network_nftables.c | 940 +++++++++ src/network/network_nftables.h | 28 + src/network/test_libvirtd_network.aug.in | 5 + src/nwfilter/nwfilter_ebiptables_driver.c | 1004 +++++----- src/util/meson.build | 1 - src/util/virebtables.c | 36 +- src/util/virfirewall.c | 820 ++++++-- src/util/virfirewall.h | 87 +- src/util/viriptables.c | 1072 ----------- src/util/viriptables.h | 155 -- .../{base.args => base.iptables} | 0 tests/networkxml2firewalldata/base.nftables | 256 +++ ...-linux.args => nat-default-linux.iptables} | 0 .../nat-default-linux.nftables | 248 +++ ...pv6-linux.args => nat-ipv6-linux.iptables} | 0 .../nat-ipv6-linux.nftables | 384 ++++ ...rgs => nat-ipv6-masquerade-linux.iptables} | 0 .../nat-ipv6-masquerade-linux.nftables | 456 +++++ ...linux.args => nat-many-ips-linux.iptables} | 0 .../nat-many-ips-linux.nftables | 472 +++++ ...-linux.args => nat-no-dhcp-linux.iptables} | 0 .../nat-no-dhcp-linux.nftables | 384 ++++ ...ftp-linux.args => nat-tftp-linux.iptables} | 0 .../nat-tftp-linux.nftables | 274 +++ ...inux.args => route-default-linux.iptables} | 0 .../route-default-linux.nftables | 162 ++ tests/networkxml2firewalltest.c | 56 +- tests/virfirewalltest.c | 424 ++--- 46 files changed, 7205 insertions(+), 2751 deletions(-) create mode 100644 src/network/libvirtd_network.aug create mode 100644 src/network/network.conf.in create mode 100644 src/network/network_iptables.c create mode 100644 src/network/network_iptables.h create mode 100644 src/network/network_nftables.c create mode 100644 src/network/network_nftables.h create mode 100644 src/network/test_libvirtd_network.aug.in delete mode 100644 src/util/viriptables.c delete mode 100644 src/util/viriptables.h rename tests/networkxml2firewalldata/{base.args => base.iptables} (100%) create mode 100644 tests/networkxml2firewalldata/base.nftables rename tests/networkxml2firewalldata/{nat-default-linux.args => nat-default-linux.iptables} (100%) create mode 100644 tests/networkxml2firewalldata/nat-default-linux.nftables rename tests/networkxml2firewalldata/{nat-ipv6-linux.args => nat-ipv6-linux.iptables} (100%) create mode 100644 tests/networkxml2firewalldata/nat-ipv6-linux.nftables rename tests/networkxml2firewalldata/{nat-ipv6-masquerade-linux.args => nat-ipv6-masquerade-linux.iptables} (100%) create mode 100644 tests/networkxml2firewalldata/nat-ipv6-masquerade-linux.nftables rename tests/networkxml2firewalldata/{nat-many-ips-linux.args => nat-many-ips-linux.iptables} (100%) create mode 100644 tests/networkxml2firewalldata/nat-many-ips-linux.nftables rename tests/networkxml2firewalldata/{nat-no-dhcp-linux.args => nat-no-dhcp-linux.iptables} (100%) create mode 100644 tests/networkxml2firewalldata/nat-no-dhcp-linux.nftables rename tests/networkxml2firewalldata/{nat-tftp-linux.args => nat-tftp-linux.iptables} (100%) create mode 100644 tests/networkxml2firewalldata/nat-tftp-linux.nftables rename tests/networkxml2firewalldata/{route-default-linux.args => route-default-linux.iptables} (100%) create mode 100644 tests/networkxml2firewalldata/route-default-linux.nftables -- 2.44.0

2 37