October 2024 - Devel - libvirt List Archives

[PATCH (RFC and a half?)] network: add rule to nftables backend that zeroes checksum of DHCP responses
by Laine Stump 24 Oct '24

24 Oct '24

Many long years ago (April 2010), soon after "vhost" in-kernel packet processing was added to the virtio-net driver, people running RHEL5 virtual machines with a virtio-net interface connected via a libvirt virtual network noticed that when vhost packet processing was enabled, their VMs could no longer get an IP address via DHCP - the guest was ignoring the DHCP response packets sent by the host. The (as danpb calls them) "gory details" of this are chronicled here: https://lists.isc.org/pipermail/dhcp-hackers/2010-April/001835.html but basically it was because the checksum of packets wasn't being fully computed on the host side (because the host had checksum offloading enabled and thought that it would be taken care of later, e.g. with NIC hardware), while these packets going from a tap device to a virtio-net NIC in a guest wouldn't get that service, and the packets would arrive with a "bad checksum". The "fix" for this ended up being that iptables added a new "--checksum-fill" action, and libvirt added an iptables rule for each virtual network to match DHCP response packets and perform --checksum-fill. In the meantime, the ISC DHCP package (which contains the dhclient program that had been rejecting the bad checksum packets) made a separate fix to their dhclient which caused it to accept packets anyway even if they didn't have a proper checksum (NB: that's not a full explanation, and possibly not accurate). The word at the time from those "in the know" was that the bad checksum problem was really specific to ISC's dhclient, and so once their fix was in use everywhere dhclient was used, the problem would be a thing of the past and the checksum fixup iptables rules would no longer be needed (but would otherwise be harmless if it was still there). Based on this information (and also due to the opinion that fixing the problem by having iptables modify the packet checksum was the wrong way to fix things), the nftables developers made the decision to not implement an equivalent to --checksum-fill in nftables. As a result, when I wrote the nftables firewall backend for libvirt virtual networks, it didn't add in any rule to "fix" broken UDP checksums (after all, that was fixed somewhere else 14 years ago, right???) Cut to last week, when Rich Jones was doing routine testing using Fedora 40 (the first Fedora release to use the nftables backend of libvirt's network driver by default) and a FreeBSD guest - for "some strange reason", the FreeBSD guest was unable to get an IP address from DHCP!! https://www.spinics.net/linux/fedora/libvirt-users/msg14356.html A few quick tests proved that it was the same old "bad checksum" problem from 2010 come back to haunt us. After some discussion with Phil Sutter and Eric Garver (nftables people), they suggested that, while nftables doesn't have an action that will *compute* the checksum of a packet, it *does* have an action that will set the checksum to 0, and that maybe we should try that. Then Phil tried it himself by manually adding such a rule to a running system, and verified that it did fix the issue at least for FreeBSD guests. So over the weekend I came up with a patch to add a checksum 0 rule to the rules setup for each virtual network. This is that patch. I have so far verified that this patch enables FreeBSD to receive the DHCP response and get an IP address, and that it hasn't *broken* this functionality for a random old Fedora image I had (Fedora 27!?!?! I really need to update my test images!!). Before pushing it I would like to verify that zeroing the checksum of DHCP response packets doesn't break any other guest, so I would appreciate the help of anyone who could build and install libvirt with this patch and let me know of both successes and failures of any guest to acquire an IP address with DHCP. Once I've received enough positive reports (and 0 negative reports!) then we can think about pushing this patch (and also backporting it downstream to Fedora 40) Signed-off-by: Laine Stump <laine(a)redhat.com> --- src/network/network_nftables.c | 69 +++++++++++++++++++ .../forward-dev-linux.nftables | 16 +++++ .../isolated-linux.nftables | 16 +++++ .../nat-default-linux.nftables | 16 +++++ .../nat-ipv6-linux.nftables | 16 +++++ .../nat-ipv6-masquerade-linux.nftables | 16 +++++ .../nat-many-ips-linux.nftables | 16 +++++ .../nat-port-range-ipv6-linux.nftables | 16 +++++ .../nat-port-range-linux.nftables | 16 +++++ .../nat-tftp-linux.nftables | 16 +++++ .../route-default-linux.nftables | 16 +++++ 11 files changed, 229 insertions(+) diff --git a/src/network/network_nftables.c b/src/network/network_nftables.c index f8b5ab665d..5523207269 100644 --- a/src/network/network_nftables.c +++ b/src/network/network_nftables.c @@ -51,6 +51,7 @@ VIR_LOG_INIT("network.nftables"); #define VIR_NFTABLES_FWD_OUT_CHAIN "guest_output" #define VIR_NFTABLES_FWD_X_CHAIN "guest_cross" #define VIR_NFTABLES_NAT_POSTROUTE_CHAIN "guest_nat" +#define VIR_NFTABLES_MANGLE_POSTROUTE_CHAIN "postroute_mangle" /* we must avoid using the standard "filter" table as used by * iptables, as any subsequent attempts to use iptables commands will @@ -106,6 +107,10 @@ nftablesGlobalChain nftablesChains[] = { /* chains for NAT rules */ {NULL, VIR_NFTABLES_NAT_POSTROUTE_CHAIN, "{ type nat hook postrouting priority 100; policy accept; }"}, + + /* chain for "mangle" rules that modify packets (e.g. 0 out UDP checksums) */ + {NULL, VIR_NFTABLES_MANGLE_POSTROUTE_CHAIN, "{ type filter hook postrouting priority 0; policy accept; }"}, + }; @@ -644,6 +649,44 @@ nftablesAddDontMasquerade(virFirewall *fw, } +/** + * nftablesAddOutputFixUdpChecksum: + * + * Add a rule to @fw that will 0 out the checksum of udp packets + * output from @iface with destination port @port. + + * Zeroing the checksum of a UDP packet tells the receiving end "you + * don't need to validate the checksum", which is useful in cases + * where the host (sender) thinks that packet checksums will be + * computed elsewhere (and so leaves a partially computed checksum in + * the packet header) while the guest (receiver) thinks that the + * checksum has already been fully computed; in the meantime none of + * the code in between has actually finished computing the + * checksum. + * + * An example of this is DHCP response packets from host to + * guest. If the checksum of each of these packets isn't zeroed, then + * many guests (e.g. FreeBSD) will drop them with reason BAD CHECKSUM; + * if the packets arrive at those guests with a checksum of 0, they + * will happily accept the packet. + */ +static void +nftablesAddOutputFixUdpChecksum(virFirewall *fw, + const char *iface, + int port) +{ + g_autofree char *portstr = g_strdup_printf("%d", port); + + virFirewallAddCmd(fw, VIR_FIREWALL_LAYER_IPV4, + "insert", "rule", "ip", + VIR_NFTABLES_PRIVATE_TABLE, + VIR_NFTABLES_MANGLE_POSTROUTE_CHAIN, + "oif", iface, "udp", "dport", portstr, + "counter", "udp", "checksum", "set", "0", + NULL); +} + + static const char networkLocalMulticastIPv4[] = "224.0.0.0/24"; static const char networkLocalMulticastIPv6[] = "ff02::/16"; static const char networkLocalBroadcast[] = "255.255.255.255/32"; @@ -901,6 +944,30 @@ nftablesAddGeneralFirewallRules(virFirewall *fw, } +static void +nftablesAddChecksumFirewallRules(virFirewall *fw, + virNetworkDef *def) +{ + size_t i; + virNetworkIPDef *ipv4def; + + /* Look for the first IPv4 address that has dhcp or tftpboot + * defined. We support dhcp config on 1 IPv4 interface only. + */ + for (i = 0; (ipv4def = virNetworkDefGetIPByIndex(def, AF_INET, i)); i++) { + if (ipv4def->nranges || ipv4def->nhosts) + break; + } + + /* If we are doing local DHCP service on this network, add a rule + * that will fixup the checksum of DHCP response packets back to + * the guests. + */ + if (ipv4def) + nftablesAddOutputFixUdpChecksum(fw, def->bridge, 68); +} + + static int nftablesAddIPSpecificFirewallRules(virFirewall *fw, virNetworkDef *def, @@ -952,6 +1019,8 @@ nftablesAddFirewallRules(virNetworkDef *def, virFirewall **fwRemoval) return -1; } + nftablesAddChecksumFirewallRules(fw, def); + if (virFirewallApply(fw) < 0) return -1; diff --git a/tests/networkxml2firewalldata/forward-dev-linux.nftables b/tests/networkxml2firewalldata/forward-dev-linux.nftables index 8badb74beb..9dea1a88a4 100644 --- a/tests/networkxml2firewalldata/forward-dev-linux.nftables +++ b/tests/networkxml2firewalldata/forward-dev-linux.nftables @@ -156,3 +156,19 @@ daddr \ 224.0.0.0/24 \ counter \ return +nft \ +-ae insert \ +rule \ +ip \ +libvirt_network \ +postroute_mangle \ +oif \ +virbr0 \ +udp \ +dport \ +68 \ +counter \ +udp \ +checksum \ +set \ +0 diff --git a/tests/networkxml2firewalldata/isolated-linux.nftables b/tests/networkxml2firewalldata/isolated-linux.nftables index d1b4dac178..67ee0a2bf5 100644 --- a/tests/networkxml2firewalldata/isolated-linux.nftables +++ b/tests/networkxml2firewalldata/isolated-linux.nftables @@ -62,3 +62,19 @@ oif \ virbr0 \ counter \ accept +nft \ +-ae insert \ +rule \ +ip \ +libvirt_network \ +postroute_mangle \ +oif \ +virbr0 \ +udp \ +dport \ +68 \ +counter \ +udp \ +checksum \ +set \ +0 diff --git a/tests/networkxml2firewalldata/nat-default-linux.nftables b/tests/networkxml2firewalldata/nat-default-linux.nftables index 28508292f9..951a5a6d60 100644 --- a/tests/networkxml2firewalldata/nat-default-linux.nftables +++ b/tests/networkxml2firewalldata/nat-default-linux.nftables @@ -142,3 +142,19 @@ daddr \ 224.0.0.0/24 \ counter \ return +nft \ +-ae insert \ +rule \ +ip \ +libvirt_network \ +postroute_mangle \ +oif \ +virbr0 \ +udp \ +dport \ +68 \ +counter \ +udp \ +checksum \ +set \ +0 diff --git a/tests/networkxml2firewalldata/nat-ipv6-linux.nftables b/tests/networkxml2firewalldata/nat-ipv6-linux.nftables index d8a9ba706d..617ed8b753 100644 --- a/tests/networkxml2firewalldata/nat-ipv6-linux.nftables +++ b/tests/networkxml2firewalldata/nat-ipv6-linux.nftables @@ -200,3 +200,19 @@ oif \ virbr0 \ counter \ accept +nft \ +-ae insert \ +rule \ +ip \ +libvirt_network \ +postroute_mangle \ +oif \ +virbr0 \ +udp \ +dport \ +68 \ +counter \ +udp \ +checksum \ +set \ +0 diff --git a/tests/networkxml2firewalldata/nat-ipv6-masquerade-linux.nftables b/tests/networkxml2firewalldata/nat-ipv6-masquerade-linux.nftables index a7f09cda59..a710d0e296 100644 --- a/tests/networkxml2firewalldata/nat-ipv6-masquerade-linux.nftables +++ b/tests/networkxml2firewalldata/nat-ipv6-masquerade-linux.nftables @@ -272,3 +272,19 @@ daddr \ ff02::/16 \ counter \ return +nft \ +-ae insert \ +rule \ +ip \ +libvirt_network \ +postroute_mangle \ +oif \ +virbr0 \ +udp \ +dport \ +68 \ +counter \ +udp \ +checksum \ +set \ +0 diff --git a/tests/networkxml2firewalldata/nat-many-ips-linux.nftables b/tests/networkxml2firewalldata/nat-many-ips-linux.nftables index b826fe6134..0be5fb7e65 100644 --- a/tests/networkxml2firewalldata/nat-many-ips-linux.nftables +++ b/tests/networkxml2firewalldata/nat-many-ips-linux.nftables @@ -366,3 +366,19 @@ daddr \ 224.0.0.0/24 \ counter \ return +nft \ +-ae insert \ +rule \ +ip \ +libvirt_network \ +postroute_mangle \ +oif \ +virbr0 \ +udp \ +dport \ +68 \ +counter \ +udp \ +checksum \ +set \ +0 diff --git a/tests/networkxml2firewalldata/nat-port-range-ipv6-linux.nftables b/tests/networkxml2firewalldata/nat-port-range-ipv6-linux.nftables index ceaed6fa40..7574356855 100644 --- a/tests/networkxml2firewalldata/nat-port-range-ipv6-linux.nftables +++ b/tests/networkxml2firewalldata/nat-port-range-ipv6-linux.nftables @@ -384,3 +384,19 @@ daddr \ ff02::/16 \ counter \ return +nft \ +-ae insert \ +rule \ +ip \ +libvirt_network \ +postroute_mangle \ +oif \ +virbr0 \ +udp \ +dport \ +68 \ +counter \ +udp \ +checksum \ +set \ +0 diff --git a/tests/networkxml2firewalldata/nat-port-range-linux.nftables b/tests/networkxml2firewalldata/nat-port-range-linux.nftables index 1dc37a26ec..127536e4db 100644 --- a/tests/networkxml2firewalldata/nat-port-range-linux.nftables +++ b/tests/networkxml2firewalldata/nat-port-range-linux.nftables @@ -312,3 +312,19 @@ oif \ virbr0 \ counter \ accept +nft \ +-ae insert \ +rule \ +ip \ +libvirt_network \ +postroute_mangle \ +oif \ +virbr0 \ +udp \ +dport \ +68 \ +counter \ +udp \ +checksum \ +set \ +0 diff --git a/tests/networkxml2firewalldata/nat-tftp-linux.nftables b/tests/networkxml2firewalldata/nat-tftp-linux.nftables index 28508292f9..951a5a6d60 100644 --- a/tests/networkxml2firewalldata/nat-tftp-linux.nftables +++ b/tests/networkxml2firewalldata/nat-tftp-linux.nftables @@ -142,3 +142,19 @@ daddr \ 224.0.0.0/24 \ counter \ return +nft \ +-ae insert \ +rule \ +ip \ +libvirt_network \ +postroute_mangle \ +oif \ +virbr0 \ +udp \ +dport \ +68 \ +counter \ +udp \ +checksum \ +set \ +0 diff --git a/tests/networkxml2firewalldata/route-default-linux.nftables b/tests/networkxml2firewalldata/route-default-linux.nftables index 282c9542a5..be9c4f5439 100644 --- a/tests/networkxml2firewalldata/route-default-linux.nftables +++ b/tests/networkxml2firewalldata/route-default-linux.nftables @@ -56,3 +56,19 @@ oif \ virbr0 \ counter \ accept +nft \ +-ae insert \ +rule \ +ip \ +libvirt_network \ +postroute_mangle \ +oif \ +virbr0 \ +udp \ +dport \ +68 \ +counter \ +udp \ +checksum \ +set \ +0 -- 2.47.0

3 7

[PATCH v2 0/4] Report CPU model blockers in domain capabilities
by Jiri Denemark 24 Oct '24

24 Oct '24

Version 2: - avoid adding the same blocker more than once for each model Jiri Denemark (4): util: Introduce virStringListRemoveDuplicates domain_capabilities: Sort CPU models domain_capabilities: Report CPU blockers NEWS: Report CPU model blockers in domain capabilities NEWS.rst | 6 + docs/formatdomaincaps.rst | 56 +- src/conf/domain_capabilities.c | 49 ++ src/conf/domain_capabilities.h | 2 + src/libvirt_private.syms | 2 + src/qemu/qemu_capabilities.c | 1 + src/util/virstring.c | 29 + src/util/virstring.h | 2 + .../domaincapsdata/qemu_5.2.0-q35.x86_64.xml | 394 ++++++++-- .../domaincapsdata/qemu_5.2.0-tcg.x86_64.xml | 649 +++++++++++++++-- .../qemu_5.2.0-virt.aarch64.xml | 56 +- tests/domaincapsdata/qemu_5.2.0.aarch64.xml | 56 +- tests/domaincapsdata/qemu_5.2.0.ppc64.xml | 4 +- tests/domaincapsdata/qemu_5.2.0.s390x.xml | 122 ++-- tests/domaincapsdata/qemu_5.2.0.x86_64.xml | 394 ++++++++-- .../domaincapsdata/qemu_6.0.0-q35.x86_64.xml | 404 +++++++++-- .../domaincapsdata/qemu_6.0.0-tcg.x86_64.xml | 681 +++++++++++++++-- .../qemu_6.0.0-virt.aarch64.xml | 56 +- tests/domaincapsdata/qemu_6.0.0.aarch64.xml | 56 +- tests/domaincapsdata/qemu_6.0.0.s390x.xml | 122 ++-- tests/domaincapsdata/qemu_6.0.0.x86_64.xml | 404 +++++++++-- .../domaincapsdata/qemu_6.1.0-q35.x86_64.xml | 405 +++++++++-- .../domaincapsdata/qemu_6.1.0-tcg.x86_64.xml | 682 ++++++++++++++++-- tests/domaincapsdata/qemu_6.1.0.x86_64.xml | 405 +++++++++-- .../domaincapsdata/qemu_6.2.0-q35.x86_64.xml | 404 +++++++++-- .../domaincapsdata/qemu_6.2.0-tcg.x86_64.xml | 680 +++++++++++++++-- .../qemu_6.2.0-virt.aarch64.xml | 58 +- tests/domaincapsdata/qemu_6.2.0.aarch64.xml | 58 +- tests/domaincapsdata/qemu_6.2.0.ppc64.xml | 4 +- tests/domaincapsdata/qemu_6.2.0.x86_64.xml | 404 +++++++++-- .../domaincapsdata/qemu_7.0.0-q35.x86_64.xml | 404 +++++++++-- .../domaincapsdata/qemu_7.0.0-tcg.x86_64.xml | 673 +++++++++++++++-- .../qemu_7.0.0-virt.aarch64.xml | 58 +- tests/domaincapsdata/qemu_7.0.0.aarch64.xml | 58 +- tests/domaincapsdata/qemu_7.0.0.ppc64.xml | 4 +- tests/domaincapsdata/qemu_7.0.0.x86_64.xml | 404 +++++++++-- .../domaincapsdata/qemu_7.1.0-q35.x86_64.xml | 368 ++++++++-- .../domaincapsdata/qemu_7.1.0-tcg.x86_64.xml | 619 ++++++++++++++-- tests/domaincapsdata/qemu_7.1.0.ppc64.xml | 4 +- tests/domaincapsdata/qemu_7.1.0.x86_64.xml | 368 ++++++++-- .../domaincapsdata/qemu_7.2.0-q35.x86_64.xml | 368 ++++++++-- .../qemu_7.2.0-tcg.x86_64+hvf.xml | 511 +++++++++++-- .../domaincapsdata/qemu_7.2.0-tcg.x86_64.xml | 511 +++++++++++-- tests/domaincapsdata/qemu_7.2.0.x86_64.xml | 368 ++++++++-- .../domaincapsdata/qemu_8.0.0-q35.x86_64.xml | 410 +++++++++-- .../domaincapsdata/qemu_8.0.0-tcg.x86_64.xml | 559 ++++++++++++-- tests/domaincapsdata/qemu_8.0.0.x86_64.xml | 410 +++++++++-- .../domaincapsdata/qemu_8.1.0-q35.x86_64.xml | 496 +++++++++++-- .../domaincapsdata/qemu_8.1.0-tcg.x86_64.xml | 570 +++++++++++++-- tests/domaincapsdata/qemu_8.1.0.s390x.xml | 282 ++++++-- tests/domaincapsdata/qemu_8.1.0.x86_64.xml | 496 +++++++++++-- .../domaincapsdata/qemu_8.2.0-q35.x86_64.xml | 496 +++++++++++-- .../domaincapsdata/qemu_8.2.0-tcg.x86_64.xml | 562 +++++++++++++-- .../qemu_8.2.0-virt.aarch64.xml | 72 +- tests/domaincapsdata/qemu_8.2.0.aarch64.xml | 72 +- tests/domaincapsdata/qemu_8.2.0.s390x.xml | 280 +++++-- tests/domaincapsdata/qemu_8.2.0.x86_64.xml | 496 +++++++++++-- .../domaincapsdata/qemu_9.0.0-q35.x86_64.xml | 496 +++++++++++-- .../domaincapsdata/qemu_9.0.0-tcg.x86_64.xml | 536 ++++++++++++-- tests/domaincapsdata/qemu_9.0.0.x86_64.xml | 496 +++++++++++-- .../domaincapsdata/qemu_9.1.0-q35.x86_64.xml | 524 ++++++++++++-- .../qemu_9.1.0-tcg-virt.riscv64.xml | 16 +- .../domaincapsdata/qemu_9.1.0-tcg.x86_64.xml | 557 ++++++++++++-- tests/domaincapsdata/qemu_9.1.0.x86_64.xml | 524 ++++++++++++-- .../domaincapsdata/qemu_9.2.0-q35.x86_64.xml | 524 ++++++++++++-- .../domaincapsdata/qemu_9.2.0-tcg.x86_64.xml | 557 ++++++++++++-- tests/domaincapsdata/qemu_9.2.0.x86_64.xml | 524 ++++++++++++-- 67 files changed, 18573 insertions(+), 2745 deletions(-) -- 2.47.0

3 6

[PATCH 0/2] network: avoid logging unnecessary and misleading errors when failing to unset a zone
by Laine Stump 24 Oct '24

24 Oct '24

While testing a recent patch that unsets the zone of bridge interfaces when a virtual network is stopped, a side effect was noticed: when firewalld reloaded its rules, this would result in an error log from libvirt complaining about attempting to unset the zone of an interface that wasn't in any zone. The two patches here fix that from different angles: * The first modifies the call to unsetZone so that it puts any error message returned from firewalld to libvirt into a virError object rather than logging it; this virError object is then silently discarded. * The second avoids even calling firewalld to unset the zone if it's just going to immediately be set again. This avoids an error message that would be logged directly by firewalld even if libvirt didn't log the message it received from firewalld. The combination of these two patches eliminate all misleading log messages about failed attempts to unset a zone. Laine Stump (2): network: ignore/don't log errors when unsetting firewalld zone network: don't unset the firewalld zone if it's going to be immediately re-set src/network/bridge_driver.c | 8 +++---- src/network/bridge_driver_linux.c | 10 +++++---- src/network/bridge_driver_nop.c | 4 +++- src/network/bridge_driver_platform.h | 3 ++- src/util/virfirewalld.c | 33 ++++++++++++++++++---------- src/util/virfirewalld.h | 2 +- 6 files changed, 38 insertions(+), 22 deletions(-) -- 2.47.0

2 3

[PATCH v2 00/20] maintainer updates (testing, gdbstub, plugins)
by Alex Bennée 23 Oct '24

23 Oct '24

This is an aggregation of three of my maintainer trees which you can also get from their respective branches (testing/next, gdbstub/next and plugins/next). I didn't include the plugins on the last post as I hadn't had a chance to do my sweep through patches before travelling. I've also updated MAINTAINERS to point at my next trees. For testing we have mostly tweaks and cleanups. I've included some tracepoints tweaks for cpu_loop_exit_atomic purely as there was no where else to but it. There are also some cleanups to the tsan support from Pierrick. The mipsel tweaks have already been applied directly to the tree. For gdbstub more cleanups as well as fixing some gdbstub breakage of the untested aarch64-be linux-user target. I've added a very basic some test to prevent silly regressions in the future. For plugins again more cleanups. The GDB trigger patch will probably not get merged and should be considered an experimental hack for now. The following still need review: plugins: add ability to register a GDB triggered callback tests/tcg: enable basic testing for aarch64_be-linux-user config/targets: update aarch64_be-linux-user gdb XML list MAINTAINERS: mention my gdbstub/next tree gitlab: make check-[dco|patch] a little more verbose scripts/ci: remove architecture checks for build-environment updates MAINTAINERS: mention my testing/next tree tests/docker: add NOFETCH env variable for testing MAINTAINERS: mention my plugins/next tree Alex Bennée (10): tests/docker: add NOFETCH env variable for testing MAINTAINERS: mention my testing/next tree scripts/ci: remove architecture checks for build-environment updates accel/tcg: add tracepoints for cpu_loop_exit_atomic gitlab: make check-[dco|patch] a little more verbose MAINTAINERS: mention my gdbstub/next tree config/targets: update aarch64_be-linux-user gdb XML list tests/tcg: enable basic testing for aarch64_be-linux-user MAINTAINERS: mention my plugins/next tree plugins: add ability to register a GDB triggered callback Gustavo Romero (2): tests/tcg/aarch64: Use raw strings for regexes in test-mte.py testing: Enhance gdb probe script Ilya Leoshkevich (2): tests/docker: Fix microblaze atomics tests/tcg/x86_64: Add cross-modifying code test Pierrick Bouvier (6): meson: hide tsan related warnings docs/devel: update tsan build documentation dockerfiles: fix default targets for debian-loongarch-cross meson: build contrib/plugins with meson contrib/plugins: remove Makefile for contrib/plugins plugins: fix qemu_plugin_reset MAINTAINERS | 3 + docs/devel/testing/main.rst | 26 +++++- configure | 23 ++--- Makefile | 10 --- configs/targets/aarch64_be-linux-user.mak | 2 +- meson.build | 14 ++- include/qemu/plugin-event.h | 1 + include/qemu/qemu-plugin.h | 16 ++++ plugins/plugin.h | 9 ++ accel/tcg/plugin-gen.c | 4 + accel/tcg/user-exec.c | 2 +- plugins/api.c | 18 ++++ plugins/core.c | 37 ++++++++ tests/tcg/aarch64_be/hello.c | 35 ++++++++ tests/tcg/plugins/mem.c | 11 ++- tests/tcg/x86_64/cross-modifying-code.c | 80 +++++++++++++++++ accel/tcg/ldst_atomicity.c.inc | 9 ++ .gitlab-ci.d/check-dco.py | 9 +- .gitlab-ci.d/check-patch.py | 9 +- accel/tcg/trace-events | 12 +++ contrib/plugins/Makefile | 87 ------------------- contrib/plugins/meson.build | 23 +++++ plugins/qemu-plugins.symbols | 1 + scripts/ci/setup/ubuntu/build-environment.yml | 2 - scripts/probe-gdb-support.py | 75 ++++++++-------- tests/docker/Makefile.include | 5 +- .../dockerfiles/debian-loongarch-cross.docker | 4 +- .../build-toolchain.sh | 8 ++ .../dockerfiles/debian-toolchain.docker | 7 ++ tests/tcg/Makefile.target | 7 +- tests/tcg/aarch64/gdbstub/test-mte.py | 4 +- tests/tcg/aarch64_be/Makefile.target | 17 ++++ tests/tcg/x86_64/Makefile.target | 4 + 33 files changed, 397 insertions(+), 177 deletions(-) create mode 100644 tests/tcg/aarch64_be/hello.c create mode 100644 tests/tcg/x86_64/cross-modifying-code.c delete mode 100644 contrib/plugins/Makefile create mode 100644 contrib/plugins/meson.build create mode 100644 tests/tcg/aarch64_be/Makefile.target -- 2.39.5

6 41

[PATCH v2 0/4] multiple memory backend support for CPR Live Updates
by mgalaxy＠akamai.com 23 Oct '24

23 Oct '24

From: Michael Galaxy <mgalaxy(a)akamai.com> CPR-based support for whole-hypervisor kexec-based live updates is now finally merged into QEMU. In support of this, we need NUMA to be supported in these kinds of environments. To do this we use a technology called PMEM (persistent memory) in Linux, which underpins the ability for CPR Live Updates to work so that QEMU memory can remain in RAM and be recovered after a kexec operationg has completed. Our systems are highly NUMA-aware, and so this patch series enables NUMA awareness for live updates. Further, we make a small change that allows live migrations to work between *non* PMEM-based systems and PMEM-based systems (and vice-versa). This allows for seemless upgrades from non-live-compatible systems to live-update-compatible sytems without any downtime. Michael Galaxy (4): qemu.conf changes to support multiple memory backend Support live migration between file-backed memory and anonymous memory. Update unit test to support multiple memory backends Update documentation to reflect memory_backing_dir change in qemu.conf NEWS.rst | 7 ++ docs/kbase/virtiofs.rst | 2 + src/qemu/qemu.conf.in | 2 + src/qemu/qemu_command.c | 8 ++- src/qemu/qemu_conf.c | 141 +++++++++++++++++++++++++++++++++++----- src/qemu/qemu_conf.h | 14 ++-- src/qemu/qemu_domain.c | 24 +++++-- src/qemu/qemu_driver.c | 29 +++++---- src/qemu/qemu_hotplug.c | 6 +- src/qemu/qemu_process.c | 44 +++++++------ src/qemu/qemu_process.h | 7 +- tests/testutilsqemu.c | 5 +- 12 files changed, 221 insertions(+), 68 deletions(-) -- 2.34.1

3 26

[PATCH v3 00/18] maintainer updates (testing, gdbstub, plugins) pre-PR
by Alex Bennée 23 Oct '24

23 Oct '24

This is an aggregation of three of my maintainer trees which you can also get from their respective branches (testing/next, gdbstub/next and plugins/next). It is now ready for a pull request. Since last post: - added r-b tags - some minor typo fixes - dropped meson plugin contrib build and gdb hook changes The last thing that still needs review: gitlab: make check-[dco|patch] a little more verbose Alex Bennée (9): tests/docker: add NOFETCH env variable for testing MAINTAINERS: mention my testing/next tree scripts/ci: remove architecture checks for build-environment updates accel/tcg: add tracepoints for cpu_loop_exit_atomic gitlab: make check-[dco|patch] a little more verbose MAINTAINERS: mention my gdbstub/next tree config/targets: update aarch64_be-linux-user gdb XML list tests/tcg: enable basic testing for aarch64_be-linux-user MAINTAINERS: mention my plugins/next tree Gustavo Romero (2): tests/tcg/aarch64: Use raw strings for regexes in test-mte.py testing: Enhance gdb probe script Ilya Leoshkevich (2): tests/docker: Fix microblaze atomics tests/tcg/x86_64: Add cross-modifying code test Pierrick Bouvier (5): meson: hide tsan related warnings docs/devel: update tsan build documentation dockerfiles: fix default targets for debian-loongarch-cross contrib/plugins: remove Makefile for contrib/plugins plugins: fix qemu_plugin_reset MAINTAINERS | 3 + docs/devel/testing/main.rst | 26 +++++- configure | 23 ++--- Makefile | 10 --- configs/targets/aarch64_be-linux-user.mak | 2 +- meson.build | 10 ++- accel/tcg/plugin-gen.c | 4 + accel/tcg/user-exec.c | 2 +- tests/tcg/aarch64_be/hello.c | 35 ++++++++ tests/tcg/x86_64/cross-modifying-code.c | 80 +++++++++++++++++ accel/tcg/ldst_atomicity.c.inc | 9 ++ .gitlab-ci.d/check-dco.py | 5 +- .gitlab-ci.d/check-patch.py | 5 +- accel/tcg/trace-events | 12 +++ contrib/plugins/Makefile | 87 ------------------- scripts/ci/setup/ubuntu/build-environment.yml | 2 - scripts/probe-gdb-support.py | 75 ++++++++-------- tests/docker/Makefile.include | 5 +- .../dockerfiles/debian-loongarch-cross.docker | 4 +- .../build-toolchain.sh | 8 ++ .../dockerfiles/debian-toolchain.docker | 7 ++ tests/tcg/Makefile.target | 7 +- tests/tcg/aarch64/gdbstub/test-mte.py | 4 +- tests/tcg/aarch64_be/Makefile.target | 17 ++++ tests/tcg/x86_64/Makefile.target | 4 + 25 files changed, 273 insertions(+), 173 deletions(-) create mode 100644 tests/tcg/aarch64_be/hello.c create mode 100644 tests/tcg/x86_64/cross-modifying-code.c delete mode 100644 contrib/plugins/Makefile create mode 100644 tests/tcg/aarch64_be/Makefile.target -- 2.39.5

2 19

[PATCH 0/9] qemu: Change CPU comparison algorithm for future models
by Jiri Denemark 23 Oct '24

23 Oct '24

When starting a domain we check whether the guest CPU definition is compatible with the host (i.e., when the host supports all features required both explicitly and by the specified CPU model) as long as check == 'partial', which is the default. We are doing so by checking our definition of the CPU model in the CPU map amending it with explicitly mentioned features and comparing it to features QEMU would enabled when started with -cpu host. But since our CPU model definitions often slightly differ from QEMU we may be checking features which are not actually needed and on the other hand not checking something that is part of the CPU model in QEMU. This patch changes the algorithm for CPU models added in the future (changing it for existing models could cause them to suddenly become incompatible with the host and domains using them would fail to start). The new algorithm uses information we probe from QEMU about features that block each model from being directly usable. If all those features are explicitly disabled in the CPU definition we consider the base model compatible with the host. Then we only need to check that all explicitly required features are supported by QEMU on the host to get the result for the whole CPU definition. After this we only use the model definitions (for newly added models) from CPU map for creating a CPU definition for host-model. Jiri Denemark (9): cpu_x86: Introduce <check> element for CPU models cpu_map: Use compat partial check for all x86 CPU models cpu: Introduce virCPUGetCheckMode qemu: Use g_autoptr in qemuConnectCompareHypervisorCPU qemu: Use virCPUCompare in qemuConnectCompareHypervisorCPU directly qemu: Separate partial CPU check into a function cpu: Introduce virCPUCompareUnusable qemu_capabilities: Introduce virQEMUCapsGetCPUBlockers qemu: Change CPU comparison algorithm for future models src/cpu/cpu.c | 104 ++++++++++++++++++ src/cpu/cpu.h | 17 +++ src/cpu/cpu_x86.c | 67 +++++++++++ src/cpu_map/x86_486.xml | 1 + src/cpu_map/x86_Broadwell-IBRS.xml | 1 + src/cpu_map/x86_Broadwell-noTSX-IBRS.xml | 1 + src/cpu_map/x86_Broadwell-noTSX.xml | 1 + src/cpu_map/x86_Broadwell.xml | 1 + src/cpu_map/x86_Cascadelake-Server-noTSX.xml | 1 + src/cpu_map/x86_Cascadelake-Server.xml | 1 + src/cpu_map/x86_Conroe.xml | 1 + src/cpu_map/x86_Cooperlake.xml | 1 + src/cpu_map/x86_Dhyana.xml | 1 + src/cpu_map/x86_EPYC-Genoa.xml | 1 + src/cpu_map/x86_EPYC-IBPB.xml | 1 + src/cpu_map/x86_EPYC-Milan.xml | 1 + src/cpu_map/x86_EPYC-Rome.xml | 1 + src/cpu_map/x86_EPYC.xml | 1 + src/cpu_map/x86_GraniteRapids.xml | 1 + src/cpu_map/x86_Haswell-IBRS.xml | 1 + src/cpu_map/x86_Haswell-noTSX-IBRS.xml | 1 + src/cpu_map/x86_Haswell-noTSX.xml | 1 + src/cpu_map/x86_Haswell.xml | 1 + src/cpu_map/x86_Icelake-Client-noTSX.xml | 1 + src/cpu_map/x86_Icelake-Client.xml | 1 + src/cpu_map/x86_Icelake-Server-noTSX.xml | 1 + src/cpu_map/x86_Icelake-Server.xml | 1 + src/cpu_map/x86_IvyBridge-IBRS.xml | 1 + src/cpu_map/x86_IvyBridge.xml | 1 + src/cpu_map/x86_Nehalem-IBRS.xml | 1 + src/cpu_map/x86_Nehalem.xml | 1 + src/cpu_map/x86_Opteron_G1.xml | 1 + src/cpu_map/x86_Opteron_G2.xml | 1 + src/cpu_map/x86_Opteron_G3.xml | 1 + src/cpu_map/x86_Opteron_G4.xml | 1 + src/cpu_map/x86_Opteron_G5.xml | 1 + src/cpu_map/x86_Penryn.xml | 1 + src/cpu_map/x86_SandyBridge-IBRS.xml | 1 + src/cpu_map/x86_SandyBridge.xml | 1 + src/cpu_map/x86_SapphireRapids.xml | 1 + src/cpu_map/x86_SierraForest.xml | 1 + src/cpu_map/x86_Skylake-Client-IBRS.xml | 1 + src/cpu_map/x86_Skylake-Client-noTSX-IBRS.xml | 1 + src/cpu_map/x86_Skylake-Client.xml | 1 + src/cpu_map/x86_Skylake-Server-IBRS.xml | 1 + src/cpu_map/x86_Skylake-Server-noTSX-IBRS.xml | 1 + src/cpu_map/x86_Skylake-Server.xml | 1 + src/cpu_map/x86_Snowridge.xml | 1 + src/cpu_map/x86_Westmere-IBRS.xml | 1 + src/cpu_map/x86_Westmere.xml | 1 + src/cpu_map/x86_athlon.xml | 1 + src/cpu_map/x86_core2duo.xml | 1 + src/cpu_map/x86_coreduo.xml | 1 + src/cpu_map/x86_cpu64-rhel5.xml | 1 + src/cpu_map/x86_cpu64-rhel6.xml | 1 + src/cpu_map/x86_kvm32.xml | 1 + src/cpu_map/x86_kvm64.xml | 1 + src/cpu_map/x86_n270.xml | 1 + src/cpu_map/x86_pentium.xml | 1 + src/cpu_map/x86_pentium2.xml | 1 + src/cpu_map/x86_pentium3.xml | 1 + src/cpu_map/x86_pentiumpro.xml | 1 + src/cpu_map/x86_phenom.xml | 1 + src/cpu_map/x86_qemu32.xml | 1 + src/cpu_map/x86_qemu64.xml | 1 + src/libvirt_private.syms | 2 + src/qemu/qemu_capabilities.c | 38 +++++++ src/qemu/qemu_capabilities.h | 4 + src/qemu/qemu_domain.c | 65 +++++++++++ src/qemu/qemu_domain.h | 7 ++ src/qemu/qemu_driver.c | 45 ++++---- src/qemu/qemu_process.c | 7 +- 72 files changed, 390 insertions(+), 28 deletions(-) -- 2.47.0

2 10

[PATCH] ci: Move definition of exit codes allowed to fail for cirrus jobs
by Peter Krempa 23 Oct '24

23 Oct '24

Update with latest lcitool. Update the build templates to move the definition of exit codes which are allowed to fail for cirrus jobs for cases when we run out of CI minutes. The previous location was overridden with the per-job 'allow_failure' value and thus didn't apply. Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- Based on: https://gitlab.com/libvirt/libvirt-ci/-/merge_requests/510 example run: https://gitlab.com/libvirt/libvirt/-/pipelines/1508364775 ci/gitlab/build-templates.yml | 2 -- ci/gitlab/builds.yml | 9 ++++++--- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/ci/gitlab/build-templates.yml b/ci/gitlab/build-templates.yml index 31a901e47c..b1e41b0783 100644 --- a/ci/gitlab/build-templates.yml +++ b/ci/gitlab/build-templates.yml @@ -282,8 +282,6 @@ image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:latest interruptible: true needs: [] - allow_failure: - exit_codes: 3 script: - set -o allexport - source ci/cirrus/$NAME.vars diff --git a/ci/gitlab/builds.yml b/ci/gitlab/builds.yml index 771fa67501..c24421378c 100644 --- a/ci/gitlab/builds.yml +++ b/ci/gitlab/builds.yml @@ -599,7 +599,8 @@ mingw64-fedora-rawhide: x86_64-freebsd-13: extends: .cirrus_build_job needs: [] - allow_failure: false + allow_failure: + exit_codes: 3 variables: CIRRUS_VM_IMAGE_NAME: freebsd-13-3 CIRRUS_VM_IMAGE_SELECTOR: image_family @@ -613,7 +614,8 @@ x86_64-freebsd-13: x86_64-freebsd-14: extends: .cirrus_build_job needs: [] - allow_failure: false + allow_failure: + exit_codes: 3 variables: CIRRUS_VM_IMAGE_NAME: freebsd-14-0 CIRRUS_VM_IMAGE_SELECTOR: image_family @@ -627,7 +629,8 @@ x86_64-freebsd-14: aarch64-macos-14: extends: .cirrus_build_job needs: [] - allow_failure: false + allow_failure: + exit_codes: 3 variables: CIRRUS_VM_IMAGE_NAME: ghcr.io/cirruslabs/macos-runner:sonoma CIRRUS_VM_IMAGE_SELECTOR: image -- 2.47.0

2 1

[PATCH v2 0/6] ch: handle events from cloud-hypervisor
by Purna Pavan Chandra Aekkaladevi 23 Oct '24

23 Oct '24

changes from v1->v2: * Rebase on latest master * Use /* */ for comments * Remove fifo file if already exists * Address other comments from Praveen Paladugu cloud-hypervisor raises various events, including VM lifecylce operations such as boot, shutdown, pause, resume, etc. Libvirt will now read these events and take the necessary actions, such as correctly updating the domain state. A FIFO file is passed to `--event-monitor` option of cloud-hypervisor. Libvirt creates a new thread that acts as the reader of the fifo file and continuously monitors for new events. Currently, shutdown events are handled by updating the domain state appropriately. Purna Pavan Chandra Aekkaladevi (6): utils: Implement virFileIsNamedPipe ch: pass --event-monitor option to cloud-hypervisor ch: start a new thread for handling ch events ch: events: Read and parse cloud-hypervisor events ch: events: facilitate lifecycle events handling NEWS: Mention event handling support in ch driver NEWS.rst | 7 + po/POTFILES | 1 + src/ch/ch_events.c | 349 +++++++++++++++++++++++++++++++++++++++ src/ch/ch_events.h | 54 ++++++ src/ch/ch_monitor.c | 52 +++++- src/ch/ch_monitor.h | 11 ++ src/ch/meson.build | 2 + src/libvirt_private.syms | 1 + src/util/virfile.c | 8 + src/util/virfile.h | 1 + 10 files changed, 479 insertions(+), 7 deletions(-) create mode 100644 src/ch/ch_events.c create mode 100644 src/ch/ch_events.h -- 2.34.1

2 10

[PATCH (pushed)] ci: Regenerate with latest lcitool to pick up 'cirrus-ci' out of credit detection
by Peter Krempa 22 Oct '24

22 Oct '24

Allow the cirrus jobs to fail on return code '3' which is emitted if we've run out of credits. Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- Pushed as CI fix. ci/gitlab/build-templates.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ci/gitlab/build-templates.yml b/ci/gitlab/build-templates.yml index b1e41b0783..31a901e47c 100644 --- a/ci/gitlab/build-templates.yml +++ b/ci/gitlab/build-templates.yml @@ -282,6 +282,8 @@ image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:latest interruptible: true needs: [] + allow_failure: + exit_codes: 3 script: - set -o allexport - source ci/cirrus/$NAME.vars -- 2.47.0

1 0