- Devel - libvirt List Archives

[PATCH 0/3] Implement support for QCOW2 data files
by Nikolai Barybin 19 Aug '24

19 Aug '24

There are use cases when the existing disks (i.e. LVM) are wanted to be used with advanced features. For this purpose QEMU allows data-file feature for qcow2 files: metadata is kept in the qcow2 file like usual, but guest data is written to an external file. These patches enable support for this feature in libvirt. Nikolai Barybin (3): conf: add data-file feature and related fields to virStorageSource storage: add qcow2 filename parsing from header qemu: enable qcow2 data-file attach to VM on start src/conf/storage_source_conf.c | 8 ++++ src/conf/storage_source_conf.h | 5 ++ src/qemu/qemu_block.c | 45 ++++++++++++++++++ src/qemu/qemu_cgroup.c | 3 ++ src/qemu/qemu_namespace.c | 6 +++ src/storage_file/storage_file_probe.c | 66 +++++++++++++++++++++++---- src/storage_file/storage_source.c | 38 ++++++++++++++- 7 files changed, 160 insertions(+), 11 deletions(-) -- 2.43.5

3 12

Shared LVM and starting/stopping volumes
by dajester2005＠gmail.com 19 Aug '24

19 Aug '24

Ran into a situation using a shared LVM2 volume group, where if there were volumes locked by a server, the pool could not be started on another server due to "vgchange -aly" not being able to acquire locks on all volumes in the pool. This was referenced in issue #38 on gitlab. Long story short, I've experimented with adding APIs for starting/stopping individual volumes in a storage pool, and implementing that API in the logical backend driver. I'm not a C developer by trade, but all the existing tests pass. I'd like to submit this for some feedback, and see what would be needed to incorporate this, or similar API: https://gitlab.com/dajester2013/libvirt/-/tree/shared-lvm-storage-mods

1 0

Plans for 10.7.0 release (freeze on Tuesday 27 Aug)
by Jiri Denemark 19 Aug '24

19 Aug '24

We are getting close to 10.7.0 release of libvirt. To aim for the release on Monday 02 Sep I suggest entering the freeze on Tuesday 27 Aug and tagging RC2 on Thursday 29 Aug. I hope this works for everyone. Jirka

1 0

[PATCH] lib: Use NULLSTR family of macros more
by Michal Privoznik 19 Aug '24

19 Aug '24

There is a family of convenient macros: NULLSTR, NULLSTR_EMPTY, NULLSTR_STAR, NULLSTR_MINUS which hides ternary operator. Generated using the following spatch (and its obvious variants): @@ expression s; @@ <+... - s ? s : "<null>" + NULLSTR(s) ...+> Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/libvirt.c | 2 +- src/network/network_iptables.c | 4 ++-- src/network/network_nftables.c | 4 ++-- src/qemu/qemu_firmware.c | 2 +- src/util/virnetdevip.c | 8 ++++---- tools/virsh-checkpoint.c | 2 +- tools/virsh-domain-monitor.c | 8 ++++---- tools/virsh-nodedev.c | 2 +- 8 files changed, 16 insertions(+), 16 deletions(-) diff --git a/src/libvirt.c b/src/libvirt.c index 26c3fe454f..1d37696d6f 100644 --- a/src/libvirt.c +++ b/src/libvirt.c @@ -168,7 +168,7 @@ virConnectAuthCallbackDefault(virConnectCredentialPtr cred, } if (cred[i].type != VIR_CRED_EXTERNAL) { - cred[i].result = bufptr ? bufptr : g_strdup(cred[i].defresult ? cred[i].defresult : ""); + cred[i].result = bufptr ? bufptr : g_strdup(NULLSTR_EMPTY(cred[i].defresult)); cred[i].resultlen = strlen(cred[i].result); } } diff --git a/src/network/network_iptables.c b/src/network/network_iptables.c index f774176b3d..e8da15426e 100644 --- a/src/network/network_iptables.c +++ b/src/network/network_iptables.c @@ -867,10 +867,10 @@ iptablesForwardMasquerade(virFirewall *fw, if (addrStartStr && addrStartStr[0]) { if (addrEndStr && addrEndStr[0]) { natRangeStr = g_strdup_printf("%s-%s%s", addrStartStr, addrEndStr, - portRangeStr ? portRangeStr : ""); + NULLSTR_EMPTY(portRangeStr)); } else { natRangeStr = g_strdup_printf("%s%s", addrStartStr, - portRangeStr ? portRangeStr : ""); + NULLSTR_EMPTY(portRangeStr)); } virFirewallCmdAddArgList(fw, fwCmd, diff --git a/src/network/network_nftables.c b/src/network/network_nftables.c index 268d1f12ca..f8b5ab665d 100644 --- a/src/network/network_nftables.c +++ b/src/network/network_nftables.c @@ -588,10 +588,10 @@ nftablesAddForwardMasquerade(virFirewall *fw, if (addrStartStr && addrStartStr[0]) { if (addrEndStr && addrEndStr[0]) { natRangeStr = g_strdup_printf("%s-%s%s", addrStartStr, addrEndStr, - portRangeStr ? portRangeStr : ""); + NULLSTR_EMPTY(portRangeStr)); } else { natRangeStr = g_strdup_printf("%s%s", addrStartStr, - portRangeStr ? portRangeStr : ""); + NULLSTR_EMPTY(portRangeStr)); } virFirewallCmdAddArgList(fw, fwCmd, "counter", "snat", "to", natRangeStr, NULL); diff --git a/src/qemu/qemu_firmware.c b/src/qemu/qemu_firmware.c index 08ca99e1ac..914f8596bd 100644 --- a/src/qemu/qemu_firmware.c +++ b/src/qemu/qemu_firmware.c @@ -1090,7 +1090,7 @@ qemuFirmwareEnsureNVRAM(virDomainDef *def, loader->nvram->path = g_strdup_printf("%s/%s_VARS%s", cfg->nvramDir, def->name, - ext ? ext : ""); + NULLSTR_EMPTY(ext)); } diff --git a/src/util/virnetdevip.c b/src/util/virnetdevip.c index 3b02fe0cec..f365c1999c 100644 --- a/src/util/virnetdevip.c +++ b/src/util/virnetdevip.c @@ -188,8 +188,8 @@ virNetDevIPAddrAdd(const char *ifname, VIR_DEBUG("Adding IP address %s/%d%s%s%s%s to %s", NULLSTR(ipStr), prefix, - peerStr ? " peer " : "", peerStr ? peerStr : "", - bcastStr ? " bcast " : "", bcastStr ? bcastStr : "", + peerStr ? " peer " : "", NULLSTR_EMPTY(peerStr), + bcastStr ? " bcast " : "", NULLSTR_EMPTY(bcastStr), ifname); if (!(nlmsg = virNetDevCreateNetlinkAddressMessage(RTM_NEWADDR, ifname, @@ -206,8 +206,8 @@ virNetDevIPAddrAdd(const char *ifname, virReportError(VIR_ERR_SYSTEM_ERROR, _("Failed to add IP address %1$s/%2$d%3$s%4$s%5$s%6$s to %7$s"), ipStr, prefix, - peerStr ? " peer " : "", peerStr ? peerStr : "", - bcastStr ? " bcast " : "", bcastStr ? bcastStr : "", + peerStr ? " peer " : "", NULLSTR_EMPTY(peerStr), + bcastStr ? " bcast " : "", NULLSTR_EMPTY(bcastStr), ifname); return -1; } diff --git a/tools/virsh-checkpoint.c b/tools/virsh-checkpoint.c index 972b2f979c..a0ac0d7f71 100644 --- a/tools/virsh-checkpoint.c +++ b/tools/virsh-checkpoint.c @@ -455,7 +455,7 @@ cmdCheckpointInfo(vshControl *ctl, _("unexpected problem querying checkpoint state")); return false; } - vshPrint(ctl, "%-15s %s\n", _("Parent:"), parent ? parent : "-"); + vshPrint(ctl, "%-15s %s\n", _("Parent:"), NULLSTR_MINUS(parent)); /* Children, Descendants. */ flags = 0; diff --git a/tools/virsh-domain-monitor.c b/tools/virsh-domain-monitor.c index 74deca2f94..37184baa69 100644 --- a/tools/virsh-domain-monitor.c +++ b/tools/virsh-domain-monitor.c @@ -712,11 +712,11 @@ cmdDomiflist(vshControl *ctl, const vshCmd *cmd) mac = virXPathString("string(./mac/@address)", ctxt); if (vshTableRowAppend(table, - target ? target : "-", + NULLSTR_MINUS(target), type, - source ? source : "-", - model ? model : "-", - mac ? mac : "-", + NULLSTR_MINUS(source), + NULLSTR_MINUS(model), + NULLSTR_MINUS(mac), NULL) < 0) return false; } diff --git a/tools/virsh-nodedev.c b/tools/virsh-nodedev.c index 910eaefc9e..145faff3e7 100644 --- a/tools/virsh-nodedev.c +++ b/tools/virsh-nodedev.c @@ -1233,7 +1233,7 @@ cmdNodeDeviceInfo(vshControl *ctl, const vshCmd *cmd) parent = virNodeDeviceGetParent(device); vshPrint(ctl, "%-15s %s\n", _("Name:"), virNodeDeviceGetName(device)); - vshPrint(ctl, "%-15s %s\n", _("Parent:"), parent ? parent : ""); + vshPrint(ctl, "%-15s %s\n", _("Parent:"), NULLSTR_EMPTY(parent)); vshPrint(ctl, "%-15s %s\n", _("Active:"), virNodeDeviceIsActive(device) ? _("yes") : _("no")); vshPrint(ctl, "%-15s %s\n", _("Persistent:"), -- 2.44.2

2 1

[PATCH 0/1] virsh: allow both --table and --uuid at the same time
by Nikolai Barybin 19 Aug '24

19 Aug '24

For now virsh does not provide a set of options to print all info about domain: --all does not print uuid, combination of --id --name --uuid does not have '--state' option. I think we should address this issue and allow user to get all available info with one command. The easiest way to do that is to allow combining --table and --uuid (the same way it now works with --title). Thus, the most diverse info would look like that: # virsh list --table --title --uuid Id Name State Title UUID ----------------------------------- Nikolai Barybin (1): virsh: allow both --table and --uuid at the same time docs/manpages/virsh.rst | 14 +++++++------- tools/virsh-domain-monitor.c | 37 ++++++++++++++++++++++++++++++++---- 2 files changed, 40 insertions(+), 11 deletions(-) -- 2.43.5

2 2

Re: [PATCH v2 4/4] virtio-net: Add support for USO features
by Thomas Huth 19 Aug '24

19 Aug '24

On 26/07/2024 08.08, Michael S. Tsirkin wrote: > On Thu, Jul 25, 2024 at 06:18:20PM -0400, Peter Xu wrote: >> On Tue, Aug 01, 2023 at 01:31:48AM +0300, Yuri Benditovich wrote: >>> USO features of virtio-net device depend on kernel ability >>> to support them, for backward compatibility by default the >>> features are disabled on 8.0 and earlier. >>> >>> Signed-off-by: Yuri Benditovich <yuri.benditovich(a)daynix.com> >>> Signed-off-by: Andrew Melnychecnko <andrew(a)daynix.com> >> >> Looks like this patch broke migration when the VM starts on a host that has >> USO supported, to another host that doesn't.. > > This was always the case with all offloads. The answer at the moment is, > don't do this. May I ask for my understanding: "don't do this" = don't automatically enable/disable virtio features in QEMU depending on host kernel features, or "don't do this" = don't try to migrate between machines that have different host kernel features? > Long term, we need to start exposing management APIs > to discover this, and management has to disable unsupported features. Ack, this likely needs some treatments from the libvirt side, too. Thomas

8 93

[PATCH 0/7] network: implement automatic subnet selection for virtual networks
by Laine Stump 17 Aug '24

17 Aug '24

The problem this is solving has a very long history. with a simple bugzilla search I found reports all the way back to 2012. The issue is that sometimes when a libvirt virtual network is started, it could be on exactly the same subnet as another interface already active on the host at libvirt install time (which was fixed long ago), or as another interface that isn't active in the install environment, but will be active when libvirt is actually started at a later time, possibly in a completely different environment than the install (this *wasn't* fixed. until now). At one point (which I've located in this bugzilla comment in 2014 - https://bugzilla.redhat.com/1146232#c17) I thought of having a config knob in virtual networks that would look for an unused network at runtime, and start the network with that subnet. Of course that would have solved the problem where the conflicting network was already active when libvirt started its networks, but wouldn't do anything about the case where the conflicting network was started *after* libvirt had started its networks. And anyway it would have required config settings in /etc/libvirt/network.conf which didn't exist at the time. Several years later (around 2020) I learned about NetworkManager dispatcher scripts, which are called any time NM brings an interface up or down, and wrote a python script that would use this to destroy any libvirt network that had a conflict with a newly activated NM interface. After sending it as an RFC patch, I promptly forgot about it (except for being on my "list of things I should get to someday"). A few weeks ago I realized that the "there is no network.conf to store config items" problem was recently solved (I had to add a network.conf file for the knob that selects nftables vs iptables), which prompted me to go back and find the branch with the NM dispatcher script, and *finally* implement the runtime auto address selection. Combining this with enhancing the NM dispatcher script to not only destroy conflicting networks, but to also then *restart* them if they have autoaddr set, gives what I think is finally a full solution to the problem - no matter what order interfaces are started vs. libvirt networks being started, a network with "autoaddr='yes'" set will *always* end up with all networks being usable. (Well, the one issue that still remains is that if there are already guests attached to the autoaddr='yes' network when its address is changed, those guests will then be disconnected, and once reconnected they will have stale IP addresses. That's another hill for another day though; I don't think anyone has complained about that. Yet.) I think this may break the record for "oldest bug to be fixed", which was previously set when danpb fixed the firewalling issue with guests on two different NATed networks being able to communicate with each other. :-) Laine Stump (7): conf: add XML config for autoaddr networks network: add configurable network autoaddr items to driver config network: reorganize the check for route collisions network: turn on autoaddr selection in bridge driver network: NetworkManager script to monitor/resolve conflicts with new interfaces network: turn on autoaddr in default network spec: stop trying to find unused network during deamon-network-config %post docs/formatnetwork.rst | 42 +++- libvirt.spec.in | 38 +--- meson.build | 14 +- meson_options.txt | 4 + src/conf/network_conf.c | 75 +++++-- src/conf/network_conf.h | 7 + src/conf/schemas/network.rng | 5 + src/network/bridge_driver.c | 201 +++++++++++++++++- src/network/bridge_driver_conf.c | 61 ++++++ src/network/bridge_driver_conf.h | 4 + src/network/bridge_driver_linux.c | 132 +++++++----- src/network/bridge_driver_nop.c | 22 +- src/network/bridge_driver_platform.h | 5 +- src/network/default.xml.in | 2 +- src/network/libvirtd_network.aug | 8 +- src/network/meson.build | 12 ++ src/network/network.conf.in | 11 + src/network/nm-dispatcher-check-nets.py | 196 +++++++++++++++++ src/network/test_libvirtd_network.aug.in | 3 + .../networkxml2xmlin/nat-network-autoaddr.xml | 11 + .../nat-network-autoaddr.xml | 11 + tests/networkxml2xmltest.c | 1 + 22 files changed, 744 insertions(+), 121 deletions(-) create mode 100755 src/network/nm-dispatcher-check-nets.py create mode 100644 tests/networkxml2xmlin/nat-network-autoaddr.xml create mode 100644 tests/networkxml2xmlout/nat-network-autoaddr.xml -- 2.45.2

2 20

[PATCH 0/2] qemu: Clarify purpose of image format settings
by Jim Fehlig 17 Aug '24

17 Aug '24

The current documentation of the various foo_image_format settings in qemu.conf subtly implies they are only used for specifying compression. Patch1 of this small series attempts to clarify and improve the description of the settings. It defines image format as a way to specify the desired layout of guest memory blocks on disk. Patch2 changes the name of the 'compressed' field of virQEMUSaveHeader to 'format', removing implication that format == compressed in the code. Jim Fehlig (2): qemu: conf: Improve the foo_image_format setting descriptions qemu: Rename compressed field of save image header src/qemu/qemu.conf.in | 46 +++++++++++++++++++++++++-------------- src/qemu/qemu_saveimage.c | 18 +++++++-------- src/qemu/qemu_saveimage.h | 4 ++-- 3 files changed, 41 insertions(+), 27 deletions(-) -- 2.35.3

1 3

[PATCH RFC 0/9] qemu: Support mapped-ram migration capability
by Jim Fehlig 16 Aug '24

16 Aug '24

This series is a RFC for support of QEMU's mapped-ram migration capability [1] for saving and restoring VMs. It implements the first part of the design approach we discussed for supporting parallel save/restore [2]. In summary, the approach is 1. Add mapped-ram migration capability 2. Steal an element from save header 'unused' for a 'features' variable and bump save version to 3. 3. Add /etc/libvirt/qemu.conf knob for the save format version, defaulting to latest v3 4. Use v3 (aka mapped-ram) by default 5. Use mapped-ram with BYPASS_CACHE for v3, old approach for v2 6. include: Define constants for parallel save/restore 7. qemu: Add support for parallel save. Implies mapped-ram, reject if v2 8. qemu: Add support for parallel restore. Implies mapped-ram. Reject if v2 9. tools: add parallel parameter to virsh save command 10. tools: add parallel parameter to virsh restore command This series implements 1-5, with the BYPASS_CACHE support in patches 8 and 9 being quite hacky. They are included to discuss approaches to make them less hacky. See the patches for details. The QEMU mapped-ram capability currently does not support directio. Fabino is working on that now [3]. This complicates merging support in libvirt. I don't think it's reasonable to enable mapped-ram by default when BYPASS_CACHE cannot be supported. Should we wait until the mapped-ram directio support is merged in QEMU before supporting mapped-ram in libvirt? For the moment, compression is ignored in the new save version. Currently, libvirt connects the output of QEMU's save stream to the specified compression program via a pipe. This approach is incompatible with mapped-ram since the fd provided to QEMU must be seekable. One option is to reopen and compress the saved image after the actual save operation has completed. This has the downside of requiring the iohelper to handle BYPASS_CACHE, which would preclude us from removing it sometime in the future. Other suggestions much welcomed. Note the logical file size of mapped-ram saved images is slightly larger than guest RAM size, so the files are often much larger than the files produced by the existing, sequential format. However, actual blocks written to disk is often lower with mapped-ram saved images. E.g. a saved image from a 30G, freshly booted, idle guest results in the following 'Size' and 'Blocks' values reported by stat(1) Size Blocks sequential 998595770 1950392 mapped-ram 34368584225 1800456 With the same guest running a workload that dirties memory Size Blocks sequential 33173330615 64791672 mapped-ram 34368578210 64706944 Thanks for any comments on this RFC! [1] https://gitlab.com/qemu-project/qemu/-/blob/master/docs/devel/migration/map… [2] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/message/K4B… [3] https://mail.gnu.org/archive/html/qemu-devel/2024-05/msg04432.html Jim Fehlig (9): qemu: Enable mapped-ram migration capability qemu_fd: Add function to retrieve fdset ID qemu: Add function to get migration params for save qemu: Add a 'features' element to save image header and bump version qemu: conf: Add setting for save image version qemu: Add support for mapped-ram on save qemu: Enable mapped-ram on restore qemu: Support O_DIRECT with mapped-ram on save qemu: Support O_DIRECT with mapped-ram on restore src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf.in | 6 + src/qemu/qemu_conf.c | 8 ++ src/qemu/qemu_conf.h | 1 + src/qemu/qemu_driver.c | 25 ++-- src/qemu/qemu_fd.c | 18 +++ src/qemu/qemu_fd.h | 3 + src/qemu/qemu_migration.c | 99 ++++++++++++++- src/qemu/qemu_migration.h | 11 +- src/qemu/qemu_migration_params.c | 20 +++ src/qemu/qemu_migration_params.h | 4 + src/qemu/qemu_monitor.c | 40 ++++++ src/qemu/qemu_monitor.h | 5 + src/qemu/qemu_process.c | 63 +++++++--- src/qemu/qemu_process.h | 16 ++- src/qemu/qemu_saveimage.c | 187 +++++++++++++++++++++++------ src/qemu/qemu_saveimage.h | 20 ++- src/qemu/qemu_snapshot.c | 12 +- src/qemu/test_libvirtd_qemu.aug.in | 1 + 19 files changed, 455 insertions(+), 85 deletions(-) -- 2.44.0

3 22

[PATCH v3 0/4] multiple memory backend support for CPR Live Updates
by mgalaxy＠akamai.com 16 Aug '24

16 Aug '24

From: Michael Galaxy <mgalaxy(a)akamai.com> CPR-based support for whole-hypervisor kexec-based live updates is now finally merged into QEMU. In support of this, we need NUMA to be supported in these kinds of environments. To do this we use a technology called PMEM (persistent memory) in Linux, which underpins the ability for CPR Live Updates to work so that QEMU memory can remain in RAM and be recovered after a kexec operationg has completed. Our systems are highly NUMA-aware, and so this patch series enables NUMA awareness for live updates. Further, we make a small change that allows live migrations to work between *non* PMEM-based systems and PMEM-based systems (and vice-versa). This allows for seemless upgrades from non-live-compatible systems to live-update-compatible sytems without any downtime. Changes since v2: 1. Corrected invalid privateData casts and re-organized variables and error codes. 2. Re-ran ninja test and corrected formatting 3. Tested that individual patches apply and build cleanly. 4. Separated out NEWS item into separate patch Michael Galaxy (4): qemu.conf changes to support multiple memory backend Support live migration between file-backed memory and anonymous memory. Update configuration file documentation to support multiple memory backends Add NEWS item for multiple memory backend support NEWS.rst | 8 +++ docs/kbase/virtiofs.rst | 2 + src/qemu/qemu.conf.in | 4 ++ src/qemu/qemu_command.c | 6 +- src/qemu/qemu_conf.c | 139 +++++++++++++++++++++++++++++++++++----- src/qemu/qemu_conf.h | 16 +++-- src/qemu/qemu_domain.c | 24 +++++-- src/qemu/qemu_driver.c | 37 +++++++---- src/qemu/qemu_hotplug.c | 6 +- src/qemu/qemu_process.c | 42 ++++++------ src/qemu/qemu_process.h | 6 +- tests/testutilsqemu.c | 5 +- 12 files changed, 224 insertions(+), 71 deletions(-) -- 2.34.1

1 4