January 2023 - Devel - libvirt List Archives

[Libvir] Hang with virt-install graphic=no on FC 6 6.93/F7 RC2
by libvirtuser 23 Dec '25

23 Dec '25

I had this problem and another user confirmed the same behavior on his machine: http://pastie.caboo.se/65417 Haven't tried it with *2925.10* yet. Also, is there a way to get more verbose info about what calls are causing the "configuration file syntax error" errors? Thanks. virt-install How large would you like the disk () to be (in gigabytes)? 4 ERROR: Must provide a file, not a directory for the disk What would you like to use as the disk (path)? /xen/images/x6.img Would you like to enable graphics support? (yes or no) no What is the install location? http://mirror.anl.gov/pub/fedora-linux-core/test/6.93/Fedora/i386/os/ Starting install... libvir: Xen Daemon error : GET operation failed: libvir: error : configuration file syntax error: expecting a name libvir: error : configuration file syntax error: expecting a name libvir: error : configuration file syntax error: expecting a name Retrieving Fedora... 276 kB 00:01 Retrieving vmlinuz... 100% |=========================| 2.1 MB 00:16 Retrieving initrd.img... 100% |=========================| 5.4 MB 00:28 libvir: Xen Daemon error : GET operation failed: libvir: error : configuration file syntax error: expecting a name libvir: error : configuration file syntax error: expecting a name libvir: error : configuration file syntax error: expecting a name Creating domain... 0 B 00:06 Linux version 2.6.20-2925.5.fc7xen (brewbuilder(a)ls20-bc1-14.build.redhat.com) (gcc version 4.1.2 20070317 (Red Hat 4.1.2-5)) #1 SMP Thu Mar 22 13:51:38 EDT 2007 BIOS-provided physical RAM map: sanitize start sanitize bail 0 copy_e820_map() start: 0000000000000000 size: 0000000020800000 end: 0000000020800000 type: 1 Xen: 0000000000000000 - 0000000020800000 (usable) 0MB HIGHMEM available. 520MB LOWMEM available. Using x86 segment limits to approximate NX protection Zone PFN ranges: DMA 0 -> 133120 Normal 133120 -> 133120 HighMem 133120 -> 133120 early_node_map[1] active PFN ranges 0: 0 -> 133120 ACPI in unprivileged domain disabled Built 1 zonelists. Total pages: 132080 Kernel command line: method=http://mirror.anl.gov/pub/fedora-linux-core/test/6.93/Fedora/i386/os/ Enabling fast FPU save and restore... done. Enabling unmasked SIMD FPU exception support... done. Initializing CPU#0 CPU 0 irqstacks, hard=c135f000 soft=c133f000 PID hash table entries: 4096 (order: 12, 16384 bytes) Xen reported: 1694.994 MHz processor. Console: colour dummy device 80x25 Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar ... MAX_LOCKDEP_SUBCLASSES: 8 ... MAX_LOCK_DEPTH: 30 ... MAX_LOCKDEP_KEYS: 2048 ... CLASSHASH_SIZE: 1024 ... MAX_LOCKDEP_ENTRIES: 8192 ... MAX_LOCKDEP_CHAINS: 16384 ... CHAINHASH_SIZE: 8192 memory used by lock dependency info: 1064 kB per task-struct memory footprint: 1200 bytes Dentry cache hash table entries: 131072 (order: 7, 524288 bytes) Inode-cache hash table entries: 65536 (order: 6, 262144 bytes) Software IO TLB disabled vmalloc area: e1000000-f4ffe000, maxmem 2d7fe000 Memory: 503936k/532480k available (2030k kernel code, 19948k reserved, 1079k data, 180k init, 0k highmem) virtual kernel memory layout: fixmap : 0xf5315000 - 0xf57fe000 (5028 kB) pkmap : 0xf5000000 - 0xf5200000 (2048 kB) vmalloc : 0xe1000000 - 0xf4ffe000 ( 319 MB) lowmem : 0xc0000000 - 0xe0800000 ( 520 MB) .init : 0xc130e000 - 0xc133b000 ( 180 kB) .data : 0xc11fb9d9 - 0xc1309714 (1079 kB) .text : 0xc1000000 - 0xc11fb9d9 (2030 kB) Checking if this processor honours the WP bit even in supervisor mode... Ok. Calibrating delay using timer specific routine.. 4262.67 BogoMIPS (lpj=8525343) Security Framework v1.0.0 initialized SELinux: Initializing. SELinux: Starting in permissive mode selinux_register_security: Registering secondary module capability Capability LSM initialized as secondary Mount-cache hash table entries: 512 CPU: Trace cache: 12K uops, L1 D cache: 8K CPU: L2 cache: 256K Checking 'hlt' instruction... OK. SMP alternatives: switching to UP code Freeing SMP alternatives: 11k freed Brought up 1 CPUs Grant table initialized NET: Registered protocol family 16 Brought up 1 CPUs PCI: Fatal: No config space access function found PCI: setting up Xen PCI frontend stub Setting up standard PCI resources ACPI: Interpreter disabled. Linux Plug and Play Support v0.97 (c) Adam Belay pnp: PnP ACPI: disabled xen_mem: Initialising balloon driver. usbcore: registered new interface driver usbfs usbcore: registered new interface driver hub usbcore: registered new device driver usb PCI: System does not support PCI PCI: System does not support PCI NetLabel: Initializing NetLabel: domain hash size = 128 NetLabel: protocols = UNLABELED CIPSOv4 NetLabel: unlabeled traffic allowed by default NET: Registered protocol family 2 IP route cache hash table entries: 32768 (order: 5, 131072 bytes) TCP established hash table entries: 131072 (order: 10, 4194304 bytes) TCP bind hash table entries: 65536 (order: 9, 2097152 bytes) TCP: Hash tables configured (established 131072 bind 65536) TCP reno registered checking if image is initramfs... it is Freeing initrd memory: 7256k freed IA-32 Microcode Update Driver: v1.14-xen <tigran(a)veritas.com> audit: initializing netlink socket (disabled) audit(1180379657.981:1): initialized VFS: Disk quotas dquot_6.5.1 Dquot-cache hash table entries: 1024 (order 0, 4096 bytes) SELinux: Registering netfilter hooks io scheduler noop registered io scheduler anticipatory registered io scheduler deadline registered io scheduler cfq registered (default) pci_hotplug: PCI Hot Plug PCI Core version: 0.5 BUG: at kernel/fork.c:994 copy_process() [<c1005d9a>] show_trace_log_lvl+0x1a/0x2f [<c1006343>] show_trace+0x12/0x14 [<c10063be>] dump_stack+0x16/0x18 [<c101d044>] copy_process+0x195/0x1245 [<c101e144>] do_fork+0x50/0x117 [<c1003351>] kernel_thread+0x8e/0x96 [<c102c701>] __call_usermodehelper+0x2d/0x46 [<c102cd4f>] run_workqueue+0x89/0x145 [<c102d711>] worker_thread+0xd5/0x102 [<c102ff27>] kthread+0xb3/0xdc [<c10058db>] kernel_thread_helper+0x7/0x10 ======================= rtc: IRQ 8 is not free. Non-volatile memory driver v1.2 Linux agpgart interface v0.101 (c) Dave Jones RAMDISK driver initialized: 16 RAM disks of 16384K size 4096 blocksize input: Macintosh mouse button emulation as /class/input/input0 Xen virtual console successfully installed as xvc0 Event-channel device installed. usbcore: registered new interface driver libusual usbcore: registered new interface driver hiddev usbcore: registered new interface driver usbhid drivers/usb/input/hid-core.c: v2.6:USB HID core driver PNP: No PS/2 controller found. Probing ports directly. i8042.c: No controller found. mice: PS/2 mouse device common for all mice TCP bic registered Initializing XFRM netlink socket NET: Registered protocol family 1 NET: Registered protocol family 17 Using IPI No-Shortcut mode XENBUS: Device with no driver: device/vbd/51712 XENBUS: Device with no driver: device/vif/0 Freeing unused kernel memory: 180k freed Write protecting the kernel read-only data: 762k ^[Domain installation still in progress. You can reconnect

2 1

[libvirt] [PATCH] Fix python error reporting for some storage operations
by Cole Robinson 07 Mar '25

07 Mar '25

In the python bindings, all vir* classes expect to be passed a virConnect object when instantiated. Before the storage stuff, these classes were only instantiated in virConnect methods, so the generator is hardcoded to pass 'self' as the connection instance to these classes. Problem is there are some methods that return pool or vol instances which aren't called from virConnect: you can lookup a storage volume's associated pool, and can lookup volumes from a pool. In these cases passing 'self' doesn't give the vir* instance a connection, so when it comes time to raise an exception crap hits the fan. Rather than rework the generator to accomodate this edge case, I just fixed the init functions for virStorage* to pull the associated connection out of the passed value if it's not a virConnect instance. Thanks, Cole diff --git a/python/generator.py b/python/generator.py index 01a17da..c706b19 100755 --- a/python/generator.py +++ b/python/generator.py @@ -962,8 +962,12 @@ def buildWrappers(): list = reference_keepers[classname] for ref in list: classes.write(" self.%s = None\n" % ref[1]) - if classname in [ "virDomain", "virNetwork", "virStoragePool", "virStorageVol" ]: + if classname in [ "virDomain", "virNetwork" ]: classes.write(" self._conn = conn\n") + elif classname in [ "virStorageVol", "virStoragePool" ]: + classes.write(" self._conn = conn\n" + \ + " if not isinstance(conn, virConnect):\n" + \ + " self._conn = conn._conn\n") classes.write(" if _obj != None:self._o = _obj;return\n") classes.write(" self._o = None\n\n"); destruct=None

4 3

Re: [libvirt] [Qemu-devel] Qemu migration with vhost-user-blk on top of local storage
by Stefan Hajnoczi 27 Oct '23

27 Oct '23

On Wed, Jan 09, 2019 at 06:23:42PM +0800, wuzhouhui wrote: > Hi everyone, > > I'm working qemu with vhost target (e.g. spdk), and I attempt to migrate VM with > 2 local storages. One local storage is a regular file, e.g. /tmp/c74.qcow2, and > the other is a malloc bdev that spdk created. This malloc bdev will exported to > VM via vhost-user-blk. When I execute following command: > > virsh migrate --live --persistent --unsafe --undefinesource --copy-storage-all \ > --p2p --auto-converge --verbose --desturi qemu+tcp://<uri>/system vm0 > > The libvirt reports: > > qemu-2.12.1: error: internal error: unable to execute QEMU command \ > 'nbd-server-add': Cannot find device=drive-virtio-disk1 nor \ > node_name=drive-virtio-disk1 Please post your libvirt domain XML. > Does it means that qemu with spdk on top of local storage don't support migration? > > QEMU: 2.12.1 > SPDK: 18.10 vhost-user-blk bypasses the QEMU block layer, so NBD storage migration at the QEMU level will not work for the vhost-user-blk disk. Stefan

4 4

[libvirt] [PATCH v3] openvswitch: Add new port VLAN mode "dot1q-tunnel"
by luzhipeng＠uniudc.com 27 Oct '23

27 Oct '23

From: ZhiPeng Lu <luzhipeng(a)uniudc.com> Signed-off-by: ZhiPeng Lu <luzhipeng(a)uniudc.com> --- v1->v2: 1. Fix "make syntax-check" failure v2->v3: 1. remove other_config when updating vlan docs/formatnetwork.html.in | 17 +++++++++-------- docs/schemas/networkcommon.rng | 1 + src/conf/netdev_vlan_conf.c | 2 +- src/util/virnetdevopenvswitch.c | 7 +++++++ src/util/virnetdevvlan.h | 1 + 5 files changed, 19 insertions(+), 9 deletions(-) diff --git a/docs/formatnetwork.html.in b/docs/formatnetwork.html.in index 363a72b..3c1ae62 100644 --- a/docs/formatnetwork.html.in +++ b/docs/formatnetwork.html.in @@ -688,16 +688,17 @@ For network connections using Open vSwitch it is also possible - to configure 'native-tagged' and 'native-untagged' VLAN modes + to configure 'native-tagged' and 'native-untagged' and 'dot1q-tunnel' + VLAN modes. Since 1.1.0. This is done with the - optional <code>nativeMode</code> attribute on - the <code><tag></code> subelement: <code>nativeMode</code> - may be set to 'tagged' or 'untagged'. The <code>id</code> - attribute of the <code><tag></code> subelement - containing <code>nativeMode</code> sets which VLAN is considered - to be the "native" VLAN for this interface, and + optional <code>nativeMode</code> attribute on the + <code><tag></code> subelement: <code>nativeMode</code> + may be set to 'tagged' or 'untagged' or 'dot1q-tunnel'. + The <code>id</code> attribute of the <code><tag></code> + subelement containing <code>nativeMode</code> sets which VLAN is + considered to be the "native" VLAN for this interface, and the <code>nativeMode</code> attribute determines whether or not - traffic for that VLAN will be tagged. + traffic for that VLAN will be tagged or QinQ. <code><vlan></code> elements can also be specified in diff --git a/docs/schemas/networkcommon.rng b/docs/schemas/networkcommon.rng index 2699555..11c48ff 100644 --- a/docs/schemas/networkcommon.rng +++ b/docs/schemas/networkcommon.rng @@ -223,6 +223,7 @@ <choice> <value>tagged</value> <value>untagged</value> + <value>dot1q-tunnel</value> </choice> </attribute> </optional> diff --git a/src/conf/netdev_vlan_conf.c b/src/conf/netdev_vlan_conf.c index dff49c6..79710d9 100644 --- a/src/conf/netdev_vlan_conf.c +++ b/src/conf/netdev_vlan_conf.c @@ -29,7 +29,7 @@ #define VIR_FROM_THIS VIR_FROM_NONE VIR_ENUM_IMPL(virNativeVlanMode, VIR_NATIVE_VLAN_MODE_LAST, - "default", "tagged", "untagged") + "default", "tagged", "untagged", "dot1q-tunnel") int virNetDevVlanParse(xmlNodePtr node, xmlXPathContextPtr ctxt, virNetDevVlanPtr def) diff --git a/src/util/virnetdevopenvswitch.c b/src/util/virnetdevopenvswitch.c index 8fe06fd..9fec30b 100644 --- a/src/util/virnetdevopenvswitch.c +++ b/src/util/virnetdevopenvswitch.c @@ -91,6 +91,11 @@ virNetDevOpenvswitchConstructVlans(virCommandPtr cmd, virNetDevVlanPtr virtVlan) virCommandAddArg(cmd, "vlan_mode=native-untagged"); virCommandAddArgFormat(cmd, "tag=%d", virtVlan->nativeTag); break; + case VIR_NATIVE_VLAN_MODE_DOT1Q_TUNNEL: + virCommandAddArg(cmd, "vlan_mode=dot1q-tunnel"); + virCommandAddArg(cmd, "other_config:qinq-ethtype=802.1q"); + virCommandAddArgFormat(cmd, "tag=%d", virtVlan->nativeTag); + break; case VIR_NATIVE_VLAN_MODE_DEFAULT: default: break; @@ -504,6 +509,8 @@ int virNetDevOpenvswitchUpdateVlan(const char *ifname, "--", "--if-exists", "clear", "Port", ifname, "tag", "--", "--if-exists", "clear", "Port", ifname, "trunk", "--", "--if-exists", "clear", "Port", ifname, "vlan_mode", + "--", "--if-exists", "remove", "Port", ifname, "other_config", + "qinq-ethtype", NULL, "--", "--if-exists", "set", "Port", ifname, NULL); if (virNetDevOpenvswitchConstructVlans(cmd, virtVlan) < 0) diff --git a/src/util/virnetdevvlan.h b/src/util/virnetdevvlan.h index be85f59..0667f9d 100644 --- a/src/util/virnetdevvlan.h +++ b/src/util/virnetdevvlan.h @@ -29,6 +29,7 @@ typedef enum { VIR_NATIVE_VLAN_MODE_DEFAULT = 0, VIR_NATIVE_VLAN_MODE_TAGGED, VIR_NATIVE_VLAN_MODE_UNTAGGED, + VIR_NATIVE_VLAN_MODE_DOT1Q_TUNNEL, VIR_NATIVE_VLAN_MODE_LAST } virNativeVlanMode; -- 1.8.3.1

3 2

[libvirt] [PATCH] Fix compile error for stable 1.2.9
by Yang hongyang 27 Oct '23

27 Oct '23

Seems a backport miss. An extra member is passed to struct virLXCBasicMountInfo. Signed-off-by: Yang hongyang <hongyang.yang(a)easystack.cn> --- src/lxc/lxc_container.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c index 28dabec..1c65fa9 100644 --- a/src/lxc/lxc_container.c +++ b/src/lxc/lxc_container.c @@ -760,7 +760,7 @@ typedef struct { static const virLXCBasicMountInfo lxcBasicMounts[] = { { "proc", "/proc", "proc", MS_NOSUID|MS_NOEXEC|MS_NODEV, false, false }, - { "/proc/sys", "/proc/sys", NULL, MS_BIND|MS_NOSUID|MS_NOEXEC|MS_NODEV|MS_RDONLY, false, false, false }, + { "/proc/sys", "/proc/sys", NULL, MS_BIND|MS_NOSUID|MS_NOEXEC|MS_NODEV|MS_RDONLY, false, false }, { "sysfs", "/sys", "sysfs", MS_NOSUID|MS_NOEXEC|MS_NODEV|MS_RDONLY, false, false }, { "securityfs", "/sys/kernel/security", "securityfs", MS_NOSUID|MS_NOEXEC|MS_NODEV|MS_RDONLY, true, true }, #if WITH_SELINUX -- 1.7.1

3 2

[libvirt] [PATCH] web docs -- macvtap mode explanation
by Stefan Berger 27 Oct '23

27 Oct '23

This adds more information about the different macvtap device modes. Regards, Stefan

3 2

[libvirt] Supporting vhost-net and macvtap in libvirt for QEMU
by Anthony Liguori 27 Oct '23

27 Oct '23

Disclaimer: I am neither an SR-IOV nor a vhost-net expert, but I've CC'd people that are who can throw tomatoes at me for getting bits wrong :-) I wanted to start a discussion about supporting vhost-net in libvirt. vhost-net has not yet been merged into qemu but I expect it will be soon so it's a good time to start this discussion. There are two modes worth supporting for vhost-net in libvirt. The first mode is where vhost-net backs to a tun/tap device. This is behaves in very much the same way that -net tap behaves in qemu today. Basically, the difference is that the virtio backend is in the kernel instead of in qemu so there should be some performance improvement. Current, libvirt invokes qemu with -net tap,fd=X where X is an already open fd to a tun/tap device. I suspect that after we merge vhost-net, libvirt could support vhost-net in this mode by just doing -net vhost,fd=X. I think the only real question for libvirt is whether to provide a user visible switch to use vhost or to just always use vhost when it's available and it makes sense. Personally, I think the later makes sense. The more interesting invocation of vhost-net though is one where the vhost-net device backs directly to a physical network card. In this mode, vhost should get considerably better performance than the current implementation. I don't know the syntax yet, but I think it's reasonable to assume that it will look something like -net tap,dev=eth0. The effect will be that eth0 is dedicated to the guest. On most modern systems, there is a small number of network devices so this model is not all that useful except when dealing with SR-IOV adapters. In that case, each physical device can be exposed as many virtual devices (VFs). There are a few restrictions here though. The biggest is that currently, you can only change the number of VFs by reloading a kernel module so it's really a parameter that must be set at startup time. I think there are a few ways libvirt could support vhost-net in this second mode. The simplest would be to introduce a new tag similar to <source network='br0'>. In fact, if you probed the device type for the network parameter, you could probably do something like <source network='eth0'> and have it Just Work. Another model would be to have libvirt see an SR-IOV adapter as a network pool whereas it handled all of the VF management. Considering how inflexible SR-IOV is today, I'm not sure whether this is the best model. Has anyone put any more thought into this problem or how this should be modeled in libvirt? Michael, could you share your current thinking for -net syntax? -- Regards, Anthony Liguori

8 26

[libvirt RFCv11 00/33] multifd save restore prototype
by Claudio Fontana 23 Oct '23

23 Oct '23

This is v11 of the multifd save prototype, which focuses on saving to a single file instead of requiring multiple separate files for multifd channels. This series demonstrates a way to save and restore from a single file by using interleaved channels of a size equal to the transfer buffer size (1MB), and relying on UNIX holes to avoid wasting physical disk space due to channels of different size. KNOWN ISSUES: a) still applies only to save/restore (no managed save etc) b) this is not not done in QEMU, where it could be possible to teach QEMU to migrate directly to a file or block device in a block-aligned way by altering the migration stream code and the state migration code of all devices. changes from v10: * virfile: add new API virFileDiskCopyChannel, which extends the existing virFileDiskCopy to work with parallel channels in the file. * drop use of virthread API, use GLIB for threads. * pass only a single FD to the multifd-helper, which will then open additional FDs as required for the multithreaded I/O. * simplify virQEMUSaveFd API, separating the initialization from the addition of extra channels. * adapt all documentation to mention a single file instead of multiple. * remove the "Lim" versions of virFileDirectRead and Write, they are not needed. --- changes from v9: * exposed virFileDirectAlign * separated the >= 2 QEMU_SAVE_VERSION change in own patch * reworked the write code to add the alignment padding to the data_len, making the on disk format compatible when loaded from an older libvirt. * reworked the read code to use direct I/O APIs only for actual direct I/O file descriptors, so as to make old images work with newer libvirt. --- changes from v8: * rebased on master * reordered patches to add more upstreamable content at the start * split introduction of virQEMUSaveFd, so the first part is multifd-free * new virQEMUSaveDataRead as a mirror of virQEMUSaveDataWrite * introduced virFileDirect API, using it in virFileDisk operations and for virQEMUSaveRead and virQEMUSaveWrite --- changes from v7: * [ base params API and iohelper refactoring upstreamed ] * extended the QEMU save image format more, to record the nr of multifd channels on save. Made the data header struct packed. * removed --parallel-connections from the restore command, as now it is useless due to QEMU save image format extension. * separate out patches to expose migration_params APIs to saveimage, including qemuMigrationParamsSetString, SetCap, SetInt. * fixed bugs in the ImageOpen patch (missing saveFd init), removed some whitespace, and fixed some convoluted code paths for return value -3. --- changes from v6: * improved error path handling, with error messages and especially cancellation of qemu process on error during restore. * split patches more and reordered them to keep general refactoring at the beginning before the --parallel stuff is introduced. * improved multifd compression support, including adding an enum and extending the QEMU save image format to record the compression used on save, and pick it up automatically on restore. --- changes from v4: * runIO renamed to virFileDiskCopy and rethought arguments * renamed new APIs from ...ParametersFlags to ...Params * introduce the new virDomainSaveParams and virDomainRestoreParams without any additional parameters, so they can be upstreamed first. * solved the issue in the gendispatch.pl script generating code that was missing the conn parameter. --- changes from v3: * reordered series to have all helper-related change at the start * solved all reported issues from ninja test, including documentation * fixed most broken migration capabilities code (still imperfect likely) * added G_GNUC_UNUSED as needed * after multifd restore, added what I think were the missing operations: qemuProcessRefreshState(), qemuProcessStartCPUs() - most importantly, virDomainObjSave() The domain now starts running after restore without further encouragement * removed the sleep(10) from the multifd-helper --- changes from v2: * added ability to restore the VM from disk using multifd * fixed the multifd-helper to work in both directions, assuming the need to listen for save, and connect for restore. * fixed a large number of bugs, and probably introduced some :-) --- Claudio Fontana (33): virfile: introduce virFileDirect APIs virfile: use virFileDirect API in runIOCopy qemu: saveimage: rework image read/write to be O_DIRECT friendly qemu: saveimage: assume future formats will also support compression virfile: virFileDiskCopy: prepare for O_DIRECT files without wrapper qemu: saveimage: introduce virQEMUSaveFd qemu: saveimage: convert qemuSaveImageCreate to use virQEMUSaveFd qemu: saveimage: convert qemuSaveImageOpen to use virQEMUSaveFd tools: prepare doSave to use parameters tools: prepare cmdRestore to use parameters libvirt: add new VIR_DOMAIN_SAVE_PARALLEL flag and parameter qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in save qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in restore virfile: add new API virFileDiskCopyChannel multifd-helper: new helper for parallel save/restore qemu: saveimage: update virQEMUSaveFd struct for parallel save qemu: saveimage: wire up saveimage code with the multifd helper qemu: capabilities: add multifd to the probed migration capabilities qemu: saveimage: add multifd related fields to save format qemu: migration_params: add APIs to set Int and Cap qemu: migration: implement qemuMigrationSrcToFilesMultiFd for save qemu: add parameter to qemuMigrationDstRun to skip waiting qemu: implement qemuSaveImageLoadMultiFd for restore tools: add parallel parameter to virsh save command tools: add parallel parameter to virsh restore command qemu: add migration parameter multifd-compression libvirt: add new VIR_DOMAIN_SAVE_PARAM_PARALLEL_COMPRESSION qemu: saveimage: add parallel compression argument to ImageCreate qemu: saveimage: add stub support for multifd compression parameter qemu: migration: expose qemuMigrationParamsSetString qemu: saveimage: implement multifd-compression in parallel save qemu: saveimage: restore compressed parallel images tools: add parallel-compression parameter to virsh save command docs/manpages/virsh.rst | 26 +- include/libvirt/libvirt-domain.h | 24 + po/POTFILES | 1 + src/libvirt_private.syms | 6 + src/qemu/qemu_capabilities.c | 4 + src/qemu/qemu_capabilities.h | 2 + src/qemu/qemu_driver.c | 146 ++-- src/qemu/qemu_migration.c | 160 ++-- src/qemu/qemu_migration.h | 16 +- src/qemu/qemu_migration_params.c | 71 +- src/qemu/qemu_migration_params.h | 15 + src/qemu/qemu_process.c | 3 +- src/qemu/qemu_process.h | 5 +- src/qemu/qemu_saveimage.c | 703 +++++++++++++----- src/qemu/qemu_saveimage.h | 69 +- src/qemu/qemu_snapshot.c | 6 +- src/util/iohelper.c | 3 + src/util/meson.build | 16 + src/util/multifd-helper.c | 359 +++++++++ src/util/virfile.c | 391 +++++++--- src/util/virfile.h | 11 + .../caps_4.0.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_4.0.0.ppc64.xml | 1 + .../caps_4.0.0.riscv32.xml | 1 + .../caps_4.0.0.riscv64.xml | 1 + .../qemucapabilitiesdata/caps_4.0.0.s390x.xml | 1 + .../caps_4.0.0.x86_64.xml | 1 + .../caps_4.1.0.x86_64.xml | 1 + .../caps_4.2.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_4.2.0.ppc64.xml | 1 + .../qemucapabilitiesdata/caps_4.2.0.s390x.xml | 1 + .../caps_4.2.0.x86_64.xml | 1 + .../caps_5.0.0.aarch64.xml | 2 + .../qemucapabilitiesdata/caps_5.0.0.ppc64.xml | 2 + .../caps_5.0.0.riscv64.xml | 2 + .../caps_5.0.0.x86_64.xml | 2 + .../qemucapabilitiesdata/caps_5.1.0.sparc.xml | 2 + .../caps_5.1.0.x86_64.xml | 2 + .../caps_5.2.0.aarch64.xml | 2 + .../qemucapabilitiesdata/caps_5.2.0.ppc64.xml | 2 + .../caps_5.2.0.riscv64.xml | 2 + .../qemucapabilitiesdata/caps_5.2.0.s390x.xml | 2 + .../caps_5.2.0.x86_64.xml | 2 + .../caps_6.0.0.aarch64.xml | 2 + .../qemucapabilitiesdata/caps_6.0.0.s390x.xml | 2 + .../caps_6.0.0.x86_64.xml | 2 + .../caps_6.1.0.x86_64.xml | 2 + .../caps_6.2.0.aarch64.xml | 2 + .../qemucapabilitiesdata/caps_6.2.0.ppc64.xml | 2 + .../caps_6.2.0.x86_64.xml | 2 + .../caps_7.0.0.aarch64.xml | 2 + .../qemucapabilitiesdata/caps_7.0.0.ppc64.xml | 2 + .../caps_7.0.0.x86_64.xml | 2 + .../caps_7.1.0.x86_64.xml | 2 + tools/virsh-domain.c | 101 ++- 55 files changed, 1748 insertions(+), 445 deletions(-) create mode 100644 src/util/multifd-helper.c -- 2.26.2

2 40

[PATCH 00/18] RFC: Remove deprecated audio features
by Martin Kletzander 15 Sep '23

15 Sep '23

I wanted to deal with https://bugzilla.redhat.com/2043498 and I got a suggesstion that removing deprecated features could actually make it easier to propagate the error. In the end (last patch) it turns out the error is still just reported with error_fatal, so it probably is not really needed, but I really wanted to dig into QEMU more and learn some of the internals for quite some time now. So I used the opportunity. The one-liner ended up being an 18 patch series which was, for someone who has just one commit in QEMU codebase, a pretty challenging task. Although I tried my best to do things properly, I am not sure whether I handled everything correctly, hence the RFC. Any comments are very much appreciated. Thanks and have a nice day ;) Martin Kletzander (18): hw/audio: Remove -soundhw support hw/input/tsc210x: Extract common init code into new function hw/audio: Simplify hda audio init hw/audio/lm4549: Add errp error reporting to init function tests/qtest: Specify audiodev= and -audiodev ui/vnc: Require audiodev= Introduce machine's default-audiodev property audio: Add easy dummy audio initialiser hw/display/xlnx_dp.c: Add audiodev property hw/input/tsc210x.c: Support machine-default audiodev with fallback hw/arm: Support machine-default audiodev with fallback hw/ppc: Support machine-default audiodev with fallback audio: Make AUD_register_card fallible and require audiodev= audio: Require AudioState in AUD_add_capture audio: Be more strict during audio backend initialisation audio: Remove legacy audio environment variables and options audio: Remove unused can_be_default audio/spiceaudio: Fail initialisation when not using spice audio/alsaaudio.c | 1 - audio/audio.c | 204 +++---- audio/audio.h | 5 +- audio/audio_int.h | 1 - audio/audio_legacy.c | 555 ------------------ audio/coreaudio.m | 1 - audio/dbusaudio.c | 1 - audio/dsoundaudio.c | 1 - audio/jackaudio.c | 1 - audio/meson.build | 1 - audio/noaudio.c | 1 - audio/ossaudio.c | 1 - audio/paaudio.c | 1 - audio/sdlaudio.c | 1 - audio/spiceaudio.c | 3 +- audio/wavaudio.c | 1 - docs/about/deprecated.rst | 24 - docs/about/removed-features.rst | 27 + docs/qdev-device-use.txt | 21 +- docs/replay.txt | 2 +- hw/arm/integratorcp.c | 8 +- hw/arm/musicpal.c | 8 +- hw/arm/omap2.c | 11 +- hw/arm/realview.c | 3 + hw/arm/spitz.c | 10 +- hw/arm/versatilepb.c | 3 + hw/arm/vexpress.c | 3 + hw/arm/xlnx-zcu102.c | 4 + hw/arm/z2.c | 12 +- hw/audio/ac97.c | 9 +- hw/audio/adlib.c | 9 +- hw/audio/cs4231a.c | 8 +- hw/audio/es1370.c | 8 +- hw/audio/gus.c | 6 +- hw/audio/hda-codec.c | 37 +- hw/audio/intel-hda.c | 25 +- hw/audio/intel-hda.h | 2 +- hw/audio/lm4549.c | 7 +- hw/audio/lm4549.h | 3 +- hw/audio/meson.build | 1 - hw/audio/pcspk.c | 15 +- hw/audio/pl041.c | 2 +- hw/audio/sb16.c | 9 +- hw/audio/soundhw.c | 177 ------ hw/audio/wm8750.c | 5 +- hw/core/machine.c | 23 + hw/display/xlnx_dp.c | 12 +- hw/input/tsc210x.c | 79 ++- hw/ppc/prep.c | 4 + hw/usb/dev-audio.c | 5 +- include/hw/audio/soundhw.h | 15 - include/hw/boards.h | 1 + qemu-options.hx | 37 -- .../codeconverter/test_regexps.py | 1 - softmmu/qdev-monitor.c | 2 - softmmu/vl.c | 10 - tests/qtest/ac97-test.c | 3 +- tests/qtest/es1370-test.c | 3 +- tests/qtest/fuzz/generic_fuzz_configs.h | 6 +- tests/qtest/intel-hda-test.c | 15 +- ui/vnc.c | 15 +- 61 files changed, 329 insertions(+), 1140 deletions(-) delete mode 100644 audio/audio_legacy.c delete mode 100644 hw/audio/soundhw.c delete mode 100644 include/hw/audio/soundhw.h -- 2.35.1

5 43

[PATCH v2] util: basic support for VFIO variant drivers
by Laine Stump 01 Jun '23

01 Jun '23

Before a PCI device can be assigned to a guest with VFIO, that device must be bound to the vfio-pci driver rather than to the device's normal driver. The vfio-pci driver provides APIs that permit QEMU to perform all the necessary operations to make the device accessible to the guest. There has been kernel work recently to support vendor/device-specific VFIO variant drivers that provide the basic vfio-pci driver functionality while adding support for device-specific operations (for example these device-specific drivers are planned to support live migration of certain devices). All that will be needed to make this functionality available will be to bind the new vendor-specific driver to the device (rather than the generic vfio-pci driver, which will continue to work just without the extra functionality). But until now libvirt has required that all PCI devices being assigned to a guest with VFIO specifically have the "vfio-pci" driver bound to the device. So even if the user manually binds a shiny new vendor-specific vfio variant driver to the device (and puts "managed='no'" in the config to prevent libvirt from changing the binding), libvirt will just fail during startup of the guest (or during hotplug) because the driver bound to the device isn't exactly "vfio-pci". This patch loosens that restriction a bit - rather than requiring that the device be bound to "vfio-pci", it also checks if the drivername contains the string "vfio" at all, and in this case allows the operation to continue. If the driver is in fact a VFIO variant, then the assignment will succeed, but if it is not a VFIO variant then QEMU will fail (and report the error back to libvirt). In the near future (possibly by kernel 6.0) there will be a formal method of identifying a VFIO variant driver by looking in sysfs; in the meantime the inexact, but simple, method in this patch will allow users of the few existing VFIO variant drivers (and developers of new VFIO variant drivers) to use their new drivers without needing to remove libvirt from their setup - they can simply pre-bind the device to the new driver, then use "managed='no'" in their libvirt config. NB: this patch does *not* handle automatically determining the proper vendor-specific driver and binding to it in the case of "managed='yes'". This will be implemented later when there is a widely available driver / device combo we can use for testing. Signed-off-by: Laine Stump <laine(a)redhat.com> --- V1 here: https://listman.redhat.com/archives/libvir-list/2022-August/233327.html Change in V2: V1 used the output of modinfo to look for "vfio_pci" as an alias for a driver to see if it was a VFIO variant driver. As a result of discussion of V1, V2 is much simpler - it just assumes that any driver with "vfio" in the name is a VFIO variant. This is okay because 1) QEMU will still catch it and libvirt will properly log the error if the driver isn't actually a VFIO variant, and 2) it's a temporary situation, just to enable use of VFIO variant drivers with libvirt until a standard method of detecting this is added to sysfs (which, according to the discussion of V1, is coming in the near future). (NB: I did implement checking of /lib/modules/`uname -r`/modules.alias as suggested by Erik, but it turned out that this caused the unit tests to call uname(3) and open the modules.alias file on the test host - for a proper unit test I would have also needed to mock these two functions, and it seemed like too much complexity for a temporary workaround. I've implemented Jason's suggestion here (accept any driver with "vfio" in the name), which is similar to danpb's suggestion (accept specifically the two drivers that are already in the upstream kernel), but will also allow for new drivers that may be under development.) src/hypervisor/virhostdev.c | 26 ++++--------- src/util/virpci.c | 76 ++++++++++++++++++++++++++++++++++--- src/util/virpci.h | 3 ++ 3 files changed, 82 insertions(+), 23 deletions(-) diff --git a/src/hypervisor/virhostdev.c b/src/hypervisor/virhostdev.c index c0ce867596..15b35fa75e 100644 --- a/src/hypervisor/virhostdev.c +++ b/src/hypervisor/virhostdev.c @@ -747,9 +747,8 @@ virHostdevPreparePCIDevicesImpl(virHostdevManager *mgr, mgr->inactivePCIHostdevs) < 0) goto reattachdevs; } else { - g_autofree char *driverPath = NULL; - g_autofree char *driverName = NULL; - int stub; + g_autofree char *drvName = NULL; + virPCIStubDriver drvType; /* Unmanaged devices should already have been marked as * inactive: if that's the case, we can simply move on */ @@ -769,18 +768,14 @@ virHostdevPreparePCIDevicesImpl(virHostdevManager *mgr, * information about active / inactive device across * daemon restarts has been implemented */ - if (virPCIDeviceGetDriverPathAndName(pci, - &driverPath, &driverName) < 0) + if (virPCIDeviceGetDriverNameAndType(pci, &drvName, &drvType) < 0) goto reattachdevs; - stub = virPCIStubDriverTypeFromString(driverName); - - if (stub > VIR_PCI_STUB_DRIVER_NONE && - stub < VIR_PCI_STUB_DRIVER_LAST) { + if (drvType > VIR_PCI_STUB_DRIVER_NONE) { /* The device is bound to a known stub driver: store this * information and add a copy to the inactive list */ - virPCIDeviceSetStubDriver(pci, stub); + virPCIDeviceSetStubDriver(pci, drvType); VIR_DEBUG("Adding PCI device %s to inactive list", virPCIDeviceGetName(pci)); @@ -2292,18 +2287,13 @@ virHostdevPrepareOneNVMeDevice(virHostdevManager *hostdev_mgr, /* Let's check if all PCI devices are NVMe disks. */ for (i = 0; i < virPCIDeviceListCount(pciDevices); i++) { virPCIDevice *pci = virPCIDeviceListGet(pciDevices, i); - g_autofree char *drvPath = NULL; g_autofree char *drvName = NULL; - int stub = VIR_PCI_STUB_DRIVER_NONE; + virPCIStubDriver drvType; - if (virPCIDeviceGetDriverPathAndName(pci, &drvPath, &drvName) < 0) + if (virPCIDeviceGetDriverNameAndType(pci, &drvName, &drvType) < 0) goto cleanup; - if (drvName) - stub = virPCIStubDriverTypeFromString(drvName); - - if (stub == VIR_PCI_STUB_DRIVER_VFIO || - STREQ_NULLABLE(drvName, "nvme")) + if (drvType == VIR_PCI_STUB_DRIVER_VFIO || STREQ_NULLABLE(drvName, "nvme")) continue; VIR_WARN("Suspicious NVMe disk assignment. PCI device " diff --git a/src/util/virpci.c b/src/util/virpci.c index 7800966963..51ccf4d9fd 100644 --- a/src/util/virpci.c +++ b/src/util/virpci.c @@ -277,6 +277,71 @@ virPCIDeviceGetDriverPathAndName(virPCIDevice *dev, char **path, char **name) } +/** + * virPCIDeviceGetDriverNameAndType: + * @dev: virPCIDevice object to examine + * @drvName: returns name of driver bound to this device (if any) + * @drvType: returns type of driver if it is a known stub driver type + * + * Find the name of the driver bound to @dev (if any) and the type of + * the driver if it is a known/recognized "stub" driver (based on the + * driver name). + * + * There are vfio "variant" drivers that provide all the basic + * functionality of the standard vfio-pci driver as well as additional + * stuff. There is a plan to add info to sysfs that will allow easily + * determining if a driver is a vfio variant driver, but that sysfs + * entry isn't yet available. In the meantime as a workaround so that + * the few existing vfio variant drivers can be used with libvirt, and + * so that driver developers can test their new vfio variant drivers + * without needing to bypass libvirt, we also check if the driver name + * contains the string "vfio"; if it does, then we consider this drier + * as type VFIO. This can lead to false positives, but that isn't a + * horrible thing, because the problem will still be caught by QEMU as + * soon as libvirt makes the request to attach the device. + * + * Return 0 on success, -1 on failure. If -1 is returned, then an error + * message has been logged. + */ +int +virPCIDeviceGetDriverNameAndType(virPCIDevice *dev, + char **drvName, + virPCIStubDriver *drvType) +{ + g_autofree char *drvPath = NULL; + int tmpType; + + if (virPCIDeviceGetDriverPathAndName(dev, &drvPath, drvName) < 0) + return -1; + + if (!*drvName) { + *drvType = VIR_PCI_STUB_DRIVER_NONE; + return 0; + } + + tmpType = virPCIStubDriverTypeFromString(*drvName); + + if (tmpType > VIR_PCI_STUB_DRIVER_NONE) { + *drvType = tmpType; + return 0; /* exact match of a known driver name (or no name) */ + } + + /* Check if the drivername contains "vfio" and count as a VFIO + * driver if so - see above for explanation. + */ + + if (strstr(*drvName, "vfio")) { + VIR_DEBUG("Driver %s is a vfio_pci driver", *drvName); + *drvType = VIR_PCI_STUB_DRIVER_VFIO; + } else { + VIR_DEBUG("Driver %s is NOT a vfio_pci driver", *drvName); + *drvType = VIR_PCI_STUB_DRIVER_NONE; + } + + return 0; +} + + static int virPCIDeviceConfigOpenInternal(virPCIDevice *dev, bool readonly, bool fatal) { @@ -1004,8 +1069,8 @@ virPCIDeviceReset(virPCIDevice *dev, virPCIDeviceList *activeDevs, virPCIDeviceList *inactiveDevs) { - g_autofree char *drvPath = NULL; g_autofree char *drvName = NULL; + virPCIStubDriver drvType; int ret = -1; int fd = -1; int hdrType = -1; @@ -1032,15 +1097,16 @@ virPCIDeviceReset(virPCIDevice *dev, * reset it whenever appropriate, so doing it ourselves would just * be redundant. */ - if (virPCIDeviceGetDriverPathAndName(dev, &drvPath, &drvName) < 0) + if (virPCIDeviceGetDriverNameAndType(dev, &drvName, &drvType) < 0) goto cleanup; - if (virPCIStubDriverTypeFromString(drvName) == VIR_PCI_STUB_DRIVER_VFIO) { - VIR_DEBUG("Device %s is bound to vfio-pci - skip reset", - dev->name); + if (drvType == VIR_PCI_STUB_DRIVER_VFIO) { + + VIR_DEBUG("Device %s is bound to %s - skip reset", dev->name, drvName); ret = 0; goto cleanup; } + VIR_DEBUG("Resetting device %s", dev->name); if ((fd = virPCIDeviceConfigOpenWrite(dev)) < 0) diff --git a/src/util/virpci.h b/src/util/virpci.h index 4d9193f24e..0532b90f90 100644 --- a/src/util/virpci.h +++ b/src/util/virpci.h @@ -280,6 +280,9 @@ int virPCIDeviceRebind(virPCIDevice *dev); int virPCIDeviceGetDriverPathAndName(virPCIDevice *dev, char **path, char **name); +int virPCIDeviceGetDriverNameAndType(virPCIDevice *dev, + char **drvName, + virPCIStubDriver *drvType); int virPCIDeviceIsPCIExpress(virPCIDevice *dev); int virPCIDeviceHasPCIExpressLink(virPCIDevice *dev); -- 2.37.1

5 13

[RFC PATCH] docs/about/deprecated: Deprecate 32-bit host systems
by Thomas Huth 05 Apr '23

05 Apr '23

Testing 32-bit host OS support takes a lot of precious time during the QEMU contiguous integration tests, and considering that many OS vendors stopped shipping 32-bit variants of their OS distributions and most hardware from the past >10 years is capable of 64-bit, keeping the 32-bit support alive is an inadequate burden for the QEMU project. Let's mark the 32-bit support as deprecated so we can drop it after a while - this will help us to cut down our limited CI minutes in the gitlab CI, for example. Signed-off-by: Thomas Huth <thuth(a)redhat.com> --- docs/about/deprecated.rst | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 9f1bbc495d..ce6463e72b 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -181,9 +181,20 @@ As Debian 10 ("Buster") moved into LTS the big endian 32 bit version of MIPS moved out of support making it hard to maintain our cross-compilation CI tests of the architecture. As we no longer have CI coverage support may bitrot away before the deprecation process -completes. The little endian variants of MIPS (both 32 and 64 bit) are +completes. The little endian variants of MIPS are still a supported host architecture. +32-bit host operating systems (since 8.0) +''''''''''''''''''''''''''''''''''''''''' + +Testing 32-bit host OS support takes a lot of precious time during the QEMU +contiguous integration tests, and considering that many OS vendors stopped +shipping 32-bit variants of their OS distributions and most hardware from +the past >10 years is capable of 64-bit, keeping the 32-bit support alive +is an inadequate burden for the QEMU project. Thus QEMU will soon drop the +support for 32-bit host systems. + + QEMU API (QAPI) events ---------------------- -- 2.31.1

16 44

[libvirt PATCH v6 0/8] Add support for 'blob' to virtio video device
by Jonathon Jongsma 03 Mar '23

03 Mar '23

Add support to libvirt for the 'blob' option for virtio video devices in qemu. Also do a little preparatory refactoring of the video device xml parsing code. This series was basically acked by Jan some time ago, but I made a couple changes and apparently I dropped the ball and forgot to send out the revised series. So here's a v6 after rebasing yet again to current master. changes in v6: - added two new patches: - conf: explicitly set VIR_DOMAIN_VIDEO_TYPE_DEFAULT to 0 - util: add virXMLPropUIntDefault() function - other minor fixes from Jan's review Jonathon Jongsma (8): conf: explicitly set VIR_DOMAIN_VIDEO_TYPE_DEFAULT to 0 conf: Refactor video model parsing util: add virXMLPropUIntDefault() function conf: switch to virXMLProp* functions for parsing video conf: use enum variable for video type conf: add support for 'blob' in virtio video device qemu: Add capability for virtio-gpu.blob qemu: Implement 'blob' support for virtio gpu docs/formatdomain.rst | 7 + src/conf/cpu_conf.c | 9 +- src/conf/domain_conf.c | 134 ++++++++---------- src/conf/domain_conf.h | 5 +- src/conf/domain_validate.c | 13 +- src/conf/numa_conf.c | 7 +- src/conf/schemas/domaincommon.rng | 5 + src/libxl/libxl_conf.c | 10 ++ src/libxl/libxl_domain.c | 11 ++ src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_cgroup.c | 22 ++- src/qemu/qemu_command.c | 3 + src/qemu/qemu_domain.h | 1 + src/qemu/qemu_monitor_json.c | 16 ++- src/qemu/qemu_namespace.c | 22 +++ src/qemu/qemu_process.c | 7 + src/qemu/qemu_validate.c | 9 ++ src/util/virxml.c | 29 +++- src/util/virxml.h | 9 ++ .../caps_6.1.0.x86_64.xml | 1 + .../caps_6.2.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_6.2.0.ppc64.xml | 1 + .../caps_6.2.0.x86_64.xml | 1 + .../caps_7.0.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_7.0.0.ppc64.xml | 1 + .../caps_7.0.0.x86_64.xml | 1 + .../qemucapabilitiesdata/caps_7.1.0.ppc64.xml | 1 + .../caps_7.1.0.x86_64.xml | 1 + .../caps_7.2.0.x86_64.xml | 1 + .../caps_8.0.0.riscv64.xml | 1 + .../caps_8.0.0.x86_64.xml | 1 + .../video-virtio-blob-off.x86_64-latest.args | 39 +++++ .../video-virtio-blob-off.xml | 36 +++++ .../video-virtio-blob-on.x86_64-latest.args | 39 +++++ .../qemuxml2argvdata/video-virtio-blob-on.xml | 36 +++++ tests/qemuxml2argvtest.c | 2 + .../video-virtio-blob-off.x86_64-latest.xml | 48 +++++++ .../video-virtio-blob-on.x86_64-latest.xml | 48 +++++++ tests/qemuxml2xmltest.c | 2 + 40 files changed, 483 insertions(+), 101 deletions(-) create mode 100644 tests/qemuxml2argvdata/video-virtio-blob-off.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/video-virtio-blob-off.xml create mode 100644 tests/qemuxml2argvdata/video-virtio-blob-on.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/video-virtio-blob-on.xml create mode 100644 tests/qemuxml2xmloutdata/video-virtio-blob-off.x86_64-latest.xml create mode 100644 tests/qemuxml2xmloutdata/video-virtio-blob-on.x86_64-latest.xml -- 2.39.0

2 10

Summary on new backup interfaces in QEMU
by Vladimir Sementsov-Ogievskiy 21 Feb '23

21 Feb '23

Hi all! Here I want to summarize new interfaces and use cases for backup in QEMU. TODO for me: convert this into good rst documentation in docs/. OK, let's begin. First, note that drive-backup qmp command is deprecated. Next, some terminology: push backup: the whole process is inside QEMU process, also may be called "internal backup" pull backup: QEMU only exports a kind of snapshot (for example by NBD), and third party software reads this export and stores it somehow, also called "external backup" copy-before-write operations: We usually do backup of active disk, guest is running and may write to the disk during the process of backup. When guest wants to rewrite data region which is not backed up yet, we must stop this guest write, and copy original data to somewhere before continuing guest write. That's a copy-before-write operation. image-fleecing: the technique that allows to export a "snapshotted" state of the active disk with help of copy-before-write operations. We create a temporary image - target for copy-before-write operations, and provide an interface to the user to read the "snapshotted" state. And for read, we do read from temporary image the data which is already changed in original active disk, and we read unchanged data directly from active disk. The temporary image itself is also called "reverse delta" or "reversed delta". == Simple push backup == Just use blockdev-backup, nothing new here. I just note some technical details, that are relatively new: 1. First, backup job inserts copy-before-write filter above source disk, to do copy-before-write operation. 2. Created copy-before-write filter shares internal block-copy state with backup job, so they work in collaboration, to not copy same things twice. == Full pull backup == Assume, we are going to do incremental backup in future, so we also need to create a dirty bitmap, to track dirtiness of active disk since full backup. 1. Create empty temporary image for fleecing. It must be of the same size as active disk. It's not necessary to be qcow2, and if it's a qcow2, you shouldn't make the original active disk a backing file for the new temporary qcow2 image (it was necessary in old fleecing scheme). Example: qemu-img create -f qcow2 temp.qcow2 64G 2. Initialize fleecing scheme and create dirty bitmap for future incremental backup. Assume, disk0 is an active disk, attached to qdev-id sda, to be backed up. qmp: transaction [ block-dirty-bitmap-add {node: disk0, name: bitmap0, persistent: true} blockdev-add* {node-name: tmp-protocol, driver: file, filename: temp.qcow2} blockdev-add {node-name: tmp, driver: qcow2, file: tmp-protocol} blockdev-add {node-name: cbw, driver: copy-before-write, file: disk0, target: tmp} blockdev-replace** {parent-type: qdev, qdev-id: sda, new-child: cbw} blockdev-add {node-name: acc, driver: snapshot-access, file: cbw} ] qmp: nbd-server-start {...} qmp: nbd-server-add {device: acc, ...} This way we create the following block-graph: [guest] [NBD export] | | | root | root v file v [copy-before-write]<------[snapshot-access] | | | file | target v v [active-disk] [temp.qcow2] * "[PATCH 0/2] blockdev-add transaction" series needed for this ** "[PATCH v3 00/11] blockdev-replace" series needed for this Note additional useful options for copy-before-write filter: "[PATCH 0/3] block: copy-before-write: on-cbw-error behavior" provides possibility of option on-cbw-error=break-snapshot, which means that on failure of CBW operation we will not break guest write, but instead all further reads by NBD client will fail, which formally means: break the backup process, not guest write. "[PATCH 0/4] block: copy-before-write: cbw-timeout" provides an option cbw-timeout, to set a timeout for CBW operations. That's very useful to avoid guest stuck. 3. Now third party backup tool can read data from NBD export NBD_CMD_TRIM (discard) operation is supported on the export, it has the following effects: 1. discard this data from temp image, if it is stored here 2. avoid further copy-before-write operation (guest is free to rewrite corresponding data with no extra latency) 3. all further read requests from discarded areas by NBD client will fail So, NBD client may discard regions that are already backed up to avoid extra latency for guest writes and to free disk space on the host. Possible TODO here is to implement NBD protocol extension, that allows to READ & DISCARD in command. In this case we avoid extra command in the wire, but lose possibility of retrying the READ operation if it failed. 4. After backup is complete, we should destroy the fleecing scheme: qmp: nbd-server-stop qmp: blockdev-del {node-name: acc} qmp: blockdev-replace {parent-type: qdev, qdev-id: sda, new-child: disk0} qmp: blockdev-del {node-name: cbw} qmp: blockdev-del {node-name: tmp} qmp: blockdev-del {node-name: tmp-protocol} 5. If backup failed, we should remove created dirty bitmap: qmp: block-dirty-bitmap-remove {node: disk0, name: bitmap0} == Incremental pull backup == OK, now we have a bitmap called bitmap0, and want to do incremental backup, accordingly to that bitmap. In short, we want: - create a new bitmap to continue dirty tracking for next incremental backup - export "snapshotted" state of disk0 through NBD - export "frozen" bitmap, so that external tool know what to copy Mostly, all points remains the same, let's go through: 1. Create empty temporary image for fleecing -- same as for full backup, no difference 2. Initialize fleecing scheme and create dirty bitmap for future incremental backup. qmp: transaction [ block-dirty-bitmap-add {node: disk0, name: bitmap1, persistent: true} block-dirty-bitmap-disable {node: disk0, name: bitmap0} blockdev-add {node-name: tmp-protocol, driver: file, filename: temp.qcow2} blockdev-add {node-name: tmp, driver: qcow2, file: tmp-protocol} blockdev-add {node-name: cbw, driver: copy-before-write, file: disk0, target: tmp, bitmap: {node: disk0, name: bitmap0}} blockdev-replace {parent-type: qdev, qdev-id: sda, new-child: cbw} blockdev-add {node-name: acc, driver: snapshot-access, file: cbw} ] qmp: nbd-server-start {...} qmp: block-export-add {type: nbd, node-name: acc, bitmaps: [{node: disk0, name: bitmap0}]} 3. Now third party backup tool can read data from NBD export - Client may negotiate meta contexts, to query exported dirty bitmap by NBD_BLOCK_STATUS commend - If client reads "not-dirty" (by bitmap0) areas, it gets an error. - NBD_CMD_TRIM (discard) works as for full backup, no difference 4. After backup is complete, we should destroy the fleecing scheme: - Same as for full backup 5. Next, we should handle dirty bitmaps: 5.1 Failure path Merge-back bitmap1 to bitmap0 and continue tracking in bitmap0: qmp: transaction [ block-dirty-bitmap-enable {node: disk0, name: bitmap0} block-dirty-bitmap-merge {node: disk0, target: bitmap0, bitmaps: ['bitmap1']} block-dirty-bitmap-remove {node: disk0, name: bitmap1} ] 5.2 Success path We have two possible user scenarios on success: 5.2.1 Continue tracking for next incremental backup in bitmap1 In this case, just remove bitmap0: qmp: block-dirty-bitmap-remove {node: disk0, name: bitmap0} Or you may not delete bitmap0 and keep it disabled, to be reused in future for differential backup (see below). 5.2.2 Continue tracking for next incremental backup in bitmap0 (assume, we always work with one bitmap and don't want any kind of differential backups, and don't associate bitmap name with stored backups) In this case, enable and clear bitmap0, merge bitmap1 to bitmap0 and remove bitmap1: qmp: transaction [ block-dirty-bitmap-enable {node: disk0, name: bitmap0} block-dirty-bitmap-clear {node: disk0, name: bitmap0} block-dirty-bitmap-merge {node: disk0, target: bitmap0, bitmaps: ['bitmap1']} block-dirty-bitmap-remove {node: disk0, name: bitmap1} ] == Push backup with fleecing full/incremental == Reasoning: the main problem of simple push backup is that guest writes may be seriously affected by copy-on-write operations, when backup target is slow. To solve this problem, we'll use the scheme like for pull backup: we create local temporary image, which is a target for copy-before-write operations, and instead of exporting the "snapshot-access" node we start internal backup from it to the target. So, the scheme and commands looks exactly the same as for full and incremental pull backup. The only difference is that we don't need to start nbd export, but instead we should add target node to qemu and start internal backup. And good thing is that it may be done in same transaction with initializing fleecing scheme: qmp: transaction [ ... initialize fleecing scheme for full or incremental backup ... # Add target node. Here is qcow2 added, but it may be nbd node or something else blockdev-add {node-name: target-protocol, driver: file, filename: target.qcow2} blockdev-add {node-name: target, driver: qcow2, file: target-protocol} # Start backup blockdev-backup {device: acc, target: target, ...} ] If it is an incremental backup, pass also bitmap parameter: blockdev-backup {..., bitmap: bitmap0, sync: incremental, bitmap-mode: never} Note bitmap-mode=never: this means that backup will do nothing with bitmap0, so we have same scheme like for pull backups (handle bitmaps by hand after backup). Still, push-backup scheme may be adopted to use other bitmap modes. What we lack here is discarding in 'acc' node after successful copying of the block to the target, to safe disk space and avoid extra copy-before-write operations. It's a TODO, should be implemented like discard-source parameter for blockdev-backup. == Differential backups == I'm not fan of this idea, but I think it should be described. Assume we have already a chain of incremental backups (represented as qcow2 chain on backup storage server, for example). They corresponds to some points in time: T0, T1, T2, T3. Assume T3 is the last backup. If we want to create usual incremental backup, it would be diff between T3 and current time (which becomes T4). Differential backup say: I want to make backup starting from T1 to current time. What's for? Maybe T2 and T3 was removed or somehow damaged.. How to do that in Qemu: on each incremental backup you start a new bitmap, and _keep_ old one as disabled. This way we have bitmap0 (which presents diff between T0 and T1), bitmap1 (diff T1 T2), bitmap2 (diff T2 T3), and bitmap3 which shows diff from T3 up to current time. bitmap3 is the only enabled bitmap and others are disabled. So, to make differential backup, use block-dirty-bitmap-merge command, to merge all bitmaps you need into one, and than use it in any backup scheme. The drawback is that all these disabled bitmaps eat RAM. Possible solution is to not keep them in RAM, it's OK to keep them in qcow2, and load only on demand. That's not realized now and that's a TODO for thous who want differential backups. -- Best regards, Vladimir

3 2

Proposal: renaming 'master' branch to 'main'
by Daniel P. Berrangé 20 Feb '23

20 Feb '23

I don't recall exactly when it first came up, but it has been a few years now since the idea of moving away from 'master' as the git default branch name became a topic in OSS communities. Far from the first mention: https://sfconservancy.org/news/2020/jun/23/gitbranchname/ Both gitlab.com and github.com now default to 'main' as the default branch when creating new repositories: https://about.gitlab.com/blog/2021/03/10/new-git-default-branch-name/ https://github.blog/changelog/2020-10-01-the-default-branch-for-newly-creat… Some notable large open source projects have done, (or at least started) to rename the default branch in their existing projects too. eg GNOME: https://gitlab.gnome.org/GNOME/glib/-/issues/2348 Fedora: https://fedoraproject.org/wiki/Changes/GitRepos-master-to-main For libosinfo we did a rename last year. There was a little disruption but nothing too terrible, IIRC, missing weblate translation branch update was our main mistake. I'd suggest it is time to libvirt to get on this train and rename our default branch to 'main' in all repositories. There are essentially two options * Rename 'master' to 'main' With this, anyone pulling from an existing checkout will get an error telling them that 'master' does not exist. It won't tell them about 'main', but at least it gives them a sign that something in their checkout probably needs changing. Downside is that any URLs pointing to source files / commits with a branch name in the URL will become 404s. * Clone 'master' to 'main' With this, anyone pulling from an existing checkout will get no updates. It is very easy for people to not realize that they are tracking a branch which is no longer used Downside is also that the undesirable term 'master' remains present in the repo, even if unused. We might also miss places which still refer to 'master' which will end up outdated Ideally, we would rename 'master' to 'main', while the git server adds 'symbolic-ref' to effectively create a symlink to 'main'. This would mean anyone pulling from 'master' would get content from 'main'. AFAICT, this is not supported by gitlab.com though. I had held off suggesting the rename, hoping such support might arrive, but I'm doubting it will happen on a timescale to be useful, if at all. In terms of impact * Developers need to checkout 'main' and delete their now stale 'master' branch * Any open merge requests need updating, hitting 'Edit' and choose the new 'main' as the target * User repo forks may wish to delete their 'master' and push 'main', but that's entirely optional, since many people never touch/look at the default branch in their forks. * CI mostly shouldn't be impacted since we use CI_DEFAULT_BRANCH most places instead of hardcoding 'master'. A few exceptions - Links to artifacts include a branch name - integration tests mention branch name - References to check-doc/cirrus-run jobs use 'master' as the docker tag * GitHub mirroring won't track the rename, it'll just add 'main' without removing 'master', so needs manual fixup. * Need to update 'protected branch' and 'default branch' fields in gitlab repo settings for each repo * Weblate needs updating to translate from 'main' instead of 'master' So the rename isn't free of cost, but it should all be one-time only costs, which I think is something we can live with. Given our never ceasing development stream there's not really any 'right' time todo such a change. Just after a release is probably as good as it gets, and January is marginally better since we skip Feb and have a 6 week gap until the March 1st release. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

5 6

[PATCH v3] cpu_s390: Implement getVendorForModel for IBM Z
by Thomas Huth 10 Feb '23

10 Feb '23

When running "virsh domcapabilities" on a s390x host, all the CPU models show up with vendor='unknown' - which sounds kind of weird since the vendor of these mainframe CPUs is well known: IBM. All CPUs starting with either "z" or "gen" match a real mainframe CPU by IBM, so let's return the string "IBM" for those now. The only remaining ones are now the artifical "qemu" and "max" models from QEMU itself, so it should be OK to get an "unknown" vendor for those two. Reviewed-by: Jiri Denemark <jdenemar(a)redhat.com> Signed-off-by: Boris Fiuczynski<fiuczy(a)linux.ibm.com> Signed-off-by: Thomas Huth <thuth(a)redhat.com> --- v3: Use STRPREFIX() for the checks (thanks to Jiri for the hint!) src/cpu/cpu_s390.c | 11 ++ tests/domaincapsdata/qemu_4.2.0.s390x.xml | 144 +++++++++++----------- tests/domaincapsdata/qemu_5.2.0.s390x.xml | 144 +++++++++++----------- tests/domaincapsdata/qemu_6.0.0.s390x.xml | 144 +++++++++++----------- 4 files changed, 227 insertions(+), 216 deletions(-) diff --git a/src/cpu/cpu_s390.c b/src/cpu/cpu_s390.c index d908a83928..81a1513ecb 100644 --- a/src/cpu/cpu_s390.c +++ b/src/cpu/cpu_s390.c @@ -109,6 +109,16 @@ virCPUs390ValidateFeatures(virCPUDef *cpu) } +static const char * +virCPUs390GetVendorForModel(const char *modelName) +{ + if (STRPREFIX(modelName, "z") || STRPREFIX(modelName, "gen")) + return "IBM"; + + return NULL; +} + + struct cpuArchDriver cpuDriverS390 = { .name = "s390", .arch = archs, @@ -119,4 +129,5 @@ struct cpuArchDriver cpuDriverS390 = { .baseline = NULL, .update = virCPUs390Update, .validateFeatures = virCPUs390ValidateFeatures, + .getVendorForModel = virCPUs390GetVendorForModel, }; diff --git a/tests/domaincapsdata/qemu_4.2.0.s390x.xml b/tests/domaincapsdata/qemu_4.2.0.s390x.xml index 4f176e2d37..66841881a1 100644 --- a/tests/domaincapsdata/qemu_4.2.0.s390x.xml +++ b/tests/domaincapsdata/qemu_4.2.0.s390x.xml @@ -83,79 +83,79 @@ <feature policy='require' name='cmm'/> </mode> <mode name='custom' supported='yes'> - <model usable='yes' vendor='unknown'>z800-base</model> - <model usable='yes' vendor='unknown'>z890.2-base</model> - <model usable='yes' vendor='unknown'>z9EC.2</model> - <model usable='yes' vendor='unknown'>z13.2</model> - <model usable='yes' vendor='unknown'>z9BC-base</model> - <model usable='yes' vendor='unknown'>z990.5-base</model> - <model usable='yes' vendor='unknown'>z890.2</model> - <model usable='yes' vendor='unknown'>z890</model> - <model usable='yes' vendor='unknown'>z9BC</model> - <model usable='yes' vendor='unknown'>z13</model> - <model usable='yes' vendor='unknown'>z196</model> - <model usable='yes' vendor='unknown'>z13s</model> - <model usable='yes' vendor='unknown'>z990.3</model> - <model usable='yes' vendor='unknown'>z13s-base</model> - <model usable='yes' vendor='unknown'>z9EC</model> - <model usable='yes' vendor='unknown'>gen15a</model> - <model usable='yes' vendor='unknown'>z14ZR1-base</model> - <model usable='yes' vendor='unknown'>z14.2-base</model> - <model usable='yes' vendor='unknown'>z900.3-base</model> - <model usable='yes' vendor='unknown'>z13.2-base</model> - <model usable='yes' vendor='unknown'>z196.2-base</model> - <model usable='yes' vendor='unknown'>zBC12-base</model> - <model usable='yes' vendor='unknown'>z9BC.2-base</model> - <model usable='yes' vendor='unknown'>z900.2-base</model> - <model usable='yes' vendor='unknown'>z9EC.3</model> - <model usable='yes' vendor='unknown'>zEC12</model> - <model usable='yes' vendor='unknown'>z900</model> - <model usable='yes' vendor='unknown'>z114-base</model> - <model usable='yes' vendor='unknown'>zEC12-base</model> - <model usable='yes' vendor='unknown'>z10EC.2</model> - <model usable='yes' vendor='unknown'>z10EC-base</model> - <model usable='yes' vendor='unknown'>z900.3</model> - <model usable='yes' vendor='unknown'>z14ZR1</model> - <model usable='yes' vendor='unknown'>z10BC</model> - <model usable='yes' vendor='unknown'>z10BC.2-base</model> - <model usable='yes' vendor='unknown'>z990.2</model> - <model usable='yes' vendor='unknown'>z9BC.2</model> - <model usable='yes' vendor='unknown'>z990</model> - <model usable='yes' vendor='unknown'>z14</model> - <model usable='yes' vendor='unknown'>gen15b-base</model> - <model usable='yes' vendor='unknown'>z990.4</model> + <model usable='yes' vendor='IBM'>z800-base</model> + <model usable='yes' vendor='IBM'>z890.2-base</model> + <model usable='yes' vendor='IBM'>z9EC.2</model> + <model usable='yes' vendor='IBM'>z13.2</model> + <model usable='yes' vendor='IBM'>z9BC-base</model> + <model usable='yes' vendor='IBM'>z990.5-base</model> + <model usable='yes' vendor='IBM'>z890.2</model> + <model usable='yes' vendor='IBM'>z890</model> + <model usable='yes' vendor='IBM'>z9BC</model> + <model usable='yes' vendor='IBM'>z13</model> + <model usable='yes' vendor='IBM'>z196</model> + <model usable='yes' vendor='IBM'>z13s</model> + <model usable='yes' vendor='IBM'>z990.3</model> + <model usable='yes' vendor='IBM'>z13s-base</model> + <model usable='yes' vendor='IBM'>z9EC</model> + <model usable='yes' vendor='IBM'>gen15a</model> + <model usable='yes' vendor='IBM'>z14ZR1-base</model> + <model usable='yes' vendor='IBM'>z14.2-base</model> + <model usable='yes' vendor='IBM'>z900.3-base</model> + <model usable='yes' vendor='IBM'>z13.2-base</model> + <model usable='yes' vendor='IBM'>z196.2-base</model> + <model usable='yes' vendor='IBM'>zBC12-base</model> + <model usable='yes' vendor='IBM'>z9BC.2-base</model> + <model usable='yes' vendor='IBM'>z900.2-base</model> + <model usable='yes' vendor='IBM'>z9EC.3</model> + <model usable='yes' vendor='IBM'>zEC12</model> + <model usable='yes' vendor='IBM'>z900</model> + <model usable='yes' vendor='IBM'>z114-base</model> + <model usable='yes' vendor='IBM'>zEC12-base</model> + <model usable='yes' vendor='IBM'>z10EC.2</model> + <model usable='yes' vendor='IBM'>z10EC-base</model> + <model usable='yes' vendor='IBM'>z900.3</model> + <model usable='yes' vendor='IBM'>z14ZR1</model> + <model usable='yes' vendor='IBM'>z10BC</model> + <model usable='yes' vendor='IBM'>z10BC.2-base</model> + <model usable='yes' vendor='IBM'>z990.2</model> + <model usable='yes' vendor='IBM'>z9BC.2</model> + <model usable='yes' vendor='IBM'>z990</model> + <model usable='yes' vendor='IBM'>z14</model> + <model usable='yes' vendor='IBM'>gen15b-base</model> + <model usable='yes' vendor='IBM'>z990.4</model> <model usable='yes' vendor='unknown'>max</model> - <model usable='yes' vendor='unknown'>z10EC.2-base</model> - <model usable='yes' vendor='unknown'>gen15a-base</model> - <model usable='yes' vendor='unknown'>z800</model> - <model usable='yes' vendor='unknown'>zEC12.2</model> - <model usable='yes' vendor='unknown'>z10EC</model> - <model usable='yes' vendor='unknown'>z990.2-base</model> - <model usable='yes' vendor='unknown'>z900-base</model> - <model usable='yes' vendor='unknown'>z10BC.2</model> - <model usable='yes' vendor='unknown'>z9EC-base</model> - <model usable='yes' vendor='unknown'>z9EC.3-base</model> - <model usable='yes' vendor='unknown'>z114</model> - <model usable='yes' vendor='unknown'>z890.3</model> - <model usable='yes' vendor='unknown'>z196-base</model> - <model usable='yes' vendor='unknown'>z9EC.2-base</model> - <model usable='yes' vendor='unknown'>z196.2</model> - <model usable='yes' vendor='unknown'>z14.2</model> - <model usable='yes' vendor='unknown'>z990-base</model> - <model usable='yes' vendor='unknown'>z900.2</model> - <model usable='yes' vendor='unknown'>z890-base</model> - <model usable='yes' vendor='unknown'>z10EC.3</model> - <model usable='yes' vendor='unknown'>z14-base</model> - <model usable='yes' vendor='unknown'>z990.4-base</model> - <model usable='yes' vendor='unknown'>z10EC.3-base</model> - <model usable='yes' vendor='unknown'>z10BC-base</model> - <model usable='yes' vendor='unknown'>z13-base</model> - <model usable='yes' vendor='unknown'>z990.3-base</model> - <model usable='yes' vendor='unknown'>zEC12.2-base</model> - <model usable='yes' vendor='unknown'>zBC12</model> - <model usable='yes' vendor='unknown'>z890.3-base</model> - <model usable='yes' vendor='unknown'>z990.5</model> - <model usable='yes' vendor='unknown'>gen15b</model> + <model usable='yes' vendor='IBM'>z10EC.2-base</model> + <model usable='yes' vendor='IBM'>gen15a-base</model> + <model usable='yes' vendor='IBM'>z800</model> + <model usable='yes' vendor='IBM'>zEC12.2</model> + <model usable='yes' vendor='IBM'>z10EC</model> + <model usable='yes' vendor='IBM'>z990.2-base</model> + <model usable='yes' vendor='IBM'>z900-base</model> + <model usable='yes' vendor='IBM'>z10BC.2</model> + <model usable='yes' vendor='IBM'>z9EC-base</model> + <model usable='yes' vendor='IBM'>z9EC.3-base</model> + <model usable='yes' vendor='IBM'>z114</model> + <model usable='yes' vendor='IBM'>z890.3</model> + <model usable='yes' vendor='IBM'>z196-base</model> + <model usable='yes' vendor='IBM'>z9EC.2-base</model> + <model usable='yes' vendor='IBM'>z196.2</model> + <model usable='yes' vendor='IBM'>z14.2</model> + <model usable='yes' vendor='IBM'>z990-base</model> + <model usable='yes' vendor='IBM'>z900.2</model> + <model usable='yes' vendor='IBM'>z890-base</model> + <model usable='yes' vendor='IBM'>z10EC.3</model> + <model usable='yes' vendor='IBM'>z14-base</model> + <model usable='yes' vendor='IBM'>z990.4-base</model> + <model usable='yes' vendor='IBM'>z10EC.3-base</model> + <model usable='yes' vendor='IBM'>z10BC-base</model> + <model usable='yes' vendor='IBM'>z13-base</model> + <model usable='yes' vendor='IBM'>z990.3-base</model> + <model usable='yes' vendor='IBM'>zEC12.2-base</model> + <model usable='yes' vendor='IBM'>zBC12</model> + <model usable='yes' vendor='IBM'>z890.3-base</model> + <model usable='yes' vendor='IBM'>z990.5</model> + <model usable='yes' vendor='IBM'>gen15b</model> <model usable='no' vendor='unknown'>qemu</model> </mode> </cpu> diff --git a/tests/domaincapsdata/qemu_5.2.0.s390x.xml b/tests/domaincapsdata/qemu_5.2.0.s390x.xml index 760f514d7b..31ddbfbc75 100644 --- a/tests/domaincapsdata/qemu_5.2.0.s390x.xml +++ b/tests/domaincapsdata/qemu_5.2.0.s390x.xml @@ -85,79 +85,79 @@ <feature policy='require' name='cmm'/> </mode> <mode name='custom' supported='yes'> - <model usable='yes' vendor='unknown'>z800-base</model> - <model usable='yes' vendor='unknown'>z890.2-base</model> - <model usable='yes' vendor='unknown'>z9EC.2</model> - <model usable='yes' vendor='unknown'>z13.2</model> - <model usable='yes' vendor='unknown'>z990.5-base</model> - <model usable='yes' vendor='unknown'>z9BC-base</model> - <model usable='yes' vendor='unknown'>z890.2</model> - <model usable='yes' vendor='unknown'>z890</model> - <model usable='yes' vendor='unknown'>z9BC</model> - <model usable='yes' vendor='unknown'>z13</model> - <model usable='yes' vendor='unknown'>z196</model> - <model usable='yes' vendor='unknown'>z13s</model> - <model usable='yes' vendor='unknown'>z990.3</model> - <model usable='yes' vendor='unknown'>z13s-base</model> - <model usable='yes' vendor='unknown'>z9EC</model> - <model usable='yes' vendor='unknown'>gen15a</model> - <model usable='yes' vendor='unknown'>z14ZR1-base</model> - <model usable='yes' vendor='unknown'>z14.2-base</model> - <model usable='yes' vendor='unknown'>z900.3-base</model> - <model usable='yes' vendor='unknown'>z13.2-base</model> - <model usable='yes' vendor='unknown'>z196.2-base</model> - <model usable='yes' vendor='unknown'>zBC12-base</model> - <model usable='yes' vendor='unknown'>z9BC.2-base</model> - <model usable='yes' vendor='unknown'>z900.2-base</model> - <model usable='yes' vendor='unknown'>z9EC.3</model> - <model usable='yes' vendor='unknown'>zEC12</model> - <model usable='yes' vendor='unknown'>z900</model> - <model usable='yes' vendor='unknown'>z114-base</model> - <model usable='yes' vendor='unknown'>zEC12-base</model> - <model usable='yes' vendor='unknown'>z10EC.2</model> - <model usable='yes' vendor='unknown'>z10EC-base</model> - <model usable='yes' vendor='unknown'>z900.3</model> - <model usable='yes' vendor='unknown'>z14ZR1</model> - <model usable='yes' vendor='unknown'>z10BC</model> - <model usable='yes' vendor='unknown'>z10BC.2-base</model> - <model usable='yes' vendor='unknown'>z9BC.2</model> - <model usable='yes' vendor='unknown'>z990</model> - <model usable='yes' vendor='unknown'>z990.2</model> - <model usable='yes' vendor='unknown'>z14</model> - <model usable='yes' vendor='unknown'>gen15b-base</model> - <model usable='yes' vendor='unknown'>z990.4</model> + <model usable='yes' vendor='IBM'>z800-base</model> + <model usable='yes' vendor='IBM'>z890.2-base</model> + <model usable='yes' vendor='IBM'>z9EC.2</model> + <model usable='yes' vendor='IBM'>z13.2</model> + <model usable='yes' vendor='IBM'>z990.5-base</model> + <model usable='yes' vendor='IBM'>z9BC-base</model> + <model usable='yes' vendor='IBM'>z890.2</model> + <model usable='yes' vendor='IBM'>z890</model> + <model usable='yes' vendor='IBM'>z9BC</model> + <model usable='yes' vendor='IBM'>z13</model> + <model usable='yes' vendor='IBM'>z196</model> + <model usable='yes' vendor='IBM'>z13s</model> + <model usable='yes' vendor='IBM'>z990.3</model> + <model usable='yes' vendor='IBM'>z13s-base</model> + <model usable='yes' vendor='IBM'>z9EC</model> + <model usable='yes' vendor='IBM'>gen15a</model> + <model usable='yes' vendor='IBM'>z14ZR1-base</model> + <model usable='yes' vendor='IBM'>z14.2-base</model> + <model usable='yes' vendor='IBM'>z900.3-base</model> + <model usable='yes' vendor='IBM'>z13.2-base</model> + <model usable='yes' vendor='IBM'>z196.2-base</model> + <model usable='yes' vendor='IBM'>zBC12-base</model> + <model usable='yes' vendor='IBM'>z9BC.2-base</model> + <model usable='yes' vendor='IBM'>z900.2-base</model> + <model usable='yes' vendor='IBM'>z9EC.3</model> + <model usable='yes' vendor='IBM'>zEC12</model> + <model usable='yes' vendor='IBM'>z900</model> + <model usable='yes' vendor='IBM'>z114-base</model> + <model usable='yes' vendor='IBM'>zEC12-base</model> + <model usable='yes' vendor='IBM'>z10EC.2</model> + <model usable='yes' vendor='IBM'>z10EC-base</model> + <model usable='yes' vendor='IBM'>z900.3</model> + <model usable='yes' vendor='IBM'>z14ZR1</model> + <model usable='yes' vendor='IBM'>z10BC</model> + <model usable='yes' vendor='IBM'>z10BC.2-base</model> + <model usable='yes' vendor='IBM'>z9BC.2</model> + <model usable='yes' vendor='IBM'>z990</model> + <model usable='yes' vendor='IBM'>z990.2</model> + <model usable='yes' vendor='IBM'>z14</model> + <model usable='yes' vendor='IBM'>gen15b-base</model> + <model usable='yes' vendor='IBM'>z990.4</model> <model usable='yes' vendor='unknown'>max</model> - <model usable='yes' vendor='unknown'>z990.2-base</model> - <model usable='yes' vendor='unknown'>z10EC.2-base</model> - <model usable='yes' vendor='unknown'>gen15a-base</model> - <model usable='yes' vendor='unknown'>z800</model> - <model usable='yes' vendor='unknown'>z10EC</model> - <model usable='yes' vendor='unknown'>zEC12.2</model> - <model usable='yes' vendor='unknown'>z900-base</model> - <model usable='yes' vendor='unknown'>z10BC.2</model> - <model usable='yes' vendor='unknown'>z9EC-base</model> - <model usable='yes' vendor='unknown'>z9EC.3-base</model> - <model usable='yes' vendor='unknown'>z114</model> - <model usable='yes' vendor='unknown'>z890.3</model> - <model usable='yes' vendor='unknown'>z196-base</model> - <model usable='yes' vendor='unknown'>z9EC.2-base</model> - <model usable='yes' vendor='unknown'>z196.2</model> - <model usable='yes' vendor='unknown'>z14.2</model> - <model usable='yes' vendor='unknown'>z990-base</model> - <model usable='yes' vendor='unknown'>z900.2</model> - <model usable='yes' vendor='unknown'>z10EC.3</model> - <model usable='yes' vendor='unknown'>z890-base</model> - <model usable='yes' vendor='unknown'>z14-base</model> - <model usable='yes' vendor='unknown'>z990.4-base</model> - <model usable='yes' vendor='unknown'>z10EC.3-base</model> - <model usable='yes' vendor='unknown'>z10BC-base</model> - <model usable='yes' vendor='unknown'>z13-base</model> - <model usable='yes' vendor='unknown'>z990.3-base</model> - <model usable='yes' vendor='unknown'>zEC12.2-base</model> - <model usable='yes' vendor='unknown'>zBC12</model> - <model usable='yes' vendor='unknown'>z890.3-base</model> - <model usable='yes' vendor='unknown'>z990.5</model> - <model usable='yes' vendor='unknown'>gen15b</model> + <model usable='yes' vendor='IBM'>z990.2-base</model> + <model usable='yes' vendor='IBM'>z10EC.2-base</model> + <model usable='yes' vendor='IBM'>gen15a-base</model> + <model usable='yes' vendor='IBM'>z800</model> + <model usable='yes' vendor='IBM'>z10EC</model> + <model usable='yes' vendor='IBM'>zEC12.2</model> + <model usable='yes' vendor='IBM'>z900-base</model> + <model usable='yes' vendor='IBM'>z10BC.2</model> + <model usable='yes' vendor='IBM'>z9EC-base</model> + <model usable='yes' vendor='IBM'>z9EC.3-base</model> + <model usable='yes' vendor='IBM'>z114</model> + <model usable='yes' vendor='IBM'>z890.3</model> + <model usable='yes' vendor='IBM'>z196-base</model> + <model usable='yes' vendor='IBM'>z9EC.2-base</model> + <model usable='yes' vendor='IBM'>z196.2</model> + <model usable='yes' vendor='IBM'>z14.2</model> + <model usable='yes' vendor='IBM'>z990-base</model> + <model usable='yes' vendor='IBM'>z900.2</model> + <model usable='yes' vendor='IBM'>z10EC.3</model> + <model usable='yes' vendor='IBM'>z890-base</model> + <model usable='yes' vendor='IBM'>z14-base</model> + <model usable='yes' vendor='IBM'>z990.4-base</model> + <model usable='yes' vendor='IBM'>z10EC.3-base</model> + <model usable='yes' vendor='IBM'>z10BC-base</model> + <model usable='yes' vendor='IBM'>z13-base</model> + <model usable='yes' vendor='IBM'>z990.3-base</model> + <model usable='yes' vendor='IBM'>zEC12.2-base</model> + <model usable='yes' vendor='IBM'>zBC12</model> + <model usable='yes' vendor='IBM'>z890.3-base</model> + <model usable='yes' vendor='IBM'>z990.5</model> + <model usable='yes' vendor='IBM'>gen15b</model> <model usable='yes' vendor='unknown'>qemu</model> </mode> </cpu> diff --git a/tests/domaincapsdata/qemu_6.0.0.s390x.xml b/tests/domaincapsdata/qemu_6.0.0.s390x.xml index b1968668db..1cb19e051b 100644 --- a/tests/domaincapsdata/qemu_6.0.0.s390x.xml +++ b/tests/domaincapsdata/qemu_6.0.0.s390x.xml @@ -86,79 +86,79 @@ <feature policy='require' name='cmm'/> </mode> <mode name='custom' supported='yes'> - <model usable='yes' vendor='unknown'>z800-base</model> - <model usable='yes' vendor='unknown'>z890.2-base</model> - <model usable='yes' vendor='unknown'>z9EC.2</model> - <model usable='yes' vendor='unknown'>z13.2</model> - <model usable='yes' vendor='unknown'>z990.5-base</model> - <model usable='yes' vendor='unknown'>z9BC-base</model> - <model usable='yes' vendor='unknown'>z890.2</model> - <model usable='yes' vendor='unknown'>z890</model> - <model usable='yes' vendor='unknown'>z9BC</model> - <model usable='yes' vendor='unknown'>z13</model> - <model usable='yes' vendor='unknown'>z196</model> - <model usable='yes' vendor='unknown'>z13s</model> - <model usable='yes' vendor='unknown'>z990.3</model> - <model usable='yes' vendor='unknown'>z13s-base</model> - <model usable='yes' vendor='unknown'>z9EC</model> - <model usable='yes' vendor='unknown'>gen15a</model> - <model usable='yes' vendor='unknown'>z14ZR1-base</model> - <model usable='yes' vendor='unknown'>z14.2-base</model> - <model usable='yes' vendor='unknown'>z900.3-base</model> - <model usable='yes' vendor='unknown'>z13.2-base</model> - <model usable='yes' vendor='unknown'>z196.2-base</model> - <model usable='yes' vendor='unknown'>zBC12-base</model> - <model usable='yes' vendor='unknown'>z9BC.2-base</model> - <model usable='yes' vendor='unknown'>z900.2-base</model> - <model usable='yes' vendor='unknown'>z9EC.3</model> - <model usable='yes' vendor='unknown'>zEC12</model> - <model usable='yes' vendor='unknown'>z900</model> - <model usable='yes' vendor='unknown'>z114-base</model> - <model usable='yes' vendor='unknown'>zEC12-base</model> - <model usable='yes' vendor='unknown'>z10EC.2</model> - <model usable='yes' vendor='unknown'>z10EC-base</model> - <model usable='yes' vendor='unknown'>z900.3</model> - <model usable='yes' vendor='unknown'>z14ZR1</model> - <model usable='yes' vendor='unknown'>z10BC</model> - <model usable='yes' vendor='unknown'>z10BC.2-base</model> - <model usable='yes' vendor='unknown'>z9BC.2</model> - <model usable='yes' vendor='unknown'>z990</model> - <model usable='yes' vendor='unknown'>z990.2</model> - <model usable='yes' vendor='unknown'>z14</model> - <model usable='yes' vendor='unknown'>gen15b-base</model> - <model usable='yes' vendor='unknown'>z990.4</model> + <model usable='yes' vendor='IBM'>z800-base</model> + <model usable='yes' vendor='IBM'>z890.2-base</model> + <model usable='yes' vendor='IBM'>z9EC.2</model> + <model usable='yes' vendor='IBM'>z13.2</model> + <model usable='yes' vendor='IBM'>z990.5-base</model> + <model usable='yes' vendor='IBM'>z9BC-base</model> + <model usable='yes' vendor='IBM'>z890.2</model> + <model usable='yes' vendor='IBM'>z890</model> + <model usable='yes' vendor='IBM'>z9BC</model> + <model usable='yes' vendor='IBM'>z13</model> + <model usable='yes' vendor='IBM'>z196</model> + <model usable='yes' vendor='IBM'>z13s</model> + <model usable='yes' vendor='IBM'>z990.3</model> + <model usable='yes' vendor='IBM'>z13s-base</model> + <model usable='yes' vendor='IBM'>z9EC</model> + <model usable='yes' vendor='IBM'>gen15a</model> + <model usable='yes' vendor='IBM'>z14ZR1-base</model> + <model usable='yes' vendor='IBM'>z14.2-base</model> + <model usable='yes' vendor='IBM'>z900.3-base</model> + <model usable='yes' vendor='IBM'>z13.2-base</model> + <model usable='yes' vendor='IBM'>z196.2-base</model> + <model usable='yes' vendor='IBM'>zBC12-base</model> + <model usable='yes' vendor='IBM'>z9BC.2-base</model> + <model usable='yes' vendor='IBM'>z900.2-base</model> + <model usable='yes' vendor='IBM'>z9EC.3</model> + <model usable='yes' vendor='IBM'>zEC12</model> + <model usable='yes' vendor='IBM'>z900</model> + <model usable='yes' vendor='IBM'>z114-base</model> + <model usable='yes' vendor='IBM'>zEC12-base</model> + <model usable='yes' vendor='IBM'>z10EC.2</model> + <model usable='yes' vendor='IBM'>z10EC-base</model> + <model usable='yes' vendor='IBM'>z900.3</model> + <model usable='yes' vendor='IBM'>z14ZR1</model> + <model usable='yes' vendor='IBM'>z10BC</model> + <model usable='yes' vendor='IBM'>z10BC.2-base</model> + <model usable='yes' vendor='IBM'>z9BC.2</model> + <model usable='yes' vendor='IBM'>z990</model> + <model usable='yes' vendor='IBM'>z990.2</model> + <model usable='yes' vendor='IBM'>z14</model> + <model usable='yes' vendor='IBM'>gen15b-base</model> + <model usable='yes' vendor='IBM'>z990.4</model> <model usable='yes' vendor='unknown'>max</model> - <model usable='yes' vendor='unknown'>z10EC.2-base</model> - <model usable='yes' vendor='unknown'>gen15a-base</model> - <model usable='yes' vendor='unknown'>z800</model> - <model usable='yes' vendor='unknown'>z10EC</model> - <model usable='yes' vendor='unknown'>zEC12.2</model> - <model usable='yes' vendor='unknown'>z990.2-base</model> - <model usable='yes' vendor='unknown'>z900-base</model> - <model usable='yes' vendor='unknown'>z10BC.2</model> - <model usable='yes' vendor='unknown'>z9EC-base</model> - <model usable='yes' vendor='unknown'>z9EC.3-base</model> - <model usable='yes' vendor='unknown'>z114</model> - <model usable='yes' vendor='unknown'>z890.3</model> - <model usable='yes' vendor='unknown'>z196-base</model> - <model usable='yes' vendor='unknown'>z9EC.2-base</model> - <model usable='yes' vendor='unknown'>z196.2</model> - <model usable='yes' vendor='unknown'>z14.2</model> - <model usable='yes' vendor='unknown'>z990-base</model> - <model usable='yes' vendor='unknown'>z900.2</model> - <model usable='yes' vendor='unknown'>z890-base</model> - <model usable='yes' vendor='unknown'>z10EC.3</model> - <model usable='yes' vendor='unknown'>z14-base</model> - <model usable='yes' vendor='unknown'>z990.4-base</model> - <model usable='yes' vendor='unknown'>z10EC.3-base</model> - <model usable='yes' vendor='unknown'>z10BC-base</model> - <model usable='yes' vendor='unknown'>z13-base</model> - <model usable='yes' vendor='unknown'>z990.3-base</model> - <model usable='yes' vendor='unknown'>zEC12.2-base</model> - <model usable='yes' vendor='unknown'>zBC12</model> - <model usable='yes' vendor='unknown'>z890.3-base</model> - <model usable='yes' vendor='unknown'>z990.5</model> - <model usable='yes' vendor='unknown'>gen15b</model> + <model usable='yes' vendor='IBM'>z10EC.2-base</model> + <model usable='yes' vendor='IBM'>gen15a-base</model> + <model usable='yes' vendor='IBM'>z800</model> + <model usable='yes' vendor='IBM'>z10EC</model> + <model usable='yes' vendor='IBM'>zEC12.2</model> + <model usable='yes' vendor='IBM'>z990.2-base</model> + <model usable='yes' vendor='IBM'>z900-base</model> + <model usable='yes' vendor='IBM'>z10BC.2</model> + <model usable='yes' vendor='IBM'>z9EC-base</model> + <model usable='yes' vendor='IBM'>z9EC.3-base</model> + <model usable='yes' vendor='IBM'>z114</model> + <model usable='yes' vendor='IBM'>z890.3</model> + <model usable='yes' vendor='IBM'>z196-base</model> + <model usable='yes' vendor='IBM'>z9EC.2-base</model> + <model usable='yes' vendor='IBM'>z196.2</model> + <model usable='yes' vendor='IBM'>z14.2</model> + <model usable='yes' vendor='IBM'>z990-base</model> + <model usable='yes' vendor='IBM'>z900.2</model> + <model usable='yes' vendor='IBM'>z890-base</model> + <model usable='yes' vendor='IBM'>z10EC.3</model> + <model usable='yes' vendor='IBM'>z14-base</model> + <model usable='yes' vendor='IBM'>z990.4-base</model> + <model usable='yes' vendor='IBM'>z10EC.3-base</model> + <model usable='yes' vendor='IBM'>z10BC-base</model> + <model usable='yes' vendor='IBM'>z13-base</model> + <model usable='yes' vendor='IBM'>z990.3-base</model> + <model usable='yes' vendor='IBM'>zEC12.2-base</model> + <model usable='yes' vendor='IBM'>zBC12</model> + <model usable='yes' vendor='IBM'>z890.3-base</model> + <model usable='yes' vendor='IBM'>z990.5</model> + <model usable='yes' vendor='IBM'>gen15b</model> <model usable='yes' vendor='unknown'>qemu</model> </mode> </cpu> -- 2.31.1

2 2

[PATCH V2 0/4] migration: add qemu parallel migration options
by Jiang Jiacheng 09 Feb '23

09 Feb '23

Add qemu parallel migration options to set multifd-compression multifd-zlib-level and multifd-zstd-level. These parameters has been supported by QEMU since 5.0. v2 of: https://listman.redhat.com/archives/libvir-list/2023-January/237088.html diff to v1: * remove VIR_MIGRATE_PARAM_PARALLEL_COMPRESSION and related message * support to reuse VIR_MIGRATE_PARAM_COMPRESSION to set parallel migration compress method * update commit message Jiang Jiacheng (4): Add public API for parallel compression method qemu: Add qemu parallel migration parameters qemu: support set parallel migration compression method virsh: Add migrate options to set parallel compress level docs/manpages/virsh.rst | 22 ++++--- include/libvirt/libvirt-domain.h | 24 +++++++- src/qemu/qemu_migration.h | 2 + src/qemu/qemu_migration_params.c | 100 ++++++++++++++++++++++++++++++- src/qemu/qemu_migration_params.h | 3 + tools/virsh-domain.c | 26 ++++++++ 6 files changed, 166 insertions(+), 11 deletions(-) -- 2.33.0

3 14

[libvirt PATCH v4 00/31] Use nbdkit for http/ftp/ssh network drives in libvirt
by Jonathon Jongsma 07 Feb '23

07 Feb '23

This is the fourth version of this patch series. See https://bugzilla.redhat.com/show_bug.cgi?id=2016527 for more information about the goal, but the summary is that RHEL does not want to ship the qemu storage plugins for curl and ssh. Handling them outside of the qemu process provides several advantages such as reduced attack surface and stability. See previous series for more info: https://listman.redhat.com/archives/libvir-list/2022-October/235052.html Note that gitlab CI will not work for this series without changes to the ci definitions due to the addition of libnbd dependency. Changes in v4: - Added new schema that makes ssh disks actually useable with nbdkit. - supports authentication with password or ssh key - enable both http and https protocols together - improve logging and error reporting - adds a dependency on libnbd to validate the storage before launching qemu - nbdkit output logged to a separate file - add missing support for hotplug - lots of smaller changes from Peter's review Jonathon Jongsma (31): schema: allow 'ssh' as a protocol for network disks qemu: Add functions for determining nbdkit availability qemu: expand nbdkit capabilities util: Allow virFileCache data to be any GObject qemu: implement basic virFileCache for nbdkit caps qemu: implement persistent file cache for nbdkit caps qemu: use file cache for nbdkit caps qemu: Add qemuNbdkitProcess qemu: query nbdkit module dir from binary qemu: add functions to start and stop nbdkit qemu: remove unused 'mode' param from qemuDomainLogContextNew() Generalize qemuDomainLogContextNew() qemu: Extract qemuDomainLogContext into a new file qemu: move qemuProcessReadLog() to qemuLogContext qemu: log error output from nbdkit tests: add ability to test various nbdkit capabilities qemu: split qemuDomainSecretStorageSourcePrepare qemu: include nbdkit state in private xml qemu: pass sensitive data to nbdkit via pipe qemu: use nbdkit to serve network disks if available util: make virCommandSetSendBuffer testable tests: add tests for nbdkit invocation qemu: add test for authenticating a https network disk qemu: Monitor nbdkit process for exit qemu: try to connect to nbdkit early to detect errors schema: add password configuration for ssh disk qemu: implement password auth for ssh disks with nbdkit schema: add configuration for host verification of ssh disks qemu: implement knownHosts for ssh disks with nbdkit schema: add keyfile configuration for ssh disks qemu: implement keyfile auth for ssh disk with nbdkit build-aux/syntax-check.mk | 2 +- docs/formatdomain.rst | 41 +- meson.build | 14 + meson_options.txt | 1 + po/POTFILES | 2 + src/conf/domain_conf.c | 32 + src/conf/schemas/domaincommon.rng | 53 + src/conf/storage_source_conf.c | 3 + src/conf/storage_source_conf.h | 6 +- src/libvirt_private.syms | 1 + src/qemu/meson.build | 3 + src/qemu/qemu_block.c | 162 +- src/qemu/qemu_conf.c | 22 + src/qemu/qemu_conf.h | 6 + src/qemu/qemu_domain.c | 415 ++---- src/qemu/qemu_domain.h | 39 +- src/qemu/qemu_driver.c | 3 + src/qemu/qemu_extdevice.c | 56 + src/qemu/qemu_hotplug.c | 7 + src/qemu/qemu_logcontext.c | 329 ++++ src/qemu/qemu_logcontext.h | 41 + src/qemu/qemu_nbdkit.c | 1326 +++++++++++++++++ src/qemu/qemu_nbdkit.h | 116 ++ src/qemu/qemu_nbdkitpriv.h | 31 + src/qemu/qemu_process.c | 119 +- src/util/vircommand.c | 17 +- src/util/vircommand.h | 8 + src/util/vircommandpriv.h | 4 + src/util/virfilecache.c | 14 +- src/util/virfilecache.h | 2 +- tests/meson.build | 1 + tests/qemublocktest.c | 2 +- ...w2-invalid.json => network-ssh-qcow2.json} | 0 ...cow2-invalid.xml => network-ssh-qcow2.xml} | 0 .../disk-cdrom-network.args.disk0 | 6 + .../disk-cdrom-network.args.disk1 | 8 + .../disk-cdrom-network.args.disk1.pipe.778 | 1 + .../disk-cdrom-network.args.disk2 | 8 + .../disk-cdrom-network.args.disk2.pipe.780 | 1 + .../disk-network-http.args.disk0 | 6 + .../disk-network-http.args.disk1 | 5 + .../disk-network-http.args.disk2 | 6 + .../disk-network-http.args.disk2.pipe.778 | 1 + .../disk-network-http.args.disk3 | 7 + .../disk-network-http.args.disk3.pipe.780 | 1 + ...work-source-curl-nbdkit-backing.args.disk0 | 7 + ...ce-curl-nbdkit-backing.args.disk0.pipe.778 | 1 + .../disk-network-source-curl.args.disk0 | 7 + ...sk-network-source-curl.args.disk0.pipe.778 | 1 + .../disk-network-source-curl.args.disk1 | 9 + ...sk-network-source-curl.args.disk1.pipe.780 | 1 + ...sk-network-source-curl.args.disk1.pipe.782 | 1 + .../disk-network-source-curl.args.disk2 | 7 + ...sk-network-source-curl.args.disk2.pipe.782 | 1 + ...sk-network-source-curl.args.disk2.pipe.784 | 1 + .../disk-network-source-curl.args.disk3 | 6 + .../disk-network-source-curl.args.disk4 | 6 + .../disk-network-ssh-key.args.disk0 | 10 + .../disk-network-ssh-password.args.disk0 | 9 + ...k-network-ssh-password.args.disk0.pipe.778 | 1 + .../disk-network-ssh.args.disk0 | 7 + .../disk-network-ssh.args.disk1 | 8 + .../disk-network-ssh.args.disk1.pipe.778 | 1 + .../disk-network-ssh.args.disk2 | 9 + tests/qemunbdkittest.c | 302 ++++ tests/qemustatusxml2xmldata/modern-in.xml | 4 + ...sk-cdrom-network-nbdkit.x86_64-latest.args | 42 + .../disk-cdrom-network-nbdkit.xml | 1 + ...isk-network-http-nbdkit.x86_64-latest.args | 45 + .../disk-network-http-nbdkit.xml | 1 + ...rce-curl-nbdkit-backing.x86_64-latest.args | 38 + ...isk-network-source-curl-nbdkit-backing.xml | 45 + ...work-source-curl-nbdkit.x86_64-latest.args | 50 + .../disk-network-source-curl-nbdkit.xml | 1 + ...isk-network-source-curl.x86_64-latest.args | 54 + .../disk-network-source-curl.xml | 74 + .../qemuxml2argvdata/disk-network-ssh-key.xml | 33 + ...disk-network-ssh-nbdkit.x86_64-latest.args | 36 + .../disk-network-ssh-nbdkit.xml | 1 + ...sk-network-ssh-password.x86_64-latest.args | 36 + .../disk-network-ssh-password.xml | 35 + .../disk-network-ssh.x86_64-latest.args | 36 + tests/qemuxml2argvdata/disk-network-ssh.xml | 32 + tests/qemuxml2argvtest.c | 19 + tests/testutilsqemu.c | 27 + tests/testutilsqemu.h | 5 + 86 files changed, 3463 insertions(+), 475 deletions(-) create mode 100644 src/qemu/qemu_logcontext.c create mode 100644 src/qemu/qemu_logcontext.h create mode 100644 src/qemu/qemu_nbdkit.c create mode 100644 src/qemu/qemu_nbdkit.h create mode 100644 src/qemu/qemu_nbdkitpriv.h rename tests/qemublocktestdata/imagecreate/{network-ssh-qcow2-invalid.json => network-ssh-qcow2.json} (100%) rename tests/qemublocktestdata/imagecreate/{network-ssh-qcow2-invalid.xml => network-ssh-qcow2.xml} (100%) create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk1.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk2 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk2.pipe.780 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk2 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk2.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk3 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk3.pipe.780 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.disk0.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk0.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.780 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.782 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.782 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.784 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk3 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk4 create mode 100644 tests/qemunbdkitdata/disk-network-ssh-key.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-ssh-password.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-ssh-password.args.disk0.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk1.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk2 create mode 100644 tests/qemunbdkittest.c create mode 100644 tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-http-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-network-http-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl.xml create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-key.xml create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-network-ssh-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-password.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-password.xml create mode 100644 tests/qemuxml2argvdata/disk-network-ssh.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-ssh.xml -- 2.39.0

2 46

[PATCH v3 0/5] logging: add log cleanup for obsolete domains
by Oleg Vasilev 06 Feb '23

06 Feb '23

Presently, logs from deleted domains remain forever. Particular motivation comes from the case when libguestfs has repeatedly created transient VMs, which in turn created plenty of logs. This takes up space and lots of files troubles filesystem navigation. More motivation in [1]. Patch solving same problem in [2]. Changes in v3: codestyle cleanup, minor fixes Changes in v2: substantial rework according to Martin Kletzander's comments v1: https://www.mail-archive.com/libvir-list@redhat.com/msg233754.html v2: https://www.spinics.net/linux/fedora/libvir/msg236081.html [1]: https://listman.redhat.com/archives/libvir-list/2022-February/228149.html [2]: https://listman.redhat.com/archives/libvir-list/2022-February/msg00865.html CC: Martin Kletzander <mkletzan(a)redhat.com> Oleg Vasilev (5): logging: refactor to store config inside log handler logging: move virLogHandler to header logging: add configuration for future log cleaner logging: add log cleanup for obsolete domains logging: use the log cleaner po/POTFILES | 1 + src/logging/log_cleaner.c | 268 +++++++++++++++++++++++++++++++ src/logging/log_cleaner.h | 29 ++++ src/logging/log_daemon.c | 6 +- src/logging/log_daemon_config.c | 9 ++ src/logging/log_daemon_config.h | 3 + src/logging/log_handler.c | 64 +++----- src/logging/log_handler.h | 50 ++++-- src/logging/meson.build | 1 + src/logging/test_virtlogd.aug.in | 2 + src/logging/virtlogd.aug | 2 + src/logging/virtlogd.conf | 14 ++ 12 files changed, 391 insertions(+), 58 deletions(-) create mode 100644 src/logging/log_cleaner.c create mode 100644 src/logging/log_cleaner.h -- 2.39.1

3 10

[PATCH 0/3] qemu: Fix setting TPM state seclabels wrt save/restore
by Michal Privoznik 06 Feb '23

06 Feb '23

*** BLURB HERE *** Michal Prívozník (3): qemuProcessStop: Fix detection of outgoing migration for external devices qemuExtTPMStop: Restore TPM state label more often qemuProcessLaunch: Tighten rules for external devices wrt incoming migration src/qemu/qemu_process.c | 11 +++++++++-- src/qemu/qemu_tpm.c | 2 +- 2 files changed, 10 insertions(+), 3 deletions(-) -- 2.39.1

3 5

[libvirt PATCH v2 0/8] Extract the integration job commands to a shell script
by Erik Skultety 06 Feb '23

06 Feb '23

Using shell scripts rather than inlining shell commands to YAML feels more natural, more readable, and will keep all different variations of execution consistent. Essentially the only disadvantage is that we won't see each command listed one-by-one in gitlab's log output (unless we set -x that is), but given that shell would complain if something was wrong with the script, it's fairly easy to identify the problem. Here's a test pipeline after the change: https://gitlab.com/eskultety/libvirt/-/pipelines/759277200 Since v1: - 3/7 - reworded commit message as requested - 4/7 was dropped - point the SCRATCH_DIR to /var/tmp instead of /tmp to not be limited by the size of ramdisk mounted in there Erik Skultety (8): syntax-check: Drop the shell's 'check for minus' rule ci: Move the SCRATCH_DIR from /tmp ci: integration: Extract several hidden job definitions to a script ci: integration: Drop the 'install-deps' hidden job and reference ci: integration-template: Drop the '-lt Fedora 35' check ci: integration.sh: Add/Rewrite/Reformat commentaries ci: integration.sh: Replace 'test' with '[' operator ci: integration.sh: Define the SCRATCH_DIR variable for local execution build-aux/syntax-check.mk | 9 -------- ci/integration-template.yml | 44 +++-------------------------------- ci/integration.sh | 46 +++++++++++++++++++++++++++++++++++++ 3 files changed, 49 insertions(+), 50 deletions(-) create mode 100644 ci/integration.sh -- 2.39.1

1 9

[PATCH 0/7] various cleanups
by Peter Krempa 01 Feb '23

01 Feb '23

Collection of cleanup patches from my previous series attempting to remove secrets clearing refactored so that it doesn't depend on the patches which were not accepted. Peter Krempa (7): virCryptoEncryptDataAESgnutls: Restructure control flow virStorageBackendISCSISetAuth: Refactor cleanup virStorageBackendISCSISetAuth: Use g_strndup to '\0' terminate data virStorageBackendISCSIDirectSetAuth: Refactor cleanup virStorageBackendISCSIDirectSetAuth: Use 'g_strndup' to '\0' terminate data libxlMakeNetworkDiskSrc: Refactor cleanup storageBackendCreateQemuImgSecretPath: Refactor cleanup src/libxl/libxl_conf.c | 17 ++++------ src/storage/storage_backend_iscsi.c | 17 ++++------ src/storage/storage_backend_iscsi_direct.c | 23 ++++++-------- src/storage/storage_util.c | 36 ++++++++++------------ src/util/vircrypto.c | 28 ++++++++--------- 5 files changed, 49 insertions(+), 72 deletions(-) -- 2.39.1

2 12

[PATCH 0/7] virDomainNetDefFormat: Modernize XML formatting
by Michal Privoznik 31 Jan '23

31 Jan '23

A bit. After these, the function is still long and could be broken into smaller ones, but let's leave that as an exercise for future us. Michal Prívozník (7): qemuxml2xmloutdata: Turn net-mtu.xml into a symlink virDomainNetDefFormat: Rename @attrBuf to @targetAttrBuf virDomainNetDefFormat: Modernize <tune/> formatting virDomainNetDefFormat: Modernize <guest/> formatting virDomainNetDefFormat: Modernize <source/> formatting virDomainNetDefFormat: Simplify @sourceAttrBuf handling for some types of VIR_DOMAIN_NET virDomainNetDefFormat: Modernize <mac/> formatting src/conf/domain_conf.c | 151 ++++++++++----------------- tests/qemuxml2argvdata/net-mtu.xml | 26 +++-- tests/qemuxml2xmloutdata/net-mtu.xml | 72 +------------ 3 files changed, 74 insertions(+), 175 deletions(-) mode change 100644 => 120000 tests/qemuxml2xmloutdata/net-mtu.xml -- 2.39.1

2 8

[PATCH 0/7] qemu: Fix hot-(un-)plug of disks with FD-passed storage
by Peter Krempa 31 Jan '23

31 Jan '23

Peter Krempa (7): qemu_fd: Remove declaration for 'qemuFDPassNewDirect' qemuStorageSourcePrivateDataFormat: Rename 'tmp' to 'objectsChildBuf' qemu: command: Handle FD passing commandline via qemuBuildBlockStorageSourceAttachDataCommandline qemuFDPassTransferCommand: Mark that FD was passed qemu: fd: Add helpers allowing storing FD set data in status XML qemu: domain: Store fdset ID for disks passed to qemu via FD qemu: block: Properly handle FD-passed disk hot-(un-)plug src/qemu/qemu_block.c | 7 +++ src/qemu/qemu_block.h | 2 + src/qemu/qemu_command.c | 26 ++--------- src/qemu/qemu_domain.c | 31 +++++++++---- src/qemu/qemu_fd.c | 43 +++++++++++++++++++ src/qemu/qemu_fd.h | 8 +++- tests/qemustatusxml2xmldata/modern-in.xml | 3 ++ .../disk-source-fd.x86_64-latest.args | 6 +-- 8 files changed, 91 insertions(+), 35 deletions(-) -- 2.39.1

2 9

[RFC PATCH] build: deprecate --enable-gprof builds and remove from CI
by Alex Bennée 31 Jan '23

31 Jan '23

As gprof relies on instrumentation you rarely get useful data compared to a real optimised build. Lets deprecate the build option and simplify the CI configuration as a result. Signed-off-by: Alex Bennée <alex.bennee(a)linaro.org> Cc: Thomas Huth <thuth(a)redhat.com> --- docs/about/deprecated.rst | 14 ++++++++++++++ meson.build | 7 ++++++- .gitlab-ci.d/buildtest.yml | 19 ++++--------------- meson_options.txt | 3 ++- 4 files changed, 26 insertions(+), 17 deletions(-) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 9f1bbc495d..87b4511535 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -20,6 +20,20 @@ they were first deprecated in the 2.10.0 release. What follows is a list of all features currently marked as deprecated. +Build options +------------- + +``gprof`` builds (since 8.0) +'''''''''''''''''''''''''''' + +The ``--enable-gprof`` configure setting relies on compiler +instrumentation to gather its data which can distort the generated +profile. As other non-instrumenting tools are available that give a +more holistic view of the system with non-instrumented binaries we are +deprecating the build option and no longer defend it in CI. The +``--enable-gcov`` build option remains for analysis test case +coverage. + System emulator command line arguments -------------------------------------- diff --git a/meson.build b/meson.build index 56320ae717..ecf44ea534 100644 --- a/meson.build +++ b/meson.build @@ -3791,7 +3791,12 @@ summary_info += {'mutex debugging': get_option('debug_mutex')} summary_info += {'memory allocator': get_option('malloc')} summary_info += {'avx2 optimization': config_host_data.get('CONFIG_AVX2_OPT')} summary_info += {'avx512f optimization': config_host_data.get('CONFIG_AVX512F_OPT')} -summary_info += {'gprof enabled': get_option('gprof')} +if get_option('gprof') + gprof_info = 'YES (deprecated)' +else + gprof_info = get_option('gprof') +endif +summary_info += {'gprof': gprof_info} summary_info += {'gcov': get_option('b_coverage')} summary_info += {'thread sanitizer': config_host.has_key('CONFIG_TSAN')} summary_info += {'CFI support': get_option('cfi')} diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml index 9a6ba1fe3b..dc8fa0a498 100644 --- a/.gitlab-ci.d/buildtest.yml +++ b/.gitlab-ci.d/buildtest.yml @@ -468,27 +468,16 @@ tsan-build: TARGETS: x86_64-softmmu ppc64-softmmu riscv64-softmmu x86_64-linux-user MAKE_CHECK_ARGS: bench V=1 -# gprof/gcov are GCC features -build-gprof-gcov: +# gcov is a GCC features +gcov: extends: .native_build_job_template needs: job: amd64-ubuntu2004-container + timeout: 80m variables: IMAGE: ubuntu2004 - CONFIGURE_ARGS: --enable-gprof --enable-gcov + CONFIGURE_ARGS: --enable-gcov TARGETS: aarch64-softmmu ppc64-softmmu s390x-softmmu x86_64-softmmu - artifacts: - expire_in: 1 days - paths: - - build - -check-gprof-gcov: - extends: .native_test_job_template - needs: - - job: build-gprof-gcov - artifacts: true - variables: - IMAGE: ubuntu2004 MAKE_CHECK_ARGS: check after_script: - cd build diff --git a/meson_options.txt b/meson_options.txt index 559a571b6b..53459c15fc 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -316,7 +316,8 @@ option('debug_stack_usage', type: 'boolean', value: false, option('qom_cast_debug', type: 'boolean', value: false, description: 'cast debugging support') option('gprof', type: 'boolean', value: false, - description: 'QEMU profiling with gprof') + description: 'QEMU profiling with gprof', + deprecated: true) option('profiler', type: 'boolean', value: false, description: 'profiler support') option('slirp_smbd', type : 'feature', value : 'auto', -- 2.34.1

3 3

Zoned storage support in libvirt
by Stefan Hajnoczi 30 Jan '23

30 Jan '23

Hi Peter, Zoned storage support (https://zonedstorage.io/docs/introduction/zoned-storage) is being added to QEMU. Given a zoned host block device, the QEMU syntax will look like this: --blockdev zoned_host_device,node-name=drive0,filename=/dev/$BDEV,... --device virtio-blk-pci,drive=drive0 Note that regular --blockdev host_device will not work. For now the virtio-blk device is the only one that supports zoned blockdevs. This brings to mind a few questions: 1. Does libvirt need domain XML syntax for zoned storage? Alternatively, it could probe /sys/block/$BDEV/queue/zoned and generate the correct QEMU command-line arguments for zoned devices when the contents of the file are not "none". 2. Should QEMU --blockdev host_device detected zoned devices so that --blockdev zoned_host_device is not necessary? That way libvirt would automatically support zoned storage without any domain XML syntax or libvirt code changes. The drawbacks I see when QEMU detects zoned storage automatically: - You can't easiy tell if a blockdev is zoned from the command-line. - It's possible to mismatch zoned and non-zoned devices across live migration. We still have time to decide on the QEMU command-line syntax for QEMU 8.0, so I wanted to raise this now. Thanks, Stefan

3 6