March 2022 - Devel - libvirt List Archives

[PATCH v2 0/8] softmmu: move and refactor -runas, -chroot and -daemonize
by Daniel P. Berrangé 04 Mar '22

04 Mar '22

This small series was motivated by my thoughts on the proposals in https://lists.gnu.org/archive/html/qemu-devel/2022-03/msg01135.html It demonstrates the approach I mention there, and has the further benefit of untangling and isolating the implementation of UID changing, chrooting and daemonizing, from the parsing of the corresponding command line options. Changed in v2: - Fix mistake that left stderr open when daemonizing - Remove use of is_daemonized from chardev - Remove use of is_daemonized from logging - Eliminate is_daemonized function Daniel P. Berrangé (8): softmmu: remove deprecated --enable-fips option os-posix: refactor code handling the -runas argument os-posix: refactor code handling the -chroot argument util: remove use of is_daemonized flag from logging code softmmu: refactor use of is_daemonized() method chardev: add API to block use of the stdio implementation softmmu: move parsing of -runas, -chroot and -daemonize code softmmu: remove is_daemonized() method chardev/char-stdio.c | 12 +- docs/about/deprecated.rst | 12 -- docs/about/removed-features.rst | 11 ++ include/chardev/char-stdio.h | 29 ++++ include/qemu/log.h | 1 + include/qemu/osdep.h | 3 - include/sysemu/os-posix.h | 6 +- include/sysemu/os-win32.h | 6 - os-posix.c | 227 ++++++++++---------------------- os-win32.c | 9 -- qemu-options.hx | 10 -- softmmu/vl.c | 92 +++++++++++-- stubs/is-daemonized.c | 9 -- stubs/meson.build | 1 - ui/vnc.c | 7 - util/log.c | 12 +- util/osdep.c | 28 ---- 17 files changed, 213 insertions(+), 262 deletions(-) create mode 100644 include/chardev/char-stdio.h delete mode 100644 stubs/is-daemonized.c -- 2.34.1

2 13

[PATCH 0/4] softmmu: move and refactor -runas, -chroot and -daemonize
by Daniel P. Berrangé 04 Mar '22

04 Mar '22

This small series was motivated by my thoughts on the proposals in https://lists.gnu.org/archive/html/qemu-devel/2022-03/msg01135.html It demostrates the approach I mention there, and has the further benefit of untangling and isolating the implementation of UID changing, chrooting and daemonized, from the parsing of the corresponding command line options. Daniel P. Berrangé (4): softmmu: remove deprecated --enable-fips option os-posix: refactor code handling the -runas argument os-posix: refactor code handling the -chroot argument softmmu: move parsing of -runas, -chroot and -daemonize code docs/about/deprecated.rst | 12 -- docs/about/removed-features.rst | 11 ++ include/qemu/osdep.h | 3 - include/sysemu/os-posix.h | 4 +- include/sysemu/os-win32.h | 1 - os-posix.c | 222 ++++++++++---------------------- os-win32.c | 9 -- qemu-options.hx | 10 -- softmmu/vl.c | 76 ++++++++++- ui/vnc.c | 7 - util/osdep.c | 28 ---- 11 files changed, 154 insertions(+), 229 deletions(-) -- 2.34.1

3 10

[libvirt PATCH v2] qemu: support multiqueue for vdpa net device
by Jonathon Jongsma 04 Mar '22

04 Mar '22

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2024406 Signed-off-by: Jonathon Jongsma <jjongsma(a)redhat.com> --- Changes in v2: - remove error message in virDomainNetDefParseXML(). The model is already checked in the post parse validation step. src/conf/domain_conf.c | 2 ++ src/qemu/qemu_domain.c | 3 +- .../net-vdpa-multiqueue.x86_64-latest.args | 36 +++++++++++++++++++ .../qemuxml2argvdata/net-vdpa-multiqueue.xml | 29 +++++++++++++++ tests/qemuxml2argvtest.c | 1 + .../net-vdpa-multiqueue.xml | 36 +++++++++++++++++++ tests/qemuxml2xmltest.c | 1 + 7 files changed, 107 insertions(+), 1 deletion(-) create mode 100644 tests/qemuxml2argvdata/net-vdpa-multiqueue.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/net-vdpa-multiqueue.xml create mode 100644 tests/qemuxml2xmloutdata/net-vdpa-multiqueue.xml diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 34fec887a3..9634c9256e 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -10552,6 +10552,8 @@ virDomainNetDefParseXML(virDomainXMLOption *xmlopt, goto error; } def->data.vdpa.devicepath = g_steal_pointer(&dev); + if (!def->model) + def->model = VIR_DOMAIN_NET_MODEL_VIRTIO; break; case VIR_DOMAIN_NET_TYPE_BRIDGE: diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index acc76c1cd6..6b61fefb8f 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -4523,7 +4523,8 @@ qemuDomainValidateActualNetDef(const virDomainNetDef *net, actualType == VIR_DOMAIN_NET_TYPE_BRIDGE || actualType == VIR_DOMAIN_NET_TYPE_DIRECT || actualType == VIR_DOMAIN_NET_TYPE_ETHERNET || - actualType == VIR_DOMAIN_NET_TYPE_VHOSTUSER)) { + actualType == VIR_DOMAIN_NET_TYPE_VHOSTUSER || + actualType == VIR_DOMAIN_NET_TYPE_VDPA)) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("interface %s - multiqueue is not supported for network interfaces of type %s"), macstr, virDomainNetTypeToString(actualType)); diff --git a/tests/qemuxml2argvdata/net-vdpa-multiqueue.x86_64-latest.args b/tests/qemuxml2argvdata/net-vdpa-multiqueue.x86_64-latest.args new file mode 100644 index 0000000000..26ef666036 --- /dev/null +++ b/tests/qemuxml2argvdata/net-vdpa-multiqueue.x86_64-latest.args @@ -0,0 +1,36 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/tmp/lib/domain--1-QEMUGuest1 \ +USER=test \ +LOGNAME=test \ +XDG_DATA_HOME=/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=QEMUGuest1,debug-threads=on \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tmp/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=off,dump-guest-core=off,memory-backend=pc.ram \ +-accel tcg \ +-cpu qemu64 \ +-m 214 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}' \ +-overcommit mem-lock=off \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,fd=1729,server=on,wait=off \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-no-acpi \ +-boot strict=on \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0x2"}' \ +-add-fd set=0,fd=1732,opaque=net0-vdpa \ +-netdev vhost-vdpa,vhostdev=/dev/fdset/0,id=hostnet0 \ +-device '{"driver":"virtio-net-pci","mq":true,"vectors":6,"netdev":"hostnet0","id":"net0","mac":"52:54:00:95:db:c0","bus":"pci.0","addr":"0x2"}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ +-msg timestamp=on diff --git a/tests/qemuxml2argvdata/net-vdpa-multiqueue.xml b/tests/qemuxml2argvdata/net-vdpa-multiqueue.xml new file mode 100644 index 0000000000..2e38c6f976 --- /dev/null +++ b/tests/qemuxml2argvdata/net-vdpa-multiqueue.xml @@ -0,0 +1,29 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='x86_64' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-x86_64</emulator> + <controller type='usb' index='0'/> + <controller type='ide' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <interface type='vdpa'> + <mac address='52:54:00:95:db:c0'/> + <source dev='/dev/vhost-vdpa-0'/> + <driver queues='2'/> + </interface> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='none'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index ce475df466..7e1167e60e 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1609,6 +1609,7 @@ mymain(void) DO_TEST_FAILURE("net-hostdev-fail", QEMU_CAPS_DEVICE_VFIO_PCI); DO_TEST_CAPS_LATEST("net-vdpa"); + DO_TEST_CAPS_LATEST("net-vdpa-multiqueue"); DO_TEST("hostdev-pci-multifunction", QEMU_CAPS_KVM, diff --git a/tests/qemuxml2xmloutdata/net-vdpa-multiqueue.xml b/tests/qemuxml2xmloutdata/net-vdpa-multiqueue.xml new file mode 100644 index 0000000000..0876d5df62 --- /dev/null +++ b/tests/qemuxml2xmloutdata/net-vdpa-multiqueue.xml @@ -0,0 +1,36 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='x86_64' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-x86_64</emulator> + <controller type='usb' index='0'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> + </controller> + <controller type='ide' index='0'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> + </controller> + <controller type='pci' index='0' model='pci-root'/> + <interface type='vdpa'> + <mac address='52:54:00:95:db:c0'/> + <source dev='/dev/vhost-vdpa-0'/> + <model type='virtio'/> + <driver queues='2'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </interface> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <audio id='1' type='none'/> + <memballoon model='none'/> + </devices> +</domain> diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index 052950b86f..2174965784 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -485,6 +485,7 @@ mymain(void) DO_TEST_NOCAPS("net-coalesce"); DO_TEST_NOCAPS("net-many-models"); DO_TEST("net-vdpa", QEMU_CAPS_NETDEV_VHOST_VDPA); + DO_TEST("net-vdpa-multiqueue", QEMU_CAPS_NETDEV_VHOST_VDPA); DO_TEST_NOCAPS("serial-tcp-tlsx509-chardev"); DO_TEST_NOCAPS("serial-tcp-tlsx509-chardev-notls"); -- 2.35.1

2 1

[PULL 25/45] hw/i386/pc_piix: Mark the machine types from version 1.4 to 1.7 as deprecated
by Michael S. Tsirkin 04 Mar '22

04 Mar '22

From: Thomas Huth <thuth(a)redhat.com> The list of machine types grows larger and larger each release ... and it is unlikely that many people still use the very old ones for live migration. QEMU v1.7 has been released more than 8 years ago, so most people should have updated their machines to a newer version in those 8 years at least once. Thus let's mark the very old 1.x machine types as deprecated now. Signed-off-by: Thomas Huth <thuth(a)redhat.com> Message-Id: <20220117191639.278497-1-thuth(a)redhat.com> Reviewed-by: Michael S. Tsirkin <mst(a)redhat.com> Signed-off-by: Michael S. Tsirkin <mst(a)redhat.com> --- hw/i386/pc_piix.c | 1 + docs/about/deprecated.rst | 8 ++++++++ 2 files changed, 9 insertions(+) diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c index 8d33cf689d..b72c03d0a6 100644 --- a/hw/i386/pc_piix.c +++ b/hw/i386/pc_piix.c @@ -757,6 +757,7 @@ static void pc_i440fx_1_7_machine_options(MachineClass *m) m->hw_version = "1.7.0"; m->default_machine_opts = NULL; m->option_rom_has_mr = true; + m->deprecation_reason = "old and unattended - use a newer version instead"; compat_props_add(m->compat_props, pc_compat_1_7, pc_compat_1_7_len); pcmc->smbios_defaults = false; pcmc->gigabyte_align = false; diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 85773db631..cf02ef6821 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -324,6 +324,14 @@ machine is hardly emulated at all (e.g. neither the LCD nor the USB part had been implemented), so there is not much value added by this board. Use the ``ref405ep`` machine instead. +``pc-i440fx-1.4`` up to ``pc-i440fx-1.7`` (since 7.0) +''''''''''''''''''''''''''''''''''''''''''''''''''''' + +These old machine types are quite neglected nowadays and thus might have +various pitfalls with regards to live migration. Use a newer machine type +instead. + + Backend options --------------- -- MST

1 0

[PATCH] rpc: Fix memory leak of fds
by Peng Liang 04 Mar '22

04 Mar '22

In virSystemdActivationClaimFDs, the memory of ent->fds has been stolen and stored in fds, but fds is never freed, which causes a memory leak. Fix it by declaring fds as g_autofree. Reported-by: Jie Tang <tangjie18(a)huawei.com> Signed-off-by: Peng Liang <liangpeng10(a)huawei.com> --- src/rpc/virnetserver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/rpc/virnetserver.c b/src/rpc/virnetserver.c index ad581a36ddf3..d9430a2cfa76 100644 --- a/src/rpc/virnetserver.c +++ b/src/rpc/virnetserver.c @@ -655,7 +655,7 @@ virNetServerAddServiceActivation(virNetServer *srv, size_t max_queued_clients, size_t nrequests_client_max) { - int *fds; + g_autofree int *fds = NULL; size_t nfds; if (act == NULL) -- 2.35.1

2 1

[PATCH 0/4] tests: Miscellaneous fixes and improvements
by Michal Privoznik 04 Mar '22

04 Mar '22

I've been trying to compile libvirt with musl lately and seen couple of tests failing. Some of them were related strictly to how musl works, but the rest are genuine bugs we have and it's by pure chance that we haven't met them with glibc (1/4 for instance). Green pipeline can be seen here: https://gitlab.com/MichalPrivoznik/libvirt/-/pipelines/483864407 Michal Prívozník (4): vircgroupmock: Be wiser about detecting fakerootdir change vircgroupmock: Make global variables static sockettest: Check for IPv4-in-IPv6 parsing and formatting tests: Update IPv4-in-IPv6 addresses tests/nwfilterxml2firewalldata/ah-ipv6-linux.args | 6 +++--- tests/nwfilterxml2firewalldata/ah-ipv6.xml | 2 +- .../nwfilterxml2firewalldata/all-ipv6-linux.args | 6 +++--- tests/nwfilterxml2firewalldata/all-ipv6.xml | 2 +- tests/nwfilterxml2firewalldata/comment-linux.args | 4 ++-- tests/nwfilterxml2firewalldata/comment.xml | 4 ++-- .../nwfilterxml2firewalldata/esp-ipv6-linux.args | 6 +++--- tests/nwfilterxml2firewalldata/esp-ipv6.xml | 2 +- .../nwfilterxml2firewalldata/hex-data-linux.args | 4 ++-- tests/nwfilterxml2firewalldata/hex-data.xml | 4 ++-- tests/nwfilterxml2firewalldata/icmpv6-linux.args | 2 +- tests/nwfilterxml2firewalldata/icmpv6.xml | 2 +- tests/nwfilterxml2firewalldata/ipv6-linux.args | 4 ++-- tests/nwfilterxml2firewalldata/ipv6.xml | 4 ++-- .../nwfilterxml2firewalldata/sctp-ipv6-linux.args | 6 +++--- tests/nwfilterxml2firewalldata/sctp-ipv6.xml | 2 +- .../nwfilterxml2firewalldata/tcp-ipv6-linux.args | 6 +++--- tests/nwfilterxml2firewalldata/tcp-ipv6.xml | 2 +- .../nwfilterxml2firewalldata/udp-ipv6-linux.args | 6 +++--- tests/nwfilterxml2firewalldata/udp-ipv6.xml | 2 +- .../udplite-ipv6-linux.args | 6 +++--- tests/nwfilterxml2firewalldata/udplite-ipv6.xml | 2 +- tests/nwfilterxml2xmlin/ah-ipv6-test-invalid.xml | 2 +- tests/nwfilterxml2xmlin/all-ipv6-test-invalid.xml | 2 +- tests/nwfilterxml2xmlin/comment-test-invalid.xml | 4 ++-- tests/nwfilterxml2xmlin/esp-ipv6-test-invalid.xml | 2 +- tests/nwfilterxml2xmlin/hex-data-test-invalid.xml | 4 ++-- tests/nwfilterxml2xmlin/icmpv6-test-invalid.xml | 2 +- tests/nwfilterxml2xmlin/ipv6-test-invalid.xml | 4 ++-- .../nwfilterxml2xmlin/sctp-ipv6-test-invalid.xml | 2 +- tests/nwfilterxml2xmlin/tcp-ipv6-test-invalid.xml | 2 +- tests/nwfilterxml2xmlin/udp-ipv6-test-invalid.xml | 2 +- .../udplite-ipv6-test-invalid.xml | 2 +- tests/nwfilterxml2xmlout/ah-ipv6-test.xml | 2 +- tests/nwfilterxml2xmlout/all-ipv6-test.xml | 2 +- tests/nwfilterxml2xmlout/comment-test.xml | 2 +- tests/nwfilterxml2xmlout/esp-ipv6-test.xml | 2 +- tests/nwfilterxml2xmlout/hex-data-test.xml | 2 +- tests/nwfilterxml2xmlout/icmpv6-test.xml | 2 +- tests/nwfilterxml2xmlout/ipv6-test.xml | 2 +- tests/nwfilterxml2xmlout/sctp-ipv6-test.xml | 2 +- tests/nwfilterxml2xmlout/tcp-ipv6-test.xml | 2 +- tests/nwfilterxml2xmlout/udp-ipv6-test.xml | 2 +- tests/nwfilterxml2xmlout/udplite-ipv6-test.xml | 2 +- tests/sockettest.c | 2 ++ tests/vircgroupmock.c | 15 ++++++--------- 46 files changed, 75 insertions(+), 76 deletions(-) -- 2.34.1

2 6

[PATCH] deprecation: x86 default machine types
by Dr. David Alan Gilbert (git) 03 Mar '22

03 Mar '22

From: "Dr. David Alan Gilbert" <dgilbert(a)redhat.com> Declare the intent to require a machine type to be specified on x86 system emulation. Signed-off-by: Dr. David Alan Gilbert <dgilbert(a)redhat.com> --- docs/about/deprecated.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 85773db631..143c60d105 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -324,6 +324,14 @@ machine is hardly emulated at all (e.g. neither the LCD nor the USB part had been implemented), so there is not much value added by this board. Use the ``ref405ep`` machine instead. +x86 default machine type +'''''''''''''''''''''''' + +x86 currently defaults to the ```pc``` machine type which is based on the very +old ```i440fx``` chipset. This default will be removed and the user will be +required to specify a machine type explicitly using -M; users are encouraged to +switch to the not quite as old ```q35``` machine types. + Backend options --------------- -- 2.35.1

4 5

[libvirt PATCH 0/4] [RFC] Functional CI - GitLab enablement
by Erik Skultety 03 Mar '22

03 Mar '22

I'd like to propose a PoC of a GitLab-driven functional CI based on GitLab's custom executor runner feature. ** TL;DR ** - there's some RH owned HW that can be utilized for upstream functional CI running workloads on Red Hat-sponsored distros (see the pipeline) - the PoC utilizes GitLab's custom executor feature [1] which allows literally any infrastructure to be plugged into the CI machinery - the PoC solution (as proposed) is relying on nested virt capability and backed by plain libvirt + QEMU/KVM - the test environment comprises of the libvirt-TCK suite [2] and Avocado test framework [3] as the harness engine (as a wrapper and future replacement of the existing Perl implementation) - example pipeline (simplified to only a couple of distros) can be found here [4] - libvirt debug logs are exposed as artifacts and can be downloaded on job failure ** LONG VERSION ** This RFC is driven by GitLab's custom executor feature which can be integrated with any VMM and hence libvirt+QEMU/KVM as well. Example pipeline can be found here [4]. Architecture ============ Platform ~~~~~~~~ It doesn't matter what platform is used with this PoC, but FWIW the one tested with this PoC is a baremetal KVM virtualization host provided and owned by Red Hat, utilizing libvirt, QEMU, a bit of lcitool [5], and libguestfs. Custom executor ~~~~~~~~~~~~~~ If you're wondering why this PoC revolves around the custom executor GitLab runner type [6], some time ago GitLab decided that instead of adding and supporting different types of virtualization runners, they'd create some kind of gateway suitable basically for any solution out there and named it 'custom executor' - it's nothing more than a bunch of shell scripts categorized to various stages of a pipeline job (which will run either on the host or the guest depending how you set it up). Provisioner ~~~~~~~~~~~ Wait, why is this section needed, just take the example scripts from [1]... If you look at the example Bash scripts in GitLab's docs on custom executor [1] you'll see that the example scripts are very straight forward and concise but there are few things that are suboptimal - creating a new disk image for every single VM instance of the same distro out of the virt-builder template - injecting a bunch of commands directly with virt-builder instead of using something more robust like Ansible when there is more than a single VM to be created - static hostname for every single VM instance - waiting for the either the VM getting an IP or for the SSH connection to be available for a quite limited fixed period of time Because I wanted to address all of the ^above pitfalls, I created a Python based provisioner relying on libvirt-NSS, transient machines, backing chain overlay images, cloud-init, etc. instead. You can find the code base for the provisioner here [7]. Child pipelines & Multiproject CI pipelines ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ child pipeline = a "downstream" (or subpipeline) of the main pipeline useful when there are multiple stages which should be part of a single stage of the main pipeline multiproject pipeline = a pipeline that is kicked off by the main pipeline of your project and runs in context of a different project which your main pipeline can then depend on Technically speaking none of the above 2 GitLab features is needed to really integrate this PoC to upstream libvirt CI, but it was a nice experiment from 3 perspectives: 1) seeing that GitLab's own public SaaS deployment has features from all tiers available 2) it made some job descriptions shorter because e.g. we didn't have to rebuild libvirt in the VM when it was done in the previous step in a container; we don't need to build the Perl bindings (which we need for libvirt-TCK) in the VM either, since we already have a project and hence a pipeline for that already, we just need to harness the results which combined makes the actual VM's purpose more obvious 3) installing libvirt and libvirt-perl using the RPM artifacts built in previous pipeline stages is closer to a real life deployment/environment than re-building everything in the VM followed by doing 'sudo make install' Artifacts ~~~~~~~~~ Apart from the RPMs being published as artifacts from an earlier stage of the pipeline as mentioned in the previous section, in order for this to be any useful to the developers (I know, NOTHING beats a local setup, just be patient with us/me for another while...), when there's a genuine test failure (no infrastructure is failproof), the job description as proposed in patch 4/5 exposes libvirt debug logs as artifacts which can be downloaded and inspected. Test framework ============== For the SW testing part a combination of libvirt-TCK [2] and Avocado [3] were used. The reasoning for this combined test environment was discussed in this thread [8], but essentially we'd like to port most of the Perl-native TCK tests to Python, as more people are familiar with the language, making maintenance of them easier as well as making it easier to contribute new tests to the somewhat ancient test suite. Avocado can definitely help here as it does provide a good foundation and building stones to achieve both. Work in progress ================ One of the things that we already know is problematic with the current hybrid TCK+Avocado setup is that if a test fails, the summary that Avocado produces doesn't quickly tell us which tests have failed and one has to either scroll through the GitLab job output or wade through Avocado job logs. -> this issue is tracked in upstream Avocado and should be addressed soon: https://github.com/avocado-framework/avocado/issues/5200 Another annoying thing in terms of test failure analysis is that TCK in its current shape cannot save libvirt debug logs on a per-test basis and so a massive dump is produced for the whole test run. This is very suboptimal, but this can only be solved with libvirt admin API which can enable/modify/redirect logs at runtime, however, the perl bindings are currently missings -> I'm already working on adding the bindings to libvirt-perl and then we need to teach TCK to use that feature Known issues ============ There are already a couple of known test failures which can be seen in the example pipeline: 1) modular daemons occasionally hang in some test scenarios (doesn't happen with monolithic) https://bugzilla.redhat.com/show_bug.cgi?id=2044379 2) QEMU tray status detection when a CD media is changed -> this one is also intermittent, but no good reproducer data to attach to an official QEMU BZ has been harnessed so far [1] https://docs.gitlab.com/runner/executors/custom.html [2] https://gitlab.com/libvirt/libvirt-tck/ [3] https://avocado-framework.readthedocs.io/ [4] https://gitlab.com/eskultety/libvirt/-/pipelines/457836923 [5] https://gitlab.com/libvirt/libvirt-ci/ [6] https://docs.gitlab.com/runner/executors/ [7] https://gitlab.com/eskultety/libvirt-gitlab-executor [8] https://listman.redhat.com/archives/libvir-list/2021-June/msg00836.html Erik Skultety (4): ci: manifest: Allow RPM builds on CentOS Stream 8 ci: containers: Add CentOS Stream 9 target ci: manifest: Publish RPMs as artifacts on CentOS Stream and Fedoras gitlab-ci: Introduce a new test 'integration' pipeline stage .gitlab-ci-integration.yml | 116 +++++++++++++++++++++++ .gitlab-ci.yml | 17 +++- ci/containers/centos-stream-9.Dockerfile | 87 +++++++++++++++++ ci/gitlab.yml | 33 ++++++- ci/manifest.yml | 26 ++++- 5 files changed, 274 insertions(+), 5 deletions(-) create mode 100644 .gitlab-ci-integration.yml create mode 100644 ci/containers/centos-stream-9.Dockerfile -- 2.34.1

5 37

[libvirt PATCH 0/2] coverity fixes
by Ján Tomko 03 Mar '22

03 Mar '22

Ján Tomko (2): remote: close outfiles faster remote: dispatch: free and close infiles too src/remote/remote_daemon_dispatch.c | 5 ++++- src/remote/remote_driver.c | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-) -- 2.34.1

2 3

[libvirt PATCH] qemu: support multiqueue for vdpa net device
by Jonathon Jongsma 03 Mar '22

03 Mar '22

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2024406 Signed-off-by: Jonathon Jongsma <jjongsma(a)redhat.com> --- src/conf/domain_conf.c | 11 ++++++ src/qemu/qemu_domain.c | 3 +- .../net-vdpa-multiqueue.x86_64-latest.args | 36 +++++++++++++++++++ .../qemuxml2argvdata/net-vdpa-multiqueue.xml | 29 +++++++++++++++ tests/qemuxml2argvtest.c | 1 + .../net-vdpa-multiqueue.xml | 36 +++++++++++++++++++ tests/qemuxml2xmltest.c | 1 + 7 files changed, 116 insertions(+), 1 deletion(-) create mode 100644 tests/qemuxml2argvdata/net-vdpa-multiqueue.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/net-vdpa-multiqueue.xml create mode 100644 tests/qemuxml2xmloutdata/net-vdpa-multiqueue.xml diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 34fec887a3..87117a2ddb 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -10552,6 +10552,17 @@ virDomainNetDefParseXML(virDomainXMLOption *xmlopt, goto error; } def->data.vdpa.devicepath = g_steal_pointer(&dev); + + if (!def->model) + def->model = VIR_DOMAIN_NET_MODEL_VIRTIO; + + if (!virDomainNetIsVirtioModel(def)) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Wrong <model> 'type' attribute specified " + "with <interface type='vdpa'/>. " + "vdpa requires the virtio-net* frontend")); + goto error; + } break; case VIR_DOMAIN_NET_TYPE_BRIDGE: diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index acc76c1cd6..6b61fefb8f 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -4523,7 +4523,8 @@ qemuDomainValidateActualNetDef(const virDomainNetDef *net, actualType == VIR_DOMAIN_NET_TYPE_BRIDGE || actualType == VIR_DOMAIN_NET_TYPE_DIRECT || actualType == VIR_DOMAIN_NET_TYPE_ETHERNET || - actualType == VIR_DOMAIN_NET_TYPE_VHOSTUSER)) { + actualType == VIR_DOMAIN_NET_TYPE_VHOSTUSER || + actualType == VIR_DOMAIN_NET_TYPE_VDPA)) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("interface %s - multiqueue is not supported for network interfaces of type %s"), macstr, virDomainNetTypeToString(actualType)); diff --git a/tests/qemuxml2argvdata/net-vdpa-multiqueue.x86_64-latest.args b/tests/qemuxml2argvdata/net-vdpa-multiqueue.x86_64-latest.args new file mode 100644 index 0000000000..26ef666036 --- /dev/null +++ b/tests/qemuxml2argvdata/net-vdpa-multiqueue.x86_64-latest.args @@ -0,0 +1,36 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/tmp/lib/domain--1-QEMUGuest1 \ +USER=test \ +LOGNAME=test \ +XDG_DATA_HOME=/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=QEMUGuest1,debug-threads=on \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tmp/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=off,dump-guest-core=off,memory-backend=pc.ram \ +-accel tcg \ +-cpu qemu64 \ +-m 214 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}' \ +-overcommit mem-lock=off \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,fd=1729,server=on,wait=off \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-no-acpi \ +-boot strict=on \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0x2"}' \ +-add-fd set=0,fd=1732,opaque=net0-vdpa \ +-netdev vhost-vdpa,vhostdev=/dev/fdset/0,id=hostnet0 \ +-device '{"driver":"virtio-net-pci","mq":true,"vectors":6,"netdev":"hostnet0","id":"net0","mac":"52:54:00:95:db:c0","bus":"pci.0","addr":"0x2"}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ +-msg timestamp=on diff --git a/tests/qemuxml2argvdata/net-vdpa-multiqueue.xml b/tests/qemuxml2argvdata/net-vdpa-multiqueue.xml new file mode 100644 index 0000000000..2e38c6f976 --- /dev/null +++ b/tests/qemuxml2argvdata/net-vdpa-multiqueue.xml @@ -0,0 +1,29 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='x86_64' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-x86_64</emulator> + <controller type='usb' index='0'/> + <controller type='ide' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <interface type='vdpa'> + <mac address='52:54:00:95:db:c0'/> + <source dev='/dev/vhost-vdpa-0'/> + <driver queues='2'/> + </interface> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='none'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index ce475df466..7e1167e60e 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1609,6 +1609,7 @@ mymain(void) DO_TEST_FAILURE("net-hostdev-fail", QEMU_CAPS_DEVICE_VFIO_PCI); DO_TEST_CAPS_LATEST("net-vdpa"); + DO_TEST_CAPS_LATEST("net-vdpa-multiqueue"); DO_TEST("hostdev-pci-multifunction", QEMU_CAPS_KVM, diff --git a/tests/qemuxml2xmloutdata/net-vdpa-multiqueue.xml b/tests/qemuxml2xmloutdata/net-vdpa-multiqueue.xml new file mode 100644 index 0000000000..0876d5df62 --- /dev/null +++ b/tests/qemuxml2xmloutdata/net-vdpa-multiqueue.xml @@ -0,0 +1,36 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='x86_64' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-x86_64</emulator> + <controller type='usb' index='0'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> + </controller> + <controller type='ide' index='0'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> + </controller> + <controller type='pci' index='0' model='pci-root'/> + <interface type='vdpa'> + <mac address='52:54:00:95:db:c0'/> + <source dev='/dev/vhost-vdpa-0'/> + <model type='virtio'/> + <driver queues='2'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </interface> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <audio id='1' type='none'/> + <memballoon model='none'/> + </devices> +</domain> diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index 052950b86f..2174965784 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -485,6 +485,7 @@ mymain(void) DO_TEST_NOCAPS("net-coalesce"); DO_TEST_NOCAPS("net-many-models"); DO_TEST("net-vdpa", QEMU_CAPS_NETDEV_VHOST_VDPA); + DO_TEST("net-vdpa-multiqueue", QEMU_CAPS_NETDEV_VHOST_VDPA); DO_TEST_NOCAPS("serial-tcp-tlsx509-chardev"); DO_TEST_NOCAPS("serial-tcp-tlsx509-chardev-notls"); -- 2.35.1

2 3