Re: [PATCH 00/21] qemu: Validate machine type and clean up test suite

I was experimenting with a qemu-5.0 feature and figured out that qemu-5.0 will drop support for machine types prior to 'pc-1.0'. What was unexpected though is that I've figured it out as some very strange and seemingly unrelated changes happened in the test output of the 'latest' tests while bumping the caps data. It turned out that the CPU driver now actually takes data from the machine type and since the machine type was no longer present the test data result has changed. This lead me down a rabbit hole of trying to make our test suite more robust and such changes easier to debug. The result is these 21 patches that fix numerous tests, clean up some stale and downstream machine types and in the end add validation of the machine type and harden the testsuite to use it properly. Note that 30k lines of the diffstat is just the qemu-5.0 capabilities data so that we can verify that nothing changes during the capbility upgrade. Some of the yak-shaving done here is incomplete though as I've figured out we have a bunch of network-device related tests which are not enabled in the xml->argv test suite, but also are not easy to enable. Peter Krempa (21): tests: qemuxml2argv: Use 64 bit qemu binary and 1.5 machine type in 'disk-cache' case tests: qemuxml2xml: Modernize 'disk-cache' test tests: qemuxml2argv: Modernize tpm-passthrough tests tests: qemuxml: Fix and enable default-video-type* tests tests: qemu: Unify fake machine types filled in for KVM and TCG caps of x86_64 tests: qemuxml2argv: Use existing machine type for 'numatune-distances' case tests: qemuxml2argv: Remove downstream machine type in 'controller-order' case tests: qemuxml2argv: Use existing non-virt machine in 'aarch64-gic-not-virt' tests: qemuxml2argv: Use proper replacement for 'pc-i440fx-wily' machine tests: qemuxml: Use existing machine type in 'iothreads-virtio-scsi-ccw' case tests: qemu: Add machine types used by the test code to fake capabilities qemu: domain: Validate that machine type is supported by qemu qemu: capabilities: Extract code from virQEMUCapsStripMachineAliases qemu: capabilities: Replace aliased machine type by copy of the canonical machine qemu: capabilities: Add accessor to qemu caps machine types presence tests: qemu: Don't add fake machine types when testing with real data tests: qemuxml: Purge prehistoric machine type 'pc-0.13' from "latest" tests tests: Add caps for upcoming qemu-5.0 tests: qemuxml: Remove prehistoric machine types from legacy tests tests: qemuxml2xml: Remove prehistoric machine types from legacy tests tests: qemu: Remove prehistoric machine types from faked data tests src/qemu/qemu_capabilities.c | 50 +- src/qemu/qemu_capabilities.h | 4 + src/qemu/qemu_capspriv.h | 3 + src/qemu/qemu_domain.c | 8 + .../domaincapsdata/qemu_5.0.0-q35.x86_64.xml | 181 + .../domaincapsdata/qemu_5.0.0-tcg.x86_64.xml | 191 + tests/domaincapsdata/qemu_5.0.0.x86_64.xml | 181 + .../caps_5.0.0.x86_64.replies | 27179 ++++++++++++++++ .../caps_5.0.0.x86_64.xml | 2697 ++ .../qemuxml2argvdata/aarch64-gic-not-virt.xml | 2 +- .../aarch64-virt-graphics.aarch64-latest.args | 1 + .../aarch64-virt-headless.aarch64-latest.args | 1 + .../channel-unix-guestfwd.x86_64-latest.args | 1 + .../console-virtio-unix.x86_64-latest.args | 1 + tests/qemuxml2argvdata/controller-order.args | 2 +- tests/qemuxml2argvdata/controller-order.xml | 2 +- .../controller-virtio-scsi.x86_64-latest.args | 1 + .../default-qemu-host-arch.args | 2 +- ...ult-video-type-aarch64.aarch64-latest.args | 35 + .../default-video-type-aarch64.xml | 1 - ...default-video-type-ppc64.ppc64-latest.args | 35 + .../default-video-type-ppc64.xml | 3 +- ...ult-video-type-riscv64.riscv64-latest.args | 34 + .../default-video-type-riscv64.xml | 1 - ...default-video-type-s390x.s390x-latest.args | 34 + .../default-video-type-s390x.xml | 3 +- .../default-video-type-x86_64-caps-test-0.xml | 3 +- .../default-video-type-x86_64-caps-test-1.xml | 3 +- .../disk-aio.x86_64-latest.args | 1 + ...-backing-chains-noindex.x86_64-latest.args | 1 + .../disk-cache.x86_64-1.5.3.args | 4 +- .../disk-cache.x86_64-2.12.0.args | 4 +- .../disk-cache.x86_64-2.6.0.args | 4 +- .../disk-cache.x86_64-2.7.0.args | 4 +- .../disk-cache.x86_64-latest.args | 4 +- tests/qemuxml2argvdata/disk-cache.xml | 4 +- .../disk-cdrom-bus-other.x86_64-latest.args | 1 + ...m-empty-network-invalid.x86_64-latest.args | 1 + tests/qemuxml2argvdata/disk-cdrom-tray.args | 2 +- .../disk-cdrom-tray.x86_64-2.12.0.args | 2 +- .../disk-cdrom-tray.x86_64-latest.args | 2 +- tests/qemuxml2argvdata/disk-cdrom-tray.xml | 2 +- .../disk-cdrom.x86_64-latest.args | 1 + tests/qemuxml2argvdata/disk-copy_on_read.args | 2 +- .../disk-copy_on_read.x86_64-2.12.0.args | 2 +- .../disk-copy_on_read.x86_64-latest.args | 2 +- tests/qemuxml2argvdata/disk-copy_on_read.xml | 2 +- .../qemuxml2argvdata/disk-detect-zeroes.args | 2 +- .../disk-detect-zeroes.x86_64-2.12.0.args | 2 +- .../disk-detect-zeroes.x86_64-latest.args | 2 +- tests/qemuxml2argvdata/disk-detect-zeroes.xml | 2 +- tests/qemuxml2argvdata/disk-discard.args | 2 +- tests/qemuxml2argvdata/disk-discard.xml | 2 +- .../disk-error-policy.x86_64-latest.args | 1 + .../disk-floppy.x86_64-latest.args | 1 + tests/qemuxml2argvdata/disk-ioeventfd.args | 2 +- tests/qemuxml2argvdata/disk-ioeventfd.xml | 2 +- .../disk-network-gluster.x86_64-latest.args | 1 + .../disk-network-iscsi.x86_64-latest.args | 1 + .../disk-network-nbd.x86_64-latest.args | 1 + .../disk-network-rbd.x86_64-latest.args | 1 + .../disk-network-sheepdog.x86_64-latest.args | 1 + ...isk-network-source-auth.x86_64-latest.args | 1 + .../disk-network-tlsx509.x86_64-latest.args | 2 +- .../disk-nvme.x86_64-latest.args | 1 + .../disk-readonly-disk.x86_64-latest.args | 1 + .../disk-scsi-device-auto.x86_64-latest.args | 1 + .../disk-scsi.x86_64-latest.args | 1 + .../disk-shared.x86_64-latest.args | 1 + ...irtio-scsi-reservations.x86_64-latest.args | 1 + .../eoi-disabled.x86_64-latest.args | 2 +- .../eoi-enabled.x86_64-latest.args | 2 +- tests/qemuxml2argvdata/event_idx.args | 2 +- tests/qemuxml2argvdata/event_idx.xml | 2 +- .../fd-memory-no-numa-topology.args | 2 +- .../fd-memory-no-numa-topology.xml | 2 +- .../fd-memory-numa-topology.args | 2 +- .../fd-memory-numa-topology.xml | 2 +- .../fd-memory-numa-topology2.args | 2 +- .../fd-memory-numa-topology2.xml | 2 +- .../fd-memory-numa-topology3.args | 2 +- .../fd-memory-numa-topology3.xml | 2 +- .../floppy-drive-fat.x86_64-latest.args | 1 + .../fs9p-ccw.s390x-latest.args | 1 + .../qemuxml2argvdata/fs9p.x86_64-latest.args | 1 + .../genid-auto.x86_64-latest.args | 1 + .../qemuxml2argvdata/genid.x86_64-latest.args | 1 + ...egl-headless-rendernode.x86_64-latest.args | 1 + .../graphics-egl-headless.x86_64-latest.args | 1 + ...pice-gl-auto-rendernode.x86_64-latest.args | 1 + ...graphics-vnc-tls-secret.x86_64-latest.args | 1 + .../graphics-vnc-tls.x86_64-latest.args | 1 + ...tdev-mdev-display-ramfb.x86_64-latest.args | 1 + ...play-spice-egl-headless.x86_64-latest.args | 1 + ...ev-display-spice-opengl.x86_64-latest.args | 1 + ...isplay-vnc-egl-headless.x86_64-latest.args | 1 + ...ostdev-mdev-display-vnc.x86_64-latest.args | 1 + ...tdev-subsys-mdev-vfio-ap.s390x-latest.args | 1 + ...ubsys-mdev-vfio-ccw-boot.s390x-latest.args | 1 + .../hugepages-nvdimm.x86_64-latest.args | 1 + .../hyperv-off.x86_64-latest.args | 1 + .../hyperv-panic.x86_64-latest.args | 2 +- .../hyperv-stimer-direct.x86_64-latest.args | 2 +- .../hyperv.x86_64-latest.args | 2 +- ...ntel-iommu-caching-mode.x86_64-latest.args | 1 + .../intel-iommu-caching-mode.xml | 3 + ...ntel-iommu-device-iotlb.x86_64-latest.args | 1 + .../intel-iommu-device-iotlb.xml | 3 + .../intel-iommu-eim.x86_64-latest.args | 1 + tests/qemuxml2argvdata/intel-iommu-eim.xml | 3 + .../intel-iommu.x86_64-2.6.0.args | 1 + .../intel-iommu.x86_64-latest.args | 1 + tests/qemuxml2argvdata/intel-iommu.xml | 3 + tests/qemuxml2argvdata/interface-driver.xml | 2 +- .../iommu-smmuv3.aarch64-latest.args | 1 + ...othreads-virtio-scsi-ccw.s390x-latest.args | 3 +- .../iothreads-virtio-scsi-ccw.xml | 2 +- ...threads-virtio-scsi-pci.x86_64-latest.args | 1 + .../kvmclock+eoi-disabled.x86_64-latest.args | 2 +- ...memory-default-hugepage.x86_64-latest.args | 3 +- .../memfd-memory-default-hugepage.xml | 2 +- .../memfd-memory-numa.x86_64-latest.args | 3 +- tests/qemuxml2argvdata/memfd-memory-numa.xml | 2 +- ...y-hotplug-nvdimm-access.x86_64-latest.args | 1 + ...ry-hotplug-nvdimm-align.x86_64-latest.args | 1 + ...ry-hotplug-nvdimm-label.x86_64-latest.args | 1 + ...ory-hotplug-nvdimm-pmem.x86_64-latest.args | 1 + ...hotplug-nvdimm-readonly.x86_64-latest.args | 1 + .../memory-hotplug-nvdimm.x86_64-latest.args | 1 + .../mlock-off.x86_64-latest.args | 1 + .../mlock-on.x86_64-latest.args | 1 + tests/qemuxml2argvdata/net-bandwidth.xml | 2 +- tests/qemuxml2argvdata/net-bandwidth2.xml | 2 +- tests/qemuxml2argvdata/net-coalesce.xml | 2 +- tests/qemuxml2argvdata/net-mtu.xml | 2 +- .../net-vhostuser.x86_64-latest.args | 1 + .../qemuxml2argvdata/numatune-distances.args | 4 +- tests/qemuxml2argvdata/numatune-distances.xml | 2 +- .../parallel-unix-chardev.x86_64-latest.args | 1 + .../ppc64-pseries-graphics.ppc64-latest.args | 1 + .../ppc64-pseries-headless.ppc64-latest.args | 1 + .../pv-spinlock-disabled.x86_64-latest.args | 2 +- .../pv-spinlock-enabled.x86_64-latest.args | 2 +- .../qemu-ns.x86_64-latest.args | 1 + .../s390x-ccw-graphics.s390x-latest.args | 1 + .../s390x-ccw-headless.s390x-latest.args | 1 + .../serial-unix-chardev.x86_64-latest.args | 1 + ...rtcard-passthrough-unix.x86_64-latest.args | 1 + .../qemuxml2argvdata/tap-vhost-incorrect.xml | 2 +- tests/qemuxml2argvdata/tap-vhost.xml | 2 +- ...=> tpm-passthrough-crb.x86_64-latest.args} | 19 +- ...rgs => tpm-passthrough.x86_64-latest.args} | 21 +- tests/qemuxml2argvdata/tpm-passthrough.xml | 2 +- .../usb-redir-unix.x86_64-latest.args | 1 + ...host-user-gpu-secondary.x86_64-latest.args | 1 + .../vhost-user-vga.x86_64-latest.args | 1 + .../vhost-vsock-ccw-auto.s390x-latest.args | 1 + .../vhost-vsock-ccw.s390x-latest.args | 1 + .../vhost-vsock.x86_64-latest.args | 2 +- tests/qemuxml2argvdata/vhost-vsock.xml | 2 +- tests/qemuxml2argvdata/vhost_queues.xml | 2 +- ...eo-bochs-display-device.x86_64-latest.args | 1 + .../video-device-pciaddr-default.args | 2 +- .../video-device-pciaddr-default.xml | 2 +- ...video-qxl-device-vram64.x86_64-latest.args | 1 + ...o-qxl-sec-device-vram64.x86_64-latest.args | 1 + ...eo-ramfb-display-device.x86_64-latest.args | 1 + tests/qemuxml2argvdata/virtio-lun.args | 2 +- tests/qemuxml2argvdata/virtio-lun.xml | 2 +- ...virtio-non-transitional.x86_64-latest.args | 1 + .../virtio-rng-egd-unix.x86_64-latest.args | 1 + .../virtio-transitional.x86_64-latest.args | 1 + .../x86_64-pc-graphics.x86_64-latest.args | 1 + .../x86_64-pc-headless.x86_64-latest.args | 1 + .../x86_64-q35-graphics.x86_64-latest.args | 1 + .../x86_64-q35-headless.x86_64-latest.args | 1 + tests/qemuxml2argvtest.c | 11 +- .../aarch64-virt-graphics.aarch64-latest.xml | 3 + .../aarch64-virt-headless.aarch64-latest.xml | 3 + .../default-qemu-host-arch.xml | 2 +- ...ault-video-type-aarch64.aarch64-latest.xml | 1 - .../default-video-type-ppc64.ppc64-latest.xml | 14 +- ...ault-video-type-riscv64.riscv64-latest.xml | 1 - .../default-video-type-s390x.s390x-latest.xml | 8 +- .../default-video-type-x86_64-caps-test-0.xml | 5 +- .../default-video-type-x86_64-caps-test-1.xml | 5 +- ...-cache.xml => disk-cache.x86_64-1.5.3.xml} | 6 +- .../disk-cache.x86_64-2.12.0.xml | 60 + .../disk-cache.x86_64-2.6.0.xml | 60 + .../disk-cache.x86_64-2.7.0.xml | 60 + .../disk-cache.x86_64-latest.xml | 63 + ...om-empty-network-invalid.x86_64-latest.xml | 3 + .../qemuxml2xmloutdata/disk-copy_on_read.xml | 2 +- tests/qemuxml2xmloutdata/disk-discard.xml | 2 +- tests/qemuxml2xmloutdata/event_idx.xml | 2 +- .../genid-active.x86_64-latest.xml | 3 + .../genid-auto-active.x86_64-latest.xml | 3 + .../genid-auto-inactive.x86_64-latest.xml | 3 + .../genid-inactive.x86_64-latest.xml | 3 + ...subsys-mdev-vfio-ccw-boot.s390x-latest.xml | 3 + .../intel-iommu.x86_64-2.6.0.xml | 3 + tests/qemuxml2xmloutdata/interface-driver.xml | 2 +- .../iommu-smmuv3.aarch64-latest.xml | 3 + .../iothreads-virtio-scsi-ccw.xml | 2 +- tests/qemuxml2xmloutdata/net-bandwidth.xml | 2 +- tests/qemuxml2xmloutdata/net-bandwidth2.xml | 2 +- tests/qemuxml2xmloutdata/net-coalesce.xml | 2 +- tests/qemuxml2xmloutdata/net-mtu.xml | 2 +- .../ppc64-pseries-graphics.ppc64-latest.xml | 3 + .../ppc64-pseries-headless.ppc64-latest.xml | 3 + .../s390x-ccw-graphics.s390x-latest.xml | 3 + .../s390x-ccw-headless.s390x-latest.xml | 3 + .../tap-vhost-incorrect.xml | 2 +- tests/qemuxml2xmloutdata/tap-vhost.xml | 2 +- .../tpm-passthrough.x86_64-latest.xml | 2 +- tests/qemuxml2xmloutdata/vhost_queues.xml | 2 +- .../video-device-pciaddr-default.xml | 2 +- tests/qemuxml2xmloutdata/virtio-lun.xml | 2 +- .../virtio-non-transitional.x86_64-latest.xml | 3 + .../virtio-transitional.x86_64-latest.xml | 3 + .../x86_64-pc-graphics.x86_64-latest.xml | 3 + .../x86_64-pc-headless.x86_64-latest.xml | 3 + .../x86_64-q35-graphics.x86_64-latest.xml | 3 + .../x86_64-q35-headless.x86_64-latest.xml | 3 + tests/qemuxml2xmltest.c | 6 +- tests/testutilsqemu.c | 71 +- tests/vircapstest.c | 4 +- 227 files changed, 31219 insertions(+), 182 deletions(-) create mode 100644 tests/domaincapsdata/qemu_5.0.0-q35.x86_64.xml create mode 100644 tests/domaincapsdata/qemu_5.0.0-tcg.x86_64.xml create mode 100644 tests/domaincapsdata/qemu_5.0.0.x86_64.xml create mode 100644 tests/qemucapabilitiesdata/caps_5.0.0.x86_64.replies create mode 100644 tests/qemucapabilitiesdata/caps_5.0.0.x86_64.xml create mode 100644 tests/qemuxml2argvdata/default-video-type-aarch64.aarch64-latest.args create mode 100644 tests/qemuxml2argvdata/default-video-type-ppc64.ppc64-latest.args create mode 100644 tests/qemuxml2argvdata/default-video-type-riscv64.riscv64-latest.args create mode 100644 tests/qemuxml2argvdata/default-video-type-s390x.s390x-latest.args rename tests/qemuxml2argvdata/{tpm-passthrough-crb.args => tpm-passthrough-crb.x86_64-latest.args} (61%) rename tests/qemuxml2argvdata/{tpm-passthrough.args => tpm-passthrough.x86_64-latest.args} (56%) rename tests/qemuxml2xmloutdata/{disk-cache.xml => disk-cache.x86_64-1.5.3.xml} (92%) create mode 100644 tests/qemuxml2xmloutdata/disk-cache.x86_64-2.12.0.xml create mode 100644 tests/qemuxml2xmloutdata/disk-cache.x86_64-2.6.0.xml create mode 100644 tests/qemuxml2xmloutdata/disk-cache.x86_64-2.7.0.xml create mode 100644 tests/qemuxml2xmloutdata/disk-cache.x86_64-latest.xml