This series makes it possible to use Secure Boot with aarch64 VMs. https://issues.redhat.com/browse/RHEL-82645 Changes from [v3]: * changes to JSON firmware descriptors shipped by the edk2 package have been merged in Fedora, so the corresponding patch is no longer marked as DONOTMERGE; * drop new varstore-specific flags from virsh, the existing NVRAM-related flags will work for varstore too; * drop some changes to firmware selection that were not related to varstore support, to be reworked and submitted again at a later date; * split, join and shuffle around patches; * tweak things according to review feedback. Changes from [v2]: * changes to the schema for JSON firmware descriptors have been queued for merge in QEMU, so the corresponding patch is no longer marked as DONOTMERGE; * improve documentation; * rebase on top of master, addressing conflicts that I have caused with some recent changes related to this work. Changes from [v1]: * rewrite based on review feedback: the <nvram> element is no longer used, and a dedicated <varstore> element is introduced instead; * additional test coverage, as well as fixes and improvements related to firmware selection and its documentation, are present as well. [v3] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/5JTQA... [v2] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/WVWT3... [v1] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/TGLFM... Andrea Bolognani (36): docs: Rename "BIOS bootloader" section to "guest firmware" docs: Improvement related to firmware selection qemu_firmware: Only set format for custom loader if path is present conf: Move type=rom default for loader to drivers tests: Rename custom JSON firmware descriptors schema: Introduce osnvram define conf: Parse and format varstore element conf: Update validation to consider varstore element qemu_capabilities: Introduce QEMU_CAPS_DEVICE_UEFI_VARS qemu: Validate presence of uefi-vars device tests: Add firmware-manual-efi-varstore-q35 tests: Add firmware-manual-efi-varstore-aarch64 tests: Add firmware-auto-efi-varstore-q35 tests: Add firmware-auto-efi-varstore-aarch64 tests: Add firmware-auto-efi-enrolled-keys-aarch64 qemu_firmware: Parse host-uefi-vars firmware feature qemu_firmware: Split sanity check qemu_firmware: Consider host-uefi-vars feature in sanity check qemu_firmware: Support extended syntax for ROM firmware descriptors qemu_firmware: Report NVRAM template path for ROMs conf: Include varstore element in domcaps qemu: Fill in varstore element in domcaps qemu_firmware: Use of NVRAM implies stateful firmware qemu_firmware: Allow matching stateful ROMs qemu_firmware: Fill in varstore information qemu: Introduce varstoreDir qemu_firmware: Generate varstore path when necessary qemu: Introduce qemuPrepareNVRAMFileCommon() qemu: Create and delete varstore file security: Mark ROMs as read only when using AppArmor security: Handle varstore file tests: Add firmware descriptors for uefi-vars builds qemu_command: Use uefi-vars device where appropriate include: Mention varstore where applicable virsh: Update for varstore handling news: Document support for uefi-vars device and firmwares NEWS.rst | 17 ++ docs/formatcaps.rst | 2 +- docs/formatdomain.rst | 47 +++-- docs/formatdomaincaps.rst | 85 +++++---- docs/kbase/secureboot.rst | 46 +++-- docs/manpages/virsh.rst | 23 +-- include/libvirt/libvirt-domain-snapshot.h | 2 +- include/libvirt/libvirt-domain.h | 4 +- libvirt.spec.in | 1 + src/conf/domain_capabilities.c | 10 + src/conf/domain_capabilities.h | 6 + src/conf/domain_conf.c | 79 +++++++- src/conf/domain_conf.h | 9 + src/conf/domain_postparse.c | 19 -- src/conf/domain_validate.c | 82 +++----- src/conf/schemas/domaincaps.rng | 9 + src/conf/schemas/domaincommon.rng | 74 +++++--- src/conf/virconftypes.h | 2 + src/libvirt_private.syms | 2 + src/libxl/libxl_domain.c | 6 + src/qemu/meson.build | 1 + src/qemu/qemu_capabilities.c | 29 ++- src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 34 ++++ src/qemu/qemu_conf.c | 4 + src/qemu/qemu_conf.h | 1 + src/qemu/qemu_driver.c | 26 ++- src/qemu/qemu_firmware.c | 177 ++++++++++++++++-- src/qemu/qemu_firmware.h | 1 + src/qemu/qemu_process.c | 84 ++++++--- src/qemu/qemu_validate.c | 20 ++ src/security/security_dac.c | 22 ++- src/security/security_selinux.c | 53 ++++-- src/security/virt-aa-helper.c | 36 +++- .../qemu_10.0.0-q35.x86_64+amdsev.xml | 1 + .../domaincapsdata/qemu_10.0.0-q35.x86_64.xml | 1 + .../qemu_10.0.0-tcg.x86_64+amdsev.xml | 1 + .../domaincapsdata/qemu_10.0.0-tcg.x86_64.xml | 1 + .../qemu_10.0.0-virt.aarch64.xml | 3 + tests/domaincapsdata/qemu_10.0.0.aarch64.xml | 3 + tests/domaincapsdata/qemu_10.0.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_10.0.0.s390x.xml | 1 + .../qemu_10.0.0.x86_64+amdsev.xml | 1 + tests/domaincapsdata/qemu_10.0.0.x86_64.xml | 1 + .../qemu_10.1.0-q35.x86_64+inteltdx.xml | 1 + .../domaincapsdata/qemu_10.1.0-q35.x86_64.xml | 1 + .../qemu_10.1.0-tcg.x86_64+inteltdx.xml | 1 + .../domaincapsdata/qemu_10.1.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_10.1.0.s390x.xml | 1 + .../qemu_10.1.0.x86_64+inteltdx.xml | 1 + tests/domaincapsdata/qemu_10.1.0.x86_64.xml | 1 + .../qemu_10.2.0-q35.x86_64+mshv.xml | 1 + .../domaincapsdata/qemu_10.2.0-q35.x86_64.xml | 1 + .../qemu_10.2.0-tcg.x86_64+mshv.xml | 1 + .../domaincapsdata/qemu_10.2.0-tcg.x86_64.xml | 1 + .../qemu_10.2.0-virt.aarch64.xml | 3 + tests/domaincapsdata/qemu_10.2.0.aarch64.xml | 3 + .../qemu_10.2.0.x86_64+mshv.xml | 1 + tests/domaincapsdata/qemu_10.2.0.x86_64.xml | 1 + .../domaincapsdata/qemu_11.0.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_11.0.0-tcg.x86_64.xml | 1 + .../qemu_11.0.0-virt.aarch64.xml | 3 + tests/domaincapsdata/qemu_11.0.0.aarch64.xml | 3 + tests/domaincapsdata/qemu_11.0.0.x86_64.xml | 1 + .../domaincapsdata/qemu_6.2.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_6.2.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_6.2.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_6.2.0.x86_64.xml | 1 + .../domaincapsdata/qemu_7.0.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_7.0.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_7.0.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_7.0.0.x86_64.xml | 1 + .../domaincapsdata/qemu_7.1.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_7.1.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_7.1.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_7.1.0.x86_64.xml | 1 + .../qemu_7.2.0-hvf.x86_64+hvf.xml | 1 + .../domaincapsdata/qemu_7.2.0-q35.x86_64.xml | 1 + .../qemu_7.2.0-tcg.x86_64+hvf.xml | 1 + .../domaincapsdata/qemu_7.2.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_7.2.0.ppc.xml | 1 + tests/domaincapsdata/qemu_7.2.0.x86_64.xml | 1 + .../domaincapsdata/qemu_8.0.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_8.0.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_8.0.0.x86_64.xml | 1 + .../domaincapsdata/qemu_8.1.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_8.1.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_8.1.0.s390x.xml | 1 + tests/domaincapsdata/qemu_8.1.0.x86_64.xml | 1 + .../domaincapsdata/qemu_8.2.0-q35.x86_64.xml | 1 + .../qemu_8.2.0-tcg-virt.loongarch64.xml | 1 + .../domaincapsdata/qemu_8.2.0-tcg.x86_64.xml | 1 + .../qemu_8.2.0-virt.aarch64.xml | 3 + .../qemu_8.2.0-virt.loongarch64.xml | 1 + tests/domaincapsdata/qemu_8.2.0.aarch64.xml | 3 + tests/domaincapsdata/qemu_8.2.0.armv7l.xml | 1 + tests/domaincapsdata/qemu_8.2.0.s390x.xml | 1 + tests/domaincapsdata/qemu_8.2.0.x86_64.xml | 1 + .../domaincapsdata/qemu_9.0.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_9.0.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_9.0.0.sparc.xml | 1 + tests/domaincapsdata/qemu_9.0.0.x86_64.xml | 1 + .../domaincapsdata/qemu_9.1.0-q35.x86_64.xml | 1 + .../qemu_9.1.0-tcg-virt.riscv64.xml | 1 + .../domaincapsdata/qemu_9.1.0-tcg.x86_64.xml | 1 + .../qemu_9.1.0-virt.riscv64.xml | 1 + tests/domaincapsdata/qemu_9.1.0.s390x.xml | 1 + tests/domaincapsdata/qemu_9.1.0.x86_64.xml | 1 + .../qemu_9.2.0-hvf.aarch64+hvf.xml | 3 + .../qemu_9.2.0-q35.x86_64+amdsev.xml | 1 + .../domaincapsdata/qemu_9.2.0-q35.x86_64.xml | 1 + .../qemu_9.2.0-tcg.x86_64+amdsev.xml | 1 + .../domaincapsdata/qemu_9.2.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_9.2.0.s390x.xml | 1 + .../qemu_9.2.0.x86_64+amdsev.xml | 1 + tests/domaincapsdata/qemu_9.2.0.x86_64.xml | 1 + .../caps_10.0.0_aarch64.xml | 1 + .../caps_10.0.0_x86_64+amdsev.xml | 1 + .../caps_10.0.0_x86_64.xml | 1 + .../caps_10.1.0_s390x.xml | 1 + .../caps_10.1.0_x86_64+inteltdx.xml | 1 + .../caps_10.1.0_x86_64.xml | 1 + .../caps_10.2.0_aarch64.xml | 1 + .../caps_10.2.0_x86_64+mshv.xml | 1 + .../caps_10.2.0_x86_64.xml | 1 + .../caps_11.0.0_aarch64.xml | 1 + .../caps_11.0.0_x86_64.xml | 1 + .../etc/qemu/firmware/20-bios.json | 1 - .../etc/qemu/firmware/20-libvirt-bios.json | 1 + .../etc/qemu/firmware/59-combined.json | 1 - .../qemu/firmware/59-libvirt-combined.json | 1 + ...{92-masked.json => 92-libvirt-masked.json} | 0 .../{10-bios.json => 10-libvirt-bios.json} | 0 .../90-edk2-aarch64-qemuvars-sb-enrolled.json | 29 +++ ...0-edk2-ovmf-qemuvars-x64-sb-enrolled.json} | 14 +- ...combined.json => 90-libvirt-combined.json} | 0 .../firmware/91-edk2-aarch64-qemuvars-sb.json | 28 +++ ...json => 91-edk2-ovmf-qemuvars-x64-sb.json} | 15 +- .../{91-bios.json => 91-libvirt-bios.json} | 0 ...{92-masked.json => 92-libvirt-masked.json} | 0 ...3-invalid.json => 93-libvirt-invalid.json} | 0 tests/qemufirmwaretest.c | 63 +++++-- ...-auto-bios-not-stateless.x86_64-latest.err | 2 +- ...-auto-bios-not-stateless.x86_64-latest.xml | 35 ++++ ...firmware-auto-bios-nvram.x86_64-latest.err | 2 +- ...fi-enrolled-keys-aarch64.aarch64-8.2.0.err | 1 + ...-enrolled-keys-aarch64.aarch64-latest.args | 32 ++++ ...i-enrolled-keys-aarch64.aarch64-latest.xml | 32 ++++ ...irmware-auto-efi-enrolled-keys-aarch64.xml | 20 ++ ...o-efi-varstore-aarch64.aarch64-latest.args | 32 ++++ ...to-efi-varstore-aarch64.aarch64-latest.xml | 32 ++++ .../firmware-auto-efi-varstore-aarch64.xml | 18 ++ ...e-auto-efi-varstore-q35.x86_64-latest.args | 35 ++++ ...re-auto-efi-varstore-q35.x86_64-latest.xml | 40 ++++ .../firmware-auto-efi-varstore-q35.xml | 18 ++ ...nual-bios-not-stateless.x86_64-latest.args | 32 ++++ ...anual-bios-not-stateless.x86_64-latest.err | 1 - ...anual-bios-not-stateless.x86_64-latest.xml | 28 +++ ...nual-efi-nvram-stateless.x86_64-latest.err | 2 +- ...nvram-template-stateless.x86_64-latest.err | 2 +- ...ware-manual-efi-rw-nvram.x86_64-latest.err | 2 +- ...ual-efi-varstore-aarch64.aarch64-8.2.0.err | 1 + ...l-efi-varstore-aarch64.aarch64-latest.args | 32 ++++ ...al-efi-varstore-aarch64.aarch64-latest.xml | 32 ++++ .../firmware-manual-efi-varstore-aarch64.xml | 19 ++ ...e-manual-efi-varstore-q35.x86_64-8.2.0.err | 1 + ...manual-efi-varstore-q35.x86_64-latest.args | 35 ++++ ...-manual-efi-varstore-q35.x86_64-latest.xml | 40 ++++ .../firmware-manual-efi-varstore-q35.xml | 19 ++ tests/qemuxmlconftest.c | 16 +- tests/testutilsqemu.c | 2 + tools/virsh-domain.c | 10 +- tools/virsh-snapshot.c | 2 +- 173 files changed, 1546 insertions(+), 307 deletions(-) delete mode 120000 tests/qemufirmwaredata/etc/qemu/firmware/20-bios.json create mode 120000 tests/qemufirmwaredata/etc/qemu/firmware/20-libvirt-bios.json delete mode 120000 tests/qemufirmwaredata/etc/qemu/firmware/59-combined.json create mode 120000 tests/qemufirmwaredata/etc/qemu/firmware/59-libvirt-combined.json rename tests/qemufirmwaredata/etc/qemu/firmware/{92-masked.json => 92-libvirt-masked.json} (100%) rename tests/qemufirmwaredata/home/user/.config/qemu/firmware/{10-bios.json => 10-libvirt-bios.json} (100%) create mode 100644 tests/qemufirmwaredata/usr/share/qemu/firmware/90-edk2-aarch64-qemuvars-sb-enrolled.json copy tests/qemufirmwaredata/usr/share/qemu/firmware/{90-combined.json => 90-edk2-ovmf-qemuvars-x64-sb-enrolled.json} (55%) copy tests/qemufirmwaredata/usr/share/qemu/firmware/{90-combined.json => 90-libvirt-combined.json} (100%) create mode 100644 tests/qemufirmwaredata/usr/share/qemu/firmware/91-edk2-aarch64-qemuvars-sb.json rename tests/qemufirmwaredata/usr/share/qemu/firmware/{90-combined.json => 91-edk2-ovmf-qemuvars-x64-sb.json} (52%) rename tests/qemufirmwaredata/usr/share/qemu/firmware/{91-bios.json => 91-libvirt-bios.json} (100%) rename tests/qemufirmwaredata/usr/share/qemu/firmware/{92-masked.json => 92-libvirt-masked.json} (100%) rename tests/qemufirmwaredata/usr/share/qemu/firmware/{93-invalid.json => 93-libvirt-invalid.json} (100%) create mode 100644 tests/qemuxmlconfdata/firmware-auto-bios-not-stateless.x86_64-latest.xml create mode 100644 tests/qemuxmlconfdata/firmware-auto-efi-enrolled-keys-aarch64.aarch64-8.2.0.err create mode 100644 tests/qemuxmlconfdata/firmware-auto-efi-enrolled-keys-aarch64.aarch64-latest.args create mode 100644 tests/qemuxmlconfdata/firmware-auto-efi-enrolled-keys-aarch64.aarch64-latest.xml create mode 100644 tests/qemuxmlconfdata/firmware-auto-efi-enrolled-keys-aarch64.xml create mode 100644 tests/qemuxmlconfdata/firmware-auto-efi-varstore-aarch64.aarch64-latest.args create mode 100644 tests/qemuxmlconfdata/firmware-auto-efi-varstore-aarch64.aarch64-latest.xml create mode 100644 tests/qemuxmlconfdata/firmware-auto-efi-varstore-aarch64.xml create mode 100644 tests/qemuxmlconfdata/firmware-auto-efi-varstore-q35.x86_64-latest.args create mode 100644 tests/qemuxmlconfdata/firmware-auto-efi-varstore-q35.x86_64-latest.xml create mode 100644 tests/qemuxmlconfdata/firmware-auto-efi-varstore-q35.xml create mode 100644 tests/qemuxmlconfdata/firmware-manual-bios-not-stateless.x86_64-latest.args delete mode 100644 tests/qemuxmlconfdata/firmware-manual-bios-not-stateless.x86_64-latest.err create mode 100644 tests/qemuxmlconfdata/firmware-manual-bios-not-stateless.x86_64-latest.xml create mode 100644 tests/qemuxmlconfdata/firmware-manual-efi-varstore-aarch64.aarch64-8.2.0.err create mode 100644 tests/qemuxmlconfdata/firmware-manual-efi-varstore-aarch64.aarch64-latest.args create mode 100644 tests/qemuxmlconfdata/firmware-manual-efi-varstore-aarch64.aarch64-latest.xml create mode 100644 tests/qemuxmlconfdata/firmware-manual-efi-varstore-aarch64.xml create mode 100644 tests/qemuxmlconfdata/firmware-manual-efi-varstore-q35.x86_64-8.2.0.err create mode 100644 tests/qemuxmlconfdata/firmware-manual-efi-varstore-q35.x86_64-latest.args create mode 100644 tests/qemuxmlconfdata/firmware-manual-efi-varstore-q35.x86_64-latest.xml create mode 100644 tests/qemuxmlconfdata/firmware-manual-efi-varstore-q35.xml -- 2.53.0