Hi, This is a follow up to the fourth RFC patchset [0] for supporting multiple vSMMU instances and using iommufd to propagate DMA mappings to kernel for VM-assigned host devices in a qemu VM. This patchset implements support for specifying multiple <iommu> devices within the VM definition when smmuv3Dev IOMMU model is specified, and is tested with Shameer's v4 qemu RFC for HW-accelerated vSMMU devices [1] Moreover, it adds a new 'iommufd' attribute for hostdev devices to be associated with the iommufd object. For instance, specifying the iommufd object and associated hostdev in a VM definition with multiple IOMMUs, configured to be routed to pcie-expander-bus controllers in a way where VFIO device to SMMUv3 associations are matched with the host: <devices> ... <controller type='pci' index='1' model='pcie-expander-bus'> <model name='pxb-pcie'/> <target busNr='252'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </controller> <controller type='pci' index='2' model='pcie-expander-bus'> <model name='pxb-pcie'/> <target busNr='248'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </controller> ... <controller type='pci' index='21' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='21' port='0x0'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </controller> <controller type='pci' index='22' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='22' port='0xa8'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> ... <hostdev mode='subsystem' type='pci' managed='no'> <driver iommufd='yes'/> <source> <address domain='0x0009' bus='0x01' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x15' slot='0x00' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='no'> <driver iommufd='yes'/> <source> <address domain='0x0019' bus='0x01' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x16' slot='0x00' function='0x0'/> </hostdev> <iommu model='smmuv3Dev'> <driver parentIdx='1' accel='on' ats='on' ril='on' pasid='on' oas='44'/> </iommu> <iommu model='smmuv3Dev'> <driver parentIdx='2' accel='on' ats='on' ril='on' pasid='on' oas='44'/> </iommu> </devices> This would get translated to a qemu command line with the arguments below. Note that libvirt will open the /dev/iommu and VFIO cdev, passing the associated fd number to qemu: -device '{"driver":"pxb-pcie","bus_nr":252,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \ -device '{"driver":"pxb-pcie","bus_nr":248,"id":"pci.2","bus":"pcie.0","addr":"0x2"}' \ -device '{"driver":"pcie-root-port","port":0,"chassis":21,"id":"pci.21","bus":"pci.1","addr":"0x0"}' \ -device '{"driver":"pcie-root-port","port":168,"chassis":22,"id":"pci.22","bus":"pci.2","addr":"0x0"}' \ -object '{"qom-type":"iommufd","id":"iommufd0","fd":"24"}' \ -device '{"driver":"arm-smmuv3","primary-bus":"pci.1","id":"smmuv3.0","accel":true,"ats":true,"ril":true,"pasid":true,"oas":44}' \ -device '{"driver":"arm-smmuv3","primary-bus":"pci.2","id":"smmuv3.1","accel":true,"ats":true,"ril":true,"pasid":true,"oas":44}' \ -device '{"driver":"vfio-pci","host":"0009:01:00.0","id":"hostdev0","iommufd":"iommufd0","fd":"22","bus":"pci.21","addr":"0x0"}' \ -device '{"driver":"vfio-pci","host":"0019:01:00.0","id":"hostdev1","iommufd":"iommufd0","fd":"25","bus":"pci.22","addr":"0x0"}' \ Changes from RFCv3: - Move smmuv3Dev-specific attributes under <driver> - Include new smmuv3Dev-specific attributes based on Shameer's v4 qemu series This series is on Github: https://github.com/NathanChenNVIDIA/libvirt/tree/smmuv3Dev-iommufd-10-01-25 Thanks, Nathan [0] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/T66FI... [1] https://lore.kernel.org/all/20250929133643.38961-1-skolothumtho@nvidia.com/#... Signed-off-by: Nathan Chen <nathanc@nvidia.com> Nathan Chen (7): qemu: add IOMMU model smmuv3Dev conf: Support multiple smmuv3Dev IOMMU devices tests: qemuxmlconfdata: provide smmuv3Dev sample XML and CLI args qemu: Implement support for associating iommufd to hostdev qemu: open iommufd FDs from libvirt backend qemu: Update Cgroup, namespace, and seclabel for qemu to access iommufd paths tests: qemuxmlconfdata: provide iommufd sample XML and CLI args docs/formatdomain.rst | 44 ++- src/conf/device_conf.c | 9 + src/conf/device_conf.h | 1 + src/conf/domain_conf.c | 177 +++++++++-- src/conf/domain_conf.h | 16 +- src/conf/domain_validate.c | 73 ++++- src/conf/schemas/basictypes.rng | 5 + src/conf/schemas/domaincommon.rng | 35 ++- src/libvirt_private.syms | 2 + src/qemu/qemu_alias.c | 15 +- src/qemu/qemu_cgroup.c | 61 ++++ src/qemu/qemu_cgroup.h | 1 + src/qemu/qemu_command.c | 275 +++++++++++++----- src/qemu/qemu_command.h | 3 +- src/qemu/qemu_domain.c | 8 + src/qemu/qemu_domain.h | 7 + src/qemu/qemu_domain_address.c | 33 ++- src/qemu/qemu_driver.c | 8 +- src/qemu/qemu_hotplug.c | 2 +- src/qemu/qemu_namespace.c | 44 +++ src/qemu/qemu_postparse.c | 11 +- src/qemu/qemu_process.c | 232 +++++++++++++++ src/qemu/qemu_validate.c | 18 +- src/security/security_apparmor.c | 15 + src/security/security_dac.c | 34 +++ src/security/security_selinux.c | 30 ++ src/util/virpci.c | 68 +++++ src/util/virpci.h | 1 + .../iommu-smmuv3Dev.aarch64-latest.args | 39 +++ .../iommu-smmuv3Dev.aarch64-latest.xml | 66 +++++ tests/qemuxmlconfdata/iommu-smmuv3Dev.xml | 53 ++++ .../iommufd-q35.x86_64-latest.args | 41 +++ .../iommufd-q35.x86_64-latest.xml | 60 ++++ tests/qemuxmlconfdata/iommufd-q35.xml | 38 +++ .../iommufd-virt.aarch64-latest.args | 33 +++ .../iommufd-virt.aarch64-latest.xml | 34 +++ tests/qemuxmlconfdata/iommufd-virt.xml | 22 ++ .../iommufd.x86_64-latest.args | 35 +++ .../qemuxmlconfdata/iommufd.x86_64-latest.xml | 38 +++ tests/qemuxmlconfdata/iommufd.xml | 30 ++ tests/qemuxmlconftest.c | 5 + 41 files changed, 1584 insertions(+), 138 deletions(-) create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3Dev.aarch64-latest.args create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3Dev.aarch64-latest.xml create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3Dev.xml create mode 100644 tests/qemuxmlconfdata/iommufd-q35.x86_64-latest.args create mode 100644 tests/qemuxmlconfdata/iommufd-q35.x86_64-latest.xml create mode 100644 tests/qemuxmlconfdata/iommufd-q35.xml create mode 100644 tests/qemuxmlconfdata/iommufd-virt.aarch64-latest.args create mode 100644 tests/qemuxmlconfdata/iommufd-virt.aarch64-latest.xml create mode 100644 tests/qemuxmlconfdata/iommufd-virt.xml create mode 100644 tests/qemuxmlconfdata/iommufd.x86_64-latest.args create mode 100644 tests/qemuxmlconfdata/iommufd.x86_64-latest.xml create mode 100644 tests/qemuxmlconfdata/iommufd.xml -- 2.43.0