Re: [RFC PATCH 0/5] qemu: Route hostdevs to multiple nested SMMUs

Hi Daniel, >Top level libvirt device representation in XML is based on the device >*class*, not the specific device impl. Adding a <nestedSmmuv3> device >type XML element in libvirt is totally inappropriate. Any configuration >must be done beneath the <iommu> element. Would keeping track of PXB <=> host SMMU nodes be better represented with a <nestedSmmuv3> PXB attribute like below, when the "nestedSmmuv3" IOMMU model is specified? This method would be simplest IMO because we could omit keeping track of the nestedSmmuv3 bus number in the virDomainDef struct since its association with a PXB controller would already be baked-in. <devices> ... <controller type='pci' index='1' model='pcie-expander-bus'> <model name='pxb-pcie'/> <target busNr='254'/> <nestedSmmuv3>smmu3.0x0000000012000000</nestedSmmuv3> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </controller> ... <iommu model='nestedSmmuv3'/> </devices> Or would it still be preferred to purely contain the host SMMU node names and bus numbers within the <iommu> element? If so, the virDomainDef struct is setup to only have a single virDomainIOMMUDef member, but we need to keep track of multiple host SMMU node names and bus numbers. Would we setup multiple virDomainIOMMUDef members? Or add "char** nestedSmmuv3" and "size_t *nestedSmmuv3Bus" members to the virDomainIOMMUDef struct to keep track of these multiple host SMMU node names and bus numbers? Or we could modify the device info member of virDomainIOMMUDef to be a variable-length array of device info structs instead of the "size_t *nestedSmmuv3Bus" member. But I'm not convinced this would be the cleanest approach, and the qemu command line doesn't specify a slot and function to arm-smmuv3-nested devices - it just specifies bus number to keep track of which PXB is associated with each SMMU node. If we need a way to associate PXB to SMMU node, we have multiple possible approaches, listed below in order of best to worst in my opinion: 1. Adding a <nestedSmmuv3> attribute for PXB controller. 2. Having a single virDomainIOMMUDef struct for virDomainDef. Adding variable-length array members to virDomainIOMMUDef for multiple SMMU node names and bus numbers. 3. Having a single virDomainIOMMUDef struct for virDomainDef. Adding a variable-length array member to virDomainIOMMUDef for SMMU node names. Changing the single virDomainDeviceInfo struct to a variable-length array of virDomainDeviceInfo structs for multiple nested SMMU bus numbers. 4. Supporting multiple virDomainIOMMUDef structs for virDomainDef. I would appreciate your thoughts on which method to go with.