[RFC PATCH 0/3] qemu: Support vCMDQ
Hi, This is a follow up to the previous RFC patchset [0] for supporting Tegra241 vCMDQ, separated out from the series for accelerated SMMUv3. This patchset is tested with Shameer's v2 qemu series [1] for vCMDQ support. The accelerated SMMUv3 series for qemu and Libvirt [2] are pre-requisites for vCMDQ support, as accelerated SMMUv3 must be enabled alongside vCMDQ. A new 'cmdqv' <iommu> driver attribute is introduced to toggle enabling vCMDQ support. An additional 'identifier' <iommu> driver attribute is introduced to specify an enumeration-independent SMMUv3 device identifier. This identifier is used so that both Tegra DSDT and SMMUv3 IORT use the same identifier. For instance, specifying hostdevs associated with multiple accelerated SMMUs + CMDQV enabled, configured to be routed to pcie-expander-bus controllers in a way where VFIO device to SMMUv3 associations are matched with the host: <devices> ... <controller type='pci' index='1' model='pcie-expander-bus'> <model name='pxb-pcie'/> <target busNr='252'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </controller> <controller type='pci' index='2' model='pcie-expander-bus'> <model name='pxb-pcie'/> <target busNr='248'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </controller> ... <controller type='pci' index='21' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='21' port='0x0'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </controller> <controller type='pci' index='22' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='22' port='0xa8'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> ... <hostdev mode='subsystem' type='pci' managed='no'> <source> <address domain='0x0009' bus='0x01' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x15' slot='0x00' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='no'> <source> <address domain='0x0019' bus='0x01' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x16' slot='0x00' function='0x0'/> </hostdev> <iommu model='smmuv3'> <driver pciBus='1' accel='on' ats='on' ril='on' ssidSize='20' oas='44' cmdqv='on' identifier='0'/> </iommu> <iommu model='smmuv3'> <driver pciBus='2' accel='on' ats='on' ril='on' ssidSize='20' oas='44' cmdqv='on' identifier='1'/> </iommu> </devices> This would get translated to a qemu command line with the arguments below: -device '{"driver":"pxb-pcie","bus_nr":252,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \ -device '{"driver":"pxb-pcie","bus_nr":248,"id":"pci.2","bus":"pcie.0","addr":"0x2"}' \ -device '{"driver":"pcie-root-port","port":0,"chassis":21,"id":"pci.21","bus":"pci.1","addr":"0x0"}' \ -device '{"driver":"pcie-root-port","port":168,"chassis":22,"id":"pci.22","bus":"pci.2","addr":"0x0"}' \ -device '{"driver":"arm-smmuv3","primary-bus":"pci.1","id":"iommu0","accel":true,"ats":true,"tegra241-cmdqv":true,"identifier":0,"ril":true,"pasid":true,"oas":44}' \ -device '{"driver":"arm-smmuv3","primary-bus":"pci.2","id":"iommu1","accel":true,"ats":true,"tegra241-cmdqv":true,"identifier":1,"ril":true,"pasid":true,"oas":44}' \ -device '{"driver":"vfio-pci","host":"0009:01:00.0","id":"hostdev0","bus":"pci.21","addr":"0x0"}' \ -device '{"driver":"vfio-pci","host":"0019:01:00.0","id":"hostdev1","bus":"pci.22","addr":"0x0"}' \ This series is on Github: https://github.com/NathanChenNVIDIA/libvirt/tree/cmdqv-02-26/ Thanks, Nathan Changes: - Support 'identifier' attribute for SMMUv3 [0] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/message/VRSS... [1] https://lore.kernel.org/qemu-devel/20260206144823.80655-1-skolothumtho@nvidi... [2] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/5GG76... Nathan Chen (3): qemu: Support NVIDIA Tegra241 CMDQV for SMMUv3 qemu: Support enumeration-independent SMMUv3 identifier tests: qemuxmlconfdata: provide cmdqv sample XML and CLI args docs/formatdomain.rst | 13 ++++ src/conf/domain_conf.c | 32 ++++++++++ src/conf/domain_conf.h | 2 + src/conf/domain_validate.c | 8 ++- src/conf/schemas/domaincommon.rng | 10 ++++ src/qemu/qemu_command.c | 2 + ...u-smmuv3-pci-bus-cmdqv.aarch64-latest.args | 40 +++++++++++++ ...mu-smmuv3-pci-bus-cmdqv.aarch64-latest.xml | 59 +++++++++++++++++++ .../iommu-smmuv3-pci-bus-cmdqv.xml | 49 +++++++++++++++ tests/qemuxmlconftest.c | 1 + 10 files changed, 215 insertions(+), 1 deletion(-) create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-cmdqv.aarch64-latest.args create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-cmdqv.aarch64-latest.xml create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-cmdqv.xml -- 2.43.0
From: Nathan Chen <nathanc@nvidia.com> Introduce support for a "cmdqv" IOMMU attribute which enables NVIDIA Tegra241 CMDQV, an extension for ARM SMMUv3. This supports passthrough of physical SMMU-CMDQ linked command queue from host space to a VM. Signed-off-by: Nathan Chen <nathanc@nvidia.com> --- docs/formatdomain.rst | 6 ++++++ src/conf/domain_conf.c | 15 +++++++++++++++ src/conf/domain_conf.h | 1 + src/conf/domain_validate.c | 5 ++++- src/conf/schemas/domaincommon.rng | 5 +++++ src/qemu/qemu_command.c | 1 + 6 files changed, 32 insertions(+), 1 deletion(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index 82788c15a2..74431838ff 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -9341,6 +9341,12 @@ Examples: The ``pciBus`` attribute notes the index of the controller that an IOMMU device is attached to. (QEMU/KVM and ``smmuv3`` model only) + ``cmdqv`` + The ``cmdqv`` attribute with possible values ``on`` and ``off`` can be used + to enable NVIDIA Tegra241 CMDQV, an extension for ARM SMMUv3 that supports + passthrough of physical SMMU-CMDQ linked command queue from host space to VM. + :since:`Since 12.1.0` (QEMU/KVM and ``smmuv3`` model only) + In case of ``virtio`` IOMMU device, the ``driver`` element can optionally contain ``granule`` subelement that allows to choose which granule will be used by default. It is useful when running guests with different page size diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 453e301041..6385420c10 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -14680,6 +14680,10 @@ virDomainIOMMUDefParseXML(virDomainXMLOption *xmlopt, &iommu->pci_bus, -1) < 0) return NULL; + if (virXMLPropTristateSwitch(driver, "cmdqv", VIR_XML_PROP_NONE, + &iommu->cmdqv) < 0) + return NULL; + if ((granule = virXPathNode("./driver/granule", ctxt))) { g_autofree char *mode = virXMLPropString(granule, "mode"); unsigned long long size; @@ -16781,6 +16785,7 @@ virDomainIOMMUDefEquals(const virDomainIOMMUDef *a, a->iotlb != b->iotlb || a->aw_bits != b->aw_bits || a->dma_translation != b->dma_translation || + a->cmdqv != b->cmdqv || a->granule != b->granule) return false; @@ -22537,6 +22542,12 @@ virDomainIOMMUDefCheckABIStability(virDomainIOMMUDef *src, dst->pci_bus, src->pci_bus); return false; } + if (src->cmdqv != dst->cmdqv) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target domain IOMMU device cmdqv value '%1$d' does not match source '%2$d'"), + dst->cmdqv, src->cmdqv); + return false; + } if (src->dma_translation != dst->dma_translation) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("Target domain IOMMU device dma translation '%1$s' does not match source '%2$s'"), @@ -28941,6 +28952,10 @@ virDomainIOMMUDefFormat(virBuffer *buf, virBufferAsprintf(&driverAttrBuf, " pciBus='%d'", iommu->pci_bus); } + if (iommu->cmdqv != VIR_TRISTATE_SWITCH_ABSENT) { + virBufferAsprintf(&driverAttrBuf, " cmdqv='%s'", + virTristateSwitchTypeToString(iommu->cmdqv)); + } if (iommu->granule != 0) { if (iommu->granule == -1) { virBufferAddLit(&driverChildBuf, "<granule mode='host'/>\n"); diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index a13f6d79e9..41dba29a4f 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -3075,6 +3075,7 @@ struct _virDomainIOMMUDef { virTristateSwitch xtsup; virTristateSwitch pt; int granule; /* -1 means 'host', 0 unset, page size in KiB otherwise */ + virTristateSwitch cmdqv; }; typedef enum { diff --git a/src/conf/domain_validate.c b/src/conf/domain_validate.c index 1ad614935f..cac5dabf06 100644 --- a/src/conf/domain_validate.c +++ b/src/conf/domain_validate.c @@ -3208,7 +3208,8 @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu) iommu->eim != VIR_TRISTATE_SWITCH_ABSENT || iommu->iotlb != VIR_TRISTATE_SWITCH_ABSENT || iommu->dma_translation != VIR_TRISTATE_SWITCH_ABSENT || - iommu->pci_bus >= 0) { + iommu->pci_bus >= 0 || + iommu->cmdqv != VIR_TRISTATE_SWITCH_ABSENT) { virReportError(VIR_ERR_XML_ERROR, _("iommu model '%1$s' doesn't support some additional attributes"), virDomainIOMMUModelTypeToString(iommu->model)); @@ -3231,6 +3232,7 @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu) iommu->aw_bits != 0 || iommu->dma_translation != VIR_TRISTATE_SWITCH_ABSENT || iommu->pci_bus >= 0 || + iommu->cmdqv != VIR_TRISTATE_SWITCH_ABSENT || iommu->granule != 0) { virReportError(VIR_ERR_XML_ERROR, _("iommu model '%1$s' doesn't support some additional attributes"), @@ -3243,6 +3245,7 @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu) if (iommu->pt != VIR_TRISTATE_SWITCH_ABSENT || iommu->xtsup != VIR_TRISTATE_SWITCH_ABSENT || iommu->pci_bus >= 0 || + iommu->cmdqv != VIR_TRISTATE_SWITCH_ABSENT || iommu->granule != 0) { virReportError(VIR_ERR_XML_ERROR, _("iommu model '%1$s' doesn't support some additional attributes"), diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincommon.rng index dafbdc63e7..2677207ae4 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -6352,6 +6352,11 @@ <data type="unsignedInt"/> </attribute> </optional> + <optional> + <attribute name="cmdqv"> + <ref name="virOnOff"/> + </attribute> + </optional> <optional> <element name="granule"> <choice> diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index a742998e4c..3735387ebd 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -6330,6 +6330,7 @@ qemuBuildPCINestedSmmuv3DevProps(const virDomainDef *def, "s:driver", "arm-smmuv3", "s:primary-bus", bus, "s:id", iommu->info.alias, + "B:tegra241-cmdqv", (iommu->cmdqv == VIR_TRISTATE_SWITCH_ON), NULL) < 0) return NULL; -- 2.43.0
-----Original Message----- From: Nathan Chen <nathanc@nvidia.com> Sent: 17 February 2026 23:20 To: devel@lists.libvirt.org Cc: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; Nicolin Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt Ochs <mochs@nvidia.com> Subject: [RFC PATCH 1/3] qemu: Support NVIDIA Tegra241 CMDQV for SMMUv3
From: Nathan Chen <nathanc@nvidia.com>
Introduce support for a "cmdqv" IOMMU attribute which enables NVIDIA Tegra241 CMDQV, an extension for ARM SMMUv3. This supports passthrough of physical SMMU-CMDQ linked command queue from host space to a VM.
Currently the v2 QEMU vCMDQ series uses the "tegra241-cmdqv" property to enable NVIDIA Tegra241 CMDQV. There is a suggestion to change this to a generic "cmdqv" property and probe the host CMDQV extension type from the kernel. Also, plan is to change it to ON_OFF_AUTO format like the suggestion for other properties like ril and ats etc I am currently looking into that. From a libvirt point of view, not sure this will have much impact, but just a heads up. Thanks, Shameer
Signed-off-by: Nathan Chen <nathanc@nvidia.com> --- docs/formatdomain.rst | 6 ++++++ src/conf/domain_conf.c | 15 +++++++++++++++ src/conf/domain_conf.h | 1 + src/conf/domain_validate.c | 5 ++++- src/conf/schemas/domaincommon.rng | 5 +++++ src/qemu/qemu_command.c | 1 + 6 files changed, 32 insertions(+), 1 deletion(-)
diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index 82788c15a2..74431838ff 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -9341,6 +9341,12 @@ Examples: The ``pciBus`` attribute notes the index of the controller that an IOMMU device is attached to. (QEMU/KVM and ``smmuv3`` model only)
+ ``cmdqv`` + The ``cmdqv`` attribute with possible values ``on`` and ``off`` can be used + to enable NVIDIA Tegra241 CMDQV, an extension for ARM SMMUv3 that supports + passthrough of physical SMMU-CMDQ linked command queue from host space to VM. + :since:`Since 12.1.0` (QEMU/KVM and ``smmuv3`` model only) + In case of ``virtio`` IOMMU device, the ``driver`` element can optionally contain ``granule`` subelement that allows to choose which granule will be used by default. It is useful when running guests with different page size diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 453e301041..6385420c10 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -14680,6 +14680,10 @@ virDomainIOMMUDefParseXML(virDomainXMLOption *xmlopt, &iommu->pci_bus, -1) < 0) return NULL;
+ if (virXMLPropTristateSwitch(driver, "cmdqv", VIR_XML_PROP_NONE, + &iommu->cmdqv) < 0) + return NULL; + if ((granule = virXPathNode("./driver/granule", ctxt))) { g_autofree char *mode = virXMLPropString(granule, "mode"); unsigned long long size; @@ -16781,6 +16785,7 @@ virDomainIOMMUDefEquals(const virDomainIOMMUDef *a, a->iotlb != b->iotlb || a->aw_bits != b->aw_bits || a->dma_translation != b->dma_translation || + a->cmdqv != b->cmdqv || a->granule != b->granule) return false;
@@ -22537,6 +22542,12 @@ virDomainIOMMUDefCheckABIStability(virDomainIOMMUDef *src, dst->pci_bus, src->pci_bus); return false; } + if (src->cmdqv != dst->cmdqv) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target domain IOMMU device cmdqv value '%1$d' does not match source '%2$d'"), + dst->cmdqv, src->cmdqv); + return false; + } if (src->dma_translation != dst->dma_translation) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("Target domain IOMMU device dma translation '%1$s' does not match source '%2$s'"), @@ -28941,6 +28952,10 @@ virDomainIOMMUDefFormat(virBuffer *buf, virBufferAsprintf(&driverAttrBuf, " pciBus='%d'", iommu->pci_bus); } + if (iommu->cmdqv != VIR_TRISTATE_SWITCH_ABSENT) { + virBufferAsprintf(&driverAttrBuf, " cmdqv='%s'", + virTristateSwitchTypeToString(iommu->cmdqv)); + } if (iommu->granule != 0) { if (iommu->granule == -1) { virBufferAddLit(&driverChildBuf, "<granule mode='host'/>\n"); diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index a13f6d79e9..41dba29a4f 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -3075,6 +3075,7 @@ struct _virDomainIOMMUDef { virTristateSwitch xtsup; virTristateSwitch pt; int granule; /* -1 means 'host', 0 unset, page size in KiB otherwise */ + virTristateSwitch cmdqv; };
typedef enum { diff --git a/src/conf/domain_validate.c b/src/conf/domain_validate.c index 1ad614935f..cac5dabf06 100644 --- a/src/conf/domain_validate.c +++ b/src/conf/domain_validate.c @@ -3208,7 +3208,8 @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu) iommu->eim != VIR_TRISTATE_SWITCH_ABSENT || iommu->iotlb != VIR_TRISTATE_SWITCH_ABSENT || iommu->dma_translation != VIR_TRISTATE_SWITCH_ABSENT || - iommu->pci_bus >= 0) { + iommu->pci_bus >= 0 || + iommu->cmdqv != VIR_TRISTATE_SWITCH_ABSENT) { virReportError(VIR_ERR_XML_ERROR, _("iommu model '%1$s' doesn't support some additional attributes"), virDomainIOMMUModelTypeToString(iommu->model)); @@ -3231,6 +3232,7 @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu) iommu->aw_bits != 0 || iommu->dma_translation != VIR_TRISTATE_SWITCH_ABSENT || iommu->pci_bus >= 0 || + iommu->cmdqv != VIR_TRISTATE_SWITCH_ABSENT || iommu->granule != 0) { virReportError(VIR_ERR_XML_ERROR, _("iommu model '%1$s' doesn't support some additional attributes"), @@ -3243,6 +3245,7 @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu) if (iommu->pt != VIR_TRISTATE_SWITCH_ABSENT || iommu->xtsup != VIR_TRISTATE_SWITCH_ABSENT || iommu->pci_bus >= 0 || + iommu->cmdqv != VIR_TRISTATE_SWITCH_ABSENT || iommu->granule != 0) { virReportError(VIR_ERR_XML_ERROR, _("iommu model '%1$s' doesn't support some additional attributes"), diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincommon.rng index dafbdc63e7..2677207ae4 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -6352,6 +6352,11 @@ <data type="unsignedInt"/> </attribute> </optional> + <optional> + <attribute name="cmdqv"> + <ref name="virOnOff"/> + </attribute> + </optional> <optional> <element name="granule"> <choice> diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index a742998e4c..3735387ebd 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -6330,6 +6330,7 @@ qemuBuildPCINestedSmmuv3DevProps(const virDomainDef *def, "s:driver", "arm-smmuv3", "s:primary-bus", bus, "s:id", iommu->info.alias, + "B:tegra241-cmdqv", (iommu->cmdqv == VIR_TRISTATE_SWITCH_ON), NULL) < 0) return NULL;
-- 2.43.0
On 2/18/2026 12:05 AM, Shameer Kolothum Thodi wrote:
Introduce support for a "cmdqv" IOMMU attribute which enables NVIDIA Tegra241 CMDQV, an extension for ARM SMMUv3. This supports passthrough of physical SMMU-CMDQ linked command queue from host space to a VM. Currently the v2 QEMU vCMDQ series uses the "tegra241-cmdqv" property to enable NVIDIA Tegra241 CMDQV. There is a suggestion to change this to a generic "cmdqv" property and probe the host CMDQV extension type from the kernel. Also, plan is to change it to ON_OFF_AUTO format like the suggestion for other properties like ril and ats etc
I am currently looking into that. From a libvirt point of view, not sure this will have much impact, but just a heads up.
Ok, I will keep track of the QEMU series refreshes and support the generic "cmdqv" property with ON_OFF_AUTO format when sent to the mailing list. Thanks, Nathan
From: Nathan Chen <nathanc@nvidia.com> Introduce support for an "identifier" IOMMU attribute that allows the CMDQV DSDT device to be correctly associated with its SMMU. Signed-off-by: Nathan Chen <nathanc@nvidia.com> --- docs/formatdomain.rst | 7 +++++++ src/conf/domain_conf.c | 17 +++++++++++++++++ src/conf/domain_conf.h | 1 + src/conf/domain_validate.c | 5 ++++- src/conf/schemas/domaincommon.rng | 5 +++++ src/qemu/qemu_command.c | 1 + 6 files changed, 35 insertions(+), 1 deletion(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index 74431838ff..c5eb4ac886 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -9347,6 +9347,13 @@ Examples: passthrough of physical SMMU-CMDQ linked command queue from host space to VM. :since:`Since 12.1.0` (QEMU/KVM and ``smmuv3`` model only) + ``identifier`` + The ``identifier`` attribute can be used to provide a stable per-device + identifier independent of enumeration order. This attribute is required for + supporting NVIDIA Tegra241 CMDQV, and is used to correctly associate the + CMDQV DSDT device with its SMMU. + :since:`Since 12.1.0` (QEMU/KVM and ``smmuv3`` model only) + In case of ``virtio`` IOMMU device, the ``driver`` element can optionally contain ``granule`` subelement that allows to choose which granule will be used by default. It is useful when running guests with different page size diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 6385420c10..0c11a3e2bc 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -2852,6 +2852,8 @@ virDomainIOMMUDefNew(void) iommu->pci_bus = -1; + iommu->identifier = -1; + return g_steal_pointer(&iommu); } @@ -14684,6 +14686,10 @@ virDomainIOMMUDefParseXML(virDomainXMLOption *xmlopt, &iommu->cmdqv) < 0) return NULL; + if (virXMLPropInt(driver, "identifier", 10, VIR_XML_PROP_NONE, + &iommu->identifier, -1) < 0) + return NULL; + if ((granule = virXPathNode("./driver/granule", ctxt))) { g_autofree char *mode = virXMLPropString(granule, "mode"); unsigned long long size; @@ -16786,6 +16792,7 @@ virDomainIOMMUDefEquals(const virDomainIOMMUDef *a, a->aw_bits != b->aw_bits || a->dma_translation != b->dma_translation || a->cmdqv != b->cmdqv || + a->identifier != b->identifier || a->granule != b->granule) return false; @@ -22548,6 +22555,12 @@ virDomainIOMMUDefCheckABIStability(virDomainIOMMUDef *src, dst->cmdqv, src->cmdqv); return false; } + if (src->identifier != dst->identifier) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target domain IOMMU device identifier '%1$d' does not match source '%2$d'"), + dst->identifier, + src->identifier); + } if (src->dma_translation != dst->dma_translation) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("Target domain IOMMU device dma translation '%1$s' does not match source '%2$s'"), @@ -28956,6 +28969,10 @@ virDomainIOMMUDefFormat(virBuffer *buf, virBufferAsprintf(&driverAttrBuf, " cmdqv='%s'", virTristateSwitchTypeToString(iommu->cmdqv)); } + if (iommu->identifier >= 0) { + virBufferAsprintf(&driverAttrBuf, " identifier='%d'", + iommu->identifier); + } if (iommu->granule != 0) { if (iommu->granule == -1) { virBufferAddLit(&driverChildBuf, "<granule mode='host'/>\n"); diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 41dba29a4f..2cf414d307 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -3076,6 +3076,7 @@ struct _virDomainIOMMUDef { virTristateSwitch pt; int granule; /* -1 means 'host', 0 unset, page size in KiB otherwise */ virTristateSwitch cmdqv; + int identifier; }; typedef enum { diff --git a/src/conf/domain_validate.c b/src/conf/domain_validate.c index cac5dabf06..9cd8efae52 100644 --- a/src/conf/domain_validate.c +++ b/src/conf/domain_validate.c @@ -3209,7 +3209,8 @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu) iommu->iotlb != VIR_TRISTATE_SWITCH_ABSENT || iommu->dma_translation != VIR_TRISTATE_SWITCH_ABSENT || iommu->pci_bus >= 0 || - iommu->cmdqv != VIR_TRISTATE_SWITCH_ABSENT) { + iommu->cmdqv != VIR_TRISTATE_SWITCH_ABSENT || + iommu->identifier >= 0) { virReportError(VIR_ERR_XML_ERROR, _("iommu model '%1$s' doesn't support some additional attributes"), virDomainIOMMUModelTypeToString(iommu->model)); @@ -3233,6 +3234,7 @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu) iommu->dma_translation != VIR_TRISTATE_SWITCH_ABSENT || iommu->pci_bus >= 0 || iommu->cmdqv != VIR_TRISTATE_SWITCH_ABSENT || + iommu->identifier >= 0 || iommu->granule != 0) { virReportError(VIR_ERR_XML_ERROR, _("iommu model '%1$s' doesn't support some additional attributes"), @@ -3246,6 +3248,7 @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu) iommu->xtsup != VIR_TRISTATE_SWITCH_ABSENT || iommu->pci_bus >= 0 || iommu->cmdqv != VIR_TRISTATE_SWITCH_ABSENT || + iommu->identifier >= 0 || iommu->granule != 0) { virReportError(VIR_ERR_XML_ERROR, _("iommu model '%1$s' doesn't support some additional attributes"), diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincommon.rng index 2677207ae4..ab142502e7 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -6357,6 +6357,11 @@ <ref name="virOnOff"/> </attribute> </optional> + <optional> + <attribute name="identifier"> + <data type="unsignedInt"/> + </attribute> + </optional> <optional> <element name="granule"> <choice> diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 3735387ebd..37bb876b66 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -6331,6 +6331,7 @@ qemuBuildPCINestedSmmuv3DevProps(const virDomainDef *def, "s:primary-bus", bus, "s:id", iommu->info.alias, "B:tegra241-cmdqv", (iommu->cmdqv == VIR_TRISTATE_SWITCH_ON), + "k:identifier", iommu->identifier, NULL) < 0) return NULL; -- 2.43.0
From: Nathan Chen <nathanc@nvidia.com> Provide sample XML and CLI args for the cmdqv and SMMUv3 identifier XML schema. Signed-off-by: Nathan Chen <nathanc@nvidia.com> --- ...u-smmuv3-pci-bus-cmdqv.aarch64-latest.args | 40 +++++++++++++ ...mu-smmuv3-pci-bus-cmdqv.aarch64-latest.xml | 59 +++++++++++++++++++ .../iommu-smmuv3-pci-bus-cmdqv.xml | 49 +++++++++++++++ tests/qemuxmlconftest.c | 1 + 4 files changed, 149 insertions(+) create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-cmdqv.aarch64-latest.args create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-cmdqv.aarch64-latest.xml create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-cmdqv.xml diff --git a/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-cmdqv.aarch64-latest.args b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-cmdqv.aarch64-latest.args new file mode 100644 index 0000000000..b144bdd08a --- /dev/null +++ b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-cmdqv.aarch64-latest.args @@ -0,0 +1,40 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/var/lib/libvirt/qemu/domain--1-guest \ +USER=test \ +LOGNAME=test \ +XDG_DATA_HOME=/var/lib/libvirt/qemu/domain--1-guest/.local/share \ +XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain--1-guest/.cache \ +XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-guest/.config \ +/usr/bin/qemu-system-aarch64 \ +-name guest=guest,debug-threads=on \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain--1-guest/master-key.aes"}' \ +-machine virt,usb=off,gic-version=2,dump-guest-core=off,memory-backend=mach-virt.ram,acpi=off \ +-accel tcg \ +-m size=1048576k \ +-object '{"qom-type":"memory-backend-ram","id":"mach-virt.ram","size":1073741824}' \ +-overcommit mem-lock=off \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid 1ccfd97d-5eb4-478a-bbe6-88d254c16db7 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,fd=1729,server=on,wait=off \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-boot strict=on \ +-device '{"driver":"pxb-pcie","bus_nr":252,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \ +-device '{"driver":"pxb-pcie","bus_nr":248,"id":"pci.2","bus":"pcie.0","addr":"0x2"}' \ +-device '{"driver":"pcie-root-port","port":0,"chassis":21,"id":"pci.3","bus":"pci.1","addr":"0x0"}' \ +-device '{"driver":"pcie-root-port","port":168,"chassis":22,"id":"pci.4","bus":"pci.2","addr":"0x0"}' \ +-device '{"driver":"arm-smmuv3","primary-bus":"pci.1","id":"iommu0","tegra241-cmdqv":true,"identifier":0}' \ +-device '{"driver":"arm-smmuv3","primary-bus":"pci.2","id":"iommu1","tegra241-cmdqv":true,"identifier":1}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-object '{"qom-type":"rng-random","id":"objrng0","filename":"/dev/urandom"}' \ +-device '{"driver":"virtio-rng-pci","rng":"objrng0","id":"rng0","bus":"pci.3","addr":"0x0"}' \ +-object '{"qom-type":"rng-random","id":"objrng1","filename":"/dev/urandom"}' \ +-device '{"driver":"virtio-rng-pci","rng":"objrng1","id":"rng1","bus":"pci.4","addr":"0x0"}' \ +-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ +-msg timestamp=on diff --git a/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-cmdqv.aarch64-latest.xml b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-cmdqv.aarch64-latest.xml new file mode 100644 index 0000000000..e7da39b33b --- /dev/null +++ b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-cmdqv.aarch64-latest.xml @@ -0,0 +1,59 @@ +<domain type='qemu'> + <name>guest</name> + <uuid>1ccfd97d-5eb4-478a-bbe6-88d254c16db7</uuid> + <memory unit='KiB'>1048576</memory> + <currentMemory unit='KiB'>1048576</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='aarch64' machine='virt'>hvm</type> + <boot dev='hd'/> + </os> + <features> + <gic version='2'/> + </features> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-aarch64</emulator> + <controller type='usb' index='0' model='none'/> + <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='252'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </controller> + <controller type='pci' index='2' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='248'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </controller> + <controller type='pci' index='3' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='21' port='0x0'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> + </controller> + <controller type='pci' index='4' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='22' port='0xa8'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> + </controller> + <audio id='1' type='none'/> + <memballoon model='none'/> + <rng model='virtio'> + <backend model='random'>/dev/urandom</backend> + <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> + </rng> + <rng model='virtio'> + <backend model='random'>/dev/urandom</backend> + <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> + </rng> + <iommu model='smmuv3'> + <driver pciBus='1' cmdqv='on' identifier='0'/> + </iommu> + <iommu model='smmuv3'> + <driver pciBus='2' cmdqv='on' identifier='1'/> + </iommu> + </devices> +</domain> diff --git a/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-cmdqv.xml b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-cmdqv.xml new file mode 100644 index 0000000000..5bf65ea26b --- /dev/null +++ b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-cmdqv.xml @@ -0,0 +1,49 @@ +<domain type='qemu'> + <name>guest</name> + <uuid>1ccfd97d-5eb4-478a-bbe6-88d254c16db7</uuid> + <memory unit='KiB'>1048576</memory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='aarch64' machine='virt'>hvm</type> + </os> + <devices> + <emulator>/usr/bin/qemu-system-aarch64</emulator> + <controller type='usb' model='none'/> + <memballoon model='none'/> + <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='252'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </controller> + <controller type='pci' index='2' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='248'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </controller> + <controller type='pci' index='3' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='21' port='0x0'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> + </controller> + <controller type='pci' index='4' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='22' port='0xa8'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> + </controller> + <rng model='virtio'> + <backend model='random'>/dev/urandom</backend> + <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> + </rng> + <rng model='virtio'> + <backend model='random'>/dev/urandom</backend> + <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> + </rng> + <iommu model='smmuv3'> + <driver pciBus='1' cmdqv='on' identifier='0'/> + </iommu> + <iommu model='smmuv3'> + <driver pciBus='2' cmdqv='on' identifier='1'/> + </iommu> + </devices> +</domain> diff --git a/tests/qemuxmlconftest.c b/tests/qemuxmlconftest.c index d2ab4a71b5..ba04534253 100644 --- a/tests/qemuxmlconftest.c +++ b/tests/qemuxmlconftest.c @@ -3116,6 +3116,7 @@ mymain(void) DO_TEST_CAPS_ARCH_LATEST("iommu-smmuv3", "aarch64"); DO_TEST_CAPS_ARCH_LATEST("iommu-smmuv3-pci-bus", "aarch64"); DO_TEST_CAPS_ARCH_LATEST("iommu-smmuv3-pci-bus-single", "aarch64"); + DO_TEST_CAPS_ARCH_LATEST("iommu-smmuv3-pci-bus-cmdqv", "aarch64"); DO_TEST_CAPS_LATEST("virtio-iommu-x86_64"); DO_TEST_CAPS_ARCH_LATEST("virtio-iommu-aarch64", "aarch64"); DO_TEST_CAPS_LATEST_PARSE_ERROR("virtio-iommu-wrong-machine"); -- 2.43.0
Hi Nathan,
-----Original Message----- From: Nathan Chen <nathanc@nvidia.com> Sent: 17 February 2026 23:20 To: devel@lists.libvirt.org Cc: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; Nicolin Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt Ochs <mochs@nvidia.com> Subject: [RFC PATCH 0/3] qemu: Support vCMDQ
Hi,
This is a follow up to the previous RFC patchset [0] for supporting Tegra241 vCMDQ, separated out from the series for accelerated SMMUv3.
This patchset is tested with Shameer's v2 qemu series [1] for vCMDQ support. The accelerated SMMUv3 series for qemu and Libvirt [2] are pre-requisites for vCMDQ support, as accelerated SMMUv3 must be enabled alongside vCMDQ.
A new 'cmdqv' <iommu> driver attribute is introduced to toggle enabling vCMDQ support. An additional 'identifier' <iommu> driver attribute is introduced to specify an enumeration-independent SMMUv3 device identifier. This identifier is used so that both Tegra DSDT and SMMUv3 IORT use the same identifier.
Though QEMU SMMUv3 introduces an additional "identifier" property for ACPI tables to use, it is not intended for users or management layer to specify. QEMU internally populates these during SMMUv3 device instance realization. In short, libvirt doesn't have to set this. In case it needs an id for something, can always use the default QOM device "id" field. eg: -device arm-smmuv3,primary-bus=pcie.1,id=smmuv3.1-pxb1,.. Thanks, Shameer
On 2/17/2026 11:53 PM, Shameer Kolothum Thodi wrote:
A new 'cmdqv' <iommu> driver attribute is introduced to toggle enabling vCMDQ support. An additional 'identifier' <iommu> driver attribute is introduced to specify an enumeration-independent SMMUv3 device identifier. This identifier is used so that both Tegra DSDT and SMMUv3 IORT use the same identifier. Though QEMU SMMUv3 introduces an additional "identifier" property for ACPI tables to use, it is not intended for users or management layer to specify. QEMU internally populates these during SMMUv3 device instance realization. In short, libvirt doesn't have to set this.
In case it needs an id for something, can always use the default QOM device "id" field.
eg: -device arm-smmuv3,primary-bus=pcie.1,id=smmuv3.1-pxb1,.
Thanks Shameer, I will remove the identifier attribute in the next revision. Nathan
participants (2)
-
Nathan Chen -
Shameer Kolothum Thodi