question about virtio related options: ats=on and iommu=on
Hi, I have a question about the virtio related options, could someone please help to confirm? In my understanding, the ats='on' should depend on the "iommu=on", but I'm not sure about it. Please check the details below. Thank you! Details: 1. set vm with intel iommu device, and enable iommu for virtio filesystem device: # virsh dumpxml rhel --xpath //iommu <iommu model="intel"> <driver intremap="on" caching_mode="on" iotlb="on"/> </iommu> Set iommu="on" ats="on" for virtio filesystem device: # virsh dumpxml rhel --xpath //filesystem <filesystem type="mount" accessmode="passthrough"> <driver type="virtiofs" queue="512" iommu="on" ats="on"/> <binary path="/usr/libexec/virtiofsd" xattr="on"> <cache mode="none"/> </binary> <source dir="/path1"/> <target dir="mount_tag1"/> <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/> </filesystem> 2. failed to start the vm since "iommu_platform=true is not supported by the device" # virsh start rhel error: Failed to start domain 'rhel' error: internal error: qemu unexpectedly closed the monitor: 2023-01-09T03:48:03.198629Z qemu-kvm: -device {"driver":"vhost-user-fs-pci","iommu_platform":true,"ats":true,"id":"fs0","chardev":"chr-vu-fs0","queue-size":512,"tag":"mount_tag1","bus":"pci.4","addr":"0x0"}: iommu_platform=true is not supported by the device 3. update the xml to be with only ats="on", vm can start successfully, which is not expected. # virsh dumpxml rhel --xpath //filesystem <filesystem type="mount" accessmode="passthrough"> <driver type="virtiofs" queue="512" ats="on"/> <binary path="/usr/libexec/virtiofsd" xattr="on"> <cache mode="none"/> </binary> <source dir="/path1"/> <target dir="mount_tag1"/> <alias name="fs0"/> <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/> </filesystem> the qemu cmd line: -chardev socket,id=chr-vu-fs0,path=/var/lib/libvirt/qemu/domain-5-rhel/fs0-fs.sock \ -device '{"driver":"vhost-user-fs-pci","ats":true,"id":"fs0","chardev":"chr-vu-fs0","queue-size":512,"tag":"mount_tag1","bus":"pci.4","addr":"0x0"}' Refer to libvirt.org: QEMU's virtio devices have some attributes related to the virtio transport under the driver element: The iommu attribute enables the use of emulated IOMMU by the device. The attribute ats controls the Address Translation Service support for PCIe devices. This is needed to make use of IOTLB support (see IOMMU devices). Possible values are on or off. In my understanding, the ats='on' should depend on the iommu='on', so if the device does not support iommu, it should not support ats, eigher. I'm not sure if the understanding is correct. Could someone please help to confirm it? Thank you! Yalan
On Mon, Jan 9, 2023 at 12:01 PM Yalan Zhang <yalzhang@redhat.com> wrote:
Hi,
I have a question about the virtio related options, could someone please help to confirm? In my understanding, the ats='on' should depend on the "iommu=on", but I'm not sure about it. Please check the details below. Thank you!
Details: 1. set vm with intel iommu device, and enable iommu for virtio filesystem device: # virsh dumpxml rhel --xpath //iommu <iommu model="intel"> <driver intremap="on" caching_mode="on" iotlb="on"/> </iommu>
Set iommu="on" ats="on" for virtio filesystem device: # virsh dumpxml rhel --xpath //filesystem <filesystem type="mount" accessmode="passthrough"> <driver type="virtiofs" queue="512" iommu="on" ats="on"/> <binary path="/usr/libexec/virtiofsd" xattr="on"> <cache mode="none"/> </binary> <source dir="/path1"/> <target dir="mount_tag1"/> <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/> </filesystem>
2. failed to start the vm since "iommu_platform=true is not supported by the device" # virsh start rhel error: Failed to start domain 'rhel' error: internal error: qemu unexpectedly closed the monitor: 2023-01-09T03:48:03.198629Z qemu-kvm: -device {"driver":"vhost-user-fs-pci","iommu_platform":true,"ats":true,"id":"fs0","chardev":"chr-vu-fs0","queue-size":512,"tag":"mount_tag1","bus":"pci.4","addr":"0x0"}: iommu_platform=true is not supported by the device
The error msg comes from the qemu commit https://gitlab.com/qemu-project/qemu/-/commit/e65902a913bf31ba79a83a3bd36211... See the commit msg for more info
3. update the xml to be with only ats="on", vm can start successfully, which is not expected. # virsh dumpxml rhel --xpath //filesystem <filesystem type="mount" accessmode="passthrough"> <driver type="virtiofs" queue="512" ats="on"/> <binary path="/usr/libexec/virtiofsd" xattr="on"> <cache mode="none"/> </binary> <source dir="/path1"/> <target dir="mount_tag1"/> <alias name="fs0"/> <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/> </filesystem>
the qemu cmd line: -chardev socket,id=chr-vu-fs0,path=/var/lib/libvirt/qemu/domain-5-rhel/fs0-fs.sock \ -device '{"driver":"vhost-user-fs-pci","ats":true,"id":"fs0","chardev":"chr-vu-fs0","queue-size":512,"tag":"mount_tag1","bus":"pci.4","addr":"0x0"}'
Refer to libvirt.org: QEMU's virtio devices have some attributes related to the virtio transport under the driver element: The iommu attribute enables the use of emulated IOMMU by the device. The attribute ats controls the Address Translation Service support for PCIe devices. This is needed to make use of IOTLB support (see IOMMU devices). Possible values are on or off.
In my understanding, the ats='on' should depend on the iommu='on', so if the device does not support iommu, it should not support ats, eigher. I'm not sure if the understanding is correct.
Not really. ats means Address Translation Services <https://www.intel.com/content/www/us/en/docs/programmable/683686/20-4/address-translation-services-ats.html> a capability for PCIe device. iommu means device can be used on IOMMU platform <https://github.com/qemu/qemu/blob/master/qapi/virtio.json#L315>
Could someone please help to confirm it? Thank you!
Yalan
participants (2)
-
Han Han -
Yalan Zhang