On 2020/7/9 下午10:10, Peter Xu wrote:
On Thu, Jul 09, 2020 at 01:58:33PM +0800, Jason Wang wrote:
>>> - If we care the performance, it's better to implement the MAP event for
>>> vhost, otherwise it could be a lot of IOTLB miss
>> I feel like these are two things.
>>
>> So far what we are talking about is whether vt-d should have knowledge about
>> what kind of events one iommu notifier is interested in. I still think we
>> should keep this as answered in question 1.
>>
>> The other question is whether we want to switch vhost from UNMAP to MAP/UNMAP
>> events even without vDMA, so that vhost can establish the mapping even before
>> IO starts. IMHO it's doable, but only if the guest runs DPDK workloads.
When
>> the guest is using dynamic iommu page mappings, I feel like that can be even
>> slower, because then the worst case is for each IO we'll need to vmexit
twice:
>>
>> - The first vmexit caused by an invalidation to MAP the page tables, so
vhost
>> will setup the page table before IO starts
>>
>> - IO/DMA triggers and completes
>>
>> - The second vmexit caused by another invalidation to UNMAP the page tables
>>
>> So it seems to be worse than when vhost only uses UNMAP like right now. At
>> least we only have one vmexit (when UNMAP). We'll have a vhost translate()
>> request from kernel to userspace, but IMHO that's cheaper than the vmexit.
>
> Right but then I would still prefer to have another notifier.
>
> Since vtd_page_walk has nothing to do with device IOTLB. IOMMU have a
> dedicated command for flushing device IOTLB. But the check for
> vtd_as_has_map_notifier is used to skip the device which can do demand
> paging via ATS or device specific way. If we have two different notifiers,
> vhost will be on the device iotlb notifier so we don't need it at all?
But we can still have iommu notifier that only registers to UNMAP even after we
introduce dev-iotlb notifier? We don't want to do page walk for them as well.
TCG should be the only one so far, but I don't know.. maybe there can still be
new ones?
I think you're right. But looking at the codes, it looks like the check
of vtd_as_has_map_notifier() was only used in:
1) vtd_iommu_replay()
2) vtd_iotlb_page_invalidate_notify() (PSI)
For the replay, it's expensive anyhow. For PSI, I think it's just about
one or few mappings, not sure it will have obvious performance impact.
And I had two questions:
1) The codes doesn't check map for DSI or GI, does this match what spec
said? (It looks to me the spec is unclear in this part)
2) for the replay() I don't see other implementations (either spapr or
generic one) that did unmap (actually they skip unmap explicitly), any
reason for doing this in intel IOMMU?
Thanks
Thanks,