Re: [PATCH RFC v2 00/13] IOMMUFD Generic interface

[Cc+ Steve, libvirt, Daniel, Laine] On Tue, 20 Sep 2022 16:56:42 -0300 Jason Gunthorpe <jgg(a)nvidia.com&gt; wrote: Steve won't be happy to hear that, QEMU support exists but isn't yet merged. I'll try to summarize the proposed change so that we can get better advice from libvirt folks, or potentially anyone else managing locked memory limits for device assignment VMs. Background: when a DMA range, ex. guest RAM, is mapped to a vfio device, we use the system IOMMU to provide GPA to HPA translation for assigned devices. Unlike CPU page tables, we don't generally have a means to demand fault these translations, therefore the memory target of the translation is pinned to prevent that it cannot be swapped or relocated, ie. to guarantee the translation is always valid. The issue is where we account these pinned pages, where accounting is necessary such that a user cannot lock an arbitrary number of pages into RAM to generate a DoS attack. Duplicate accounting should be resolved by iommufd, but is outside the scope of this discussion. Currently, vfio tests against the mm_struct.locked_vm relative to rlimit(RLIMIT_MEMLOCK), which reads task->signal->rlim[limit].rlim_cur, where task is the current process. This is the same limit set via the setrlimit syscall used by prlimit(1) and reported via 'ulimit -l'. Note that in both cases above, we're dealing with a task, or process limit and both prlimit and ulimit man pages describe them as such. iommufd supposes instead, and references existing kernel implementations, that despite the descriptions above these limits are actually meant to be user limits and therefore instead charges pinned pages against user_struct.locked_vm and also marks them in mm_struct.pinned_vm. The proposed algorithm is to read the _task_ locked memory limit, then attempt to charge the _user_ locked_vm, such that user_struct.locked_vm cannot exceed the task locked memory limit. This obviously has implications. AFAICT, any management tool that doesn't instantiate assigned device VMs under separate users are essentially untenable. For example, if we launch VM1 under userA and set a locked memory limit of 4GB via prlimit to account for an assigned device, that works fine, until we launch VM2 from userA as well. In that case we can't simply set a 4GB limit on the VM2 task because there's already 4GB charged against user_struct.locked_vm for VM1. So we'd need to set the VM2 task limit to 8GB to be able to launch VM2. But not only that, we'd need to go back and also set VM1's task limit to 8GB or else it will fail if a DMA mapped memory region is transient and needs to be re-mapped. Effectively any task under the same user and requiring pinned memory needs to have a locked memory limit set, and updated, to account for all tasks using pinned memory by that user. How does this affect known current use cases of locked memory management for assigned device VMs? Does qemu://system by default sandbox into per VM uids or do they all use the qemu user by default. I imagine qemu://session mode is pretty screwed by this, but I also don't know who/where locked limits are lifted for such VMs. Boxes, who I think now supports assigned device VMs, could also be affected. If a vfio compatible interface isn't transparently compatible, then I have a hard time understanding its value. Please correct my above description and implications, but I suspect these are not just theoretical ABI compat issues. Thanks, Alex