On 2/6/19 1:51 PM, Erik Skultety wrote:
On Wed, Feb 06, 2019 at 01:44:53PM -0200, Daniel Henrique Barboza wrote:
QEMU virtual machines with PCI passthrough of certain devices, such as the NVIDIA Tesla V100 GPU, requires allocation of an amount of memory pages that can break KVM limits. When that Wow, ^this means only 1 thing, this Tesla card requires more than 1GB extra memory to be locked, is that right?
Yes. Just checked in the dmesg of the guest out of curiosity. The very fist big DMA call (the one that causes the IPC_LOCK issue) is a bit larger than that already. There are a few nuances to be considered. As I mentioned earlier in Alex's reply, this is a custom QEMU that has NVLink2 passthrough support. The V100 card has 16GB of RAM that is allocated in its own NUMA node, and the pseries machine emulates this behavior. So I am not sure if these DMAs are locking regular host RAM or the GPU memory to populate the NUMA node the guest uses. If the latter, then a case can be made about allowing QEMU running with IPC_LOCK - even if we limit it to the pseries machine assigning a Tesla V100 device (although Yuval reported a similar issue with a Mellanox mlx5 card replied in the previous thread ...). Thanks, DHB
Erik