Thanks, but I don't understand why nobody mentions actually WHY it is being locked. I eventually found the answer by listening to Alex Williamson's talk on the subject, where he mentions that IOMMU chips lack "page faulting" machanism. He also suggests that it may be possible that in the future it will be added to x86 PCs, so I think that it is important that we talk about it rather than just take this memory pinning situation for granted.

Thanks for mentioning memory hotplugging - I've looked into virtio-mem project, which seems to address this, but as you mentioned (and thank you for stating it clearly as it took me a long time to figure it out) it requires manual interventions from the admin.

Best wishes,

Daniel

On Fri, Sep 15, 2023 at 02:41:50AM +0100, Daniel Krajnik wrote:

Hi

I'm trying to Passthrough a GPU to a VM and have it still share its free
memory with the host (just like it would with virtio-balloon). Is this
something virtio-mem is capable of? I've tried to run it, but it immediately
took over 9GB of memory:

virt-install --name test --cpu
cell0.cpus=0,cell0.memory=4194304,cell1.cpus=1,cell1.memory=4194304 --memory
maxMemory=65536,maxMemory.slots=8 --memdev
model=virtio-mem,target.node=0,target.block=2048,target.size=1000 --disk
size=400 --network network=ai
--location=/tmp/debian-12.1.0-amd64-netinst.iso --os-variant=debian11
--graphics none --extra-args "console=ttyS0

When using PCI device assignment, the VFIO driver requires that libvit
pin guest memory into RAM

  https://gitlab.com/libvirt/libvirt/-/blob/19484ccac5cb1586f9d10d3e6eb9b93ee82885c6/src/qemu/qemu_domain.c#L9567

so that's why you'll see the immediate memory usage. I dont' think there's
any way to workaround this limitation automatically. Closest you can get is
to actually hot-unplug the RAM your guest isn't using, but that needs manual
work to re-hot-plug it if the guest later needs more RAM, so that's not very
friendly.

With regards,
Daniel