8:41 p.m.
Hi
I'm trying to Passthrough a GPU to a VM and have it still share its free
memory with the host (just like it would with virtio-balloon). Is this
something virtio-mem is capable of? I've tried to run it, but it
immediately took over 9GB of memory:
virt-install --name test --cpu
cell0.cpus=0,cell0.memory=4194304,cell1.cpus=1,cell1.memory=4194304
--memory maxMemory=65536,maxMemory.slots=8 --memdev
model=virtio-mem,target.node=0,target.block=2048,target.size=1000 --disk
size=400 --network network=ai
--location=/tmp/debian-12.1.0-amd64-netinst.iso --os-variant=debian11
--graphics none --extra-args "console=ttyS0
Best wishes,
Daniel
10:59 a.m.
On Fri, Sep 15, 2023 at 02:41:50AM +0100, Daniel Krajnik wrote:
Hi
I'm trying to Passthrough a GPU to a VM and have it still share its free
memory with the host (just like it would with virtio-balloon). Is this
something virtio-mem is capable of? I've tried to run it, but it immediately
took over 9GB of memory:
virt-install --name test --cpu
cell0.cpus=0,cell0.memory=4194304,cell1.cpus=1,cell1.memory=4194304 --memory
maxMemory=65536,maxMemory.slots=8 --memdev
model=virtio-mem,target.node=0,target.block=2048,target.size=1000 --disk
size=400 --network network=ai
--location=/tmp/debian-12.1.0-amd64-netinst.iso --os-variant=debian11
--graphics none --extra-args "console=ttyS0
When using PCI device assignment, the VFIO driver requires that libvit
pin guest memory into RAM
https://gitlab.com/libvirt/libvirt/-/blob/19484ccac5cb1586f9d10d3e6eb9b93...
so that's why you'll see the immediate memory usage. I dont' think
there's
any way to workaround this limitation automatically. Closest you can get is
to actually hot-unplug the RAM your guest isn't using, but that needs manual
work to re-hot-plug it if the guest later needs more RAM, so that's not very
friendly.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
11:10 a.m.
Thanks, but I don't understand why nobody mentions actually WHY it is
being locked. I eventually found the answer by listening to Alex
Williamson <https://www.youtube.com/watch?v=WFkdTFTOTpA&t=1099s>'s talk
on the subject, where he mentions that IOMMU chips lack "page faulting"
machanism. He also suggests that it may be possible that in the future
it will be added to x86 PCs, so I think that it is important that we
talk about it rather than just take this memory pinning situation for
granted.
Thanks for mentioning memory hotplugging - I've looked into virtio-mem
project, which seems to address this, but as you mentioned (and thank
you for stating it clearly as it took me a long time to figure it out)
it requires manual interventions from the admin.
Best wishes,
Daniel
On 20/09/2023 16:59, Daniel P. Berrangé wrote:
On Fri, Sep 15, 2023 at 02:41:50AM +0100, Daniel Krajnik wrote:
> Hi
>
> I'm trying to Passthrough a GPU to a VM and have it still share its free
> memory with the host (just like it would with virtio-balloon). Is this
> something virtio-mem is capable of? I've tried to run it, but it immediately
> took over 9GB of memory:
>
> virt-install --name test --cpu
> cell0.cpus=0,cell0.memory=4194304,cell1.cpus=1,cell1.memory=4194304 --memory
> maxMemory=65536,maxMemory.slots=8 --memdev
> model=virtio-mem,target.node=0,target.block=2048,target.size=1000 --disk
> size=400 --network network=ai
> --location=/tmp/debian-12.1.0-amd64-netinst.iso --os-variant=debian11
> --graphics none --extra-args "console=ttyS0
When using PCI device assignment, the VFIO driver requires that libvit
pin guest memory into RAM
https://gitlab.com/libvirt/libvirt/-/blob/19484ccac5cb1586f9d10d3e6eb9b93...
so that's why you'll see the immediate memory usage. I dont' think
there's
any way to workaround this limitation automatically. Closest you can get is
to actually hot-unplug the RAM your guest isn't using, but that needs manual
work to re-hot-plug it if the guest later needs more RAM, so that's not very
friendly.
With regards,
Daniel
429
days inactive
434
days old
2 comments
2 participants
participants (2)
-
Daniel Krajnik -
Daniel P. Berrangé