4:28 a.m.
Hello!
vhost-user has a small limitation: guest memory must be shared. However, this simple
requirement is satisfied by Libvirt only in
very complicated case:
1. We have to specify NUMA configuration, because we can have "shared" attribute
only for node descriptors inside "NUMA" section.
2. We have to specify huge page size, because memory-backend-file is used only in this
case.
Isn't it a problem? In order to do a simple thing (use vhost-user) we have to add two
more quote unobvious things, making the whole
stuff significantly more complicated. This creates even more problems on the level above,
for example in order to get OpenStack
working with userspace networking, we have to edit all flavors and rebuild all instances.
And no single documentation mentions it.
Shouldn't Libvirt simply detect usage of vhost-user, and build some minimal
configuration with shared memory? For example, it could
be memory-backend-file on /dev/mem.
If the community agrees that it's a good idea, improving the usability, i can propose
patches.
Kind regards,
Pavel Fedin
Senior Engineer
Samsung Electronics Research center Russia
4:37 a.m.
On Thu, Feb 11, 2016 at 01:28:47PM +0300, Pavel Fedin wrote:
Hello!
vhost-user has a small limitation: guest memory must be shared. However, this simple
requirement is satisfied by Libvirt only in
very complicated case:
1. We have to specify NUMA configuration, because we can have "shared"
attribute only for node descriptors inside "NUMA" section.
2. We have to specify huge page size, because memory-backend-file is used only in this
case.
Isn't it a problem? In order to do a simple thing (use vhost-user) we have to add
two more quote unobvious things, making the whole
stuff significantly more complicated. This creates even more problems on the level above,
for example in order to get OpenStack
working with userspace networking, we have to edit all flavors and rebuild all instances.
And no single documentation mentions it.
Shouldn't Libvirt simply detect usage of vhost-user, and build some minimal
configuration with shared memory? For example, it could
be memory-backend-file on /dev/mem.
If the community agrees that it's a good idea, improving the usability, i can
propose patches.
Historically QEMU had a pointless check on the path passed in, to enforce
that it was only hugetlbfs, so could not just pass in a regular tmpfs
file. I think we removed that in QEMU 2.5. I think it is a valid enhance
<memoryBacking> to allow specification of "shared" memory backing which
would be mapping to a regular tmpfs.
I don't think we should magically do anything based on existance of
vhost-user though - changes in way the guest memory is allocated should
always require explicit user configuration.
We could however report an error VIR_ERR_CONFIG_UNSUPPORTED if the user
provided a vhost-user device and forgot to request shared memory, given
that its an unusable combination.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
4:54 a.m.
Hello!
Historically QEMU had a pointless check on the path passed in, to
enforce
that it was only hugetlbfs, so could not just pass in a regular tmpfs
file. I think we removed that in QEMU 2.5. I think it is a valid enhance
<memoryBacking> to allow specification of "shared" memory backing which
would be mapping to a regular tmpfs.
I don't think we should magically do anything based on existance of
vhost-user though - changes in way the guest memory is allocated should
always require explicit user configuration.
Ok, then would it be a good compromise if we require <memoryBacking>, and only
implicitly add "shared" if we have vhost-user
devices? This way we would not change the way the guest memory is allocated.
IMHO being able to manually specify "shared" both in <numa> and in
<memoryBacking> would be ambiguous.
Kind regards,
Pavel Fedin
Senior Engineer
Samsung Electronics Research center Russia
8:11 a.m.
On Thu, Feb 11, 2016 at 01:54:49PM +0300, Pavel Fedin wrote:
Hello!
> Historically QEMU had a pointless check on the path passed in, to enforce
> that it was only hugetlbfs, so could not just pass in a regular tmpfs
> file. I think we removed that in QEMU 2.5. I think it is a valid enhance
> <memoryBacking> to allow specification of "shared" memory backing
which
> would be mapping to a regular tmpfs.
>
> I don't think we should magically do anything based on existance of
> vhost-user though - changes in way the guest memory is allocated should
> always require explicit user configuration.
Ok, then would it be a good compromise if we require <memoryBacking>, and only
implicitly add "shared" if we have vhost-user
devices? This way we would not change the way the guest memory is allocated.
Adding shared implicitly *will* change the way guest memory is allocated,
as it will have to use tmpfs to make it shared.
IMHO being able to manually specify "shared" both in
<numa> and
in <memoryBacking> would be ambiguous.
That's not really any different to what we have already with NUMA.
The top level setting would apply as the default, and the NUMA level
settings override it if needed.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
8:35 a.m.
Hello!
> Ok, then would it be a good compromise if we require
<memoryBacking>, and only implicitly
add "shared" if we have vhost-user
> devices? This way we would not change the way the guest memory is allocated.
Adding shared implicitly *will* change the way guest memory is allocated,
as it will have to use tmpfs to make it shared.
You perhaps didn't get my idea. I meant - we will still need to specify
<memoryBacking> with huge pages, just no <numa>. Therefore, the memory will be
allocated via file backend from hugetlbfs. Only mode will be changed implicitly (private
-> shared).
> IMHO being able to manually specify "shared" both in
<numa> and
> in <memoryBacking> would be ambiguous.
That's not really any different to what we have already with NUMA.
The top level setting would apply as the default, and the NUMA level
settings override it if needed.
Well, the only little drawback would be necessity to add "shared" by itself.
This would require additional patching to clients (e. g. openstack).
Kind regards,
Pavel Fedin
Senior Engineer
Samsung Electronics Research center Russia
3236
days inactive
3236
days old
4 comments
2 participants
participants (2)
-
Daniel P. Berrange -
Pavel Fedin