Re: [libvirt] Globally Reserve Resources for Host
On Wed, Nov 28, 2012 at 3:26 PM, Daniel P. Berrange <berrange(a)redhat.com> wrote:
On Wed, Nov 28, 2012 at 02:27:53PM -0500, Dusty Mabe wrote:
> On Wed, Nov 14, 2012 at 11:22 AM, Dusty Mabe <dustymabe(a)gmail.com> wrote:
> > On Thu, Nov 1, 2012 at 11:32 PM, Dusty Mabe <dustymabe(a)gmail.com> wrote:
Sorry for not replying before. I've been thinking about this
today
and am specifically wondering about the possible implications for
a change we need to make to cgroups setup in libvirt
No worries, thank you for responding!!
The core issue is that it has become apparent that nesting cgroups
can cause some very significant performance / scalability problems
for the kernel. They have requested / recommended that libvirt make
its cgroup hierarchy as flat as possible to avoid this problem.
That makes sense. Does this affect how you feel about
https://www.redhat.com/archives/libvir-list/2011-March/msg01546.html ?
1. Match current hardcoded layout:
cgroup_layout="/::PROCESS::/libvirt/::DRIVER::/::VMNAME::"
2. Remove first level when used with systemd:
cgroup_layout="/::PROCESS::/::DRIVER::/::VMNAME::"
3. Combine 3rd/4th levels too
cgroup_layout="/::PROCESS::/::DRIVER::-::VMNAME::"
4. Ignore current libvirtd placement completely and create in the root:
cgroup_layout="/libvirt/::DRIVER::-::VMNAME::"
5. Use UUID instead of VM name
cgroup_layout="/libvirt/::DRIVER::-::VMUUID::"
A couple more options:
6. cgroup_layout="/libvirt/::DRIVER::/VMNAME"
7. cgroup_layout="/libvirt/::DRIVER::/VMUUID"
I'm sure you've noticed that this plan doesn't leave much
scope for the
approach you outlined above, since the 'qemu' level will disappear by
default.
It looks like you are right, by default there would be no driver level cgroup.
What if we still had settings in qemu.conf for something like
"reservedHostMem" and "reservedHostCpus" (maybe those names don't
make
sense, but I think you know what i mean) that would only be enforced
if the cgroup_layout had a */::DRIVER::/* component (cases 1,2,6,7).
In this case there would be a 'qemu' level. We could explain this
dependency in the conf file and print out an error/warning in the logs
if it wasn't followed.
It is on my plate to get these changes done for Fedora 19 / RHEL-7
releases.
Daniel
Thanks again for responding! I appreciate your input and it is helpful
to know the path that libvirt is going down. I know what I am
suggesting above may be a long shot but i figure it's worth at least
airing the idea. I like it because too often my hosts get
oversubscribed from other people creating guests and can cause wacky
behavior (my problem, not yours).
Dusty Mabe