Re: [libvirt] [PATCH RESEND RFC v4 1/6] Introduce the function virCgroupForVcpu
On 07/21/2011 08:34 AM, Daniel P. Berrange wrote:
On Thu, Jul 21, 2011 at 07:54:05AM -0500, Adam Litke wrote:
> Added Anthony to give him the opportunity to address the finer points of
> this one especially with respect to the qemu IO thread(s).
>
> This feature is really about capping the compute performance of a VM
> such that we get consistent top end performance. Yes, qemu has non-VCPU
> threads that this patch set doesn't govern, but that's the point. We
> are not attempting to throttle IO or device emulation with this feature.
> It's true that an IO-intensive guest may consume more host resources
> than a compute intensive guest, but they should still have equal top-end
> CPU performance when viewed from the guest's perspective.
I could be mis-understanding, what you're trying to achieve,
here, so perhaps we should consider an example.
- A machine has 4 physical CPUs
- There are 4 guests on the machine
- Each guest has 2 virtual CPUs
So we've overcommit the host CPU resources x2 here.
Lets say that we want to use this feature to ensure consistent
top end performance of every guest, splitting the host pCPUs
resources evenly across all guests, so each guest is ensured
1 pCPU worth of CPU time overall.
This patch lets you do this by assigning caps per VCPU. So
in this example, each VCPU cgroup would have to be configured
to cap the VCPUs at 50% of a single pCPU.
This leaves the other QEMU threads uncapped / unaccounted
for. If any one guest causes non-trivial compute load in
a non-VCPU thread, this can/will impact the top-end compute
performance of all the other guests on the machine.
But this is not undesirable behavior. You're mixing up consistency and
top end performance. They are totally different things and I think most
consumers of capping really only care about the later.
If we did caps per VM, then you could set the VM cgroup
such that the VM as a whole had 100% of a single pCPU.
Consistent performance is very hard to achieve. The desire is to cap
performance not just within a box, but also across multiple different
boxes with potentially different versions of KVM. The I/O threads are
basically hypervisor overhead. That's going to change over time.
If a guest is 100% compute bound, it can use its full
100% of a pCPU allocation in vCPU threads. If any other
guest is causing CPU time in a non-VCPU thread, it cannot
impact the top end compute performance of VCPU threads in
the other guests.
A per-VM cap would, however, mean a guest with 2 vCPUs
could have unequal scheduling, where one vCPU claimed 75%
of the pCPU and the othe vCPU got left with only 25%.
So AFAICT, per-VM cgroups is better for ensuring top
end compute performance of a guest as a whole, but
per-VCPU cgroups can ensure consistent top end performance
across vCPUs within a guest.
And the later is the primary use case as far as I can tell.
If I'm a user, I'll be very confused if I have a 4 VCPU guest, run 1
instance of specint, and get X as the result. I then run 4 instances of
specint and get X as the result. My expectation is to get 4X as the
result because I'm running 4 instances.
Getting X as the result is going to appear to be a bug. I cannot
imagine that any normal user would expect this to be the behavior.
Regards,
Anthony Liguori
IMHO, per-VM cgroups is the more useful because it is the
only way to stop guests impacting each other, but there
could be additional benefits of *also* have per-VCPU cgroups
if you want to ensure fairness of top-end performance across
vCPUs inside a single VM.
Regards,
Daniel