[libvirt-users] Hard limit for the cpu usage of a VM
Hi , I was previously using xen and currently moved to KVM. I am using libvirt to manage these VMs. In den's credit scheduler , I had the ability to set a cap on the cpu usage for a VM. But I was not able to find a similar substitute in KVM. I find that we can use cgroups to provide shares for VM but that will be more like weight based and it doesn't set a hard cap for that VM. I tried using cpulimit but I find it inaccurate and we can give values only between 0-100. Thus, I think it cannot support multi core environments. Can any one suggest a method to set a hard limit on a VM's cpu usage? Thank you. - Regards, Sethuraman Subbiah Graduate Student - NC state University M.S in Computer Science
On 10/24/2011 01:50 PM, sethuraman subbiah wrote:
Hi ,
I was previously using xen and currently moved to KVM. I am using libvirt to manage these VMs. In den's credit scheduler , I had the ability to set a cap on the cpu usage for a VM. But I was not able to find a similar substitute in KVM. I find that we can use cgroups to provide shares for VM but that will be more like weight based and it doesn't set a hard cap for that VM. I tried using cpulimit but I find it inaccurate and we can give values only between 0-100. Thus, I think it cannot support multi core environments. Can any one suggest a method to set a hard limit on a VM's cpu usage? Thank you.
Libvirt 0.9.4 added CPU bandwidth limitation support, via <cputune>: http://libvirt.org/formatdomain.html#elementsCPUTuning. <period> and <quota> map to hard-coded caps via cgroups, nicer than the weighted numbers via <shares>. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
Hi, - Regards, Sethuraman Subbiah Graduate Student - NC state University M.S in Computer Science On Oct 24, 2011, at 4:20 PM, Eric Blake wrote:
On 10/24/2011 01:50 PM, sethuraman subbiah wrote:
Hi ,
I was previously using xen and currently moved to KVM. I am using libvirt to manage these VMs. In den's credit scheduler , I had the ability to set a cap on the cpu usage for a VM. But I was not able to find a similar substitute in KVM. I find that we can use cgroups to provide shares for VM but that will be more like weight based and it doesn't set a hard cap for that VM. I tried using cpulimit but I find it inaccurate and we can give values only between 0-100. Thus, I think it cannot support multi core environments. Can any one suggest a method to set a hard limit on a VM's cpu usage? Thank you.
Libvirt 0.9.4 added CPU bandwidth limitation support, via <cputune>: http://libvirt.org/formatdomain.html#elementsCPUTuning. <period> and <quota> map to hard-coded caps via cgroups, nicer than the weighted numbers via <shares>.
Great, I have one more question. Is it possible to set the quota during run time from the host os ? Using virsh ? Thank you.
-- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
On 10/24/2011 02:24 PM, sethuraman subbiah wrote:
Great, I have one more question. Is it possible to set the quota during run time from the host os ? Using virsh ? Thank you.
virsh schedinfo can both query and alter these cpu tunables, and that includes doing it while the guest is running. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
Hi, On Oct 24, 2011, at 4:30 PM, Eric Blake wrote:
On 10/24/2011 02:24 PM, sethuraman subbiah wrote:
Great, I have one more question. Is it possible to set the quota during run time from the host os ? Using virsh ? Thank you.
virsh schedinfo can both query and alter these cpu tunables, and that includes doing it while the guest is running.
I tried the virsh schedinfo. Here is the information regarding my libvirt : [root@sbc11 ssubbiah]# virsh version Compiled against library: libvir 0.9.4 Using library: libvir 0.9.4 Using API: QEMU 0.9.4 Running hypervisor: QEMU 0.12.1 When i use "[root@hgcc11 ssubbiah]# virsh schedinfo rubis_ws --cap 50" , I dont see the cap value being set and I am able to see only cpu shares. Scheduler : posix cpu_shares : 1024
-- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
On 10/24/2011 02:39 PM, sethuraman subbiah wrote:
Hi,
On Oct 24, 2011, at 4:30 PM, Eric Blake wrote:
On 10/24/2011 02:24 PM, sethuraman subbiah wrote:
Great, I have one more question. Is it possible to set the quota during run time from the host os ? Using virsh ? Thank you.
virsh schedinfo can both query and alter these cpu tunables, and that includes doing it while the guest is running.
I tried the virsh schedinfo. Here is the information regarding my libvirt :
[root@sbc11 ssubbiah]# virsh version Compiled against library: libvir 0.9.4 Using library: libvir 0.9.4 Using API: QEMU 0.9.4 Running hypervisor: QEMU 0.12.1
When i use "[root@hgcc11 ssubbiah]# virsh schedinfo rubis_ws --cap 50" , I dont see the cap value being set and I am able to see only cpu shares.
schedinfo --cap is obsolete (it's xen-only). Rather, you want: virsh schedinfo domain to list the current settings, and virsh schedinfo domain --set vcpu_period $value --set vcpu_quota $value I haven't used it myself; I'm only going off the documentation provided by Wen Congyang, who oversaw the addition of the features. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
On Oct 24, 2011, at 4:46 PM, Eric Blake wrote:
On 10/24/2011 02:39 PM, sethuraman subbiah wrote:
Hi,
When i use "[root@hgcc11 ssubbiah]# virsh schedinfo rubis_ws --cap 50" , I dont see the cap value being set and I am able to see only cpu shares.
schedinfo --cap is obsolete (it's xen-only). Rather, you want:
virsh schedinfo domain
to list the current settings, and
virsh schedinfo domain --set vcpu_period $value --set vcpu_quota $value
I haven't used it myself; I'm only going off the documentation provided by Wen Congyang, who oversaw the addition of the features.
Sorry, I noticed it late about using a xen command here. I tried virsh schedinfo domain. I dont think I have the capability to set quota : [root@sbc11 ssubbiah]# virsh schedinfo rubis_ws Scheduler : posix cpu_shares : 1024 [root@sbc11 ssubbiah]# virsh schedinfo rubis_ws --set vcpu_quota=50 Scheduler : posix error: invalid scheduler option: vcpu_quota=50 I get the same error for "virsh schedinfo rubies_ws --set vcpu_quota 50" . I find that my version is 0.9.4. [root@sbc11 ssubbiah]# virsh --version 0.9.4 Should I do some modifications,enable options or add some patch to get the capability to set cap ? Thanks for all your help.
-- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
On 10/24/2011 02:54 PM, sethuraman subbiah wrote:
[root@sbc11 ssubbiah]# virsh schedinfo rubis_ws --set vcpu_quota=50 Scheduler : posix error: invalid scheduler option: vcpu_quota=50
I get the same error for "virsh schedinfo rubies_ws --set vcpu_quota 50" .
I find that my version is 0.9.4.
[root@sbc11 ssubbiah]# virsh --version 0.9.4
Should I do some modifications,enable options or add some patch to get the capability to set cap ? Thanks for all your help.
I found this comment in the code: /* If we does not know VCPU<->PID mapping or all vcpu runs in the same * thread, we cannot control each vcpu. So we only modify cpu bandwidth * when each vcpu has a separated thread. I think that means that in order to use CPU caps, you have to also use <vcpupin> to call out the host CPU pinning to be used by the guest VCPUS. The original implementation was here, although I don't see much documentation in that patch series. https://www.redhat.com/archives/libvir-list/2011-July/msg01389.html Wen, can you help out here? -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
I find that my version is 0.9.4.
[root@sbc11 ssubbiah]# virsh --version 0.9.4
Should I do some modifications,enable options or add some patch to get the capability to set cap ? Thanks for all your help.
I found this comment in the code:
/* If we does not know VCPU<->PID mapping or all vcpu runs in the same * thread, we cannot control each vcpu. So we only modify cpu bandwidth * when each vcpu has a separated thread.
I think that means that in order to use CPU caps, you have to also use <vcpupin> to call out the host CPU pinning to be used by the guest VCPUS.
The original implementation was here, although I don't see much documentation in that patch series. https://www.redhat.com/archives/libvir-list/2011-July/msg01389.html
Wen, can you help out here?
Thanks Eric and Hi Wen. I did use <vcpupin> now . This is that part of my config xml file : <vcpu>3</vcpu> <cputune> <vcpupin vcpu='2' cpuset='0-2'/> </cputune> This shows the vcpu affinity : [root@hgcc11 cloning_script]# virsh vcpuinfo rubis_ws VCPU: 0 CPU: 2 State: running CPU time: 5.6s CPU Affinity: yyyy VCPU: 1 CPU: 1 State: running CPU time: 0.1s CPU Affinity: yyyy VCPU: 2 CPU: 0 State: running CPU time: 0.1s CPU Affinity: yyy- [root@hgcc11 cloning_script]# virsh schedinfo rubis_ws Scheduler : posix cpu_shares : 1024 Still I get it as an invalid option : [root@hgcc11 cloning_script]# virsh schedinfo rubis_ws --set vcpu_quota=1100 Scheduler : posix error: invalid scheduler option: vcpu_quota=1100 If I try to add the quota option to the xml file : <vcpu>3</vcpu> <cputune> <vcpupin vcpu='2' cpuset='0-2'/> <quota>-1</quota> </cputune> I get : [root@hgcc11 cloning_script]# virsh create images/rubis_ws.xml error: Failed to create domain from images/rubis_ws.xml error: Unable to set cpu bandwidth quota: No such file or directory Eventhough I am using libvirt 0.9.4, I think I am missing something.
-- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
- Regards, Sethuraman Subbiah Graduate Student - NC state University M.S in Computer Science
At 10/25/2011 05:13 AM, Eric Blake Write:
On 10/24/2011 02:54 PM, sethuraman subbiah wrote:
[root@sbc11 ssubbiah]# virsh schedinfo rubis_ws --set vcpu_quota=50 Scheduler : posix error: invalid scheduler option: vcpu_quota=50
I get the same error for "virsh schedinfo rubies_ws --set vcpu_quota 50" .
I find that my version is 0.9.4.
[root@sbc11 ssubbiah]# virsh --version 0.9.4
Should I do some modifications,enable options or add some patch to get the capability to set cap ? Thanks for all your help.
I found this comment in the code:
/* If we does not know VCPU<->PID mapping or all vcpu runs in the same * thread, we cannot control each vcpu. So we only modify cpu bandwidth * when each vcpu has a separated thread.
I think that means that in order to use CPU caps, you have to also use <vcpupin> to call out the host CPU pinning to be used by the guest VCPUS.
VCPU<->PID mapping is the thread id for each vcpu. qemu-kvm supports this feature. We use monitor command query-cpus to get VCPU<->PID mapping in libvirt(the function is: qemuProcessDetectVcpuPIDs()).
The original implementation was here, although I don't see much documentation in that patch series. https://www.redhat.com/archives/libvir-list/2011-July/msg01389.html
Wen, can you help out here?
It seems that the kernel does not support it. The cfs bandwidth was merged into kernel tree about some months ago. Thanks Wen Congyang
participants (3)
-
Eric Blake -
sethuraman subbiah -
Wen Congyang