2:50 p.m.
Hi ,
I was previously using xen and currently moved to KVM. I am using libvirt to manage these
VMs. In den's credit scheduler , I had the ability to set a cap on the cpu usage for a
VM. But I was not able to find a similar substitute in KVM. I find that we can use cgroups
to provide shares for VM but that will be more like weight based and it doesn't set a
hard cap for that VM. I tried using cpulimit but I find it inaccurate and we can give
values only between 0-100. Thus, I think it cannot support multi core environments. Can
any one suggest a method to set a hard limit on a VM's cpu usage? Thank you.
-
Regards,
Sethuraman Subbiah
Graduate Student - NC state University
M.S in Computer Science
3:20 p.m.
On 10/24/2011 01:50 PM, sethuraman subbiah wrote:
Hi ,
I was previously using xen and currently moved to KVM. I am using libvirt to manage these
VMs. In den's credit scheduler , I had the ability to set a cap on the cpu usage for a
VM. But I was not able to find a similar substitute in KVM. I find that we can use cgroups
to provide shares for VM but that will be more like weight based and it doesn't set a
hard cap for that VM. I tried using cpulimit but I find it inaccurate and we can give
values only between 0-100. Thus, I think it cannot support multi core environments. Can
any one suggest a method to set a hard limit on a VM's cpu usage? Thank you.
Libvirt 0.9.4 added CPU bandwidth limitation support, via <cputune>:
http://libvirt.org/formatdomain.html#elementsCPUTuning. <period> and
<quota> map to hard-coded caps via cgroups, nicer than the weighted
numbers via <shares>.
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
3:24 p.m.
Hi,
-
Regards,
Sethuraman Subbiah
Graduate Student - NC state University
M.S in Computer Science
On Oct 24, 2011, at 4:20 PM, Eric Blake wrote:
On 10/24/2011 01:50 PM, sethuraman subbiah wrote:
> Hi ,
>
> I was previously using xen and currently moved to KVM. I am using libvirt to manage
these VMs. In den's credit scheduler , I had the ability to set a cap on the cpu usage
for a VM. But I was not able to find a similar substitute in KVM. I find that we can use
cgroups to provide shares for VM but that will be more like weight based and it
doesn't set a hard cap for that VM. I tried using cpulimit but I find it inaccurate
and we can give values only between 0-100. Thus, I think it cannot support multi core
environments. Can any one suggest a method to set a hard limit on a VM's cpu usage?
Thank you.
Libvirt 0.9.4 added CPU bandwidth limitation support, via <cputune>:
http://libvirt.org/formatdomain.html#elementsCPUTuning. <period> and <quota>
map to hard-coded caps via cgroups, nicer than the weighted numbers via <shares>.
Great, I have one more question. Is it possible to set the quota during run time from the
host os ? Using virsh ? Thank you.
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
3:30 p.m.
On 10/24/2011 02:24 PM, sethuraman subbiah wrote:
Great, I have one more question. Is it possible to set the quota
during run time from the host os ? Using virsh ? Thank you.
virsh schedinfo can both query and alter these cpu tunables, and that
includes doing it while the guest is running.
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
3:39 p.m.
Hi,
On Oct 24, 2011, at 4:30 PM, Eric Blake wrote:
On 10/24/2011 02:24 PM, sethuraman subbiah wrote:
> Great, I have one more question. Is it possible to set the quota during run time from
the host os ? Using virsh ? Thank you.
virsh schedinfo can both query and alter these cpu tunables, and that includes doing it
while the guest is running.
I tried the virsh schedinfo. Here is the information regarding my libvirt :
[root@sbc11 ssubbiah]# virsh version
Compiled against library: libvir 0.9.4
Using library: libvir 0.9.4
Using API: QEMU 0.9.4
Running hypervisor: QEMU 0.12.1
When i use "[root@hgcc11 ssubbiah]# virsh schedinfo rubis_ws --cap 50" , I dont
see the cap value being set and I am able to see only cpu shares.
Scheduler : posix
cpu_shares : 1024
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
3:46 p.m.
On 10/24/2011 02:39 PM, sethuraman subbiah wrote:
Hi,
On Oct 24, 2011, at 4:30 PM, Eric Blake wrote:
> On 10/24/2011 02:24 PM, sethuraman subbiah wrote:
>> Great, I have one more question. Is it possible to set the quota during run time
from the host os ? Using virsh ? Thank you.
>
> virsh schedinfo can both query and alter these cpu tunables, and that includes doing
it while the guest is running.
>
I tried the virsh schedinfo. Here is the information regarding my libvirt :
[root@sbc11 ssubbiah]# virsh version
Compiled against library: libvir 0.9.4
Using library: libvir 0.9.4
Using API: QEMU 0.9.4
Running hypervisor: QEMU 0.12.1
When i use "[root@hgcc11 ssubbiah]# virsh schedinfo rubis_ws --cap 50" , I dont
see the cap value being set and I am able to see only cpu shares.
schedinfo --cap is obsolete (it's xen-only). Rather, you want:
virsh schedinfo domain
to list the current settings, and
virsh schedinfo domain --set vcpu_period $value --set vcpu_quota $value
I haven't used it myself; I'm only going off the documentation provided
by Wen Congyang, who oversaw the addition of the features.
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
3:54 p.m.
On Oct 24, 2011, at 4:46 PM, Eric Blake wrote:
On 10/24/2011 02:39 PM, sethuraman subbiah wrote:
> Hi,
>
>
>
> When i use "[root@hgcc11 ssubbiah]# virsh schedinfo rubis_ws --cap 50" , I
dont see the cap value being set and I am able to see only cpu shares.
schedinfo --cap is obsolete (it's xen-only). Rather, you want:
virsh schedinfo domain
to list the current settings, and
virsh schedinfo domain --set vcpu_period $value --set vcpu_quota $value
I haven't used it myself; I'm only going off the documentation provided by Wen
Congyang, who oversaw the addition of the features.
Sorry, I noticed it late about using a xen command here. I tried virsh schedinfo domain. I
dont think I have the capability to set quota :
[root@sbc11 ssubbiah]# virsh schedinfo rubis_ws
Scheduler : posix
cpu_shares : 1024
[root@sbc11 ssubbiah]# virsh schedinfo rubis_ws --set vcpu_quota=50
Scheduler : posix
error: invalid scheduler option: vcpu_quota=50
I get the same error for "virsh schedinfo rubies_ws --set vcpu_quota 50" .
I find that my version is 0.9.4.
[root@sbc11 ssubbiah]# virsh --version
0.9.4
Should I do some modifications,enable options or add some patch to get the capability to
set cap ? Thanks for all your help.
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
4:13 p.m.
On 10/24/2011 02:54 PM, sethuraman subbiah wrote:
[root@sbc11 ssubbiah]# virsh schedinfo rubis_ws --set vcpu_quota=50
Scheduler : posix
error: invalid scheduler option: vcpu_quota=50
I get the same error for "virsh schedinfo rubies_ws --set vcpu_quota 50" .
I find that my version is 0.9.4.
[root@sbc11 ssubbiah]# virsh --version
0.9.4
Should I do some modifications,enable options or add some patch to get the capability to
set cap ? Thanks for all your help.
I found this comment in the code:
/* If we does not know VCPU<->PID mapping or all vcpu runs in the same
* thread, we cannot control each vcpu. So we only modify cpu bandwidth
* when each vcpu has a separated thread.
I think that means that in order to use CPU caps, you have to also use
<vcpupin> to call out the host CPU pinning to be used by the guest VCPUS.
The original implementation was here, although I don't see much
documentation in that patch series.
https://www.redhat.com/archives/libvir-list/2011-July/msg01389.html
Wen, can you help out here?
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
4:49 p.m.
>
> I find that my version is 0.9.4.
>
> [root@sbc11 ssubbiah]# virsh --version
> 0.9.4
>
> Should I do some modifications,enable options or add some patch to get the capability
to set cap ? Thanks for all your help.
I found this comment in the code:
/* If we does not know VCPU<->PID mapping or all vcpu runs in the same
* thread, we cannot control each vcpu. So we only modify cpu bandwidth
* when each vcpu has a separated thread.
I think that means that in order to use CPU caps, you have to also use <vcpupin> to
call out the host CPU pinning to be used by the guest VCPUS.
The original implementation was here, although I don't see much documentation in that
patch series. https://www.redhat.com/archives/libvir-list/2011-July/msg01389.html
Wen, can you help out here?
Thanks Eric and Hi Wen. I did use <vcpupin> now . This is that part of my config xml
file :
<vcpu>3</vcpu>
<cputune>
<vcpupin vcpu='2' cpuset='0-2'/>
</cputune>
This shows the vcpu affinity :
[root@hgcc11 cloning_script]# virsh vcpuinfo rubis_ws
VCPU: 0
CPU: 2
State: running
CPU time: 5.6s
CPU Affinity: yyyy
VCPU: 1
CPU: 1
State: running
CPU time: 0.1s
CPU Affinity: yyyy
VCPU: 2
CPU: 0
State: running
CPU time: 0.1s
CPU Affinity: yyy-
[root@hgcc11 cloning_script]# virsh schedinfo rubis_ws
Scheduler : posix
cpu_shares : 1024
Still I get it as an invalid option :
[root@hgcc11 cloning_script]# virsh schedinfo rubis_ws --set vcpu_quota=1100
Scheduler : posix
error: invalid scheduler option: vcpu_quota=1100
If I try to add the quota option to the xml file :
<vcpu>3</vcpu>
<cputune>
<vcpupin vcpu='2' cpuset='0-2'/>
<quota>-1</quota>
</cputune>
I get :
[root@hgcc11 cloning_script]# virsh create images/rubis_ws.xml
error: Failed to create domain from images/rubis_ws.xml
error: Unable to set cpu bandwidth quota: No such file or directory
Eventhough I am using libvirt 0.9.4, I think I am missing something.
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
-
Regards,
Sethuraman Subbiah
Graduate Student - NC state University
M.S in Computer Science
8:56 p.m.
At 10/25/2011 05:13 AM, Eric Blake Write:
On 10/24/2011 02:54 PM, sethuraman subbiah wrote:
> [root@sbc11 ssubbiah]# virsh schedinfo rubis_ws --set vcpu_quota=50
> Scheduler : posix
> error: invalid scheduler option: vcpu_quota=50
>
> I get the same error for "virsh schedinfo rubies_ws --set vcpu_quota
> 50" .
>
> I find that my version is 0.9.4.
>
> [root@sbc11 ssubbiah]# virsh --version
> 0.9.4
>
> Should I do some modifications,enable options or add some patch to get
> the capability to set cap ? Thanks for all your help.
I found this comment in the code:
/* If we does not know VCPU<->PID mapping or all vcpu runs in the same
* thread, we cannot control each vcpu. So we only modify cpu bandwidth
* when each vcpu has a separated thread.
I think that means that in order to use CPU caps, you have to also use
<vcpupin> to call out the host CPU pinning to be used by the guest VCPUS.
VCPU<->PID mapping is the thread id for each vcpu. qemu-kvm supports this
feature. We use monitor command query-cpus to get VCPU<->PID mapping in
libvirt(the function is: qemuProcessDetectVcpuPIDs()).
The original implementation was here, although I don't see much
documentation in that patch series.
https://www.redhat.com/archives/libvir-list/2011-July/msg01389.html
Wen, can you help out here?
It seems that the kernel does not support it. The cfs bandwidth was merged into
kernel tree about some months ago.
Thanks
Wen Congyang
4770
days inactive
4778
days old
9 comments
3 participants
participants (3)
-
Eric Blake -
sethuraman subbiah -
Wen Congyang