On 09/03/2012 08:06 AM, Tang Chen wrote:
It seems that libvirt is not cpu hotplug aware. Please refer to the following problem.
1. At first, we have 2 cpus. # cat /cgroup/cpuset/cpuset.cpus 0-1 # cat /cgroup/cpuset/libvirt/qemu/cpuset.cpus 0-1
2. And we have a vm1 with following configuration. <cputune> <vcpupin vcpu='0' cpuset='1'/> <hypervisorpin cpuset='1'/> </cputune>
3. Offline cpu1. # echo 0 > /sys/devices/system/cpu/cpu1/online # cat /sys/devices/system/cpu/cpu1/online 0 # cat /cgroup/cpuset/cpuset.cpus 0 # cat /cgroup/cpuset/libvirt/qemu/cpuset.cpus 0 # cat /cgroup/cpuset/libvirt/lxc/cpuset.cpus 0
4. Online cpu1. # echo 1 > /sys/devices/system/cpu/cpu1/online # cat /sys/devices/system/cpu/cpu1/online 1 # cat /cgroup/cpuset/cpuset.cpus 0-1 # cat /cgroup/cpuset/libvirt/cpuset.cpus 0 # cat /cgroup/cpuset/libvirt/qemu/cpuset.cpus 0 # cat /cgroup/cpuset/libvirt/lxc/cpuset.cpus 0
Here，cgroup updated cpuset.cpus，but not for libvirt directory，and also qemu and lxc directory. vm1 cannot be started again. # virsh start vm1 error: Failed to start domain vm1 error: Unable to set cpuset.cpus: Permission denied
And libvird gave the following errors. 2012-07-17 07:30:22.478+0000: 3118: error : qemuSetupCgroupVcpuPin:498 : Unable to set cpuset.cpus: Permission denied
These patches resolves this problem by listening on the netlink for cpu hotplug event. When the netlink service gets the cpu hotplug event, it will attract the cpuid in the message, and add it into cpuset.cpus in: /cgroup/cpuset/libvirt /cgroup/cpuset/libvirt/qemu /cgroup/cpuset/libvirt/lxc
Hi, this approach requires that libvirtd keeps running through the entire lifecycle of a guest. That is something that cannot be safely assumed and therefore hotplug events can be missed. That means that libvirt must synchronize the hypervisors cpusets with the host's current CPU states. You could do that for instance when registering the callback. -- Mit freundlichen Grüßen/Kind Regards Viktor Mihajlovski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martin Jetter Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft: Böblingen Registergericht: Amtsgericht Stuttgart, HRB 243294