On 2012年12月13日 14:59, Hu Tao wrote:
On Thu, Dec 13, 2012 at 12:21:29PM +0800, Osier Yang wrote:
On 2012年11月07日 18:38, Hu Tao wrote:
The reason to disable cgroup cpuset by default is: currently, sub-directories of cgroup cpuset won't be auto-updated if cpu hot-plug happens. For example, if we now have 0-3,5-7 pcpus online, the value of /sys/fs/cgroup/cpuset/libvirt/qemu/mydom/cpuset.cpus is 0-3,5-7, then pcpu 4 is hot-plugged, but the value of cpuset.cpus is still 0-3,5-7. This will cause a problem that we can't pin to a hot-plugged pcpu
IMHO it's a kernel bug and should be fixed in kernel instead.
The problem was discussed before, see
http://www.redhat.com/archives/libvir-list/2012-September/msg00165.html http://www.redhat.com/archives/libvir-list/2012-September/msg00221.html
Okay, read through the threads. It's fine for emulatorpin and vcpupin, because they can still use sched_setaffinity. But APIs like domain{Get,Set}NumaParameters can't be used anymore. Will document the problem somewhere and clarify it's not recommended to use be enough? Or do we really want to disable it by default, one can disable any controller by changing the default configurations in qemu.conf as long as the bug is fixed: https://www.redhat.com/archives/libvir-list/2012-December/msg00774.html
Users can still enable cpuset by editing qemu.conf.
And per users can enable/disable the cpuset cgroup in qemu.conf, why do we need to disable it by default? On one hand, it has to be enabled again once the kernel bug is fixed; On the other hand, disabling it affects many functions, it's not deserved to disable it with the only problem (actually the libvirt pinning just behaves correctly, as the hotplug pCPU is not visible yet, the root cause is in kernel) on pinning from my p.o.v.
Hu Tao (6): cgroup: Add a flag VIR_CGROUP_DISABLE_CPUSET create cgroup controllers for driver according to config create cgroup controllers for domain according to config create cgroup controllers for vcpu according to config create cgroup controllers for emulator according to config disable cgroup cpuset by default
src/lxc/lxc_cgroup.c | 4 +-- src/lxc/lxc_driver.c | 22 ++++++------ src/lxc/lxc_process.c | 4 +-- src/qemu/qemu.conf | 2 +- src/qemu/qemu_cgroup.c | 15 ++++---- src/qemu/qemu_conf.c | 1 - src/qemu/qemu_driver.c | 92 +++++++++++++++++++++++++++++------------------ src/qemu/qemu_hotplug.c | 9 +++-- src/qemu/qemu_migration.c | 2 +- src/util/cgroup.c | 49 +++++++++++++++++++------ src/util/cgroup.h | 12 ++++--- 11 files changed, 136 insertions(+), 76 deletions(-)