On 10/19/2012 05:45 AM, Osier Yang wrote:
On 2012年10月17日 23:38, Martin Kletzander wrote:
According to our recent changes (clarifications), we should be pinning qemu's emulator processes using the<vcpu> 'cpuset' attribute in case there is no<emulatorpin> specified. This however doesn't work entirely as expected and this patch should resolve all the remaining issues. --- src/qemu/qemu_cgroup.c | 25 ++++++++++++++++--------- src/qemu/qemu_cgroup.h | 4 ++-- src/qemu/qemu_driver.c | 3 ++- src/qemu/qemu_process.c | 16 ++++++++-------- 4 files changed, 28 insertions(+), 20 deletions(-)
[...]
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index a94e9c4..e08ec67 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -2024,11 +2024,12 @@ cleanup: return ret; }
-/* Set CPU affinities for emulator threads if emulatorpin xml provided. */ +/* Set CPU affinities for emulator threads. */ static int qemuProcessSetEmulatorAffinites(virConnectPtr conn, virDomainObjPtr vm) { + virBitmapPtr cpumask; virDomainDefPtr def = vm->def; virNodeInfo nodeinfo; int ret = -1; @@ -2036,15 +2037,14 @@ qemuProcessSetEmulatorAffinites(virConnectPtr conn, if (virNodeGetInfo(conn,&nodeinfo) != 0) return -1;
- if (!def->cputune.emulatorpin) - return 0; - - if (virProcessInfoSetAffinity(vm->pid, - def->cputune.emulatorpin->cpumask)< 0) { + if (def->cputune.emulatorpin) + cpumask = def->cputune.emulatorpin->cpumask; + else if (def->cpumask) + cpumask = def->cpumask; + else goto cleanup; - }
- ret = 0; + ret = virProcessInfoSetAffinity(vm->pid, cpumask); cleanup: return ret; }
This patch actually duplicates the affinity setting on domain process with sched_setaffinity.
Assume "cpuset" of <vcpu> is specified, and no "<emulatorpin>".
qemuProcessInitCpuAffinity is excuted between the fork and exec, with this patch, qemuProcessSetEmulatorAffinites will set the affinity again using sched_setaffinity.
Cgroup setting is fine though, as there is no previous setting with cgroup.
I'm wondering why the domain process is not pinned to pCPUs specified "cpuset" of "<vcpu>" by qemuProcessInitCpuAffinity, when <emulatorpin> is not specified.
Regards, Osier
I'm glad you pointed it out. To be honest, it took me quite a while to see where this should be set to fix the problem with as few lines as possible. As Peter says, there is a mess with the pinning, cgroups and affinities. Rewriting of these things (using affinity even though crgoups are working on the system, etc.) is in my mind, but not right now. Martin