[libvirt] [PATCH v1 rebase 0/6] disable cgroup cpuset
This is merely a rebased version, so still tagged with v1. The reason to disable cgroup cpuset by default is: currently, sub-directories of cgroup cpuset won't be auto-updated if cpu hot-plug happens. For example, if we now have 0-3,5-7 pcpus online, the value of /sys/fs/cgroup/cpuset/libvirt/qemu/mydom/cpuset.cpus is 0-3,5-7, then pcpu 4 is hot-plugged, but the value of cpuset.cpus is still 0-3,5-7. This will cause a problem that we can't pin to a hot-plugged pcpu. Users can still enable cpuset by editing qemu.conf. Hu Tao (6): cgroup: Add a flag VIR_CGROUP_DISABLE_CPUSET create cgroup controllers for driver according to config create cgroup controllers for domain according to config create cgroup controllers for vcpu according to config create cgroup controllers for emulator according to config disable cgroup cpuset by default src/lxc/lxc_cgroup.c | 4 +- src/lxc/lxc_driver.c | 22 +++++------ src/lxc/lxc_process.c | 4 +- src/qemu/qemu.conf | 2 +- src/qemu/qemu_cgroup.c | 15 ++++---- src/qemu/qemu_conf.c | 1 - src/qemu/qemu_driver.c | 93 ++++++++++++++++++++++++++++++----------------- src/qemu/qemu_hotplug.c | 9 +++-- src/qemu/qemu_migration.c | 2 +- src/util/cgroup.c | 49 ++++++++++++++++++++----- src/util/cgroup.h | 12 ++++-- 11 files changed, 137 insertions(+), 76 deletions(-) -- 1.8.0.1.240.ge8a1f5a
Add a flag VIR_CGROUP_DISABLE_CPUSET to disable cgroup cpuset. This flag inhibits making of directory under cpuset. --- src/util/cgroup.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/util/cgroup.c b/src/util/cgroup.c index f867fb7..e955a22 100644 --- a/src/util/cgroup.c +++ b/src/util/cgroup.c @@ -72,6 +72,7 @@ typedef enum { */ VIR_CGROUP_VCPU = 1 << 1, /* create subdir only under the cgroup cpu, * cpuacct and cpuset if possible. */ + VIR_CGROUP_DISABLE_CPUSET = 1 << 2, } virCgroupFlags; /** @@ -542,6 +543,12 @@ static int virCgroupMakeGroup(virCgroupPtr parent, if (!group->controllers[i].mountPoint) continue; + if ((flags & VIR_CGROUP_DISABLE_CPUSET) && + i == VIR_CGROUP_CONTROLLER_CPUSET) { + group->controllers[i].mountPoint = NULL; + continue; + } + /* We need to control cpu bandwidth for each vcpu now */ if ((flags & VIR_CGROUP_VCPU) && (i != VIR_CGROUP_CONTROLLER_CPU && -- 1.8.0.1.240.ge8a1f5a
On 2012年12月13日 15:01, Hu Tao wrote:
Add a flag VIR_CGROUP_DISABLE_CPUSET to disable cgroup cpuset. This flag inhibits making of directory under cpuset. --- src/util/cgroup.c | 7 +++++++ 1 file changed, 7 insertions(+)
diff --git a/src/util/cgroup.c b/src/util/cgroup.c index f867fb7..e955a22 100644 --- a/src/util/cgroup.c +++ b/src/util/cgroup.c @@ -72,6 +72,7 @@ typedef enum { */ VIR_CGROUP_VCPU = 1<< 1, /* create subdir only under the cgroup cpu, * cpuacct and cpuset if possible. */ + VIR_CGROUP_DISABLE_CPUSET = 1<< 2, } virCgroupFlags;
/** @@ -542,6 +543,12 @@ static int virCgroupMakeGroup(virCgroupPtr parent, if (!group->controllers[i].mountPoint) continue;
+ if ((flags& VIR_CGROUP_DISABLE_CPUSET)&& + i == VIR_CGROUP_CONTROLLER_CPUSET) { + group->controllers[i].mountPoint = NULL; + continue; + }
This actually exposes an existed bug, regardless of whether the user will configure qemu.conf without the controller or not. It will always be created. As it iterates over the all the internal-defined controllers without honoring the configuration in qemu.conf. (see the 'for' loop) However, It's expected to not create the cgroup as long as it's not configured in qemu.conf. The right way is to fix virCgroupMakeCgroup to honor the configuration in qemu.conf instead, a new flag here only fixes the case "cpuset" is not configured, it doesn't fix the root cause, and the new flags is not needed if the root cause is fixed. I see this set passes driver->cgroupContollers to virCgroupMakeCgroup, which is good, but it still iterates over the internal-defined all controllers. Correct me if I'm wrong. Regards, Osier
On Thu, Dec 13, 2012 at 05:10:47PM +0800, Osier Yang wrote:
On 2012年12月13日 15:01, Hu Tao wrote:
Add a flag VIR_CGROUP_DISABLE_CPUSET to disable cgroup cpuset. This flag inhibits making of directory under cpuset. --- src/util/cgroup.c | 7 +++++++ 1 file changed, 7 insertions(+)
diff --git a/src/util/cgroup.c b/src/util/cgroup.c index f867fb7..e955a22 100644 --- a/src/util/cgroup.c +++ b/src/util/cgroup.c @@ -72,6 +72,7 @@ typedef enum { */ VIR_CGROUP_VCPU = 1<< 1, /* create subdir only under the cgroup cpu, * cpuacct and cpuset if possible. */ + VIR_CGROUP_DISABLE_CPUSET = 1<< 2, } virCgroupFlags;
/** @@ -542,6 +543,12 @@ static int virCgroupMakeGroup(virCgroupPtr parent, if (!group->controllers[i].mountPoint) continue;
+ if ((flags& VIR_CGROUP_DISABLE_CPUSET)&& + i == VIR_CGROUP_CONTROLLER_CPUSET) { + group->controllers[i].mountPoint = NULL; + continue; + }
This actually exposes an existed bug, regardless of whether the user will configure qemu.conf without the controller or not. It will always be created.
As it iterates over the all the internal-defined controllers without honoring the configuration in qemu.conf. (see the 'for' loop)
However, It's expected to not create the cgroup as long as it's not configured in qemu.conf.
Agreed. I don't think those directories should be created if not in use, neither.
The right way is to fix virCgroupMakeCgroup to honor the configuration in qemu.conf instead, a new flag here only fixes the case "cpuset" is not configured, it doesn't fix the root cause, and the new flags is not needed if the root cause is fixed.
I see this set passes driver->cgroupContollers to virCgroupMakeCgroup, which is good, but it still iterates over the internal-defined all controllers.
I'm thinking about refactoring cgroup code to create cgroup dir as needed. -- Hu Tao
On Thu, Dec 13, 2012 at 05:10:47PM +0800, Osier Yang wrote:
On 2012年12月13日 15:01, Hu Tao wrote:
Add a flag VIR_CGROUP_DISABLE_CPUSET to disable cgroup cpuset. This flag inhibits making of directory under cpuset. --- src/util/cgroup.c | 7 +++++++ 1 file changed, 7 insertions(+)
diff --git a/src/util/cgroup.c b/src/util/cgroup.c index f867fb7..e955a22 100644 --- a/src/util/cgroup.c +++ b/src/util/cgroup.c @@ -72,6 +72,7 @@ typedef enum { */ VIR_CGROUP_VCPU = 1<< 1, /* create subdir only under the cgroup cpu, * cpuacct and cpuset if possible. */ + VIR_CGROUP_DISABLE_CPUSET = 1<< 2, } virCgroupFlags;
/** @@ -542,6 +543,12 @@ static int virCgroupMakeGroup(virCgroupPtr parent, if (!group->controllers[i].mountPoint) continue;
+ if ((flags& VIR_CGROUP_DISABLE_CPUSET)&& + i == VIR_CGROUP_CONTROLLER_CPUSET) { + group->controllers[i].mountPoint = NULL; + continue; + }
This actually exposes an existed bug, regardless of whether the user will configure qemu.conf without the controller or not. It will always be created.
As it iterates over the all the internal-defined controllers without honoring the configuration in qemu.conf. (see the 'for' loop)
However, It's expected to not create the cgroup as long as it's not configured in qemu.conf.
The right way is to fix virCgroupMakeCgroup to honor the configuration in qemu.conf instead, a new flag here only fixes the case "cpuset" is not configured, it doesn't fix the root cause, and the new flags is not needed if the root cause is fixed.
I see this set passes driver->cgroupContollers to virCgroupMakeCgroup, which is good, but it still iterates over the internal-defined all controllers.
Yep, this is an existing bug. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Add a parameter to virCgroupForDriver to indicate which cgroup controllers to create. The value of the parameter is read from user config. --- src/lxc/lxc_cgroup.c | 2 +- src/lxc/lxc_driver.c | 2 +- src/qemu/qemu_driver.c | 13 +++++++------ src/util/cgroup.c | 12 +++++++++--- src/util/cgroup.h | 3 ++- 5 files changed, 20 insertions(+), 12 deletions(-) diff --git a/src/lxc/lxc_cgroup.c b/src/lxc/lxc_cgroup.c index 767ef26..0af9dcc 100644 --- a/src/lxc/lxc_cgroup.c +++ b/src/lxc/lxc_cgroup.c @@ -371,7 +371,7 @@ int virLXCCgroupSetup(virDomainDefPtr def) int ret = -1; int rc; - rc = virCgroupForDriver("lxc", &driver, 1, 0); + rc = virCgroupForDriver("lxc", &driver, 1, 0, ~0); if (rc != 0) { virReportSystemError(-rc, "%s", _("Unable to get cgroup for driver")); diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c index e0e76e6..3f5aa80 100644 --- a/src/lxc/lxc_driver.c +++ b/src/lxc/lxc_driver.c @@ -1448,7 +1448,7 @@ static int lxcStartup(bool privileged, lxc_driver->log_libvirtd = 0; /* by default log to container logfile */ lxc_driver->have_netns = lxcCheckNetNsSupport(); - rc = virCgroupForDriver("lxc", &lxc_driver->cgroup, privileged, 1); + rc = virCgroupForDriver("lxc", &lxc_driver->cgroup, privileged, 1, ~0); if (rc < 0) { char buf[1024] ATTRIBUTE_UNUSED; VIR_DEBUG("Unable to create cgroup for LXC driver: %s", diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 1228f6e..d155ea7 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -797,17 +797,18 @@ qemuStartup(bool privileged, VIR_FREE(base); - rc = virCgroupForDriver("qemu", &qemu_driver->cgroup, privileged, 1); - if (rc < 0) { - VIR_INFO("Unable to create cgroup for driver: %s", - virStrerror(-rc, ebuf, sizeof(ebuf))); - } - if (qemuLoadDriverConfig(qemu_driver, driverConf) < 0) { goto error; } VIR_FREE(driverConf); + rc = virCgroupForDriver("qemu", &qemu_driver->cgroup, privileged, 1, + qemu_driver->cgroupControllers); + if (rc < 0) { + VIR_INFO("Unable to create cgroup for driver: %s", + virStrerror(-rc, ebuf, sizeof(ebuf))); + } + /* Allocate bitmap for remote display port reservations. We cannot * do this before the config is loaded properly, since the port * numbers are configurable now */ diff --git a/src/util/cgroup.c b/src/util/cgroup.c index e955a22..37a5c08 100644 --- a/src/util/cgroup.c +++ b/src/util/cgroup.c @@ -934,11 +934,13 @@ cleanup: int virCgroupForDriver(const char *name, virCgroupPtr *group, bool privileged, - bool create) + bool create, + int controllers) { int rc; char *path = NULL; virCgroupPtr rootgrp = NULL; + unsigned flags = VIR_CGROUP_NONE; rc = virCgroupAppRoot(privileged, &rootgrp, create); if (rc != 0) @@ -949,11 +951,14 @@ int virCgroupForDriver(const char *name, goto out; } + if (!(controllers & (1UL << VIR_CGROUP_CONTROLLER_CPUSET))) + flags |= VIR_CGROUP_DISABLE_CPUSET; + rc = virCgroupNew(path, group); VIR_FREE(path); if (rc == 0) { - rc = virCgroupMakeGroup(rootgrp, *group, create, VIR_CGROUP_NONE); + rc = virCgroupMakeGroup(rootgrp, *group, create, flags); if (rc != 0) virCgroupFree(group); } @@ -967,7 +972,8 @@ out: int virCgroupForDriver(const char *name ATTRIBUTE_UNUSED, virCgroupPtr *group ATTRIBUTE_UNUSED, bool privileged ATTRIBUTE_UNUSED, - bool create ATTRIBUTE_UNUSED) + bool create ATTRIBUTE_UNUSED, + int controllers) { /* Claim no support */ return -ENXIO; diff --git a/src/util/cgroup.h b/src/util/cgroup.h index fc9e409..3e3be91 100644 --- a/src/util/cgroup.h +++ b/src/util/cgroup.h @@ -45,7 +45,8 @@ VIR_ENUM_DECL(virCgroupController); int virCgroupForDriver(const char *name, virCgroupPtr *group, bool privileged, - bool create); + bool create, + int controllers); int virCgroupGetAppRoot(virCgroupPtr *group); -- 1.8.0.1.240.ge8a1f5a
Add a parameter to virCgroupForDomain to indicate which cgroup controllers to create for domain. The value of the parameter is read from user config. --- src/lxc/lxc_cgroup.c | 2 +- src/lxc/lxc_driver.c | 20 +++++++++--------- src/lxc/lxc_process.c | 4 ++-- src/qemu/qemu_cgroup.c | 10 ++++----- src/qemu/qemu_driver.c | 53 ++++++++++++++++++++++++++++++----------------- src/qemu/qemu_hotplug.c | 9 +++++--- src/qemu/qemu_migration.c | 2 +- src/util/cgroup.c | 9 ++++++-- src/util/cgroup.h | 3 ++- 9 files changed, 68 insertions(+), 44 deletions(-) diff --git a/src/lxc/lxc_cgroup.c b/src/lxc/lxc_cgroup.c index 0af9dcc..59007da 100644 --- a/src/lxc/lxc_cgroup.c +++ b/src/lxc/lxc_cgroup.c @@ -378,7 +378,7 @@ int virLXCCgroupSetup(virDomainDefPtr def) goto cleanup; } - rc = virCgroupForDomain(driver, def->name, &cgroup, 1); + rc = virCgroupForDomain(driver, def->name, &cgroup, 1, ~0); if (rc != 0) { virReportSystemError(-rc, _("Unable to create cgroup for domain %s"), diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c index 3f5aa80..0530af2 100644 --- a/src/lxc/lxc_driver.c +++ b/src/lxc/lxc_driver.c @@ -544,7 +544,7 @@ static int lxcDomainGetInfo(virDomainPtr dom, info->cpuTime = 0; info->memory = vm->def->mem.cur_balloon; } else { - if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, ~0) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Unable to get cgroup for %s"), vm->def->name); goto cleanup; @@ -737,7 +737,7 @@ static int lxcDomainSetMemory(virDomainPtr dom, unsigned long newmem) { goto cleanup; } - if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, ~0) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Unable to get cgroup for %s"), vm->def->name); goto cleanup; @@ -794,7 +794,7 @@ lxcDomainSetMemoryParameters(virDomainPtr dom, goto cleanup; } - if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, ~0) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("cannot find cgroup for domain %s"), vm->def->name); goto cleanup; @@ -871,7 +871,7 @@ lxcDomainGetMemoryParameters(virDomainPtr dom, goto cleanup; } - if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, ~0) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Unable to get cgroup for %s"), vm->def->name); goto cleanup; @@ -1795,7 +1795,7 @@ lxcSetSchedulerParametersFlags(virDomainPtr dom, "%s", _("cgroup CPU controller is not mounted")); goto cleanup; } - if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0, ~0) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("cannot find cgroup for domain %s"), vm->def->name); @@ -1939,7 +1939,7 @@ lxcGetSchedulerParametersFlags(virDomainPtr dom, goto cleanup; } - if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0, ~0) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("cannot find cgroup for domain %s"), vm->def->name); goto cleanup; @@ -2044,7 +2044,7 @@ lxcDomainSetBlkioParameters(virDomainPtr dom, goto cleanup; } - if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0, ~0) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("cannot find cgroup for domain %s"), vm->def->name); goto cleanup; @@ -2149,7 +2149,7 @@ lxcDomainGetBlkioParameters(virDomainPtr dom, goto cleanup; } - if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0, ~0) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("cannot find cgroup for domain %s"), vm->def->name); goto cleanup; @@ -2383,7 +2383,7 @@ static int lxcFreezeContainer(virLXCDriverPtr driver, virDomainObjPtr vm) virCgroupPtr cgroup = NULL; if (!(driver->cgroup && - virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0) == 0)) + virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, ~0) == 0)) return -1; /* From here on, we know that cgroup != NULL. */ @@ -2521,7 +2521,7 @@ static int lxcUnfreezeContainer(virLXCDriverPtr driver, virDomainObjPtr vm) virCgroupPtr cgroup = NULL; if (!(driver->cgroup && - virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0) == 0)) + virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, ~0) == 0)) return -1; ret = virCgroupSetFreezerState(cgroup, "THAWED"); diff --git a/src/lxc/lxc_process.c b/src/lxc/lxc_process.c index 3e7fcb8..383ca41 100644 --- a/src/lxc/lxc_process.c +++ b/src/lxc/lxc_process.c @@ -272,7 +272,7 @@ static void virLXCProcessCleanup(virLXCDriverPtr driver, virDomainConfVMNWFilterTeardown(vm); if (driver->cgroup && - virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0) == 0) { + virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, ~0) == 0) { virCgroupRemove(cgroup); virCgroupFree(&cgroup); } @@ -720,7 +720,7 @@ int virLXCProcessStop(virLXCDriverPtr driver, VIR_FREE(vm->def->seclabels[0]->imagelabel); } - if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0) == 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0, ~0) == 0) { rc = virCgroupKillPainfully(group); if (rc < 0) { virReportSystemError(-rc, "%s", diff --git a/src/qemu/qemu_cgroup.c b/src/qemu/qemu_cgroup.c index 30cd1d6..b868269 100644 --- a/src/qemu/qemu_cgroup.c +++ b/src/qemu/qemu_cgroup.c @@ -203,7 +203,7 @@ int qemuSetupCgroup(virQEMUDriverPtr driver, if (driver->cgroup == NULL) return 0; /* Not supported, so claim success */ - rc = virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 1); + rc = virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 1, driver->cgroupControllers); if (rc != 0) { virReportSystemError(-rc, _("Unable to create cgroup for %s"), @@ -558,7 +558,7 @@ int qemuSetupCgroupForVcpu(virQEMUDriverPtr driver, virDomainObjPtr vm) if (driver->cgroup == NULL) return 0; - rc = virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0); + rc = virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, driver->cgroupControllers); if (rc != 0) { virReportSystemError(-rc, _("Unable to find cgroup for %s"), @@ -661,7 +661,7 @@ int qemuSetupCgroupForEmulator(virQEMUDriverPtr driver, if (driver->cgroup == NULL) return 0; /* Not supported, so claim success */ - rc = virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0); + rc = virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, driver->cgroupControllers); if (rc != 0) { virReportSystemError(-rc, _("Unable to find cgroup for %s"), @@ -748,7 +748,7 @@ int qemuRemoveCgroup(virQEMUDriverPtr driver, if (driver->cgroup == NULL) return 0; /* Not supported, so claim success */ - rc = virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0); + rc = virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, driver->cgroupControllers); if (rc != 0) { if (!quiet) virReportError(VIR_ERR_INTERNAL_ERROR, @@ -772,7 +772,7 @@ int qemuAddToCgroup(virQEMUDriverPtr driver, if (driver->cgroup == NULL) return 0; /* Not supported, so claim success */ - rc = virCgroupForDomain(driver->cgroup, def->name, &cgroup, 0); + rc = virCgroupForDomain(driver->cgroup, def->name, &cgroup, 0, driver->cgroupControllers); if (rc != 0) { virReportSystemError(-rc, _("unable to find cgroup for domain %s"), diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index d155ea7..2b915ff 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -3822,7 +3822,7 @@ static int qemuDomainHotplugVcpus(virQEMUDriverPtr driver, } cgroup_available = (virCgroupForDomain(driver->cgroup, vm->def->name, - &cgroup, 0) == 0); + &cgroup, 0, driver->cgroupControllers) == 0); if (nvcpus > oldvcpus) { for (i = oldvcpus; i < nvcpus; i++) { @@ -4148,7 +4148,8 @@ qemuDomainPinVcpuFlags(virDomainPtr dom, /* Configure the corresponding cpuset cgroup before set affinity. */ if (qemuCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_CPUSET)) { - if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup_dom, 0) == 0 && + if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup_dom, 0, + driver->cgroupControllers) == 0 && virCgroupForVcpu(cgroup_dom, vcpu, &cgroup_vcpu, 0) == 0 && qemuSetupCgroupVcpuPin(cgroup_vcpu, newVcpuPin, newVcpuPinNum, vcpu) < 0) { virReportError(VIR_ERR_OPERATION_INVALID, @@ -4413,7 +4414,7 @@ qemuDomainPinEmulator(virDomainPtr dom, * If no cgroup for domain or hypervisor exists, do nothing. */ if (virCgroupForDomain(driver->cgroup, vm->def->name, - &cgroup_dom, 0) == 0) { + &cgroup_dom, 0, driver->cgroupControllers) == 0) { if (virCgroupForEmulator(cgroup_dom, &cgroup_emulator, 0) == 0) { if (qemuSetupCgroupEmulatorPin(cgroup_emulator, newVcpuPin[0]->cpumask) < 0) { @@ -5997,7 +5998,8 @@ qemuDomainAttachDeviceDiskLive(virConnectPtr conn, goto end; if (qemuCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_DEVICES)) { - if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0)) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, + driver->cgroupControllers)) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Unable to find cgroup for %s"), vm->def->name); @@ -6227,8 +6229,8 @@ qemuDomainChangeDiskMediaLive(virDomainObjPtr vm, goto end; if (qemuCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_DEVICES)) { - if (virCgroupForDomain(driver->cgroup, - vm->def->name, &cgroup, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, + driver->cgroupControllers) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Unable to find cgroup for %s"), vm->def->name); @@ -7092,7 +7094,8 @@ qemuDomainSetBlkioParameters(virDomainPtr dom, goto cleanup; } - if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0, + driver->cgroupControllers) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("cannot find cgroup for domain %s"), vm->def->name); @@ -7253,7 +7256,8 @@ qemuDomainGetBlkioParameters(virDomainPtr dom, goto cleanup; } - if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0, + driver->cgroupControllers) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("cannot find cgroup for domain %s"), vm->def->name); goto cleanup; @@ -7442,7 +7446,8 @@ qemuDomainSetMemoryParameters(virDomainPtr dom, goto cleanup; } - if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0, + driver->cgroupControllers) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("cannot find cgroup for domain %s"), vm->def->name); goto cleanup; @@ -7599,7 +7604,8 @@ qemuDomainGetMemoryParameters(virDomainPtr dom, goto cleanup; } - if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0, + driver->cgroupControllers) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("cannot find cgroup for domain %s"), vm->def->name); goto cleanup; @@ -7761,7 +7767,8 @@ qemuDomainSetNumaParameters(virDomainPtr dom, goto cleanup; } - if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0, + driver->cgroupControllers) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("cannot find cgroup for domain %s"), vm->def->name); @@ -7917,7 +7924,8 @@ qemuDomainGetNumaParameters(virDomainPtr dom, goto cleanup; } - if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0, + driver->cgroupControllers) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("cannot find cgroup for domain %s"), vm->def->name); @@ -8122,7 +8130,8 @@ qemuSetSchedulerParametersFlags(virDomainPtr dom, "%s", _("cgroup CPU controller is not mounted")); goto cleanup; } - if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0, + driver->cgroupControllers) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("cannot find cgroup for domain %s"), vm->def->name); @@ -8409,7 +8418,8 @@ qemuGetSchedulerParametersFlags(virDomainPtr dom, goto cleanup; } - if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0, + driver->cgroupControllers) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("cannot find cgroup for domain %s"), vm->def->name); goto cleanup; @@ -11275,7 +11285,8 @@ qemuDomainSnapshotCreateDiskActive(virQEMUDriverPtr driver, } if (qemuCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_DEVICES) && - virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0)) { + virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, + driver->cgroupControllers)) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Unable to find cgroup for %s"), vm->def->name); @@ -13017,7 +13028,8 @@ qemuDomainBlockPivot(virConnectPtr conn, * we know for sure that there is a backing chain. */ if (disk->mirrorFormat && disk->mirrorFormat != VIR_STORAGE_FILE_RAW && qemuCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_DEVICES) && - virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0) < 0) { + virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, + driver->cgroupControllers) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Unable to find cgroup for %s"), vm->def->name); @@ -13330,7 +13342,8 @@ qemuDomainBlockCopy(virDomainPtr dom, const char *path, goto cleanup; } if (qemuCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_DEVICES) && - virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0) < 0) { + virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, + driver->cgroupControllers) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Unable to find cgroup for %s"), vm->def->name); @@ -13614,7 +13627,8 @@ qemuDomainBlockCommit(virDomainPtr dom, const char *path, const char *base, * operation succeeds, but doing that requires tracking the * operation in XML across libvirtd restarts. */ if (qemuCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_DEVICES) && - virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0) < 0) { + virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, + driver->cgroupControllers) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Unable to find cgroup for %s"), vm->def->name); @@ -14546,7 +14560,8 @@ qemuDomainGetCPUStats(virDomainPtr domain, goto cleanup; } - if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0, + driver->cgroupControllers) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("cannot find cgroup for domain %s"), vm->def->name); goto cleanup; diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 3dfdb65..3584a47 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -1097,7 +1097,8 @@ int qemuDomainAttachHostUsbDevice(virQEMUDriverPtr driver, usbDevice *usb; qemuCgroupData data; - if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, + driver->cgroupControllers) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Unable to find cgroup for %s"), vm->def->name); @@ -1995,7 +1996,8 @@ int qemuDomainDetachPciDiskDevice(virQEMUDriverPtr driver, } if (qemuCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_DEVICES)) { - if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, + driver->cgroupControllers) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Unable to find cgroup for %s"), vm->def->name); @@ -2108,7 +2110,8 @@ int qemuDomainDetachDiskDevice(virQEMUDriverPtr driver, } if (qemuCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_DEVICES)) { - if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0) != 0) { + if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0, + driver->cgroupControllers) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Unable to find cgroup for %s"), vm->def->name); diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 5dcbb07..12dc719 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -3617,7 +3617,7 @@ qemuMigrationToFile(virQEMUDriverPtr driver, virDomainObjPtr vm, if (qemuCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_DEVICES)) { if (virCgroupForDomain(driver->cgroup, vm->def->name, - &cgroup, 0) != 0) { + &cgroup, 0, driver->cgroupControllers) != 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Unable to find cgroup for %s"), vm->def->name); diff --git a/src/util/cgroup.c b/src/util/cgroup.c index 37a5c08..f3e4185 100644 --- a/src/util/cgroup.c +++ b/src/util/cgroup.c @@ -1009,14 +1009,19 @@ int virCgroupGetAppRoot(virCgroupPtr *group) int virCgroupForDomain(virCgroupPtr driver, const char *name, virCgroupPtr *group, - bool create) + bool create, + int controllers) { + unsigned flags = VIR_CGROUP_MEM_HIERACHY; int rc; char *path; if (driver == NULL) return -EINVAL; + if (!(controllers & (1UL << VIR_CGROUP_CONTROLLER_CPUSET))) + flags |= VIR_CGROUP_DISABLE_CPUSET; + if (virAsprintf(&path, "%s/%s", driver->path, name) < 0) return -ENOMEM; @@ -1034,7 +1039,7 @@ int virCgroupForDomain(virCgroupPtr driver, * a group for driver, is to avoid overhead to track * cumulative usage that we don't need. */ - rc = virCgroupMakeGroup(driver, *group, create, VIR_CGROUP_MEM_HIERACHY); + rc = virCgroupMakeGroup(driver, *group, create, flags); if (rc != 0) virCgroupFree(group); } diff --git a/src/util/cgroup.h b/src/util/cgroup.h index 3e3be91..f632240 100644 --- a/src/util/cgroup.h +++ b/src/util/cgroup.h @@ -53,7 +53,8 @@ int virCgroupGetAppRoot(virCgroupPtr *group); int virCgroupForDomain(virCgroupPtr driver, const char *name, virCgroupPtr *group, - bool create); + bool create, + int controllers); int virCgroupForVcpu(virCgroupPtr driver, int vcpuid, -- 1.8.0.1.240.ge8a1f5a
Add a parameter to virCgroupForVcpu to indicate which cgroup controllers to create. The value of the parameter is read from user config. --- src/qemu/qemu_cgroup.c | 3 ++- src/qemu/qemu_driver.c | 18 ++++++++++++------ src/util/cgroup.c | 9 +++++++-- src/util/cgroup.h | 3 ++- 4 files changed, 23 insertions(+), 10 deletions(-) diff --git a/src/qemu/qemu_cgroup.c b/src/qemu/qemu_cgroup.c index b868269..5a1bf7e 100644 --- a/src/qemu/qemu_cgroup.c +++ b/src/qemu/qemu_cgroup.c @@ -576,7 +576,8 @@ int qemuSetupCgroupForVcpu(virQEMUDriverPtr driver, virDomainObjPtr vm) } for (i = 0; i < priv->nvcpupids; i++) { - rc = virCgroupForVcpu(cgroup, i, &cgroup_vcpu, 1); + rc = virCgroupForVcpu(cgroup, i, &cgroup_vcpu, 1, + driver->cgroupControllers); if (rc < 0) { virReportSystemError(-rc, _("Unable to create vcpu cgroup for %s(vcpu:" diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 2b915ff..005dc8a 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -3829,7 +3829,8 @@ static int qemuDomainHotplugVcpus(virQEMUDriverPtr driver, if (cgroup_available) { int rv = -1; /* Create cgroup for the onlined vcpu */ - rv = virCgroupForVcpu(cgroup, i, &cgroup_vcpu, 1); + rv = virCgroupForVcpu(cgroup, i, &cgroup_vcpu, 1, + driver->cgroupControllers); if (rv < 0) { virReportSystemError(-rv, _("Unable to create vcpu cgroup for %s(vcpu:" @@ -3903,7 +3904,8 @@ static int qemuDomainHotplugVcpus(virQEMUDriverPtr driver, if (cgroup_available) { int rv = -1; - rv = virCgroupForVcpu(cgroup, i, &cgroup_vcpu, 0); + rv = virCgroupForVcpu(cgroup, i, &cgroup_vcpu, 0, + driver->cgroupControllers); if (rv < 0) { virReportSystemError(-rv, _("Unable to access vcpu cgroup for %s(vcpu:" @@ -4150,7 +4152,8 @@ qemuDomainPinVcpuFlags(virDomainPtr dom, if (qemuCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_CPUSET)) { if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup_dom, 0, driver->cgroupControllers) == 0 && - virCgroupForVcpu(cgroup_dom, vcpu, &cgroup_vcpu, 0) == 0 && + virCgroupForVcpu(cgroup_dom, vcpu, &cgroup_vcpu, 0, + driver->cgroupControllers) == 0 && qemuSetupCgroupVcpuPin(cgroup_vcpu, newVcpuPin, newVcpuPinNum, vcpu) < 0) { virReportError(VIR_ERR_OPERATION_INVALID, _("failed to set cpuset.cpus in cgroup" @@ -8005,7 +8008,8 @@ qemuSetVcpusBWLive(virDomainObjPtr vm, virCgroupPtr cgroup, */ if (priv->nvcpupids != 0 && priv->vcpupids[0] != vm->pid) { for (i = 0; i < priv->nvcpupids; i++) { - rc = virCgroupForVcpu(cgroup, i, &cgroup_vcpu, 0); + rc = virCgroupForVcpu(cgroup, i, &cgroup_vcpu, 0, + qemu_driver->cgroupControllers); if (rc < 0) { virReportSystemError(-rc, _("Unable to find vcpu cgroup for %s(vcpu:" @@ -8296,7 +8300,8 @@ qemuGetVcpusBWLive(virDomainObjPtr vm, virCgroupPtr cgroup, } /* get period and quota for vcpu0 */ - rc = virCgroupForVcpu(cgroup, 0, &cgroup_vcpu, 0); + rc = virCgroupForVcpu(cgroup, 0, &cgroup_vcpu, 0, + qemu_driver->cgroupControllers); if (!cgroup_vcpu) { virReportSystemError(-rc, _("Unable to find vcpu cgroup for %s(vcpu: 0)"), @@ -14386,7 +14391,8 @@ getSumVcpuPercpuStats(virCgroupPtr group, unsigned long long tmp; int j; - if (virCgroupForVcpu(group, i, &group_vcpu, 0) < 0) { + if (virCgroupForVcpu(group, i, &group_vcpu, 0, + qemu_driver->cgroupControllers) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("error accessing cgroup cpuacct for vcpu")); goto cleanup; diff --git a/src/util/cgroup.c b/src/util/cgroup.c index f3e4185..8b117f3 100644 --- a/src/util/cgroup.c +++ b/src/util/cgroup.c @@ -1069,14 +1069,19 @@ int virCgroupForDomain(virCgroupPtr driver ATTRIBUTE_UNUSED, int virCgroupForVcpu(virCgroupPtr driver, int vcpuid, virCgroupPtr *group, - bool create) + bool create, + int controllers) { + unsigned flags = VIR_CGROUP_VCPU; int rc; char *path; if (driver == NULL) return -EINVAL; + if (!(controllers & (1UL << VIR_CGROUP_CONTROLLER_CPUSET))) + flags |= VIR_CGROUP_DISABLE_CPUSET; + if (virAsprintf(&path, "%s/vcpu%d", driver->path, vcpuid) < 0) return -ENOMEM; @@ -1084,7 +1089,7 @@ int virCgroupForVcpu(virCgroupPtr driver, VIR_FREE(path); if (rc == 0) { - rc = virCgroupMakeGroup(driver, *group, create, VIR_CGROUP_VCPU); + rc = virCgroupMakeGroup(driver, *group, create, flags); if (rc != 0) virCgroupFree(group); } diff --git a/src/util/cgroup.h b/src/util/cgroup.h index f632240..4b5cc5c 100644 --- a/src/util/cgroup.h +++ b/src/util/cgroup.h @@ -59,7 +59,8 @@ int virCgroupForDomain(virCgroupPtr driver, int virCgroupForVcpu(virCgroupPtr driver, int vcpuid, virCgroupPtr *group, - bool create); + bool create, + int controllers); int virCgroupForEmulator(virCgroupPtr driver, virCgroupPtr *group, -- 1.8.0.1.240.ge8a1f5a
Add a parameter to virCgroupForEmulator to indicate which cgroup controllers to create. The value of the parameter is read from user config. --- src/qemu/qemu_cgroup.c | 2 +- src/qemu/qemu_driver.c | 9 ++++++--- src/util/cgroup.c | 12 +++++++++--- src/util/cgroup.h | 3 ++- 4 files changed, 18 insertions(+), 8 deletions(-) diff --git a/src/qemu/qemu_cgroup.c b/src/qemu/qemu_cgroup.c index 5a1bf7e..5738e93 100644 --- a/src/qemu/qemu_cgroup.c +++ b/src/qemu/qemu_cgroup.c @@ -670,7 +670,7 @@ int qemuSetupCgroupForEmulator(virQEMUDriverPtr driver, goto cleanup; } - rc = virCgroupForEmulator(cgroup, &cgroup_emulator, 1); + rc = virCgroupForEmulator(cgroup, &cgroup_emulator, 1, driver->cgroupControllers); if (rc < 0) { virReportSystemError(-rc, _("Unable to create emulator cgroup for %s"), diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 005dc8a..a6034c2 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -4418,7 +4418,8 @@ qemuDomainPinEmulator(virDomainPtr dom, */ if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup_dom, 0, driver->cgroupControllers) == 0) { - if (virCgroupForEmulator(cgroup_dom, &cgroup_emulator, 0) == 0) { + if (virCgroupForEmulator(cgroup_dom, &cgroup_emulator, 0, + driver->cgroupControllers) == 0) { if (qemuSetupCgroupEmulatorPin(cgroup_emulator, newVcpuPin[0]->cpumask) < 0) { virReportError(VIR_ERR_OPERATION_INVALID, "%s", @@ -8047,7 +8048,8 @@ qemuSetEmulatorBandwidthLive(virDomainObjPtr vm, virCgroupPtr cgroup, return 0; } - rc = virCgroupForEmulator(cgroup, &cgroup_emulator, 0); + rc = virCgroupForEmulator(cgroup, &cgroup_emulator, 0, + qemu_driver->cgroupControllers); if (rc < 0) { virReportSystemError(-rc, _("Unable to find emulator cgroup for %s"), @@ -8339,7 +8341,8 @@ qemuGetEmulatorBandwidthLive(virDomainObjPtr vm, virCgroupPtr cgroup, } /* get period and quota for emulator */ - rc = virCgroupForEmulator(cgroup, &cgroup_emulator, 0); + rc = virCgroupForEmulator(cgroup, &cgroup_emulator, 0, + qemu_driver->cgroupControllers); if (!cgroup_emulator) { virReportSystemError(-rc, _("Unable to find emulator cgroup for %s"), diff --git a/src/util/cgroup.c b/src/util/cgroup.c index 8b117f3..2e5a7e1 100644 --- a/src/util/cgroup.c +++ b/src/util/cgroup.c @@ -1117,14 +1117,19 @@ int virCgroupForVcpu(virCgroupPtr driver ATTRIBUTE_UNUSED, #if defined HAVE_MNTENT_H && defined HAVE_GETMNTENT_R int virCgroupForEmulator(virCgroupPtr driver, virCgroupPtr *group, - bool create) + bool create, + int controllers) { + unsigned flags = VIR_CGROUP_VCPU; int rc; char *path; if (driver == NULL) return -EINVAL; + if (!(controllers & (1UL << VIR_CGROUP_CONTROLLER_CPUSET))) + flags |= VIR_CGROUP_DISABLE_CPUSET; + if (virAsprintf(&path, "%s/emulator", driver->path) < 0) return -ENOMEM; @@ -1132,7 +1137,7 @@ int virCgroupForEmulator(virCgroupPtr driver, VIR_FREE(path); if (rc == 0) { - rc = virCgroupMakeGroup(driver, *group, create, VIR_CGROUP_VCPU); + rc = virCgroupMakeGroup(driver, *group, create, flags); if (rc != 0) virCgroupFree(group); } @@ -1142,7 +1147,8 @@ int virCgroupForEmulator(virCgroupPtr driver, #else int virCgroupForEmulator(virCgroupPtr driver ATTRIBUTE_UNUSED, virCgroupPtr *group ATTRIBUTE_UNUSED, - bool create ATTRIBUTE_UNUSED) + bool create ATTRIBUTE_UNUSED, + int controllers ATTRIBUTE_UNUSED) { return -ENXIO; } diff --git a/src/util/cgroup.h b/src/util/cgroup.h index 4b5cc5c..0ffdd84 100644 --- a/src/util/cgroup.h +++ b/src/util/cgroup.h @@ -64,7 +64,8 @@ int virCgroupForVcpu(virCgroupPtr driver, int virCgroupForEmulator(virCgroupPtr driver, virCgroupPtr *group, - bool create); + bool create, + int controllers); int virCgroupPathOfController(virCgroupPtr group, int controller, -- 1.8.0.1.240.ge8a1f5a
The reason ti disable cgroup cpuset by default is: currently, sub-directories of cgroup cpuset won't be auto-updated if cpu hot-plug happens. For example, if we now have 0-3,5-7 pcpus online, the value of /sys/fs/cgroup/cpuset/libvirt/qemu/mydom/cpuset.cpus is 0-3,5-7, then pcpu 4 is hot-plugged, but the value of cpuset.cpus is still 0-3,5-7. This will cause a problem that we can't pin to a hot-plugged pcpu. Users can still enable cpuset by editing qemu.conf. --- src/qemu/qemu.conf | 2 +- src/qemu/qemu_conf.c | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/src/qemu/qemu.conf b/src/qemu/qemu.conf index dd853c8..b853852 100644 --- a/src/qemu/qemu.conf +++ b/src/qemu/qemu.conf @@ -226,7 +226,7 @@ # can be mounted in different locations. libvirt will detect # where they are located. # -#cgroup_controllers = [ "cpu", "devices", "memory", "blkio", "cpuset", "cpuacct" ] +#cgroup_controllers = [ "cpu", "devices", "memory", "blkio", "cpuacct" ] # This is the basic set of devices allowed / required by # all virtual machines. diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index e95609c..96b14a0 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -287,7 +287,6 @@ int qemuLoadDriverConfig(virQEMUDriverPtr driver, (1 << VIR_CGROUP_CONTROLLER_DEVICES) | (1 << VIR_CGROUP_CONTROLLER_MEMORY) | (1 << VIR_CGROUP_CONTROLLER_BLKIO) | - (1 << VIR_CGROUP_CONTROLLER_CPUSET) | (1 << VIR_CGROUP_CONTROLLER_CPUACCT); } for (i = 0 ; i < VIR_CGROUP_CONTROLLER_LAST ; i++) { -- 1.8.0.1.240.ge8a1f5a
On 12/13/12 08:01, Hu Tao wrote:
This is merely a rebased version, so still tagged with v1.
The reason to disable cgroup cpuset by default is: currently, sub-directories of cgroup cpuset won't be auto-updated if cpu hot-plug happens. For example, if we now have 0-3,5-7 pcpus online, the value of /sys/fs/cgroup/cpuset/libvirt/qemu/mydom/cpuset.cpus is 0-3,5-7, then pcpu 4 is hot-plugged, but the value of cpuset.cpus is still 0-3,5-7. This will cause a problem that we can't pin to a hot-plugged pcpu.
Users can still enable cpuset by editing qemu.conf.
CPU hotplug isn't really a common thing in hosts right now. I'd rather go for cpusets enabled by default and disabling them only if it poses problems. Peter
participants (4)
-
Daniel P. Berrange -
Hu Tao -
Osier Yang -
Peter Krempa