This affects only cgroups v2 where enabled controllers are not based on available mount points but on the list provided in cgroup.controllers file. However, moving it will fill in placement as well, so it needs to be freed together with mount point if we don't need that controller. Before this patch we were assuming that all controllers available in root cgroup where available in all other sub-cgroups which was wrong. In order to fix it we need to move the cgroup controllers detection after cgroup placement was prepared in order to build correct path for cgroup.controllers file. Signed-off-by: Pavel Hrdina <phrdina(a)redhat.com&gt; --- Notes: Changes in v3: - Moving the detection after placement means we need to free it together with mount point if that controller is not required. src/util/vircgroup.c | 32 ++++++++++++++++---------------- src/util/vircgroupv1.c | 1 + src/util/vircgroupv2.c | 5 +++-- 3 files changed, 20 insertions(+), 18 deletions(-) diff --git a/src/util/vircgroup.c b/src/util/vircgroup.c index b7e5f03521..da506fc0b0 100644 --- a/src/util/vircgroup.c +++ b/src/util/vircgroup.c @@ -381,22 +381,6 @@ virCgroupDetect(virCgroupPtr group, return -1; } - for (i = 0; i < VIR_CGROUP_BACKEND_TYPE_LAST; i++) { - if (group->backends[i]) { - int rc = group->backends[i]->detectControllers(group, controllers, parent); - if (rc < 0) - return -1; - controllersAvailable |= rc; - } - } - - /* Check that at least 1 controller is available */ - if (controllersAvailable == 0) { - virReportSystemError(ENXIO, "%s", - _("At least one cgroup controller is required")); - return -1; - } - /* In some cases we can copy part of the placement info * based on the parent cgroup... */ @@ -421,6 +405,22 @@ virCgroupDetect(virCgroupPtr group, } } + for (i = 0; i < VIR_CGROUP_BACKEND_TYPE_LAST; i++) { + if (group->backends[i]) { + int rc = group->backends[i]->detectControllers(group, controllers, parent); + if (rc < 0) + return -1; + controllersAvailable |= rc; + } + } + + /* Check that at least 1 controller is available */ + if (controllersAvailable == 0) { + virReportSystemError(ENXIO, "%s", + _("At least one cgroup controller is required")); + return -1; + } + return 0; } diff --git a/src/util/vircgroupv1.c b/src/util/vircgroupv1.c index fb3e0b2d47..4231d8d6fa 100644 --- a/src/util/vircgroupv1.c +++ b/src/util/vircgroupv1.c @@ -464,6 +464,7 @@ virCgroupV1DetectControllers(virCgroupPtr group, } } VIR_FREE(group->legacy[i].mountPoint); + VIR_FREE(group->legacy[i].placement); } } } else { diff --git a/src/util/vircgroupv2.c b/src/util/vircgroupv2.c index 2d09d77a29..29b5806a01 100644 --- a/src/util/vircgroupv2.c +++ b/src/util/vircgroupv2.c @@ -250,8 +250,9 @@ virCgroupV2ParseControllersFile(virCgroupPtr group) char **contList = NULL; char **tmp; - if (virAsprintf(&contFile, "%s/cgroup.controllers", - group->unified.mountPoint) < 0) + if (virAsprintf(&contFile, "%s%s/cgroup.controllers", + group->unified.mountPoint, + NULLSTR_EMPTY(group->unified.placement)) < 0) return -1; rc = virFileReadAll(contFile, 1024 * 1024, &contStr); -- 2.21.0

The assumption that CPU controller would be always enabled is wrong, we should use any available controller to create a new sub-cgroup. Signed-off-by: Pavel Hrdina <phrdina(a)redhat.com&gt; Reviewed-by: Ján Tomko <jtomko(a)redhat.com&gt; --- src/util/vircgroupv2.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/util/vircgroupv2.c b/src/util/vircgroupv2.c index 29b5806a01..6bcbb7e1a0 100644 --- a/src/util/vircgroupv2.c +++ b/src/util/vircgroupv2.c @@ -399,7 +399,7 @@ virCgroupV2MakeGroup(virCgroupPtr parent ATTRIBUTE_UNUSED, if (create) { if (flags & VIR_CGROUP_THREAD) { - if (virCgroupSetValueStr(group, VIR_CGROUP_CONTROLLER_CPU, + if (virCgroupSetValueStr(group, controller, "cgroup.type", "threaded") < 0) { return -1; } -- 2.21.0

In order to skip controllers that we are not able to activate we need to return different return value so the caller can decide what to do. Signed-off-by: Pavel Hrdina <phrdina(a)redhat.com&gt; --- Notes: Introduced in v2 src/util/vircgroupv2.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/src/util/vircgroupv2.c b/src/util/vircgroupv2.c index 3f4548b532..133a8e0e66 100644 --- a/src/util/vircgroupv2.c +++ b/src/util/vircgroupv2.c @@ -353,22 +353,37 @@ virCgroupV2PathOfController(virCgroupPtr group, } +/** + * virCgroupV2EnableController: + * + * Returns: -1 on fatal error + * -2 if we failed to write into cgroup.subtree_control + * 0 on success + */ static int virCgroupV2EnableController(virCgroupPtr parent, int controller) { VIR_AUTOFREE(char *) val = NULL; + VIR_AUTOFREE(char *) path = NULL; if (virAsprintf(&val, "+%s", virCgroupV2ControllerTypeToString(controller)) < 0) { return -1; } - if (virCgroupSetValueStr(parent, controller, - "cgroup.subtree_control", val) < 0) { + if (virCgroupPathOfController(parent, controller, + "cgroup.subtree_control", &path) < 0) { return -1; } + if (virFileWriteStr(path, val, 0) < 0) { + virReportSystemError(errno, + _("Failed to enable controller '%s' for '%s'"), + val, path); + return -2; + } + return 0; } -- 2.21.0

Currently CPU controller cannot be enabled if there is any real-time task running and is assigned to non-root cgroup which is the case on several distributions with graphical environment. Instead of erroring out treat it as the controller is not available. Signed-off-by: Pavel Hrdina <phrdina(a)redhat.com&gt; --- src/util/vircgroupv2.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/src/util/vircgroupv2.c b/src/util/vircgroupv2.c index 133a8e0e66..348c12d5c6 100644 --- a/src/util/vircgroupv2.c +++ b/src/util/vircgroupv2.c @@ -433,6 +433,8 @@ virCgroupV2MakeGroup(virCgroupPtr parent ATTRIBUTE_UNUSED, } else { size_t i; for (i = 0; i < VIR_CGROUP_CONTROLLER_LAST; i++) { + int rc; + if (!virCgroupV2HasController(parent, i)) continue; @@ -440,8 +442,17 @@ virCgroupV2MakeGroup(virCgroupPtr parent ATTRIBUTE_UNUSED, if (i == VIR_CGROUP_CONTROLLER_CPUACCT) continue; - if (virCgroupV2EnableController(parent, i) < 0) + rc = virCgroupV2EnableController(parent, i); + if (rc < 0) { + if (rc == -2) { + virResetLastError(); + VIR_DEBUG("failed to enable '%s' controller, skipping", + virCgroupV2ControllerTypeToString(i)); + group->unified.controllers &= ~(1 << i); + continue; + } return -1; + } } } } -- 2.21.0

On Wed, Jun 26, 2019 at 09:21:59AM +0200, Peter Krempa wrote: Works for me, this was based on our in person discussion but I guess there was some misunderstanding. Do you need v4? Pavel

On Wed, Jun 26, 2019 at 10:27:31AM +0200, Peter Krempa wrote: Thanks, all the issues are fixed and pushed now. Pavel

On Tue, Jun 25, 2019 at 13:16:26 +0200, Pavel Hrdina wrote: The use of 'int' here for 'controllers' makes it super non-obvious and super sketchy in what's happening here. Especially since you are then doing bitwise operations with it afterwards. It's pre-existing though. Still super ugly. ACK