[libvirt PATCH 0/2] fix cgroups BPF FDs leak
Pavel Hrdina (2): vircgroupv2: properly free BPF prog and map FDs vircgroupv2devices: refactor virCgroupV2DevicesRemoveProg src/libvirt_private.syms | 2 +- src/util/vircgroupv2.c | 3 ++- src/util/vircgroupv2devices.c | 14 ++++---------- src/util/vircgroupv2devices.h | 2 +- 4 files changed, 8 insertions(+), 13 deletions(-) -- 2.30.2
When nested cgroup was introduced it did not properly free file descriptors for BPF prog and map. With nested cgroups we create the BPF bits in the nested cgroup instead of the VM root cgroup. This would leak the FDs which would be the last reference to the prog and map so kernel would not remove the resources as well. It would only happen once libvirtd process exits. Fixes: 184245f53b94fc84f727eb6e8a2aa52df02d69c0 Reported-by: Eric Farman <farman@linux.ibm.com> Signed-off-by: Pavel Hrdina <phrdina@redhat.com> --- src/util/vircgroupv2.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/util/vircgroupv2.c b/src/util/vircgroupv2.c index 248d4047e5..4664492c34 100644 --- a/src/util/vircgroupv2.c +++ b/src/util/vircgroupv2.c @@ -523,6 +523,7 @@ static int virCgroupV2Remove(virCgroupPtr group) { g_autofree char *grppath = NULL; + virCgroupPtr parent = virCgroupGetNested(group); int controller; /* Don't delete the root group, if we accidentally @@ -534,7 +535,7 @@ virCgroupV2Remove(virCgroupPtr group) if (virCgroupV2PathOfController(group, controller, "", &grppath) < 0) return 0; - if (virCgroupV2DevicesRemoveProg(group) < 0) + if (virCgroupV2DevicesRemoveProg(parent) < 0) return -1; return virCgroupRemoveRecursively(grppath); -- 2.30.2
When running on systemd host the cgroup itself is removed by machined so when we reach this code the directory no longer exist. If libvirtd was running the whole time between starting and destroying VM the detection is skipped because we still have both FD in memory. But if libvirtd was restarted and no operation requiring cgroup devices executed the FDs would be 0 and libvirt would try to detect them using the cgroup directory. This results in reporting following errors: libvirtd[955]: unable to open '/sys/fs/cgroup/machine.slice/machine-qemu\x2d1\x2dguest.scope/': No such file or directory libvirtd[955]: Failed to remove cgroup for guest When running on non-systemd host where we handle cgroups manually this would not happen. When destroying VM it is not necessary to detect the BPF prog and map because the following code only closes the FDs without doing anything else. We could run code that would try to detach the BPF prog from the cgroup but that is not necessary as well. If the cgroup is removed and there is no other FD open to the prog kernel will cleanup the prog and map eventually. Reported-by: Eric Farman <farman@linux.ibm.com> Signed-off-by: Pavel Hrdina <phrdina@redhat.com> --- src/libvirt_private.syms | 2 +- src/util/vircgroupv2.c | 2 +- src/util/vircgroupv2devices.c | 14 ++++---------- src/util/vircgroupv2devices.h | 2 +- 4 files changed, 7 insertions(+), 13 deletions(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 119768496b..0ccde7e1b4 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1963,12 +1963,12 @@ virCgroupV2Register; # util/vircgroupv2devices.h virCgroupV2DevicesAvailable; +virCgroupV2DevicesCloseProg; virCgroupV2DevicesCreateProg; virCgroupV2DevicesDetectProg; virCgroupV2DevicesGetKey; virCgroupV2DevicesGetPerms; virCgroupV2DevicesPrepareProg; -virCgroupV2DevicesRemoveProg; # util/vircommand.h virCommandAbort; diff --git a/src/util/vircgroupv2.c b/src/util/vircgroupv2.c index 4664492c34..5bf087b39b 100644 --- a/src/util/vircgroupv2.c +++ b/src/util/vircgroupv2.c @@ -535,7 +535,7 @@ virCgroupV2Remove(virCgroupPtr group) if (virCgroupV2PathOfController(group, controller, "", &grppath) < 0) return 0; - if (virCgroupV2DevicesRemoveProg(parent) < 0) + if (virCgroupV2DevicesCloseProg(parent) < 0) return -1; return virCgroupRemoveRecursively(grppath); diff --git a/src/util/vircgroupv2devices.c b/src/util/vircgroupv2devices.c index 4bcc1d52fe..98ab6e8f5a 100644 --- a/src/util/vircgroupv2devices.c +++ b/src/util/vircgroupv2devices.c @@ -548,18 +548,12 @@ virCgroupV2DevicesPrepareProg(virCgroupPtr group) int -virCgroupV2DevicesRemoveProg(virCgroupPtr group) +virCgroupV2DevicesCloseProg(virCgroupPtr group) { - if (virCgroupV2DevicesDetectProg(group) < 0) - return -1; - - if (group->unified.devices.progfd <= 0 && group->unified.devices.mapfd <= 0) - return 0; - - if (group->unified.devices.mapfd >= 0) + if (group->unified.devices.mapfd > 0) VIR_FORCE_CLOSE(group->unified.devices.mapfd); - if (group->unified.devices.progfd >= 0) + if (group->unified.devices.progfd > 0) VIR_FORCE_CLOSE(group->unified.devices.progfd); return 0; @@ -629,7 +623,7 @@ virCgroupV2DevicesPrepareProg(virCgroupPtr group G_GNUC_UNUSED) int -virCgroupV2DevicesRemoveProg(virCgroupPtr group G_GNUC_UNUSED) +virCgroupV2DevicesCloseProg(virCgroupPtr group G_GNUC_UNUSED) { return 0; } diff --git a/src/util/vircgroupv2devices.h b/src/util/vircgroupv2devices.h index 9d91256120..17ab07afa0 100644 --- a/src/util/vircgroupv2devices.h +++ b/src/util/vircgroupv2devices.h @@ -38,7 +38,7 @@ int virCgroupV2DevicesPrepareProg(virCgroupPtr group); int -virCgroupV2DevicesRemoveProg(virCgroupPtr group); +virCgroupV2DevicesCloseProg(virCgroupPtr group); uint32_t virCgroupV2DevicesGetPerms(int perms, -- 2.30.2
On Mon, 2021-04-12 at 22:37 +0200, Pavel Hrdina wrote:
Pavel Hrdina (2): vircgroupv2: properly free BPF prog and map FDs vircgroupv2devices: refactor virCgroupV2DevicesRemoveProg
src/libvirt_private.syms | 2 +- src/util/vircgroupv2.c | 3 ++- src/util/vircgroupv2devices.c | 14 ++++---------- src/util/vircgroupv2devices.h | 2 +- 4 files changed, 8 insertions(+), 13 deletions(-)
Works well for me. Series: Tested-by: Eric Farman <farman@linux.ibm.com> Reviewed-by: Eric Farman <farman@linux.ibm.com>
On 4/12/21 10:37 PM, Pavel Hrdina wrote:
Pavel Hrdina (2): vircgroupv2: properly free BPF prog and map FDs vircgroupv2devices: refactor virCgroupV2DevicesRemoveProg
src/libvirt_private.syms | 2 +- src/util/vircgroupv2.c | 3 ++- src/util/vircgroupv2devices.c | 14 ++++---------- src/util/vircgroupv2devices.h | 2 +- 4 files changed, 8 insertions(+), 13 deletions(-)
Reviewed-by: Michal Privoznik <mprivozn@redhat.com> Michal
participants (3)
-
Eric Farman -
Michal Privoznik -
Pavel Hrdina