[libvirt PATCH] apparmor: Allow running /usr/libexec/qemu-kvm
Distros that use AppArmor, such as Debian and Ubuntu, install QEMU under /usr/bin/qemu-system-*, and our AppArmor profile is written with that assumption in mind. If you try to run the RHEL or CentOS version of libvirt and QEMU inside a privileged container on such distros, however, that will result in an error, because the path /usr/libexec/qemu-kvm is used instead. In particular, this prevents upstream KubeVirt releases (which are based on CentOS) from running on Debian/Ubuntu nodes. See https://github.com/kubevirt/kubevirt/pull/8692 and the issues referenced therein for additional details. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- src/security/apparmor/usr.sbin.libvirtd.in | 4 ++++ src/security/apparmor/usr.sbin.virtqemud.in | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/src/security/apparmor/usr.sbin.libvirtd.in b/src/security/apparmor/usr.sbin.libvirtd.in index 886f1ad518..2994de5ec9 100644 --- a/src/security/apparmor/usr.sbin.libvirtd.in +++ b/src/security/apparmor/usr.sbin.libvirtd.in @@ -99,6 +99,10 @@ profile libvirtd @sbindir@/libvirtd flags=(attach_disconnected) { # read and run an ebtables script. /var/lib/libvirt/virtd* ixr, + # Needed when running the RHEL/CentOS version of libvirt and QEMU + # inside a privileged container on a Debian/Ubuntu host + /usr/libexec/qemu-kvm PUx, + # force the use of virt-aa-helper audit deny /{usr/,}sbin/apparmor_parser rwxl, audit deny /etc/apparmor.d/libvirt/** wxl, diff --git a/src/security/apparmor/usr.sbin.virtqemud.in b/src/security/apparmor/usr.sbin.virtqemud.in index 3de03d49fc..b3f33b9471 100644 --- a/src/security/apparmor/usr.sbin.virtqemud.in +++ b/src/security/apparmor/usr.sbin.virtqemud.in @@ -94,6 +94,10 @@ profile virtqemud @sbindir@/virtqemud flags=(attach_disconnected) { # read and run an ebtables script. /var/lib/libvirt/virtd* ixr, + # Needed when running the RHEL/CentOS version of libvirt and QEMU + # inside a privileged container on a Debian/Ubuntu host + /usr/libexec/qemu-kvm PUx, + # force the use of virt-aa-helper audit deny /{usr/,}sbin/apparmor_parser rwxl, audit deny /etc/apparmor.d/libvirt/** wxl, -- 2.38.1
On Thu, Nov 03, 2022 at 12:13:53PM +0100, Andrea Bolognani wrote:
Distros that use AppArmor, such as Debian and Ubuntu, install QEMU under /usr/bin/qemu-system-*, and our AppArmor profile is written with that assumption in mind.
If you try to run the RHEL or CentOS version of libvirt and QEMU inside a privileged container on such distros, however, that will result in an error, because the path /usr/libexec/qemu-kvm is used instead.
In particular, this prevents upstream KubeVirt releases (which are based on CentOS) from running on Debian/Ubuntu nodes. See
https://github.com/kubevirt/kubevirt/pull/8692
and the issues referenced therein for additional details.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- src/security/apparmor/usr.sbin.libvirtd.in | 4 ++++ src/security/apparmor/usr.sbin.virtqemud.in | 4 ++++ 2 files changed, 8 insertions(+)
[...]
+ # Needed when running the RHEL/CentOS version of libvirt and QEMU + # inside a privileged container on a Debian/Ubuntu host + /usr/libexec/qemu-kvm PUx,
Jim and Christian, can you please take a look and confirm that this is sane? Thanks in advance! -- Andrea Bolognani / Red Hat / Virtualization
On 11/3/22 05:13, Andrea Bolognani wrote:
Distros that use AppArmor, such as Debian and Ubuntu, install QEMU under /usr/bin/qemu-system-*, and our AppArmor profile is written with that assumption in mind.
If you try to run the RHEL or CentOS version of libvirt and QEMU inside a privileged container on such distros, however, that will result in an error, because the path /usr/libexec/qemu-kvm is used instead.
In particular, this prevents upstream KubeVirt releases (which are based on CentOS) from running on Debian/Ubuntu nodes. See
https://github.com/kubevirt/kubevirt/pull/8692
and the issues referenced therein for additional details.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- src/security/apparmor/usr.sbin.libvirtd.in | 4 ++++ src/security/apparmor/usr.sbin.virtqemud.in | 4 ++++ 2 files changed, 8 insertions(+)
diff --git a/src/security/apparmor/usr.sbin.libvirtd.in b/src/security/apparmor/usr.sbin.libvirtd.in index 886f1ad518..2994de5ec9 100644 --- a/src/security/apparmor/usr.sbin.libvirtd.in +++ b/src/security/apparmor/usr.sbin.libvirtd.in @@ -99,6 +99,10 @@ profile libvirtd @sbindir@/libvirtd flags=(attach_disconnected) { # read and run an ebtables script. /var/lib/libvirt/virtd* ixr,
+ # Needed when running the RHEL/CentOS version of libvirt and QEMU + # inside a privileged container on a Debian/Ubuntu host + /usr/libexec/qemu-kvm PUx,
Based on the comment, agree this should be explicit vs @libexecdir@/qemu-kvm PUx,
+ # force the use of virt-aa-helper audit deny /{usr/,}sbin/apparmor_parser rwxl, audit deny /etc/apparmor.d/libvirt/** wxl, diff --git a/src/security/apparmor/usr.sbin.virtqemud.in b/src/security/apparmor/usr.sbin.virtqemud.in index 3de03d49fc..b3f33b9471 100644 --- a/src/security/apparmor/usr.sbin.virtqemud.in +++ b/src/security/apparmor/usr.sbin.virtqemud.in @@ -94,6 +94,10 @@ profile virtqemud @sbindir@/virtqemud flags=(attach_disconnected) { # read and run an ebtables script. /var/lib/libvirt/virtd* ixr,
+ # Needed when running the RHEL/CentOS version of libvirt and QEMU + # inside a privileged container on a Debian/Ubuntu host + /usr/libexec/qemu-kvm PUx, + # force the use of virt-aa-helper audit deny /{usr/,}sbin/apparmor_parser rwxl, audit deny /etc/apparmor.d/libvirt/** wxl,
Do you also need the path in src/security/apparmor/libvirt-qemu? Regards, Jim
On Thu, Nov 03, 2022 at 08:24:37AM -0600, Jim Fehlig wrote:
On 11/3/22 05:13, Andrea Bolognani wrote:
+ # Needed when running the RHEL/CentOS version of libvirt and QEMU + # inside a privileged container on a Debian/Ubuntu host + /usr/libexec/qemu-kvm PUx,
Do you also need the path in src/security/apparmor/libvirt-qemu?
Good question :) IIUC usr.sbin.{libvirtd,virtqemud}.in is the profile that is used for the daemon and libvirt-qemu the one that's used for the QEMU process itself, right? If that's the case, I don't really understand why we would need to list the various QEMU binaries in there? Once the QEMU process has been started, it shouldn't really need to access any other QEMU binary, should it? Or am I missing something obvious? -- Andrea Bolognani / Red Hat / Virtualization
On 11/3/22 09:02, Andrea Bolognani wrote:
On Thu, Nov 03, 2022 at 08:24:37AM -0600, Jim Fehlig wrote:
On 11/3/22 05:13, Andrea Bolognani wrote:
+ # Needed when running the RHEL/CentOS version of libvirt and QEMU + # inside a privileged container on a Debian/Ubuntu host + /usr/libexec/qemu-kvm PUx,
Do you also need the path in src/security/apparmor/libvirt-qemu?
Good question :)
IIUC usr.sbin.{libvirtd,virtqemud}.in is the profile that is used for the daemon and libvirt-qemu the one that's used for the QEMU process itself, right?
Correct.
If that's the case, I don't really understand why we would need to list the various QEMU binaries in there? Once the QEMU process has been started, it shouldn't really need to access any other QEMU binary, should it?
Good question :-P. I don't know why all the various qemu binaries are listed in that file. Maybe someone more familiar with the history of libvirt apparmor support can clarify. I simply noticed they were there and wondered if the /usr/libexec/qemu-kvm path should be added too. Regards, Jim
On Thu, Nov 03, 2022 at 12:13:53 +0100, Andrea Bolognani wrote:
Distros that use AppArmor, such as Debian and Ubuntu, install QEMU under /usr/bin/qemu-system-*, and our AppArmor profile is written with that assumption in mind.
If you try to run the RHEL or CentOS version of libvirt and QEMU inside a privileged container on such distros, however, that will result in an error, because the path /usr/libexec/qemu-kvm is used instead.
So IIUC by this patch you modify the profile which gets installed into the Debian/Ubuntu host system by the Debian/Ubuntu package which then in turn allows the non-Debian/Ubuntu libvirt in the container to do it's job? I'm basing the above on the fact that the RHEL/Centos package is compiled with: -Dapparmor=disabled \ -Dapparmor_profiles=disabled \ -Dsecdriver_apparmor=disabled \ By extension, does that mean that you have to install libvirt on your host so that you can in turn run a container (which I'd presume is opaque) with libvirt bundled inside?
On Thu, Nov 03, 2022 at 03:39:44PM +0100, Peter Krempa wrote:
On Thu, Nov 03, 2022 at 12:13:53 +0100, Andrea Bolognani wrote:
Distros that use AppArmor, such as Debian and Ubuntu, install QEMU under /usr/bin/qemu-system-*, and our AppArmor profile is written with that assumption in mind.
If you try to run the RHEL or CentOS version of libvirt and QEMU inside a privileged container on such distros, however, that will result in an error, because the path /usr/libexec/qemu-kvm is used instead.
So IIUC by this patch you modify the profile which gets installed into the Debian/Ubuntu host system by the Debian/Ubuntu package which then in turn allows the non-Debian/Ubuntu libvirt in the container to do it's job?
Pretty much.
I'm basing the above on the fact that the RHEL/Centos package is compiled with:
-Dapparmor=disabled \ -Dapparmor_profiles=disabled \ -Dsecdriver_apparmor=disabled \
By extension, does that mean that you have to install libvirt on your host so that you can in turn run a container (which I'd presume is opaque) with libvirt bundled inside?
It's actually the other way around :) If you don't have libvirt installed on the Debian/Ubuntu host, then the AppArmor profile won't be present and the containerized CentOS libvirt will be allowed to start the containerized CentOS QEMU. If you *do* have libvirt installed on the Debian/Ubuntu host, then the AppArmor profile will also be applied to the containerized CentOS libvirt and running the containerized CentOS QEMU will be forbidden. Patching the AppArmor policy is supposed to help with the second scenario. Please check out the discussion at https://github.com/kubevirt/kubevirt/pull/8692 if you haven't already, it's not very long and might help clear things up :) -- Andrea Bolognani / Red Hat / Virtualization
On Thu, Nov 03, 2022 at 12:35:15PM -0400, Andrea Bolognani wrote:
On Thu, Nov 03, 2022 at 03:39:44PM +0100, Peter Krempa wrote:
On Thu, Nov 03, 2022 at 12:13:53 +0100, Andrea Bolognani wrote:
Distros that use AppArmor, such as Debian and Ubuntu, install QEMU under /usr/bin/qemu-system-*, and our AppArmor profile is written with that assumption in mind.
If you try to run the RHEL or CentOS version of libvirt and QEMU inside a privileged container on such distros, however, that will result in an error, because the path /usr/libexec/qemu-kvm is used instead.
So IIUC by this patch you modify the profile which gets installed into the Debian/Ubuntu host system by the Debian/Ubuntu package which then in turn allows the non-Debian/Ubuntu libvirt in the container to do it's job?
Pretty much.
I'm basing the above on the fact that the RHEL/Centos package is compiled with:
-Dapparmor=disabled \ -Dapparmor_profiles=disabled \ -Dsecdriver_apparmor=disabled \
By extension, does that mean that you have to install libvirt on your host so that you can in turn run a container (which I'd presume is opaque) with libvirt bundled inside?
It's actually the other way around :)
If you don't have libvirt installed on the Debian/Ubuntu host, then the AppArmor profile won't be present and the containerized CentOS libvirt will be allowed to start the containerized CentOS QEMU.
If you *do* have libvirt installed on the Debian/Ubuntu host, then the AppArmor profile will also be applied to the containerized CentOS libvirt and running the containerized CentOS QEMU will be forbidden.
Patching the AppArmor policy is supposed to help with the second scenario.
I don't see how this can work properly. If running with AppArmor, I would expect libvirtd itself needs to be built with AppArmor, so that when launching a VM it can spawn virt-aa-helper to create the per-VM customized profile. The CentOS based libvirt running inside the container will be built without virt-aa-helper, so won't load this. I would rather expect that AppArmor does not attempt to control any processes inside the containers, other than with a generic 'docker' AppArmor profile. It makes no sense for profiles from the host OS install to apply to stuff in containers, as we can't assume the host + container installs are the same versions. Are you sure the KubeVirt problem isn't simply a mis-configuration of the host environment allowing AppArmor to leak inside the container. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 11/3/22 11:23, Daniel P. Berrangé wrote:
On Thu, Nov 03, 2022 at 12:35:15PM -0400, Andrea Bolognani wrote:
On Thu, Nov 03, 2022 at 03:39:44PM +0100, Peter Krempa wrote:
On Thu, Nov 03, 2022 at 12:13:53 +0100, Andrea Bolognani wrote:
Distros that use AppArmor, such as Debian and Ubuntu, install QEMU under /usr/bin/qemu-system-*, and our AppArmor profile is written with that assumption in mind.
If you try to run the RHEL or CentOS version of libvirt and QEMU inside a privileged container on such distros, however, that will result in an error, because the path /usr/libexec/qemu-kvm is used instead.
So IIUC by this patch you modify the profile which gets installed into the Debian/Ubuntu host system by the Debian/Ubuntu package which then in turn allows the non-Debian/Ubuntu libvirt in the container to do it's job?
Pretty much.
I'm basing the above on the fact that the RHEL/Centos package is compiled with:
-Dapparmor=disabled \ -Dapparmor_profiles=disabled \ -Dsecdriver_apparmor=disabled \
By extension, does that mean that you have to install libvirt on your host so that you can in turn run a container (which I'd presume is opaque) with libvirt bundled inside?
It's actually the other way around :)
If you don't have libvirt installed on the Debian/Ubuntu host, then the AppArmor profile won't be present and the containerized CentOS libvirt will be allowed to start the containerized CentOS QEMU.
If you *do* have libvirt installed on the Debian/Ubuntu host, then the AppArmor profile will also be applied to the containerized CentOS libvirt and running the containerized CentOS QEMU will be forbidden.
Patching the AppArmor policy is supposed to help with the second scenario.
I don't see how this can work properly.
Agree this scenario is a little suspect, but does this patch still have value? Is it possible to build/enable apparmor on a CentOS host, or is that impractical? Regards, Jim
On Fri, Nov 04, 2022 at 10:21:53AM -0600, Jim Fehlig wrote:
On 11/3/22 11:23, Daniel P. Berrangé wrote:
On Thu, Nov 03, 2022 at 12:35:15PM -0400, Andrea Bolognani wrote:
On Thu, Nov 03, 2022 at 03:39:44PM +0100, Peter Krempa wrote:
On Thu, Nov 03, 2022 at 12:13:53 +0100, Andrea Bolognani wrote:
Distros that use AppArmor, such as Debian and Ubuntu, install QEMU under /usr/bin/qemu-system-*, and our AppArmor profile is written with that assumption in mind.
If you try to run the RHEL or CentOS version of libvirt and QEMU inside a privileged container on such distros, however, that will result in an error, because the path /usr/libexec/qemu-kvm is used instead.
So IIUC by this patch you modify the profile which gets installed into the Debian/Ubuntu host system by the Debian/Ubuntu package which then in turn allows the non-Debian/Ubuntu libvirt in the container to do it's job?
Pretty much.
I'm basing the above on the fact that the RHEL/Centos package is compiled with:
-Dapparmor=disabled \ -Dapparmor_profiles=disabled \ -Dsecdriver_apparmor=disabled \
By extension, does that mean that you have to install libvirt on your host so that you can in turn run a container (which I'd presume is opaque) with libvirt bundled inside?
It's actually the other way around :)
If you don't have libvirt installed on the Debian/Ubuntu host, then the AppArmor profile won't be present and the containerized CentOS libvirt will be allowed to start the containerized CentOS QEMU.
If you *do* have libvirt installed on the Debian/Ubuntu host, then the AppArmor profile will also be applied to the containerized CentOS libvirt and running the containerized CentOS QEMU will be forbidden.
Patching the AppArmor policy is supposed to help with the second scenario.
I don't see how this can work properly.
Agree this scenario is a little suspect, but does this patch still have value? Is it possible to build/enable apparmor on a CentOS host, or is that impractical?
Anything's possible, but you'd be building a new kernel, and creating packages for the apparmor library, and rebuilding libvirt to enable apparmor too. Seems unlikely With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Thu, Nov 03, 2022 at 05:23:27PM +0000, Daniel P. Berrangé wrote:
On Thu, Nov 03, 2022 at 12:35:15PM -0400, Andrea Bolognani wrote:
On Thu, Nov 03, 2022 at 03:39:44PM +0100, Peter Krempa wrote:
On Thu, Nov 03, 2022 at 12:13:53 +0100, Andrea Bolognani wrote:
Distros that use AppArmor, such as Debian and Ubuntu, install QEMU under /usr/bin/qemu-system-*, and our AppArmor profile is written with that assumption in mind.
If you try to run the RHEL or CentOS version of libvirt and QEMU inside a privileged container on such distros, however, that will result in an error, because the path /usr/libexec/qemu-kvm is used instead.
So IIUC by this patch you modify the profile which gets installed into the Debian/Ubuntu host system by the Debian/Ubuntu package which then in turn allows the non-Debian/Ubuntu libvirt in the container to do it's job?
Pretty much.
I'm basing the above on the fact that the RHEL/Centos package is compiled with:
-Dapparmor=disabled \ -Dapparmor_profiles=disabled \ -Dsecdriver_apparmor=disabled \
By extension, does that mean that you have to install libvirt on your host so that you can in turn run a container (which I'd presume is opaque) with libvirt bundled inside?
It's actually the other way around :)
If you don't have libvirt installed on the Debian/Ubuntu host, then the AppArmor profile won't be present and the containerized CentOS libvirt will be allowed to start the containerized CentOS QEMU.
If you *do* have libvirt installed on the Debian/Ubuntu host, then the AppArmor profile will also be applied to the containerized CentOS libvirt and running the containerized CentOS QEMU will be forbidden.
Patching the AppArmor policy is supposed to help with the second scenario.
I don't see how this can work properly.
If running with AppArmor, I would expect libvirtd itself needs to be built with AppArmor, so that when launching a VM it can spawn virt-aa-helper to create the per-VM customized profile. The CentOS based libvirt running inside the container will be built without virt-aa-helper, so won't load this.
I would rather expect that AppArmor does not attempt to control any processes inside the containers, other than with a generic 'docker' AppArmor profile. It makes no sense for profiles from the host OS install to apply to stuff in containers, as we can't assume the host + container installs are the same versions. Are you sure the KubeVirt problem isn't simply a mis-configuration of the host environment allowing AppArmor to leak inside the container.
IIUC a specific profile (cri-containerd.apparmor.d) is used for unprivileged containers such as virt-launcher, but a privileged one such as virt-handler falls under the same profile as the host. This makes some amount of sense to me: unprivileged containers are already limited in what they can do by the usual restrictions on user processes. Privileged containers, on the other hand, are effectively root processes, so it's advisable to be significantly more cautious with them. Note that this is just my current understanding of the situation, and I'm far from an expert when it comes to containers in general and their interactions with AppArmor in particular. I recommend taking a look at https://github.com/kubevirt/kubevirt/pull/8692 and the issues linked therein, which will provide more context coming from people who actually know what they're talking about :) Now that I've typed all of the above, I wonder if the problem wouldn't be better solved by making sure that KubeVirt runs the libvirtd instance used to figure out node capabilities in an unprivileged container? Maybe there's something that prevents them from doing so. I'll bring up the idea and see what they think of it. -- Andrea Bolognani / Red Hat / Virtualization
On Fri, Nov 04, 2022 at 02:56:51PM -0400, Andrea Bolognani wrote:
On Thu, Nov 03, 2022 at 05:23:27PM +0000, Daniel P. Berrangé wrote:
On Thu, Nov 03, 2022 at 12:35:15PM -0400, Andrea Bolognani wrote:
On Thu, Nov 03, 2022 at 03:39:44PM +0100, Peter Krempa wrote:
On Thu, Nov 03, 2022 at 12:13:53 +0100, Andrea Bolognani wrote:
Distros that use AppArmor, such as Debian and Ubuntu, install QEMU under /usr/bin/qemu-system-*, and our AppArmor profile is written with that assumption in mind.
If you try to run the RHEL or CentOS version of libvirt and QEMU inside a privileged container on such distros, however, that will result in an error, because the path /usr/libexec/qemu-kvm is used instead.
So IIUC by this patch you modify the profile which gets installed into the Debian/Ubuntu host system by the Debian/Ubuntu package which then in turn allows the non-Debian/Ubuntu libvirt in the container to do it's job?
Pretty much.
I'm basing the above on the fact that the RHEL/Centos package is compiled with:
-Dapparmor=disabled \ -Dapparmor_profiles=disabled \ -Dsecdriver_apparmor=disabled \
By extension, does that mean that you have to install libvirt on your host so that you can in turn run a container (which I'd presume is opaque) with libvirt bundled inside?
It's actually the other way around :)
If you don't have libvirt installed on the Debian/Ubuntu host, then the AppArmor profile won't be present and the containerized CentOS libvirt will be allowed to start the containerized CentOS QEMU.
If you *do* have libvirt installed on the Debian/Ubuntu host, then the AppArmor profile will also be applied to the containerized CentOS libvirt and running the containerized CentOS QEMU will be forbidden.
Patching the AppArmor policy is supposed to help with the second scenario.
I don't see how this can work properly.
If running with AppArmor, I would expect libvirtd itself needs to be built with AppArmor, so that when launching a VM it can spawn virt-aa-helper to create the per-VM customized profile. The CentOS based libvirt running inside the container will be built without virt-aa-helper, so won't load this.
I would rather expect that AppArmor does not attempt to control any processes inside the containers, other than with a generic 'docker' AppArmor profile. It makes no sense for profiles from the host OS install to apply to stuff in containers, as we can't assume the host + container installs are the same versions. Are you sure the KubeVirt problem isn't simply a mis-configuration of the host environment allowing AppArmor to leak inside the container.
IIUC a specific profile (cri-containerd.apparmor.d) is used for unprivileged containers such as virt-launcher, but a privileged one such as virt-handler falls under the same profile as the host.
This makes some amount of sense to me: unprivileged containers are already limited in what they can do by the usual restrictions on user processes. Privileged containers, on the other hand, are effectively root processes, so it's advisable to be significantly more cautious with them.
I still consider that situation to be broken by design. If the privileged container is running a completely differnt software stack from the host OS, using the host OS apparmour profile to confined the container binary is never going to be a reliable setup. Either the privileged container has to run without confinement, or it needs to be confined using policy provided by the container (which is likely not viable anyway).
Note that this is just my current understanding of the situation, and I'm far from an expert when it comes to containers in general and their interactions with AppArmor in particular. I recommend taking a look at
https://github.com/kubevirt/kubevirt/pull/8692
and the issues linked therein, which will provide more context coming from people who actually know what they're talking about :)
I did read that and it didn't give me any more confidence that this setup is sensible.
Now that I've typed all of the above, I wonder if the problem wouldn't be better solved by making sure that KubeVirt runs the libvirtd instance used to figure out node capabilities in an unprivileged container? Maybe there's something that prevents them from doing so.
I'll bring up the idea and see what they think of it.
With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Wed, Nov 09, 2022 at 06:14:58PM +0000, Daniel P. Berrangé wrote:
On Fri, Nov 04, 2022 at 02:56:51PM -0400, Andrea Bolognani wrote:
IIUC a specific profile (cri-containerd.apparmor.d) is used for unprivileged containers such as virt-launcher, but a privileged one such as virt-handler falls under the same profile as the host.
This makes some amount of sense to me: unprivileged containers are already limited in what they can do by the usual restrictions on user processes. Privileged containers, on the other hand, are effectively root processes, so it's advisable to be significantly more cautious with them.
I still consider that situation to be broken by design. If the privileged container is running a completely differnt software stack from the host OS, using the host OS apparmour profile to confined the container binary is never going to be a reliable setup. Either the privileged container has to run without confinement, or it needs to be confined using policy provided by the container (which is likely not viable anyway).
Note that this is just my current understanding of the situation, and I'm far from an expert when it comes to containers in general and their interactions with AppArmor in particular. I recommend taking a look at
https://github.com/kubevirt/kubevirt/pull/8692
and the issues linked therein, which will provide more context coming from people who actually know what they're talking about :)
I did read that and it didn't give me any more confidence that this setup is sensible.
Closing the loop on this one. After some discussion[1], we have reached the agreement that the proper way to solve this is to have the node-labeller run in an unprivileged container, just as the actual VM workload would. Since doing that requires reworking KubeVirt in non-trivial ways, I have filed an issue[2] to track this. In the meantime, the user guide now recommends[3] simply uninstalling libvirt from the host, which is a simple and effective workaround. tl;dr SNACK Thanks everyone for the input! [1] https://github.com/kubevirt/kubevirt/pull/8692#issuecomment-1305956638 [2] https://github.com/kubevirt/kubevirt/issues/8744 [3] https://github.com/kubevirt/user-guide/pull/618 -- Andrea Bolognani / Red Hat / Virtualization
participants (4)
-
Andrea Bolognani -
Daniel P. Berrangé -
Jim Fehlig -
Peter Krempa