AppArmor confinement for qemu:///session VMs
An issue was recently reported[1] with running unprivileged VMs configured to use passt on Debian with AppArmor confinement enabled. After looking into the situation, I am convinced that AppArmor confinement never really worked for unprivileged VMs. The whole mechanism is built around the concept of per-VM profiles that are dynamically generated and registered, but doing so requires write access to /etc/apparmor.d/ and in general permissions that unprivileged libvirt will by design not have. Of course it's unfortunate that unprivileged VMs would be forced to miss out on the potential benefits of AppArmor isolation, and even more unfortunate that passt won't work out of the box for unprivileged VMs, since those are the ones where it makes the most sense to use passt in the first place. Stefano suggested introducing a generic "libvirt-user" profile that would be attached to unprivileged VMs and would be more liberal than the one used for privileged VMs, since we wouldn't be able to tailor it to the specifics of the VM, but would at least prevent the worst of the abuse; specifically, it would only allow R/W access to files in the current user's home directory. Does that sound like a reasonable direction? Any other ideas? In the meantime, Stefano has posted a workaround[2] that, when applied to passt's AppArmor profile, would allow these VMs to at least start. CC'ing people with AppArmor knowledge for awareness. [1] https://archives.passt.top/passt-dev/20250129104112.0756df5c@elisabeth/T/#u [2] https://archives.passt.top/passt-dev/20250205163101.3793658-1-sbrivio@redhat... -- Andrea Bolognani / Red Hat / Virtualization
On 2/5/25 10:22, Andrea Bolognani wrote:
An issue was recently reported[1] with running unprivileged VMs configured to use passt on Debian with AppArmor confinement enabled.
A long thread that's hard to read 'flat' or 'nested' via the referenced link :-). But yes, SUSE and openSUSE distros would be similarly affected if confinement of VMs is enabled. FYI, future SUSE and openSUSE distros will default to using selinux. E.g. here's a somewhat recent update on the progress in Tumbleweed https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/E...
After looking into the situation, I am convinced that AppArmor confinement never really worked for unprivileged VMs. The whole mechanism is built around the concept of per-VM profiles that are dynamically generated and registered, but doing so requires write access to /etc/apparmor.d/ and in general permissions that unprivileged libvirt will by design not have.
Likely it has never worked. I suspect the session daemon wasn't considered when per-VM confinement was developed.
Of course it's unfortunate that unprivileged VMs would be forced to miss out on the potential benefits of AppArmor isolation, and even more unfortunate that passt won't work out of the box for unprivileged VMs, since those are the ones where it makes the most sense to use passt in the first place.
Stefano suggested introducing a generic "libvirt-user" profile that would be attached to unprivileged VMs and would be more liberal than the one used for privileged VMs, since we wouldn't be able to tailor it to the specifics of the VM, but would at least prevent the worst of the abuse; specifically, it would only allow R/W access to files in the current user's home directory.
Does that sound like a reasonable direction? Any other ideas?
Sounds good to me. But like you, my apparmor knowledge is limited to bits and pieces gained through helping maintain the libvirt integration. Regards, Jim
On Mon, 10 Feb 2025 17:04:01 -0700 Jim Fehlig <jfehlig@suse.com> wrote:
On 2/5/25 10:22, Andrea Bolognani wrote:
An issue was recently reported[1] with running unprivileged VMs configured to use passt on Debian with AppArmor confinement enabled.
A long thread that's hard to read 'flat' or 'nested' via the referenced link :-).
It's the usual public-inbox thing, 'flat' and 'nested' links are just (presumably) one page scroll away? Or here is the mbox: https://archives.passt.top/passt-dev/20250129104112.0756df5c@elisabeth/t.mbo... This is Andrea's analysis of the issue, perhaps the most important message on that thread: https://archives.passt.top/passt-dev/CABJz62OUge8uNdXHVGU+YOW1_MfGbUMHnnb3DW... -- Stefano
On Wed, Feb 5, 2025 at 6:22 PM Andrea Bolognani <abologna@redhat.com> wrote:
An issue was recently reported[1] with running unprivileged VMs configured to use passt on Debian with AppArmor confinement enabled.
Hi Andrea (and Stefano), thank you for the depth and work on the topic!
After looking into the situation, I am convinced that AppArmor confinement never really worked for unprivileged VMs. The whole mechanism is built around the concept of per-VM profiles that are dynamically generated and registered, but doing so requires write access to /etc/apparmor.d/ and in general permissions that unprivileged libvirt will by design not have.
It becomes clear that, while it works well for the use cases that existed when Jaimie was developing it, it no longer does so for many modern approaches. From recent discussions about hotplug [1] to older demands to better handle pools [2] and various in-betweens - they all tend to come to a compromise or big-rework approach. So unless/until there is a major spike to rework the apparmor/libvirt interaction we have to make compromises like the one you went for below :-/ For the time being I think we should be ok with (non-too awkward) compromise solutions. While it reduces the pain that would force to do the big rework, it keeps users functional and that is what we should care about the most. I'm even tempted to suggest accepting [1] without insisting on too big of a rework for the same reasons. FWIW I've added the unprivileged user session handling to my list of known "one should do for apparmor/libvirt ..." tracking list. [1]: https://gitlab.com/libvirt/libvirt/-/issues/692 [2]: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1677398
Of course it's unfortunate that unprivileged VMs would be forced to miss out on the potential benefits of AppArmor isolation, and even more unfortunate that passt won't work out of the box for unprivileged VMs, since those are the ones where it makes the most sense to use passt in the first place.
Stefano suggested introducing a generic "libvirt-user" profile that would be attached to unprivileged VMs and would be more liberal than the one used for privileged VMs, since we wouldn't be able to tailor it to the specifics of the VM, but would at least prevent the worst of the abuse; specifically, it would only allow R/W access to files in the current user's home directory.
Does that sound like a reasonable direction? Any other ideas?
I've read it probably the fourth time now, each time before concluding "I need to think more, maybe something comes to mind if I don't read in a rush" :-) But even after the fourth time I like the compromise that you proposed. It is much better than not isolating them at all, after all libvirtd itself also has a liberal profile and despite being so "open" has prevented quite some already.
In the meantime, Stefano has posted a workaround[2] that, when applied to passt's AppArmor profile, would allow these VMs to at least start.
CC'ing people with AppArmor knowledge for awareness.
[1] https://archives.passt.top/passt-dev/20250129104112.0756df5c@elisabeth/T/#u [2] https://archives.passt.top/passt-dev/20250205163101.3793658-1-sbrivio@redhat... -- Andrea Bolognani / Red Hat / Virtualization
-- Christian Ehrhardt Director of Engineering, Ubuntu Server Canonical Ltd
On Thu, 13 Feb 2025 09:16:33 +0100 Christian Ehrhardt <christian.ehrhardt@canonical.com> wrote:
On Wed, Feb 5, 2025 at 6:22 PM Andrea Bolognani <abologna@redhat.com> wrote:
An issue was recently reported[1] with running unprivileged VMs configured to use passt on Debian with AppArmor confinement enabled.
Hi Andrea (and Stefano), thank you for the depth and work on the topic!
After looking into the situation, I am convinced that AppArmor confinement never really worked for unprivileged VMs. The whole mechanism is built around the concept of per-VM profiles that are dynamically generated and registered, but doing so requires write access to /etc/apparmor.d/ and in general permissions that unprivileged libvirt will by design not have.
It becomes clear that, while it works well for the use cases that existed when Jaimie was developing it, it no longer does so for many modern approaches. From recent discussions about hotplug [1] to older demands to better handle pools [2] and various in-betweens - they all tend to come to a compromise or big-rework approach.
So unless/until there is a major spike to rework the apparmor/libvirt interaction we have to make compromises like the one you went for below :-/
So, about passt, the compromise we now have upstream (Debian packages coming in a bit) is probably the quickest way to restore functionality, but it's also the most... compromising, because passt isn't necessarily started by libvirt, and yet it's now going to allow read-write access to @{run}/user/[0-9]*/libvirt/qemu/run/passt/*, libvirt or not, and to execute itself. The compromise described below would avoid this, while being reasonably simple.
For the time being I think we should be ok with (non-too awkward) compromise solutions. While it reduces the pain that would force to do the big rework, it keeps users functional and that is what we should care about the most. I'm even tempted to suggest accepting [1] without insisting on too big of a rework for the same reasons.
FWIW I've added the unprivileged user session handling to my list of known "one should do for apparmor/libvirt ..." tracking list.
[1]: https://gitlab.com/libvirt/libvirt/-/issues/692 [2]: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1677398
Of course it's unfortunate that unprivileged VMs would be forced to miss out on the potential benefits of AppArmor isolation, and even more unfortunate that passt won't work out of the box for unprivileged VMs, since those are the ones where it makes the most sense to use passt in the first place.
Stefano suggested introducing a generic "libvirt-user" profile that would be attached to unprivileged VMs and would be more liberal than the one used for privileged VMs, since we wouldn't be able to tailor it to the specifics of the VM, but would at least prevent the worst of the abuse; specifically, it would only allow R/W access to files in the current user's home directory.
Does that sound like a reasonable direction? Any other ideas?
I've read it probably the fourth time now, each time before concluding "I need to think more, maybe something comes to mind if I don't read in a rush" :-)
But even after the fourth time I like the compromise that you proposed. It is much better than not isolating them at all, after all libvirtd itself also has a liberal profile and despite being so "open" has prevented quite some already.
...and, by the way, at least nothing is running as root with that profile. In this case, a strong isolation between users is already provided even without AppArmor. We just wouldn't isolate between VMs run by the same user.
In the meantime, Stefano has posted a workaround[2] that, when applied to passt's AppArmor profile, would allow these VMs to at least start.
CC'ing people with AppArmor knowledge for awareness.
[1] https://archives.passt.top/passt-dev/20250129104112.0756df5c@elisabeth/T/#u [2] https://archives.passt.top/passt-dev/20250205163101.3793658-1-sbrivio@redhat...
-- Stefano
On Thu, Feb 13, 2025 at 03:36:00PM +0100, Stefano Brivio wrote:
On Thu, 13 Feb 2025 09:16:33 +0100 Christian Ehrhardt <christian.ehrhardt@canonical.com> wrote:
On Wed, Feb 5, 2025 at 6:22 PM Andrea Bolognani <abologna@redhat.com> wrote:
After looking into the situation, I am convinced that AppArmor confinement never really worked for unprivileged VMs. The whole mechanism is built around the concept of per-VM profiles that are dynamically generated and registered, but doing so requires write access to /etc/apparmor.d/ and in general permissions that unprivileged libvirt will by design not have.
It becomes clear that, while it works well for the use cases that existed when Jaimie was developing it, it no longer does so for many modern approaches.
Session libvirt has just not received as much love as system libvirt in general over the years. This is changing somewhat now that we have a very prominent user (KubeVirt) and major improvements to external components (passt) make using it a more viable prospect.
So, about passt, the compromise we now have upstream (Debian packages coming in a bit) is probably the quickest way to restore functionality, but it's also the most... compromising, because passt isn't necessarily started by libvirt, and yet it's now going to allow read-write access to @{run}/user/[0-9]*/libvirt/qemu/run/passt/*, libvirt or not, and to execute itself.
Agreed. passt is just not the right component to address this. The workaround you've applied works in a pinch, but it's very obviously a kludge that we should aim to make unnecessary sooner rather than later.
For the time being I think we should be ok with (non-too awkward) compromise solutions. While it reduces the pain that would force to do the big rework, it keeps users functional and that is what we should care about the most. I'm even tempted to suggest accepting [1] without insisting on too big of a rework for the same reasons.
I've already expressed my opinion about that patch series. Believe me, I don't enjoy blocking the fix any more than you and Georgia do, but I'm genuinely convinced that changing how the driver works in such a fundamental way just to address a single, somewhat narrow failure scenario would prove to be a mistake in the long run :(
But even after the fourth time I like the compromise that you proposed. It is much better than not isolating them at all, after all libvirtd itself also has a liberal profile and despite being so "open" has prevented quite some already.
...and, by the way, at least nothing is running as root with that profile.
In this case, a strong isolation between users is already provided even without AppArmor. We just wouldn't isolate between VMs run by the same user.
To be clear, I like the proposed approach too. It doesn't really change how the driver operates - if anything, it makes session mode and system mode more alike than ever - and takes us pretty much as far as we can reasonably go in terms of isolation given the requirement to keep things unprivileged. It should be reasonably straightforward to implement too. Is anyone willing to take a stab at it? I'm afraid I can't commit the time myself for the foreseeable future, though I should be able to accommodate at least reviewing any patches that might show up. -- Andrea Bolognani / Red Hat / Virtualization
participants (4)
-
Andrea Bolognani -
Christian Ehrhardt -
Jim Fehlig -
Stefano Brivio