[libvirt] [sandbox] Weird apparmor problems
Hi all, I'm seeing weird apparmor errors when running virt-sandbox here. Here are the log entries: apparmor="ALLOWED" operation="mknod" parent=1 profile="libvirt-634ed189-cca0-4126-830c-4e4a76846b25" name="/var/lib/libvirt/qemu/sandbox.monitor" pid=2251 comm="qemu-system-x86" requested_mask="c" denied_mask="c" fsuid=493 ouid=493 apparmor="ALLOWED" operation="open" parent=1 profile="libvirt-634ed189-cca0-4126-830c-4e4a76846b25" name="/dev/ptmx" pid=2251 comm="qemu-system-x86" requested_mask="w" denied_mask="w" fsuid=493 ouid=0 apparmor="ALLOWED" operation="open" parent=1 profile="libvirt-634ed189-cca0-4126-830c-4e4a76846b25" name="/dev/pts/2" pid=2251 comm="qemu-system-x86" requested_mask="w" denied_mask="w" fsuid=493 ouid=493 apparmor="ALLOWED" operation="file_perm" parent=1 profile="libvirt-634ed189-cca0-4126-830c-4e4a76846b25" name="/var/log/libvirt/qemu/sandbox.log" pid=2251 comm="qemu-system-x86" requested_mask="w" denied_mask="w" fsuid=493 ouid=0 apparmor="ALLOWED" operation="open" parent=1 profile="libvirt-634ed189-cca0-4126-830c-4e4a76846b25" name="/dev/ptmx" pid=2251 comm="qemu-system-x86" requested_mask="w" denied_mask="w" fsuid=493 ouid=0 apparmor="ALLOWED" operation="open" parent=1 profile="libvirt-634ed189-cca0-4126-830c-4e4a76846b25" name="/dev/pts/3" pid=2251 comm="qemu-system-x86" requested_mask="w" denied_mask="w" fsuid=493 ouid=493 apparmor="ALLOWED" operation="file_perm" parent=1 profile="libvirt-634ed189-cca0-4126-830c-4e4a76846b25" name="/var/log/libvirt/qemu/sandbox.log" pid=2251 comm="qemu-system-x86" requested_mask="w" denied_mask="w" fsuid=493 ouid=0 apparmor="ALLOWED" operation="open" parent=1 profile="libvirt-634ed189-cca0-4126-830c-4e4a76846b25" name="/dev/kvm" pid=2251 comm="qemu-system-x86" requested_mask="w" denied_mask="w" fsuid=493 ouid=0 The weird thing is that /dev/kvm, /var/log/libvirt/qemu/sandbox.log and /var/lib/libvirt/qemu/sandbox.monitor already have rules. And I'm wondering if it's normal to have write access to /dev/pts/* and /dev/ptmx. Any idea? -- Cedric
On Thu, Oct 29, 2015 at 05:35:23PM +0100, Cedric Bosdonnat wrote:
Hi all,
I'm seeing weird apparmor errors when running virt-sandbox here. Here are the log entries:
apparmor="ALLOWED" operation="mknod" parent=1 profile="libvirt-634ed189-cca0-4126-830c-4e4a76846b25" name="/var/lib/libvirt/qemu/sandbox.monitor" pid=2251 comm="qemu-system-x86" requested_mask="c" denied_mask="c" fsuid=493 ouid=493 apparmor="ALLOWED" operation="open" parent=1 profile="libvirt-634ed189-cca0-4126-830c-4e4a76846b25" name="/dev/ptmx" pid=2251 comm="qemu-system-x86" requested_mask="w" denied_mask="w" fsuid=493 ouid=0 apparmor="ALLOWED" operation="open" parent=1 profile="libvirt-634ed189-cca0-4126-830c-4e4a76846b25" name="/dev/pts/2" pid=2251 comm="qemu-system-x86" requested_mask="w" denied_mask="w" fsuid=493 ouid=493 apparmor="ALLOWED" operation="file_perm" parent=1 profile="libvirt-634ed189-cca0-4126-830c-4e4a76846b25" name="/var/log/libvirt/qemu/sandbox.log" pid=2251 comm="qemu-system-x86" requested_mask="w" denied_mask="w" fsuid=493 ouid=0 apparmor="ALLOWED" operation="open" parent=1 profile="libvirt-634ed189-cca0-4126-830c-4e4a76846b25" name="/dev/ptmx" pid=2251 comm="qemu-system-x86" requested_mask="w" denied_mask="w" fsuid=493 ouid=0 apparmor="ALLOWED" operation="open" parent=1 profile="libvirt-634ed189-cca0-4126-830c-4e4a76846b25" name="/dev/pts/3" pid=2251 comm="qemu-system-x86" requested_mask="w" denied_mask="w" fsuid=493 ouid=493 apparmor="ALLOWED" operation="file_perm" parent=1 profile="libvirt-634ed189-cca0-4126-830c-4e4a76846b25" name="/var/log/libvirt/qemu/sandbox.log" pid=2251 comm="qemu-system-x86" requested_mask="w" denied_mask="w" fsuid=493 ouid=0 apparmor="ALLOWED" operation="open" parent=1 profile="libvirt-634ed189-cca0-4126-830c-4e4a76846b25" name="/dev/kvm" pid=2251 comm="qemu-system-x86" requested_mask="w" denied_mask="w" fsuid=493 ouid=0
The weird thing is that /dev/kvm, /var/log/libvirt/qemu/sandbox.log and /var/lib/libvirt/qemu/sandbox.monitor already have rules.
And I'm wondering if it's normal to have write access to /dev/pts/* and /dev/ptmx.
NB in containers we have two PTYs involved. The libvirt_lxc process opens one pty in the host context and that is used to communicate between virsh console & libvirt_lxc. The libvirt_lxc process opens one pty in the guest context and that is used to commnuicate between libvirt_lxc and the container master console. Libvirt_lxc forwards data between the two PTYs. So, yes, it is normal for libvirt_lxc to access /dev/ptmx to create a new master PTY and to read/write to /dev/pts/NN associated with the file descriptor retrieved from /dev/ptmx. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Fri, 2015-10-30 at 09:15 +0900, Daniel P. Berrange wrote:
NB in containers we have two PTYs involved. The libvirt_lxc process opens one pty in the host context and that is used to communicate between virsh console & libvirt_lxc. The libvirt_lxc process opens one pty in the guest context and that is used to commnuicate between libvirt_lxc and the container master console. Libvirt_lxc forwards data between the two PTYs.
So, yes, it is normal for libvirt_lxc to access /dev/ptmx to create a new master PTY and to read/write to /dev/pts/NN associated with the file descriptor retrieved from /dev/ptmx.
After checking more carefully, all rules are already in the profile... and are concerning the qemu builder. I haven't checked if it happens with lxc yet. The question now is why does it happen with virt-sandbox and not with a normal libvirt qemu domain. -- Cedric
On Fri, 2015-10-30 at 09:15 +0900, Daniel P. Berrange wrote:
So, yes, it is normal for libvirt_lxc to access /dev/ptmx to create a new master PTY and to read/write to /dev/pts/NN associated with the file descriptor retrieved from /dev/ptmx.
After some more debugging and help from jjohansen, the problem happens to be this commit: http://libvirt.org/git/?p=libvirt.git;a=commit;h=d0d4b8ad76d3e8a859ee90701a2... When having the not-so-silly idea to mount the host / readonly in a qemu guest (like what virt-sandbox is doing), we are adding a "deny /** w" rule taking precedence over all rules giving write access to files inside that path. Would there be a clean solution for that problem? I can already teach virt-sandbox to add the host / mount only if there is nothing else to be mounted as /, but that wouldn't cover all cases. -- Cedric
On 10/30/2015 08:27 AM, Cedric Bosdonnat wrote:
On Fri, 2015-10-30 at 09:15 +0900, Daniel P. Berrange wrote:
So, yes, it is normal for libvirt_lxc to access /dev/ptmx to create a new master PTY and to read/write to /dev/pts/NN associated with the file descriptor retrieved from /dev/ptmx.
After some more debugging and help from jjohansen, the problem happens to be this commit:
http://libvirt.org/git/?p=libvirt.git;a=commit;h=d0d4b8ad76d3e8a859ee90701a2...
This commit isn't the actual issue. It is the logic in this commit combined with: http://libvirt.org/git/?p=libvirt.git;a=commitdiff;h=29ea8a9b64aac60251d283f... In the original commit, presumably only 'deny / w,' would have been added, but with the second commit, this turns into 'deny /** w,'.
When having the not-so-silly idea to mount the host / readonly in a qemu guest (like what virt-sandbox is doing), we are adding a "deny /** w" rule taking precedence over all rules giving write access to files inside that path.
Would there be a clean solution for that problem? I can already teach virt-sandbox to add the host / mount only if there is nothing else to be mounted as /, but that wouldn't cover all cases.
There is nothing that can be done with additional rules if the 'deny /** w,' is being added because deny rules always take precedence over other rules. The only course of action is to rework the logic introduced in 29ea8a9b64aac60251d283f74d57690e4edb5a6b. One option might be to revert it and then add the glob rule conditionally on if it is a 9p filesystem. I'm not sure if this is even valid for 9p filesystems though. -- Jamie Strandboge http://www.ubuntu.com/
participants (3)
-
Cedric Bosdonnat -
Daniel P. Berrange -
Jamie Strandboge