SELinux labels change in libvirt
Hello all, tl;dr, can you point me to the point in the libvirt repo where it's trying to change a tap-device's SELinux label? I am trying to create a tap device with libvirt on a super-privileged container, and then use it on another, unprivileged container with libvirt. User wise, I know I need the super-privileged container to open the tap device with the user of the unprivileged one - that I already did and it's not the issue. But I have a problem when I open the tap device in the non-privileged container: the tap device currently has the spc_t label since the tun_socket inherited the selinux context from the super-privileged container who creates it. then libvirt is trying to change the SELinux labels, and since it's not privileged then it fails. But I didn't find where and how libvirt is trying to change the tap device's label. Can you point me to that specific code on libvirt? Ram Lavi Senior Software Engineer Red Hat Israel <https://www.redhat.com/> Yerushalaim Road 34, Ra'anana ralavi@redhat.com IM: ralavi @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com/>
On Tue, Jul 14, 2020 at 03:21:17PM +0300, Ram Lavi wrote:
Hello all,
tl;dr, can you point me to the point in the libvirt repo where it's trying to change a tap-device's SELinux label?
I am trying to create a tap device with libvirt on a super-privileged container, and then use it on another, unprivileged container with libvirt. User wise, I know I need the super-privileged container to open the tap device with the user of the unprivileged one - that I already did and it's not the issue. But I have a problem when I open the tap device in the non-privileged container: the tap device currently has the spc_t label since the tun_socket inherited the selinux context from the super-privileged container who creates it. then libvirt is trying to change the SELinux labels, and since it's not privileged then it fails. But I didn't find where and how libvirt is trying to change the tap device's label.
Can you point me to that specific code on libvirt?
If the SELinux policy that libvirtd is running under prevents it from re-labelling, then TAP devices label failure is just going to be one out of 100's of labelling failures. Either the SELinux policy needs to be changed to allow libvirtd to relabel stuff in the normal manner, or you will have to turn off SELinux support in libvirtd. in /etc/libvirt/qemu.conf via the param security_driver = "none". If you turn off SELinux in libvirt, then you no longer have separation of QEMU processes which may be a security flaw depending on your deplyoment scenario. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Tue, Jul 14, 2020 at 3:33 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
Hello all,
tl;dr, can you point me to the point in the libvirt repo where it's
On Tue, Jul 14, 2020 at 03:21:17PM +0300, Ram Lavi wrote: trying
to change a tap-device's SELinux label?
I am trying to create a tap device with libvirt on a super-privileged container, and then use it on another, unprivileged container with libvirt. User wise, I know I need the super-privileged container to open the tap device with the user of the unprivileged one - that I already did and it's not the issue. But I have a problem when I open the tap device in the non-privileged container: the tap device currently has the spc_t label since the tun_socket inherited the selinux context from the super-privileged container who creates it. then libvirt is trying to change the SELinux labels, and since it's not privileged then it fails. But I didn't find where and how libvirt is trying to change the tap device's label.
Can you point me to that specific code on libvirt?
If the SELinux policy that libvirtd is running under prevents it from re-labelling, then TAP devices label failure is just going to be one out of 100's of labelling failures.
IIUC normally libvirtd would use devices created by itself, so there shouldn't be relabeling failures, right?
Either the SELinux policy needs to be changed to allow libvirtd to relabel stuff in the normal manner, or you will have to turn off SELinux support in libvirtd. in /etc/libvirt/qemu.conf via the param security_driver = "none". If you turn off SELinux in libvirt, then you no longer have separation of QEMU processes which may be a security flaw depending on your deplyoment scenario.
turning SELinux in libvirtd off or allowing libvirt to relabel are tempting options but it is not an option I'm afraid due to security concerns. Our plan (or at least this particular effort) is to try to relabel the tun-socket after created in the super-privileged container to be the same one as the one used in the unprivileged one, then it won't have an issue consuming it. btw the change I (or rather Migule's my teammate) made is in this PR, where I want to add a tap device in virt-handler (i.e. the super privileged container) to be further uses in virt-launcher (i.e. the non-privileged container): https://github.com/kubevirt/kubevirt/pull/3290
Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Tue, Jul 14, 2020 at 04:02:17PM +0300, Ram Lavi wrote:
On Tue, Jul 14, 2020 at 3:33 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
Hello all,
tl;dr, can you point me to the point in the libvirt repo where it's
On Tue, Jul 14, 2020 at 03:21:17PM +0300, Ram Lavi wrote: trying
to change a tap-device's SELinux label?
I am trying to create a tap device with libvirt on a super-privileged container, and then use it on another, unprivileged container with libvirt. User wise, I know I need the super-privileged container to open the tap device with the user of the unprivileged one - that I already did and it's not the issue. But I have a problem when I open the tap device in the non-privileged container: the tap device currently has the spc_t label since the tun_socket inherited the selinux context from the super-privileged container who creates it. then libvirt is trying to change the SELinux labels, and since it's not privileged then it fails. But I didn't find where and how libvirt is trying to change the tap device's label.
Can you point me to that specific code on libvirt?
If the SELinux policy that libvirtd is running under prevents it from re-labelling, then TAP devices label failure is just going to be one out of 100's of labelling failures.
IIUC normally libvirtd would use devices created by itself, so there shouldn't be relabeling failures, right?
Whether or not there are relabeling failures is determined by what SELinux policy libvirtd is running under.
Either the SELinux policy needs to be changed to allow libvirtd to relabel stuff in the normal manner, or you will have to turn off SELinux support in libvirtd. in /etc/libvirt/qemu.conf via the param security_driver = "none". If you turn off SELinux in libvirt, then you no longer have separation of QEMU processes which may be a security flaw depending on your deplyoment scenario.
turning SELinux in libvirtd off or allowing libvirt to relabel are tempting options but it is not an option I'm afraid due to security concerns. Our plan (or at least this particular effort) is to try to relabel the tun-socket after created in the super-privileged container to be the same one as the one used in the unprivileged one, then it won't have an issue consuming it.
btw the change I (or rather Migule's my teammate) made is in this PR, where I want to add a tap device in virt-handler (i.e. the super privileged container) to be further uses in virt-launcher (i.e. the non-privileged container): https://github.com/kubevirt/kubevirt/pull/3290
In normal host OS deployment, libvirtd runs under virtd_t, and when it spawns QEMU, it will relabel files to svirt_image_t:s0:$MCS, and spawn QEMU as svirt_t:s0:$MCS. My understanding is what in kubevirt, things work differently. Docker (or podman), launch the container as container_t:s0:$MCS. libvirtd *and* QEMU thus both run as container_t:s0:$MCS. ie All the labelling is setup when the container is launched and libvirtd should not do anything. So I'm really not sure why you have libvirtd configured to do relabelling at all ? I'd be expecting it to have security_driver=none in the qemu.conf file so that libvirtd doesn't do anything. If libvirtd is doing relabelling, I'm not sure how it works for anything, let alone tap devices. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Tue, Jul 14, 2020 at 6:03 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
On Tue, Jul 14, 2020 at 04:02:17PM +0300, Ram Lavi wrote:
On Tue, Jul 14, 2020 at 3:33 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
Hello all,
tl;dr, can you point me to the point in the libvirt repo where it's
On Tue, Jul 14, 2020 at 03:21:17PM +0300, Ram Lavi wrote: trying
to change a tap-device's SELinux label?
I am trying to create a tap device with libvirt on a super-privileged container, and then use it on another, unprivileged container with libvirt. User wise, I know I need the super-privileged container to open the tap device with the user of the unprivileged one - that I already did and it's not the issue. But I have a problem when I open the tap device in the non-privileged container: the tap device currently has the spc_t label since the tun_socket inherited the selinux context from the super-privileged container who creates it. then libvirt is trying to change the SELinux labels, and since it's not privileged then it fails. But I didn't find where and how libvirt is trying to change the tap device's label.
Can you point me to that specific code on libvirt?
If the SELinux policy that libvirtd is running under prevents it from re-labelling, then TAP devices label failure is just going to be one out of 100's of labelling failures.
IIUC normally libvirtd would use devices created by itself, so there shouldn't be relabeling failures, right?
Whether or not there are relabeling failures is determined by what SELinux policy libvirtd is running under.
Either the SELinux policy needs to be changed to allow libvirtd to relabel stuff in the normal manner, or you will have to turn off SELinux support in libvirtd. in /etc/libvirt/qemu.conf via the param security_driver = "none". If you turn off SELinux in libvirt, then you no longer have separation of QEMU processes which may be a security flaw depending on your deplyoment scenario.
turning SELinux in libvirtd off or allowing libvirt to relabel are tempting options but it is not an option I'm afraid due to security concerns. Our plan (or at least this particular effort) is to try to relabel the tun-socket after created in the super-privileged container to be the same one as the one used in the unprivileged one, then it won't have an issue consuming it.
btw the change I (or rather Migule's my teammate) made is in this PR, where I want to add a tap device in virt-handler (i.e. the super privileged container) to be further uses in virt-launcher (i.e. the non-privileged container): https://github.com/kubevirt/kubevirt/pull/3290
In normal host OS deployment, libvirtd runs under virtd_t, and when it spawns QEMU, it will relabel files to svirt_image_t:s0:$MCS, and spawn QEMU as svirt_t:s0:$MCS.
My understanding is what in kubevirt, things work differently. Docker (or podman), launch the container as container_t:s0:$MCS. libvirtd *and* QEMU thus both run as container_t:s0:$MCS. ie All the labelling is setup when the container is launched and libvirtd should not do anything.
So I'm really not sure why you have libvirtd configured to do relabelling at all ? I'd be expecting it to have security_driver=none in the qemu.conf file so that libvirtd doesn't do anything.
I checked the dumpxml of the virt-launcher pod (that runs the qemu in kubevirt) - it has dynamic policy. <seclabel type='dynamic' model='dac' relabel='yes'> <label>+107:+107</label> <imagelabel>+107:+107</imagelabel> </seclabel> Are you saying this a wrong configuration for a kubevirt vmi?
If libvirtd is doing relabelling, I'm not sure how it works for anything, let alone tap devices.
I want to emphasize that in this current situation the reason we fail is that we are trying to give libvirt in a non-privileged pod (i.e. virt-launcher) a tap device that was created externally (by a privileged pod - virt-handler). I think the reason it works so far is that libvirt was using only devices which were self created so there was no issue to relabel. Our thought was that if we know how libvirt is relabeling then we could also do it so that the externally created tap's label will match the virt-launcher's. Is this were libvirt does the relabeling https://github.com/libvirt/libvirt/blob/e71e13488dc1aa65456e54a4b41bc925821b... ? btw the error we get is (from audit) type=AVC msg=audit(1586956552.265:513): avc: denied { relabelfrom } for pid=27423 comm="libvirtd" scontext=system_u:system_r:container_t:s0:c143,c582 tcontext=system_u:system_r:spc_t:s0 tclass=tun_socket permissive=0
Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Thu, Jul 16, 2020 at 11:22:26AM +0300, Ram Lavi wrote:
On Tue, Jul 14, 2020 at 6:03 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
I checked the dumpxml of the virt-launcher pod (that runs the qemu in kubevirt) - it has dynamic policy.
<seclabel type='dynamic' model='dac' relabel='yes'> <label>+107:+107</label> <imagelabel>+107:+107</imagelabel> </seclabel>
Are you saying this a wrong configuration for a kubevirt vmi?
No, this is exactly what I'm expecting and matches the behaviour I mentioned in my previous mail, where SELinux is not enabled in libvirtd. This setting is completely unrelated to SELinux. This seclabel is for the "dac" model, aka Discretionary Access Control, aka plain old POSIX users and groups. It is used when libvirtd runs as root, and then launches QEMU as a separate "qemu:qemu" user:group pair. If selinux / sVirt was in use you would see a separate <seclabel> with model="selinux". If you don't see that, then libvirtd is *not* doing any SELinux labelling at all. This makes me wonder how you hit the relabelling AVC ? It feels very much like your custom work to deal with passing in TAP devices has mistakenly changed something else and thus incorrectly turned on SELinux/sVirt in libvirtd.
If libvirtd is doing relabelling, I'm not sure how it works for anything, let alone tap devices.
I want to emphasize that in this current situation the reason we fail is that we are trying to give libvirt in a non-privileged pod (i.e. virt-launcher) a tap device that was created externally (by a privileged pod - virt-handler). I think the reason it works so far is that libvirt was using only devices which were self created so there was no issue to relabel.
I believe the reason it works historically is that libvirtd is not using SELinux, per your <seclabel> config above, and so was not attempting todo any relabelling in the first place.
Our thought was that if we know how libvirt is relabeling then we could also do it so that the externally created tap's label will match the virt-launcher's.
You almost certainly *will* need to relabel the tap device when passing it from your privileged container into the unprivileged, but this is not due to libvirtd. This is because podman is likely using a different SELinux label for the two containers as they have different privilege levels. The tap device needs to be given the SELinux label matching the unprivileged container.
Is this were libvirt does the relabeling https://github.com/libvirt/libvirt/blob/e71e13488dc1aa65456e54a4b41bc925821b... ?
That's for setting contexts on files, for FDs you need to call fsetfilecon_raw
btw the error we get is (from audit)
type=AVC msg=audit(1586956552.265:513): avc: denied { relabelfrom } for pid=27423 comm="libvirtd" scontext=system_u:system_r:container_t:s0:c143,c582 tcontext=system_u:system_r:spc_t:s0 tclass=tun_socket permissive=0
This just suggests libvirtd has been misconfigured in some way that causes SELinux to become enabled again. You need to do relabelling of the TAP device, but that isn't going to avoid this AVC - it will just change what you see as the tcontext in the AVC. You still need to fix the configuration of libvirtd I believe. If you run 'virsh capabilities' in your container that is showing this problem, I expect it will say: <secmodel> <model>selinux</model> <doi>0</doi> <baselabel type='kvm'>system_u:system_r:svirt_t:s0</baselabel> <baselabel type='qemu'>system_u:system_r:svirt_tcg_t:s0</baselabel> </secmodel> If you compare with the "virsh capabilities" from a correctly operating existing KubeVirt install, then I expect selinux won't be reported at all, which would match your <seclabel> example from earlier. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
participants (2)
-
Daniel P. Berrangé -
Ram Lavi