virtiofs mounted filesystems & SELinux
I am mounting a filesystem into a domain using the virtiofs driver. <filesystem accessmode="passthrough" type="mount"> <source dir="/home"/> <target dir="/home"/> <driver type="virtiofs"/> </filesystem> Both my host (Fedora 34) and guest (CentOS 8.4) are running with SELinux enforcing. From my host, I can see that the SELinux context type is set to user_home_dir_t. $ ls -ldZ /home/link drwxr-xr-x. 61 link link system_u:object_r:user_home_dir_t:s0 8192 May 21 12:41 /home/link
From within the guest however, the volume is unlabeled_t
$ ls -lZd /home/link drwxr-xr-x. 61 link link system_u:object_r:unlabeled_t:s0 8192 May 21 12:53 / home/link Is there a way to pass the SELinux context through to the guest? Or mount the volume with the correct options to map SELinux contexts?
On 5/21/21 11:59 AM, Link Dupont wrote: Adding the virtio-fs mailing list.
I am mounting a filesystem into a domain using the virtiofs driver.
<filesystem accessmode="passthrough" type="mount"> <source dir="/home"/> <target dir="/home"/> <driver type="virtiofs"/> </filesystem>
Both my host (Fedora 34) and guest (CentOS 8.4) are running with SELinux enforcing. From my host, I can see that the SELinux context type is set to user_home_dir_t.
$ ls -ldZ /home/link drwxr-xr-x. 61 link link system_u:object_r:user_home_dir_t:s0 8192 May 21 12:41 /home/link
From within the guest however, the volume is unlabeled_t
$ ls -lZd /home/link drwxr-xr-x. 61 link link system_u:object_r:unlabeled_t:s0 8192 May 21 12:53 / home/link
Is there a way to pass the SELinux context through to the guest? Or mount the volume with the correct options to map SELinux contexts?
Hi, I'm afraid I actually don't know that much about SELinux but I read that it relies on using extended attributes in the file system to accomplish its labeling. Do you still experience this issue when you enable extended attribute support[1] in virtiofsd? The example in the optional parameters snippet enables extended attributes with the xattr='on' element. Connor [1] https://libvirt.org/kbase/virtiofs.html#optional-parameters
Adding the <binary xattr='on'> element to the <filesystem> device does seem to spawn virtiofsd with the option string "source=/home,xattr". My guest can no longer mount the device though. It errors with: [ 170.225553] 9pnet_virtio: no channels available mount: mount(2) failed: No such file or directory I think what this is doing is causing libvirt to create the device as a virtiofs device instead of a 9p device. The EL7 kernel doesn't have a virtiofs driver, so it can't mount virtiofs devices. My knowledge is unfortunately limited about the nuances between 9p and virtiofs. So I'm mostly experimenting by trial-and-error here. On Wed, Jun 2 2021 at 03:55:40 PM -0500, Connor Kuehl <ckuehl@redhat.com> wrote:
On 5/21/21 11:59 AM, Link Dupont wrote:
Adding the virtio-fs mailing list.
I am mounting a filesystem into a domain using the virtiofs driver.
<filesystem accessmode="passthrough" type="mount"> <source dir="/home"/> <target dir="/home"/> <driver type="virtiofs"/> </filesystem>
Both my host (Fedora 34) and guest (CentOS 8.4) are running with SELinux enforcing. From my host, I can see that the SELinux context type is set to user_home_dir_t.
$ ls -ldZ /home/link drwxr-xr-x. 61 link link system_u:object_r:user_home_dir_t:s0 8192 May 21 12:41 /home/link
From within the guest however, the volume is unlabeled_t
$ ls -lZd /home/link drwxr-xr-x. 61 link link system_u:object_r:unlabeled_t:s0 8192 May 21 12:53 / home/link
Is there a way to pass the SELinux context through to the guest? Or mount the volume with the correct options to map SELinux contexts?
Hi,
I'm afraid I actually don't know that much about SELinux but I read that it relies on using extended attributes in the file system to accomplish its labeling.
Do you still experience this issue when you enable extended attribute support[1] in virtiofsd? The example in the optional parameters snippet enables extended attributes with the xattr='on' element.
Connor
[1] https://libvirt.org/kbase/virtiofs.html#optional-parameters
* Link Dupont (link@sub-pop.net) wrote:
Adding the <binary xattr='on'> element to the <filesystem> device does seem to spawn virtiofsd with the option string "source=/home,xattr". My guest can no longer mount the device though.
It errors with:
[ 170.225553] 9pnet_virtio: no channels available mount: mount(2) failed: No such file or directory
I think what this is doing is causing libvirt to create the device as a virtiofs device instead of a 9p device. The EL7 kernel doesn't have a virtiofs driver, so it can't mount virtiofs devices.
My knowledge is unfortunately limited about the nuances between 9p and virtiofs. So I'm mostly experimenting by trial-and-error here.
They're almost entirely different implementations; if you have a virtiofsd then you're running virtiofs, not 9p, and yes RHEL7 won't like that. (I'm not sure el7 had 9p either??) Dave
On Wed, Jun 2 2021 at 03:55:40 PM -0500, Connor Kuehl <ckuehl@redhat.com> wrote:
On 5/21/21 11:59 AM, Link Dupont wrote:
Adding the virtio-fs mailing list.
I am mounting a filesystem into a domain using the virtiofs driver.
<filesystem accessmode="passthrough" type="mount"> <source dir="/home"/> <target dir="/home"/> <driver type="virtiofs"/> </filesystem>
Both my host (Fedora 34) and guest (CentOS 8.4) are running with SELinux enforcing. From my host, I can see that the SELinux context type is set to user_home_dir_t.
$ ls -ldZ /home/link drwxr-xr-x. 61 link link system_u:object_r:user_home_dir_t:s0 8192 May 21 12:41 /home/link
From within the guest however, the volume is unlabeled_t
$ ls -lZd /home/link drwxr-xr-x. 61 link link system_u:object_r:unlabeled_t:s0 8192 May 21 12:53 / home/link
Is there a way to pass the SELinux context through to the guest? Or mount the volume with the correct options to map SELinux contexts?
Hi,
I'm afraid I actually don't know that much about SELinux but I read that it relies on using extended attributes in the file system to accomplish its labeling.
Do you still experience this issue when you enable extended attribute support[1] in virtiofsd? The example in the optional parameters snippet enables extended attributes with the xattr='on' element.
Connor
[1] https://libvirt.org/kbase/virtiofs.html#optional-parameters
_______________________________________________ Virtio-fs mailing list Virtio-fs@redhat.com https://listman.redhat.com/mailman/listinfo/virtio-fs -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
On Thu, Jun 3 2021 at 08:24:02 PM +0100, Dr. David Alan Gilbert <dgilbert@redhat.com> wrote:
(I'm not sure el7 had 9p either??)
Oh drat. I'm conflating my issues. I have an EL7 guest that is having a similar problem, except the guest sees the files as type nfs_t. I guess the symptoms are the same, but the solutions won't necessarily. Let me try and scale these down to reproducible scenarios.
On Thu, Jun 3 2021 at 08:56:46 PM -0400, Link Dupont <link@sub-pop.net> wrote:
reproducible scenarios
Alright. I reran my tests with a CentOS 8 guest. On CentOS 8 (with a virtiofs filesystem and with xattr on), the type of files in the mounted hierarchy are unlabeled_t. I can work around that by switching SELinux in the guest to permissive or disabled. With a CentOS 7 guest, things get less usable. I digested this to a reproducible scenario. Build a disk image with `virt-builder`, configuring the CentOS Plus kernel to get 9p support. virt-builder centos-7.8 \ --root-password password:centos \ --output centos-7.8.qcow2 \ --install yum-utils \ --run-command 'yum-config-manager --enable centosplus' \ --run-command 'sed -ie "s/DEFAULTKERNEL=kernel/DEFAULTKERNEL=kernel-plus/" /etc/sysconfig/kernel' \ --append-line '/etc/dracut.conf.d/virtio.conf:add_drivers+="virtio_scsi virtio_pci virtio_console"' \ --append-line '/etc/modules-load.d/9pnet_virtio.conf:9pnet_virtio' \ --install kernel-plus \ --append-line '/etc/fstab:home /home 9p trans=virtio,version=9p2000.L 0 0' Install the volume into the `default` pool. sudo install -m644 centos-7.8.qcow2 /var/lib/libvirt/images Next, define a domain using the disk image (using `virt-install` here for "easy mode"). virt-install \ --import \ --os-variant centos7.0 \ --name centos \ --ram 2048 \ --disk path=/var/lib/libvirt/images/centos-7.8.qcow2 \ --memorybacking access.mode=shared \ --filesystem source=/home,target=home,accessmode=passthrough \ --autoconsole none Now with SELinux enforcing, I cannot list the contents of the directories in the mounted hierarchy. [root@localhost ~]# ls -lZ /home/link ls: cannot open directory /home/link: Permission denied
On Thu, Jun 03, 2021 at 10:14:24PM -0400, Link Dupont wrote:
On Thu, Jun 3 2021 at 08:56:46 PM -0400, Link Dupont <link@sub-pop.net> wrote:
reproducible scenarios
Alright. I reran my tests with a CentOS 8 guest. On CentOS 8 (with a virtiofs filesystem and with xattr on), the type of files in the mounted hierarchy are unlabeled_t. I can work around that by switching SELinux in the guest to permissive or disabled.
cc Dan Walsh. I was discussing this with Dan Walsh yesterday in general. In general, if we want to enable SELinux both on host and guest, then both host and guest should have same SELinux policy. Otherwise there will be lot of different kind of conflicts because both host and guest will try to work with same selinux label. I guess that in practice this will be very hard to achieve as people will run different host and guest flavors and these might have different policies. So another option is to rename selinux xattr in virtiofs so that any selinux xattr coming from guest is saved as user.virtiofs.security.selinux xattr on host. That way host and guest can have their separate labels without interfering with each other. David Gilbert already has added support for this. I can't remember the exact syntax but you can figure it out from documentation here in xattr remappig section. https://github.com/qemu/qemu/blob/master/docs/tools/virtiofsd.rst But I have question with selinux xattr remapping. What will happen to initial labels when fs is exported. I mean until and unless some process in guest labels all the exported files, they all with either be unlabeled or pick some generic label for all the files. Another option is, can we use a single label for whole of the virtiofs (using context=<label>) option in guest. That way nothing is saved in files as such. But this means that processes in guest can't have different selinux labels on different virtiofs dir/files. Dan, what do you think? Thanks Vivek
With a CentOS 7 guest, things get less usable. I digested this to a reproducible scenario.
Build a disk image with `virt-builder`, configuring the CentOS Plus kernel to get 9p support.
virt-builder centos-7.8 \ --root-password password:centos \ --output centos-7.8.qcow2 \ --install yum-utils \ --run-command 'yum-config-manager --enable centosplus' \ --run-command 'sed -ie "s/DEFAULTKERNEL=kernel/DEFAULTKERNEL=kernel-plus/" /etc/sysconfig/kernel' \ --append-line '/etc/dracut.conf.d/virtio.conf:add_drivers+="virtio_scsi virtio_pci virtio_console"' \ --append-line '/etc/modules-load.d/9pnet_virtio.conf:9pnet_virtio' \ --install kernel-plus \ --append-line '/etc/fstab:home /home 9p trans=virtio,version=9p2000.L 0 0'
Install the volume into the `default` pool.
sudo install -m644 centos-7.8.qcow2 /var/lib/libvirt/images
Next, define a domain using the disk image (using `virt-install` here for "easy mode").
virt-install \ --import \ --os-variant centos7.0 \ --name centos \ --ram 2048 \ --disk path=/var/lib/libvirt/images/centos-7.8.qcow2 \ --memorybacking access.mode=shared \ --filesystem source=/home,target=home,accessmode=passthrough \ --autoconsole none
Now with SELinux enforcing, I cannot list the contents of the directories in the mounted hierarchy.
[root@localhost ~]# ls -lZ /home/link ls: cannot open directory /home/link: Permission denied
_______________________________________________ Virtio-fs mailing list Virtio-fs@redhat.com https://listman.redhat.com/mailman/listinfo/virtio-fs
On Fri, Jun 04, 2021 at 09:44:39AM -0400, Vivek Goyal wrote:
On Thu, Jun 03, 2021 at 10:14:24PM -0400, Link Dupont wrote:
On Thu, Jun 3 2021 at 08:56:46 PM -0400, Link Dupont <link@sub-pop.net> wrote:
reproducible scenarios
Alright. I reran my tests with a CentOS 8 guest. On CentOS 8 (with a virtiofs filesystem and with xattr on), the type of files in the mounted hierarchy are unlabeled_t. I can work around that by switching SELinux in the guest to permissive or disabled.
cc Dan Walsh. I was discussing this with Dan Walsh yesterday in general.
In general, if we want to enable SELinux both on host and guest, then both host and guest should have same SELinux policy. Otherwise there will be lot of different kind of conflicts because both host and guest will try to work with same selinux label. I guess that in practice this will be very hard to achieve as people will run different host and guest flavors and these might have different policies.
Yeah, I think there's little to no chance of people keeping the same SELinux policy in host/guest, except in very tightly controlled narrow use cases where the host admin exerts direct control over the precise guest config.
So another option is to rename selinux xattr in virtiofs so that any selinux xattr coming from guest is saved as user.virtiofs.security.selinux xattr on host. That way host and guest can have their separate labels without interfering with each other. David Gilbert already has added support for this. I can't remember the exact syntax but you can figure it out from documentation here in xattr remappig section.
For general purpose virt usage, I think remapping in some way is likely to be needed as the default strategy.
https://github.com/qemu/qemu/blob/master/docs/tools/virtiofsd.rst
But I have question with selinux xattr remapping. What will happen to initial labels when fs is exported. I mean until and unless some process in guest labels all the exported files, they all with either be unlabeled or pick some generic label for all the files.
I'd say you need some mechanism to force a re-label inside the guest. Normally a relabel will be done in /.autorelabel file is present, or in certain other scenarios like selinux policy RPM updates. We wouldn't want to force a relabel neccesarily for the entire FS if we're just hotplugging a new virtiofs export though. So perhaps there's scope for supporting usage of a per-mount point relabel trigger. eg Host creates $VIRTIOFS-ROOT/.autorelabel and whenever the guest sees a new virtiofs export arriving, it can look for $VIRTIOFS-MOUNT-POINT/.autorelabel
Another option is, can we use a single label for whole of the virtiofs (using context=<label>) option in guest. That way nothing is saved in files as such. But this means that processes in guest can't have different selinux labels on different virtiofs dir/files.
Forcing a single label for the entire export is passable as a fallback plan. This is what people have done for years with NFS v3 mounts. It has annoying usage limitations though, so if at all possible remapping is a preferrable approach.
Dan, what do you think?
Thanks Vivek
With a CentOS 7 guest, things get less usable. I digested this to a reproducible scenario.
Build a disk image with `virt-builder`, configuring the CentOS Plus kernel to get 9p support.
virt-builder centos-7.8 \ --root-password password:centos \ --output centos-7.8.qcow2 \ --install yum-utils \ --run-command 'yum-config-manager --enable centosplus' \ --run-command 'sed -ie "s/DEFAULTKERNEL=kernel/DEFAULTKERNEL=kernel-plus/" /etc/sysconfig/kernel' \ --append-line '/etc/dracut.conf.d/virtio.conf:add_drivers+="virtio_scsi virtio_pci virtio_console"' \ --append-line '/etc/modules-load.d/9pnet_virtio.conf:9pnet_virtio' \ --install kernel-plus \ --append-line '/etc/fstab:home /home 9p trans=virtio,version=9p2000.L 0 0'
Install the volume into the `default` pool.
sudo install -m644 centos-7.8.qcow2 /var/lib/libvirt/images
Next, define a domain using the disk image (using `virt-install` here for "easy mode").
virt-install \ --import \ --os-variant centos7.0 \ --name centos \ --ram 2048 \ --disk path=/var/lib/libvirt/images/centos-7.8.qcow2 \ --memorybacking access.mode=shared \ --filesystem source=/home,target=home,accessmode=passthrough \ --autoconsole none
Now with SELinux enforcing, I cannot list the contents of the directories in the mounted hierarchy.
[root@localhost ~]# ls -lZ /home/link ls: cannot open directory /home/link: Permission denied
_______________________________________________ Virtio-fs mailing list Virtio-fs@redhat.com https://listman.redhat.com/mailman/listinfo/virtio-fs
Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Fri, Jun 04, 2021 at 02:59:29PM +0100, Daniel P. Berrangé wrote:
On Fri, Jun 04, 2021 at 09:44:39AM -0400, Vivek Goyal wrote:
On Thu, Jun 03, 2021 at 10:14:24PM -0400, Link Dupont wrote:
On Thu, Jun 3 2021 at 08:56:46 PM -0400, Link Dupont <link@sub-pop.net> wrote:
reproducible scenarios
Alright. I reran my tests with a CentOS 8 guest. On CentOS 8 (with a virtiofs filesystem and with xattr on), the type of files in the mounted hierarchy are unlabeled_t. I can work around that by switching SELinux in the guest to permissive or disabled.
cc Dan Walsh. I was discussing this with Dan Walsh yesterday in general.
In general, if we want to enable SELinux both on host and guest, then both host and guest should have same SELinux policy. Otherwise there will be lot of different kind of conflicts because both host and guest will try to work with same selinux label. I guess that in practice this will be very hard to achieve as people will run different host and guest flavors and these might have different policies.
Yeah, I think there's little to no chance of people keeping the same SELinux policy in host/guest, except in very tightly controlled narrow use cases where the host admin exerts direct control over the precise guest config.
So another option is to rename selinux xattr in virtiofs so that any selinux xattr coming from guest is saved as user.virtiofs.security.selinux xattr on host. That way host and guest can have their separate labels without interfering with each other. David Gilbert already has added support for this. I can't remember the exact syntax but you can figure it out from documentation here in xattr remappig section.
For general purpose virt usage, I think remapping in some way is likely to be needed as the default strategy.
https://github.com/qemu/qemu/blob/master/docs/tools/virtiofsd.rst
But I have question with selinux xattr remapping. What will happen to initial labels when fs is exported. I mean until and unless some process in guest labels all the exported files, they all with either be unlabeled or pick some generic label for all the files.
I'd say you need some mechanism to force a re-label inside the guest. Normally a relabel will be done in /.autorelabel file is present, or in certain other scenarios like selinux policy RPM updates.
We wouldn't want to force a relabel neccesarily for the entire FS if we're just hotplugging a new virtiofs export though. So perhaps there's scope for supporting usage of a per-mount point relabel trigger. eg Host creates $VIRTIOFS-ROOT/.autorelabel and whenever the guest sees a new virtiofs export arriving, it can look for $VIRTIOFS-MOUNT-POINT/.autorelabel
Per mount point auto relabel seems interesting. Will it relabel everytime virtiofs export shows up. Or it will intelligence to figure out exported fs is already labeled (say from previous boot) and no need to relabel again.
Another option is, can we use a single label for whole of the virtiofs (using context=<label>) option in guest. That way nothing is saved in files as such. But this means that processes in guest can't have different selinux labels on different virtiofs dir/files.
Forcing a single label for the entire export is passable as a fallback plan. This is what people have done for years with NFS v3 mounts.
Aha, good to know.
It has annoying usage limitations though, so if at all possible remapping is a preferrable approach.
Agreed that this appraoch will have more limitations as comapred to selinux xattr remapping + autorelabel. At the same time, this is the simplest approach to begin with. And one does not pay the penalty of relabeling files. Maybe this can be the default and we document that how to enable proper selinux support by using xattr remapping + autorelabel. Thanks Vivek
Dan, what do you think?
Thanks Vivek
With a CentOS 7 guest, things get less usable. I digested this to a reproducible scenario.
Build a disk image with `virt-builder`, configuring the CentOS Plus kernel to get 9p support.
virt-builder centos-7.8 \ --root-password password:centos \ --output centos-7.8.qcow2 \ --install yum-utils \ --run-command 'yum-config-manager --enable centosplus' \ --run-command 'sed -ie "s/DEFAULTKERNEL=kernel/DEFAULTKERNEL=kernel-plus/" /etc/sysconfig/kernel' \ --append-line '/etc/dracut.conf.d/virtio.conf:add_drivers+="virtio_scsi virtio_pci virtio_console"' \ --append-line '/etc/modules-load.d/9pnet_virtio.conf:9pnet_virtio' \ --install kernel-plus \ --append-line '/etc/fstab:home /home 9p trans=virtio,version=9p2000.L 0 0'
Install the volume into the `default` pool.
sudo install -m644 centos-7.8.qcow2 /var/lib/libvirt/images
Next, define a domain using the disk image (using `virt-install` here for "easy mode").
virt-install \ --import \ --os-variant centos7.0 \ --name centos \ --ram 2048 \ --disk path=/var/lib/libvirt/images/centos-7.8.qcow2 \ --memorybacking access.mode=shared \ --filesystem source=/home,target=home,accessmode=passthrough \ --autoconsole none
Now with SELinux enforcing, I cannot list the contents of the directories in the mounted hierarchy.
[root@localhost ~]# ls -lZ /home/link ls: cannot open directory /home/link: Permission denied
_______________________________________________ Virtio-fs mailing list Virtio-fs@redhat.com https://listman.redhat.com/mailman/listinfo/virtio-fs
Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Fri, Jun 04, 2021 at 10:43:13AM -0400, Vivek Goyal wrote:
On Fri, Jun 04, 2021 at 02:59:29PM +0100, Daniel P. Berrangé wrote:
On Fri, Jun 04, 2021 at 09:44:39AM -0400, Vivek Goyal wrote:
On Thu, Jun 03, 2021 at 10:14:24PM -0400, Link Dupont wrote:
On Thu, Jun 3 2021 at 08:56:46 PM -0400, Link Dupont <link@sub-pop.net> wrote:
reproducible scenarios
Alright. I reran my tests with a CentOS 8 guest. On CentOS 8 (with a virtiofs filesystem and with xattr on), the type of files in the mounted hierarchy are unlabeled_t. I can work around that by switching SELinux in the guest to permissive or disabled.
cc Dan Walsh. I was discussing this with Dan Walsh yesterday in general.
In general, if we want to enable SELinux both on host and guest, then both host and guest should have same SELinux policy. Otherwise there will be lot of different kind of conflicts because both host and guest will try to work with same selinux label. I guess that in practice this will be very hard to achieve as people will run different host and guest flavors and these might have different policies.
Yeah, I think there's little to no chance of people keeping the same SELinux policy in host/guest, except in very tightly controlled narrow use cases where the host admin exerts direct control over the precise guest config.
So another option is to rename selinux xattr in virtiofs so that any selinux xattr coming from guest is saved as user.virtiofs.security.selinux xattr on host. That way host and guest can have their separate labels without interfering with each other. David Gilbert already has added support for this. I can't remember the exact syntax but you can figure it out from documentation here in xattr remappig section.
For general purpose virt usage, I think remapping in some way is likely to be needed as the default strategy.
https://github.com/qemu/qemu/blob/master/docs/tools/virtiofsd.rst
But I have question with selinux xattr remapping. What will happen to initial labels when fs is exported. I mean until and unless some process in guest labels all the exported files, they all with either be unlabeled or pick some generic label for all the files.
I'd say you need some mechanism to force a re-label inside the guest. Normally a relabel will be done in /.autorelabel file is present, or in certain other scenarios like selinux policy RPM updates.
We wouldn't want to force a relabel neccesarily for the entire FS if we're just hotplugging a new virtiofs export though. So perhaps there's scope for supporting usage of a per-mount point relabel trigger. eg Host creates $VIRTIOFS-ROOT/.autorelabel and whenever the guest sees a new virtiofs export arriving, it can look for $VIRTIOFS-MOUNT-POINT/.autorelabel
Per mount point auto relabel seems interesting. Will it relabel everytime virtiofs export shows up. Or it will intelligence to figure out exported fs is already labeled (say from previous boot) and no need to relabel again.
Normal behaviour with the existing global /.autorelabel is that selinux deletes the file once the relabel is complete, so it is a one-time thing. Essentially anytime you need to force[1] a one-time only relabel you just do "touch /.autorelabel && reboot" DanW does however recommend that you configure your system such that the labels are always correct and thus don't need the global relabel. https://danwalsh.livejournal.com/38157.html None the less I think the virtiofs situation is slightly different and justifies a way to force relabel, provided it can be scoped to just that one filesystem mount. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 6/4/21 09:59, Daniel P. Berrangé wrote:
On Thu, Jun 03, 2021 at 10:14:24PM -0400, Link Dupont wrote:
reproducible scenarios Alright. I reran my tests with a CentOS 8 guest. On CentOS 8 (with a virtiofs filesystem and with xattr on), the type of files in the mounted hierarchy are unlabeled_t. I can work around that by switching SELinux in
On Thu, Jun 3 2021 at 08:56:46 PM -0400, Link Dupont <link@sub-pop.net> wrote: the guest to permissive or disabled. cc Dan Walsh. I was discussing this with Dan Walsh yesterday in general.
In general, if we want to enable SELinux both on host and guest, then both host and guest should have same SELinux policy. Otherwise there will be lot of different kind of conflicts because both host and guest will try to work with same selinux label. I guess that in practice this will be very hard to achieve as people will run different host and guest flavors and these might have different policies. Yeah, I think there's little to no chance of people keeping the same SELinux policy in host/guest, except in very tightly controlled narrow use cases where the host admin exerts direct control over
On Fri, Jun 04, 2021 at 09:44:39AM -0400, Vivek Goyal wrote: the precise guest config.
So another option is to rename selinux xattr in virtiofs so that any selinux xattr coming from guest is saved as user.virtiofs.security.selinux xattr on host. That way host and guest can have their separate labels without interfering with each other. David Gilbert already has added support for this. I can't remember the exact syntax but you can figure it out from documentation here in xattr remappig section. For general purpose virt usage, I think remapping in some way is likely to be needed as the default strategy.
https://github.com/qemu/qemu/blob/master/docs/tools/virtiofsd.rst
But I have question with selinux xattr remapping. What will happen to initial labels when fs is exported. I mean until and unless some process in guest labels all the exported files, they all with either be unlabeled or pick some generic label for all the files. I'd say you need some mechanism to force a re-label inside the guest. Normally a relabel will be done in /.autorelabel file is present, or in certain other scenarios like selinux policy RPM updates.
We wouldn't want to force a relabel neccesarily for the entire FS if we're just hotplugging a new virtiofs export though. So perhaps there's scope for supporting usage of a per-mount point relabel trigger. eg Host creates $VIRTIOFS-ROOT/.autorelabel and whenever the guest sees a new virtiofs export arriving, it can look for $VIRTIOFS-MOUNT-POINT/.autorelabel
Another option is, can we use a single label for whole of the virtiofs (using context=<label>) option in guest. That way nothing is saved in files as such. But this means that processes in guest can't have different selinux labels on different virtiofs dir/files. Forcing a single label for the entire export is passable as a fallback plan. This is what people have done for years with NFS v3 mounts. It has annoying usage limitations though, so if at all possible remapping is a preferrable approach.
Dan, what do you think?
Thanks Vivek
With a CentOS 7 guest, things get less usable. I digested this to a reproducible scenario.
Build a disk image with `virt-builder`, configuring the CentOS Plus kernel to get 9p support.
virt-builder centos-7.8 \ --root-password password:centos \ --output centos-7.8.qcow2 \ --install yum-utils \ --run-command 'yum-config-manager --enable centosplus' \ --run-command 'sed -ie "s/DEFAULTKERNEL=kernel/DEFAULTKERNEL=kernel-plus/" /etc/sysconfig/kernel' \ --append-line '/etc/dracut.conf.d/virtio.conf:add_drivers+="virtio_scsi virtio_pci virtio_console"' \ --append-line '/etc/modules-load.d/9pnet_virtio.conf:9pnet_virtio' \ --install kernel-plus \ --append-line '/etc/fstab:home /home 9p trans=virtio,version=9p2000.L 0 0'
Install the volume into the `default` pool.
sudo install -m644 centos-7.8.qcow2 /var/lib/libvirt/images
Next, define a domain using the disk image (using `virt-install` here for "easy mode").
virt-install \ --import \ --os-variant centos7.0 \ --name centos \ --ram 2048 \ --disk path=/var/lib/libvirt/images/centos-7.8.qcow2 \ --memorybacking access.mode=shared \ --filesystem source=/home,target=home,accessmode=passthrough \ --autoconsole none
Now with SELinux enforcing, I cannot list the contents of the directories in the mounted hierarchy.
[root@localhost ~]# ls -lZ /home/link ls: cannot open directory /home/link: Permission denied
_______________________________________________ Virtio-fs mailing list Virtio-fs@redhat.com https://listman.redhat.com/mailman/listinfo/virtio-fs
Regards, Daniel
The separate XAttr support would also allow virtiofsd to work with labels inside of the container when the virtiofsd is confined on the host. I would guess/hope virtiofsd on the host will be confined and only able to write certain labels like container_file_t or svirt_image_t. If you want to allow virt file systems within the "VM" to be able to set labels, these labels will have to be something other then SELinux labels on the host, or SELinux will prevent them from being set.
On Mon, Jun 7 2021 at 09:01:08 AM -0400, Daniel Walsh <dwalsh@redhat.com> wrote:
On 6/4/21 09:59, Daniel P. Berrangé wrote:
On Thu, Jun 03, 2021 at 10:14:24PM -0400, Link Dupont wrote:
reproducible scenarios Alright. I reran my tests with a CentOS 8 guest. On CentOS 8 (with a virtiofs filesystem and with xattr on), the type of files in the mounted hierarchy are unlabeled_t. I can work around that by switching SELinux in
On Thu, Jun 3 2021 at 08:56:46 PM -0400, Link Dupont <link@sub-pop.net> wrote: the guest to permissive or disabled. cc Dan Walsh. I was discussing this with Dan Walsh yesterday in general.
In general, if we want to enable SELinux both on host and guest, then both host and guest should have same SELinux policy. Otherwise there will be lot of different kind of conflicts because both host and guest will try to work with same selinux label. I guess that in practice this will be very hard to achieve as people will run different host and guest flavors and these might have different policies. Yeah, I think there's little to no chance of people keeping the same SELinux policy in host/guest, except in very tightly controlled narrow use cases where the host admin exerts direct control over
On Fri, Jun 04, 2021 at 09:44:39AM -0400, Vivek Goyal wrote: the precise guest config.
So another option is to rename selinux xattr in virtiofs so that any selinux xattr coming from guest is saved as user.virtiofs.security.selinux xattr on host. That way host and guest can have their separate labels without interfering with each other. David Gilbert already has added support for this. I can't remember the exact syntax but you can figure it out from documentation here in xattr remappig section. For general purpose virt usage, I think remapping in some way is likely to be needed as the default strategy.
https://github.com/qemu/qemu/blob/master/docs/tools/virtiofsd.rst
But I have question with selinux xattr remapping. What will happen to initial labels when fs is exported. I mean until and unless some process in guest labels all the exported files, they all with either be unlabeled or pick some generic label for all the files. I'd say you need some mechanism to force a re-label inside the guest. Normally a relabel will be done in /.autorelabel file is present, or in certain other scenarios like selinux policy RPM updates.
We wouldn't want to force a relabel neccesarily for the entire FS if we're just hotplugging a new virtiofs export though. So perhaps there's scope for supporting usage of a per-mount point relabel trigger. eg Host creates $VIRTIOFS-ROOT/.autorelabel and whenever the guest sees a new virtiofs export arriving, it can look for $VIRTIOFS-MOUNT-POINT/.autorelabel
Another option is, can we use a single label for whole of the virtiofs (using context=<label>) option in guest. That way nothing is saved in files as such. But this means that processes in guest can't have different selinux labels on different virtiofs dir/files. Forcing a single label for the entire export is passable as a fallback plan. This is what people have done for years with NFS v3 mounts. It has annoying usage limitations though, so if at all possible remapping is a preferrable approach.
Dan, what do you think?
Thanks Vivek
With a CentOS 7 guest, things get less usable. I digested this to a reproducible scenario.
Build a disk image with `virt-builder`, configuring the CentOS Plus kernel to get 9p support.
virt-builder centos-7.8 \ --root-password password:centos \ --output centos-7.8.qcow2 \ --install yum-utils \ --run-command 'yum-config-manager --enable centosplus' \ --run-command 'sed -ie "s/DEFAULTKERNEL=kernel/DEFAULTKERNEL=kernel-plus/" /etc/sysconfig/kernel' \ --append-line '/etc/dracut.conf.d/virtio.conf:add_drivers+="virtio_scsi virtio_pci virtio_console"' \ --append-line '/etc/modules-load.d/9pnet_virtio.conf:9pnet_virtio' \ --install kernel-plus \ --append-line '/etc/fstab:home /home 9p trans=virtio,version=9p2000.L 0 0'
Install the volume into the `default` pool.
sudo install -m644 centos-7.8.qcow2 /var/lib/libvirt/images
Next, define a domain using the disk image (using `virt-install` here for "easy mode").
virt-install \ --import \ --os-variant centos7.0 \ --name centos \ --ram 2048 \ --disk path=/var/lib/libvirt/images/centos-7.8.qcow2 \ --memorybacking access.mode=shared \ --filesystem source=/home,target=home,accessmode=passthrough \ --autoconsole none
Now with SELinux enforcing, I cannot list the contents of the directories in the mounted hierarchy.
[root@localhost ~]# ls -lZ /home/link ls: cannot open directory /home/link: Permission denied
_______________________________________________ Virtio-fs mailing list Virtio-fs@redhat.com https://listman.redhat.com/mailman/listinfo/virtio-fs
Regards, Daniel
The separate XAttr support would also allow virtiofsd to work with labels inside of the container when the virtiofsd is confined on the host.
I would guess/hope virtiofsd on the host will be confined and only able to write certain labels like container_file_t or svirt_image_t. If you want to allow virt file systems within the "VM" to be able to set labels, these labels will have to be something other then SELinux labels on the host, or SELinux will prevent them from being set.
I don't think the guest needs to be able to set labels on the virtiofs filesystems; as long as it can read labels that grant read and write access as the guest user, I'd call it a success. The way I use this feature, I'm treating my host filesystem as the "source", and really only mounting it into the guest as a zero-delay synchronization. (I previously did this using sync solutions like rsync or lsyncd, but those have a slight delay before files sync over.) If the labels as seen by the guest allow for reading and writing files from the guest to the host filesystem, I'd be happy. Doing any relabeling from the host is totally fine.
On Fri, Jun 04, 2021 at 02:59:29PM +0100, Daniel P. Berrangé wrote:
On Fri, Jun 04, 2021 at 09:44:39AM -0400, Vivek Goyal wrote:
On Thu, Jun 03, 2021 at 10:14:24PM -0400, Link Dupont wrote:
On Thu, Jun 3 2021 at 08:56:46 PM -0400, Link Dupont <link@sub-pop.net> wrote:
reproducible scenarios
Alright. I reran my tests with a CentOS 8 guest. On CentOS 8 (with a virtiofs filesystem and with xattr on), the type of files in the mounted hierarchy are unlabeled_t. I can work around that by switching SELinux in the guest to permissive or disabled.
cc Dan Walsh. I was discussing this with Dan Walsh yesterday in general.
In general, if we want to enable SELinux both on host and guest, then both host and guest should have same SELinux policy. Otherwise there will be lot of different kind of conflicts because both host and guest will try to work with same selinux label. I guess that in practice this will be very hard to achieve as people will run different host and guest flavors and these might have different policies.
Yeah, I think there's little to no chance of people keeping the same SELinux policy in host/guest, except in very tightly controlled narrow use cases where the host admin exerts direct control over the precise guest config.
So another option is to rename selinux xattr in virtiofs so that any selinux xattr coming from guest is saved as user.virtiofs.security.selinux xattr on host. That way host and guest can have their separate labels without interfering with each other. David Gilbert already has added support for this. I can't remember the exact syntax but you can figure it out from documentation here in xattr remappig section.
For general purpose virt usage, I think remapping in some way is likely to be needed as the default strategy.
https://github.com/qemu/qemu/blob/master/docs/tools/virtiofsd.rst
But I have question with selinux xattr remapping. What will happen to initial labels when fs is exported. I mean until and unless some process in guest labels all the exported files, they all with either be unlabeled or pick some generic label for all the files.
I'd say you need some mechanism to force a re-label inside the guest. Normally a relabel will be done in /.autorelabel file is present, or in certain other scenarios like selinux policy RPM updates.
We wouldn't want to force a relabel neccesarily for the entire FS if we're just hotplugging a new virtiofs export though. So perhaps there's scope for supporting usage of a per-mount point relabel trigger. eg Host creates $VIRTIOFS-ROOT/.autorelabel and whenever the guest sees a new virtiofs export arriving, it can look for $VIRTIOFS-MOUNT-POINT/.autorelabel
Another option is, can we use a single label for whole of the virtiofs (using context=<label>) option in guest. That way nothing is saved in files as such. But this means that processes in guest can't have different selinux labels on different virtiofs dir/files.
Forcing a single label for the entire export is passable as a fallback plan. This is what people have done for years with NFS v3 mounts. It has annoying usage limitations though, so if at all possible remapping is a preferrable approach.
Another thing we should bear in mind is that it is entirely plausible for users to want SELinux in their guest, wihle the host is not using SELinux at all. eg running a RHEL guest, on a cloud running Ubuntu hosts. In such a case SELinux essentially won't exist from the POV of the host OS / admin using. Anything related to labelling will have to be exclusively done by the guest. If we provide a general purpose way to remap XAttrs, then this ought to be viable by having a generic remap rule for anything the guest does. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
participants (6)
-
Connor Kuehl -
Daniel P. Berrangé -
Daniel Walsh -
Dr. David Alan Gilbert -
Link Dupont -
Vivek Goyal