2:01 p.m.
We run unprivileged lxc containers (libvirt based) with next config:
...
<idmap>
<uid start='0' target='65535' count='65535'/>
<gid start='0' target='65535' count='65535'/>
</idmap>
...
<devices>
<emulator>/usr/libexec/libvirt_lxc</emulator>
<filesystem type='mount' accessmode='passthrough'>
<source dir='/var/lib/libvirt/lxc/test1'/>
<target dir='/'/>
</filesystem>
...
</devices>
Before start we need mount container's LVM:
mount /dev/data/test1 /var/lib/libvirt/lxc/test1
And shift uid/gid:
./uidmapshift -b /var/lib/libvirt/lxc/test1 0 65535 65535
(uidmapshift.c is simple utility, found in LXD community)
As result, our FS permissions look from hardware node as:
[root(a)ops-node01.infra]# ls -la /var/lib/libvirt/lxc/test1/
total 8
dr-xr-xr-x 19 65535 65535 275 Apr 9 18:55 .
drwxrwxrwx 3 root root 26 Apr 9 19:32 ..
-rw-r--r-- 1 65535 65535 0 Mar 30 18:20 .autorelabel
lrwxrwxrwx 1 65535 65535 7 Mar 2 2017 bin -> usr/bin
dr-xr-xr-x 2 65535 65535 6 Nov 5 2016 boot
drwxr-xr-x 4 65535 65535 206 Mar 30 18:03 dev
drwxr-xr-x 59 65535 65535 4096 Apr 9 18:57 etc
drwxr-xr-x 2 65535 65535 6 Nov 5 2016 home
lrwxrwxrwx 1 65535 65535 7 Mar 2 2017 lib -> usr/lib
lrwxrwxrwx 1 65535 65535 9 Mar 2 2017 lib64 -> usr/lib64
drwxr-xr-x 2 65535 65535 6 Nov 5 2016 media
drwxr-xr-x 2 65535 65535 6 Nov 5 2016 mnt
drwxr-xr-x 2 65535 65535 6 Apr 9 18:55 .oldroot
drwxr-xr-x 3 65535 65535 24 Mar 2 2017 opt
dr-xr-xr-x 2 65535 65535 6 Nov 5 2016 proc
dr-xr-x--- 4 65535 65535 172 Apr 9 18:57 root
drwxr-xr-x 13 65535 65535 178 Mar 30 18:03 run
lrwxrwxrwx 1 65535 65535 8 Mar 2 2017 sbin -> usr/sbin
drwxr-xr-x 2 65535 65535 21 Mar 30 18:03 selinux
drwxr-xr-x 2 65535 65535 6 Nov 5 2016 srv
dr-xr-xr-x 2 65535 65535 6 Nov 5 2016 sys
drwxrwxrwt 7 65535 65535 93 Apr 6 21:08 tmp
drwxr-xr-x 13 65535 65535 155 Mar 2 2017 usr
drwxr-xr-x 18 65535 65535 254 Mar 30 18:20 var
Container running, but some FS objects inside get owner 65534.65534:
[root@test1 ~]# find / -uid 65534 -ls | wc -l
40084
It's mostly /sys and /proc objects:
[root@test1 ~]# ls -la /
total 8
dr-xr-xr-x 19 root root 275 Apr 9 18:55 .
dr-xr-xr-x 19 root root 275 Apr 9 18:55 ..
-rw-r--r-- 1 root root 0 Mar 30 18:20 .autorelabel
drwxr-xr-x 2 root root 6 Apr 9 18:55 .oldroot
lrwxrwxrwx 1 root root 7 Mar 2 2017 bin -> usr/bin
dr-xr-xr-x 2 root root 6 Nov 5 2016 boot
drwxr-xr-x 6 root root 440 Apr 9 20:40 dev
drwxr-xr-x 59 root root 4096 Apr 9 18:57 etc
drwxr-xr-x 2 root root 6 Nov 5 2016 home
lrwxrwxrwx 1 root root 7 Mar 2 2017 lib -> usr/lib
lrwxrwxrwx 1 root root 9 Mar 2 2017 lib64 -> usr/lib64
drwxr-xr-x 2 root root 6 Nov 5 2016 media
drwxr-xr-x 2 root root 6 Nov 5 2016 mnt
drwxr-xr-x 3 root root 24 Mar 2 2017 opt
dr-xr-xr-x 348 65534 65534 0 Apr 9 20:40 proc
dr-xr-x--- 4 root root 172 Apr 9 18:57 root
drwxrwxrwt 15 root root 360 Apr 9 20:40 run
lrwxrwxrwx 1 root root 8 Mar 2 2017 sbin -> usr/sbin
drwxr-xr-x 2 root root 21 Mar 30 18:03 selinux
drwxr-xr-x 2 root root 6 Nov 5 2016 srv
dr-xr-xr-x 13 65534 65534 0 Apr 9 20:40 sys
drwxrwxrwt 7 root root 93 Apr 6 21:08 tmp
drwxr-xr-x 13 root root 155 Mar 2 2017 usr
drwxr-xr-x 18 root root 254 Mar 30 18:20 var
[root@test1 ~]# find /sys/ -uid 65534 -ls | wc -l
36871
[root@t1 ~]# find /proc -uid 65534 -ls | wc -l
3200
[root@test1 ~]# find /dev -uid 65534 -ls
2121875 0 drwxrwxrwt 2 65534 65534 40 Apr 10 13:57 /dev/mqueue
It's feature ? Look like libvirt make all this objects from real root
with owner root.root
You can't call virLXCControllerSetupUserns() before make this objects, right ?
Because you need root permissions for some operations.
After you set uid range for process 65535-131070, we can't show object
with uid.gid 0 and then kernel use values from
/proc/sys/kernel/overflowuid and /proc/sys/kernel/overflowgid.
It's can probably affect some process in container ?
As solution is shift uid/gid _after_ make all objects and _before_
enter userNS ?
As workaround we can try set 0 for overflowuid and overflowgid
p.s.
We use type='mount' instead type='block' because in unprivileged
container mount() call failed, more detailes in
https://bugzilla.redhat.com/show_bug.cgi?id=1328946
b.r.
Maxim Kozin