[libvirt-users] Problems with user namespaces
Hello! I'm testing user namespaces and I have quite some problem getting them to work. First of all, I have user namespaces support enabled in kernel: offlinehacker:~/ $ uname -r 3.10.10 offlinehacker:~/ $ ls /proc/self/ns/ ipc@ mnt@ net@ pid@ user@ uts@ I created simple ubuntu rootfs and when I start container without idmap, so without user namespace mappings, it works just fine: Libivrt config: <domain type='lxc'> <name>helloworld</name> <memory>102400</memory> <os> <type>exe</type> <init>/bin/dash</init> </os> <!--<idmap> <uid start='0' target='499' count='10'/> <gid start='0' target='100' count='10'/> </idmap>--> <devices> <console type='pty'/> <filesystem type='mount'> <source dir='/home/offlinehacker/rootfs'/> <target dir='/'/> </filesystem> </devices> </domain> This is how my rootfs looks like: offlinehacker:~/ $ ls -la rootfs drwxr-xr-x 23 offlinehacker users 4096 sep 5 19:06 ./ drwxr-xr-x 59 offlinehacker users 4096 sep 5 19:06 ../ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 bin/ drwxr-xr-x 3 offlinehacker users 4096 avg 27 14:11 boot/ drwxr-xr-x 4 offlinehacker users 4096 avg 27 14:11 dev/ drwxr-xr-x 86 offlinehacker users 4096 sep 5 18:20 etc/ drwxr-xr-x 3 offlinehacker users 4096 avg 27 14:11 home/ lrwxrwxrwx 1 offlinehacker users 33 avg 27 14:10 initrd.img -> /boot/initrd.img-3.2.0-52-virtual drwxr-xr-x 18 offlinehacker users 4096 avg 27 14:10 lib/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:10 lib64/ drwx------ 2 offlinehacker users 4096 avg 27 14:11 lost+found/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 media/ drwxr-xr-x 2 offlinehacker users 4096 apr 19 2012 mnt/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 opt/ -rw-r--r-- 1 offlinehacker vboxusers 231671365 avg 27 14:12 precise-server-cloudimg-amd64-root.tar.gz drwxr-xr-x 2 offlinehacker users 4096 apr 19 2012 proc/ drwx------ 2 offlinehacker users 4096 sep 3 23:47 root/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 run/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 sbin/ drwxr-xr-x 2 offlinehacker users 4096 mar 5 2012 selinux/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 srv/ drwxr-xr-x 2 offlinehacker users 4096 apr 14 2012 sys/ drwxrwxrwt 2 offlinehacker users 4096 sep 4 01:39 tmp/ drwxr-xr-x 10 offlinehacker users 4096 avg 27 14:09 usr/ drwxr-xr-x 12 offlinehacker users 4096 sep 5 18:10 var/ lrwxrwxrwx 1 offlinehacker users 29 avg 27 14:10 vmlinuz -> boot/vmlinuz-3.2.0-52-virtual And this is who I am: offlinehacker:~/ $ id uid=499(offlinehacker) gid=67(libvirtd) groups=100(users),1(wheel),57(networkmanager),67(libvirtd) When I create the container with idmap uncommented I get the following error: offlinehacker:~/ $ virsh -c lxc:/// create helloworld.xml error: Failed to create domain from helloworld.xml error: internal error: guest failed to start: 2013-09-05 19:08:57.781+0000: 19036: debug And these are the logs: sep 05 19:08:52 laptop libvirtd[1542]: server=0x7fc8a60ddd60 client=0x7fc8a60e68d0 msg=0x7fc8a60e9380 rerr=0x7fc89a32cd40 args=0x7fc88800b4a0 ret=0x7fc88800a1c0 sep 05 19:08:52 laptop libvirtd[1542]: priv=0x7fc8a60e91f0 conn=(nil) sep 05 19:08:52 laptop libvirtd[1542]: name=lxc:/// sep 05 19:08:57 laptop libvirtd[1542]: Cannot recv data: Connection reset by peer sep 05 19:08:58 laptop libvirtd[1542]: internal error: guest failed to start: 2013-09-05 19:08:57.781+0000: 19036: debug Looks like .oldroot, dev, proc and sys gets created with mapped permissions(499:100), but container fails to start. Any help or direction how to get this working would be appriciated, Thanks, Jaka Hudoklin!
On 09/06/2013 03:15 AM, Jaka Hudoklin wrote:
Hello!
I'm testing user namespaces and I have quite some problem getting them to work.
First of all, I have user namespaces support enabled in kernel:
offlinehacker:~/ $ uname -r 3.10.10 offlinehacker:~/ $ ls /proc/self/ns/ ipc@ mnt@ net@ pid@ user@ uts@
I created simple ubuntu rootfs and when I start container without idmap, so without user namespace mappings, it works just fine:
Libivrt config:
<domain type='lxc'> <name>helloworld</name> <memory>102400</memory> <os> <type>exe</type> <init>/bin/dash</init> </os> <!--<idmap> <uid start='0' target='499' count='10'/> <gid start='0' target='100' count='10'/> </idmap>--> <devices> <console type='pty'/> <filesystem type='mount'> <source dir='/home/offlinehacker/rootfs'/> <target dir='/'/> </filesystem> </devices> </domain>
Your configuration looks good.
This is how my rootfs looks like:
offlinehacker:~/ $ ls -la rootfs drwxr-xr-x 23 offlinehacker users 4096 sep 5 19:06 ./ drwxr-xr-x 59 offlinehacker users 4096 sep 5 19:06 ../ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 bin/ drwxr-xr-x 3 offlinehacker users 4096 avg 27 14:11 boot/ drwxr-xr-x 4 offlinehacker users 4096 avg 27 14:11 dev/ drwxr-xr-x 86 offlinehacker users 4096 sep 5 18:20 etc/ drwxr-xr-x 3 offlinehacker users 4096 avg 27 14:11 home/ lrwxrwxrwx 1 offlinehacker users 33 avg 27 14:10 initrd.img -> /boot/initrd.img-3.2.0-52-virtual drwxr-xr-x 18 offlinehacker users 4096 avg 27 14:10 lib/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:10 lib64/ drwx------ 2 offlinehacker users 4096 avg 27 14:11 lost+found/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 media/ drwxr-xr-x 2 offlinehacker users 4096 apr 19 2012 mnt/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 opt/ -rw-r--r-- 1 offlinehacker vboxusers 231671365 avg 27 14:12 precise-server-cloudimg-amd64-root.tar.gz drwxr-xr-x 2 offlinehacker users 4096 apr 19 2012 proc/ drwx------ 2 offlinehacker users 4096 sep 3 23:47 root/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 run/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 sbin/ drwxr-xr-x 2 offlinehacker users 4096 mar 5 2012 selinux/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 srv/ drwxr-xr-x 2 offlinehacker users 4096 apr 14 2012 sys/ drwxrwxrwt 2 offlinehacker users 4096 sep 4 01:39 tmp/ drwxr-xr-x 10 offlinehacker users 4096 avg 27 14:09 usr/ drwxr-xr-x 12 offlinehacker users 4096 sep 5 18:10 var/ lrwxrwxrwx 1 offlinehacker users 29 avg 27 14:10 vmlinuz -> boot/vmlinuz-3.2.0-52-virtual
And this is who I am: offlinehacker:~/ $ id uid=499(offlinehacker) gid=67(libvirtd) groups=100(users),1(wheel),57(networkmanager),67(libvirtd)
Can this user exec //home/offlinehacker/rootfs/bin/dash successfully ?
When I create the container with idmap uncommented I get the following error:
offlinehacker:~/ $ virsh -c lxc:/// create helloworld.xml error: Failed to create domain from helloworld.xml error: internal error: guest failed to start: 2013-09-05 19:08:57.781+0000: 19036: debug
And these are the logs: sep 05 19:08:52 laptop libvirtd[1542]: server=0x7fc8a60ddd60 client=0x7fc8a60e68d0 msg=0x7fc8a60e9380 rerr=0x7fc89a32cd40 args=0x7fc88800b4a0 ret=0x7fc88800a1c0 sep 05 19:08:52 laptop libvirtd[1542]: priv=0x7fc8a60e91f0 conn=(nil) sep 05 19:08:52 laptop libvirtd[1542]: name=lxc:/// sep 05 19:08:57 laptop libvirtd[1542]: Cannot recv data: Connection reset by peer sep 05 19:08:58 laptop libvirtd[1542]: internal error: guest failed to start: 2013-09-05 19:08:57.781+0000: 19036: debug
Looks like .oldroot, dev, proc and sys gets created with mapped permissions(499:100), but container fails to start.
Please enable debug mode, I need more information. http://libvirt.org/logging.html Thansk
Hello! Okay i tried again with only staticly linked busybox: offlinehacker:~/ $ /home/offlinehacker/busybox/busybox BusyBox v1.17.1 (Debian 1:1.17.1-8) multi-call binary. Copyright (C) 1998-2009 Erik Andersen, Rob Landley, Denys Vlasenko and others. Licensed under GPLv2. See source distribution for full notice. .... Again my id: uid=499(offlinehacker) gid=100(users) groups=100(users),1(wheel),57(networkmanager) My rootfs tree(/home/offlinehacker/busybox): busybox ├── [offlineh users ] busybox └── [offlineh users ] busybox-static_1.17.1-8_amd64.deb It works just fine as root and these folders gets created: busybox ├── [offlineh users ] busybox ├── [offlineh users ] busybox-static_1.17.1-8_amd64.deb ├── [root root ] dev ├── [root root ] .oldroot ├── [root root ] proc └── [root root ] sys When i start it with idmap with clean rootfs(dev proc sys and .oldroot deleted) i get this error, and it is a little bit different now: error: Failed to create domain from helloworld.xml error: internal error: guest failed to start: 2013-09-06 11:24:57.088+0000: 5794: debug : virFileC And log is pretty similar: sep 06 11:24:56 laptop libvirtd[1542]: EVENT_POLL_UPDATE_HANDLE: watch=241 events=1 sep 06 11:24:57 laptop libvirtd[1542]: Skip interrupt, 1 140499747788544 sep 06 11:24:57 laptop libvirtd[1542]: OBJECT_REF: obj=0x7fc878000c90 sep 06 11:24:57 laptop libvirtd[1542]: OBJECT_REF: obj=0x7fc878000c90 sep 06 11:24:57 laptop libvirtd[1542]: server=0x7fc8a60ddd60 client=0x7fc8a60e8bb0 msg=0x7fc8a60e6970 rerr=0x7fc89a32cd40 args=0x7fc8880160a0 ret=0x7fc888016030 sep 06 11:24:57 laptop libvirtd[1542]: priv=0x7fc8a60ea3a0 conn=(nil) sep 06 11:24:57 laptop libvirtd[1542]: name=lxc:/// sep 06 11:24:57 laptop libvirtd[1542]: Cannot recv data: Connection reset by peer sep 06 11:24:57 laptop libvirtd[1542]: internal error: guest failed to start: 2013-09-06 11:24:57.088+0000: 5794: debug : virFileC Rootfs after failed creation looks like this: busybox ├── [offlineh users ] busybox ├── [offlineh users ] busybox-static_1.17.1-8_amd64.deb ├── [offlineh users ] .oldroot ├── [offlineh users ] proc └── [offlineh users ] sys I have debugging enabled, at least LIBVIRT_DEBUG is set to 1 and i get much more messages. If there's any my granular debug please let me know. PS: I forgot to mention my version of libvirt is 1.1.2 Thanks, Jaka! On Fri, Sep 6, 2013 at 3:41 AM, Gao feng <gaofeng@cn.fujitsu.com> wrote:
On 09/06/2013 03:15 AM, Jaka Hudoklin wrote:
Hello!
I'm testing user namespaces and I have quite some problem getting them to work.
First of all, I have user namespaces support enabled in kernel:
offlinehacker:~/ $ uname -r 3.10.10 offlinehacker:~/ $ ls /proc/self/ns/ ipc@ mnt@ net@ pid@ user@ uts@
I created simple ubuntu rootfs and when I start container without idmap, so without user namespace mappings, it works just fine:
Libivrt config:
<domain type='lxc'> <name>helloworld</name> <memory>102400</memory> <os> <type>exe</type> <init>/bin/dash</init> </os> <!--<idmap> <uid start='0' target='499' count='10'/> <gid start='0' target='100' count='10'/> </idmap>--> <devices> <console type='pty'/> <filesystem type='mount'> <source dir='/home/offlinehacker/rootfs'/> <target dir='/'/> </filesystem> </devices> </domain>
Your configuration looks good.
This is how my rootfs looks like:
offlinehacker:~/ $ ls -la rootfs drwxr-xr-x 23 offlinehacker users 4096 sep 5 19:06 ./ drwxr-xr-x 59 offlinehacker users 4096 sep 5 19:06 ../ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 bin/ drwxr-xr-x 3 offlinehacker users 4096 avg 27 14:11 boot/ drwxr-xr-x 4 offlinehacker users 4096 avg 27 14:11 dev/ drwxr-xr-x 86 offlinehacker users 4096 sep 5 18:20 etc/ drwxr-xr-x 3 offlinehacker users 4096 avg 27 14:11 home/ lrwxrwxrwx 1 offlinehacker users 33 avg 27 14:10 initrd.img -> /boot/initrd.img-3.2.0-52-virtual drwxr-xr-x 18 offlinehacker users 4096 avg 27 14:10 lib/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:10 lib64/ drwx------ 2 offlinehacker users 4096 avg 27 14:11 lost+found/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 media/ drwxr-xr-x 2 offlinehacker users 4096 apr 19 2012 mnt/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 opt/ -rw-r--r-- 1 offlinehacker vboxusers 231671365 avg 27 14:12 precise-server-cloudimg-amd64-root.tar.gz drwxr-xr-x 2 offlinehacker users 4096 apr 19 2012 proc/ drwx------ 2 offlinehacker users 4096 sep 3 23:47 root/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 run/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 sbin/ drwxr-xr-x 2 offlinehacker users 4096 mar 5 2012 selinux/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 srv/ drwxr-xr-x 2 offlinehacker users 4096 apr 14 2012 sys/ drwxrwxrwt 2 offlinehacker users 4096 sep 4 01:39 tmp/ drwxr-xr-x 10 offlinehacker users 4096 avg 27 14:09 usr/ drwxr-xr-x 12 offlinehacker users 4096 sep 5 18:10 var/ lrwxrwxrwx 1 offlinehacker users 29 avg 27 14:10 vmlinuz -> boot/vmlinuz-3.2.0-52-virtual
And this is who I am: offlinehacker:~/ $ id uid=499(offlinehacker) gid=67(libvirtd) groups=100(users),1(wheel),57(networkmanager),67(libvirtd)
Can this user exec //home/offlinehacker/rootfs/bin/dash successfully ?
When I create the container with idmap uncommented I get the following
error:
offlinehacker:~/ $ virsh -c lxc:/// create helloworld.xml error: Failed to create domain from helloworld.xml error: internal error: guest failed to start: 2013-09-05
19:08:57.781+0000: 19036: debug
And these are the logs: sep 05 19:08:52 laptop libvirtd[1542]: server=0x7fc8a60ddd60
sep 05 19:08:52 laptop libvirtd[1542]: priv=0x7fc8a60e91f0 conn=(nil) sep 05 19:08:52 laptop libvirtd[1542]: name=lxc:/// sep 05 19:08:57 laptop libvirtd[1542]: Cannot recv data: Connection reset by peer sep 05 19:08:58 laptop libvirtd[1542]: internal error: guest failed to start: 2013-09-05 19:08:57.781+0000: 19036: debug
Looks like .oldroot, dev, proc and sys gets created with mapped
client=0x7fc8a60e68d0 msg=0x7fc8a60e9380 rerr=0x7fc89a32cd40 args=0x7fc88800b4a0 ret=0x7fc88800a1c0 permissions(499:100), but container fails to start.
Please enable debug mode, I need more information.
http://libvirt.org/logging.html
Thansk
And if needed, i can recompile from source and place custom debug handlers, but i need some pointers where it could have go wrong. Thanks, Jaka! On Fri, Sep 6, 2013 at 1:32 PM, Jaka Hudoklin <jakahudoklin@gmail.com>wrote:
Hello!
Okay i tried again with only staticly linked busybox: offlinehacker:~/ $ /home/offlinehacker/busybox/busybox BusyBox v1.17.1 (Debian 1:1.17.1-8) multi-call binary. Copyright (C) 1998-2009 Erik Andersen, Rob Landley, Denys Vlasenko and others. Licensed under GPLv2. See source distribution for full notice. ....
Again my id: uid=499(offlinehacker) gid=100(users) groups=100(users),1(wheel),57(networkmanager)
My rootfs tree(/home/offlinehacker/busybox): busybox ├── [offlineh users ] busybox └── [offlineh users ] busybox-static_1.17.1-8_amd64.deb
It works just fine as root and these folders gets created: busybox ├── [offlineh users ] busybox ├── [offlineh users ] busybox-static_1.17.1-8_amd64.deb ├── [root root ] dev ├── [root root ] .oldroot ├── [root root ] proc └── [root root ] sys
When i start it with idmap with clean rootfs(dev proc sys and .oldroot deleted) i get this error, and it is a little bit different now: error: Failed to create domain from helloworld.xml error: internal error: guest failed to start: 2013-09-06 11:24:57.088+0000: 5794: debug : virFileC
And log is pretty similar: sep 06 11:24:56 laptop libvirtd[1542]: EVENT_POLL_UPDATE_HANDLE: watch=241 events=1 sep 06 11:24:57 laptop libvirtd[1542]: Skip interrupt, 1 140499747788544 sep 06 11:24:57 laptop libvirtd[1542]: OBJECT_REF: obj=0x7fc878000c90 sep 06 11:24:57 laptop libvirtd[1542]: OBJECT_REF: obj=0x7fc878000c90 sep 06 11:24:57 laptop libvirtd[1542]: server=0x7fc8a60ddd60 client=0x7fc8a60e8bb0 msg=0x7fc8a60e6970 rerr=0x7fc89a32cd40 args=0x7fc8880160a0 ret=0x7fc888016030 sep 06 11:24:57 laptop libvirtd[1542]: priv=0x7fc8a60ea3a0 conn=(nil) sep 06 11:24:57 laptop libvirtd[1542]: name=lxc:/// sep 06 11:24:57 laptop libvirtd[1542]: Cannot recv data: Connection reset by peer sep 06 11:24:57 laptop libvirtd[1542]: internal error: guest failed to start: 2013-09-06 11:24:57.088+0000: 5794: debug : virFileC
Rootfs after failed creation looks like this: busybox ├── [offlineh users ] busybox ├── [offlineh users ] busybox-static_1.17.1-8_amd64.deb ├── [offlineh users ] .oldroot ├── [offlineh users ] proc └── [offlineh users ] sys
I have debugging enabled, at least LIBVIRT_DEBUG is set to 1 and i get much more messages. If there's any my granular debug please let me know.
PS: I forgot to mention my version of libvirt is 1.1.2
Thanks, Jaka!
On Fri, Sep 6, 2013 at 3:41 AM, Gao feng <gaofeng@cn.fujitsu.com> wrote:
On 09/06/2013 03:15 AM, Jaka Hudoklin wrote:
Hello!
I'm testing user namespaces and I have quite some problem getting them to work.
First of all, I have user namespaces support enabled in kernel:
offlinehacker:~/ $ uname -r 3.10.10 offlinehacker:~/ $ ls /proc/self/ns/ ipc@ mnt@ net@ pid@ user@ uts@
I created simple ubuntu rootfs and when I start container without idmap, so without user namespace mappings, it works just fine:
Libivrt config:
<domain type='lxc'> <name>helloworld</name> <memory>102400</memory> <os> <type>exe</type> <init>/bin/dash</init> </os> <!--<idmap> <uid start='0' target='499' count='10'/> <gid start='0' target='100' count='10'/> </idmap>--> <devices> <console type='pty'/> <filesystem type='mount'> <source dir='/home/offlinehacker/rootfs'/> <target dir='/'/> </filesystem> </devices> </domain>
Your configuration looks good.
This is how my rootfs looks like:
offlinehacker:~/ $ ls -la rootfs drwxr-xr-x 23 offlinehacker users 4096 sep 5 19:06 ./ drwxr-xr-x 59 offlinehacker users 4096 sep 5 19:06 ../ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 bin/ drwxr-xr-x 3 offlinehacker users 4096 avg 27 14:11 boot/ drwxr-xr-x 4 offlinehacker users 4096 avg 27 14:11 dev/ drwxr-xr-x 86 offlinehacker users 4096 sep 5 18:20 etc/ drwxr-xr-x 3 offlinehacker users 4096 avg 27 14:11 home/ lrwxrwxrwx 1 offlinehacker users 33 avg 27 14:10 initrd.img -> /boot/initrd.img-3.2.0-52-virtual drwxr-xr-x 18 offlinehacker users 4096 avg 27 14:10 lib/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:10 lib64/ drwx------ 2 offlinehacker users 4096 avg 27 14:11 lost+found/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 media/ drwxr-xr-x 2 offlinehacker users 4096 apr 19 2012 mnt/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 opt/ -rw-r--r-- 1 offlinehacker vboxusers 231671365 avg 27 14:12 precise-server-cloudimg-amd64-root.tar.gz drwxr-xr-x 2 offlinehacker users 4096 apr 19 2012 proc/ drwx------ 2 offlinehacker users 4096 sep 3 23:47 root/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 run/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 sbin/ drwxr-xr-x 2 offlinehacker users 4096 mar 5 2012 selinux/ drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 srv/ drwxr-xr-x 2 offlinehacker users 4096 apr 14 2012 sys/ drwxrwxrwt 2 offlinehacker users 4096 sep 4 01:39 tmp/ drwxr-xr-x 10 offlinehacker users 4096 avg 27 14:09 usr/ drwxr-xr-x 12 offlinehacker users 4096 sep 5 18:10 var/ lrwxrwxrwx 1 offlinehacker users 29 avg 27 14:10 vmlinuz -> boot/vmlinuz-3.2.0-52-virtual
And this is who I am: offlinehacker:~/ $ id uid=499(offlinehacker) gid=67(libvirtd) groups=100(users),1(wheel),57(networkmanager),67(libvirtd)
Can this user exec //home/offlinehacker/rootfs/bin/dash successfully ?
When I create the container with idmap uncommented I get the following
error:
offlinehacker:~/ $ virsh -c lxc:/// create helloworld.xml error: Failed to create domain from helloworld.xml error: internal error: guest failed to start: 2013-09-05
19:08:57.781+0000: 19036: debug
And these are the logs: sep 05 19:08:52 laptop libvirtd[1542]: server=0x7fc8a60ddd60
sep 05 19:08:52 laptop libvirtd[1542]: priv=0x7fc8a60e91f0 conn=(nil) sep 05 19:08:52 laptop libvirtd[1542]: name=lxc:/// sep 05 19:08:57 laptop libvirtd[1542]: Cannot recv data: Connection reset by peer sep 05 19:08:58 laptop libvirtd[1542]: internal error: guest failed to start: 2013-09-05 19:08:57.781+0000: 19036: debug
Looks like .oldroot, dev, proc and sys gets created with mapped
client=0x7fc8a60e68d0 msg=0x7fc8a60e9380 rerr=0x7fc89a32cd40 args=0x7fc88800b4a0 ret=0x7fc88800a1c0 permissions(499:100), but container fails to start.
Please enable debug mode, I need more information.
http://libvirt.org/logging.html
Thansk
On 09/06/2013 07:32 PM, Jaka Hudoklin wrote:
Hello!
Okay i tried again with only staticly linked busybox: offlinehacker:~/ $ /home/offlinehacker/busybox/busybox BusyBox v1.17.1 (Debian 1:1.17.1-8) multi-call binary. Copyright (C) 1998-2009 Erik Andersen, Rob Landley, Denys Vlasenko and others. Licensed under GPLv2. See source distribution for full notice. ....
Again my id: uid=499(offlinehacker) gid=100(users) groups=100(users),1(wheel),57(networkmanager)
My rootfs tree(/home/offlinehacker/busybox): busybox ├── [offlineh users ] busybox └── [offlineh users ] busybox-static_1.17.1-8_amd64.deb
It works just fine as root and these folders gets created: busybox ├── [offlineh users ] busybox ├── [offlineh users ] busybox-static_1.17.1-8_amd64.deb ├── [root root ] dev ├── [root root ] .oldroot ├── [root root ] proc └── [root root ] sys
When i start it with idmap with clean rootfs(dev proc sys and .oldroot deleted) i get this error, and it is a little bit different now: error: Failed to create domain from helloworld.xml error: internal error: guest failed to start: 2013-09-06 11:24:57.088+0000: 5794: debug : virFileC
And log is pretty similar: sep 06 11:24:56 laptop libvirtd[1542]: EVENT_POLL_UPDATE_HANDLE: watch=241 events=1 sep 06 11:24:57 laptop libvirtd[1542]: Skip interrupt, 1 140499747788544 sep 06 11:24:57 laptop libvirtd[1542]: OBJECT_REF: obj=0x7fc878000c90 sep 06 11:24:57 laptop libvirtd[1542]: OBJECT_REF: obj=0x7fc878000c90 sep 06 11:24:57 laptop libvirtd[1542]: server=0x7fc8a60ddd60 client=0x7fc8a60e8bb0 msg=0x7fc8a60e6970 rerr=0x7fc89a32cd40 args=0x7fc8880160a0 ret=0x7fc888016030 sep 06 11:24:57 laptop libvirtd[1542]: priv=0x7fc8a60ea3a0 conn=(nil) sep 06 11:24:57 laptop libvirtd[1542]: name=lxc:/// sep 06 11:24:57 laptop libvirtd[1542]: Cannot recv data: Connection reset by peer sep 06 11:24:57 laptop libvirtd[1542]: internal error: guest failed to start: 2013-09-06 11:24:57.088+0000: 5794: debug : virFileC
Rootfs after failed creation looks like this: busybox ├── [offlineh users ] busybox ├── [offlineh users ] busybox-static_1.17.1-8_amd64.deb ├── [offlineh users ] .oldroot ├── [offlineh users ] proc └── [offlineh users ] sys
I have debugging enabled, at least LIBVIRT_DEBUG is set to 1 and i get much more messages. If there's any my granular debug please let me know.
PS: I forgot to mention my version of libvirt is 1.1.2
OK, I get it, Maybe you need this patch 1583dfda7c4e5ad71efe0615c06e5676528d8203 LXC: Don't mount securityfs when user namespace enabled Thanks
I applied your patch, but no success. What bothers me is that connection gets reseted. By the way, i'm using systemd, with process started in forking mode and as daemon. Could this cause any problems? This is my libvirtd.conf, if it helps anything: unix_sock_group = "libvirtd" unix_sock_rw_perms = "0770" auth_unix_ro = "none" auth_unix_rw = "none" Can you please tell me easy to setup distro with systemd that user namespaces work, so i can compare. Thanks! On Mon, Sep 9, 2013 at 3:08 AM, Gao feng <gaofeng@cn.fujitsu.com> wrote:
On 09/06/2013 07:32 PM, Jaka Hudoklin wrote:
Hello!
Okay i tried again with only staticly linked busybox: offlinehacker:~/ $ /home/offlinehacker/busybox/busybox BusyBox v1.17.1 (Debian 1:1.17.1-8) multi-call binary. Copyright (C) 1998-2009 Erik Andersen, Rob Landley, Denys Vlasenko and others. Licensed under GPLv2. See source distribution for full notice. ....
Again my id: uid=499(offlinehacker) gid=100(users) groups=100(users),1(wheel),57(networkmanager)
My rootfs tree(/home/offlinehacker/busybox): busybox ├── [offlineh users ] busybox └── [offlineh users ] busybox-static_1.17.1-8_amd64.deb
It works just fine as root and these folders gets created: busybox ├── [offlineh users ] busybox ├── [offlineh users ] busybox-static_1.17.1-8_amd64.deb ├── [root root ] dev ├── [root root ] .oldroot ├── [root root ] proc └── [root root ] sys
When i start it with idmap with clean rootfs(dev proc sys and .oldroot deleted) i get this error, and it is a little bit different now: error: Failed to create domain from helloworld.xml error: internal error: guest failed to start: 2013-09-06 11:24:57.088+0000: 5794: debug : virFileC
And log is pretty similar: sep 06 11:24:56 laptop libvirtd[1542]: EVENT_POLL_UPDATE_HANDLE: watch=241 events=1 sep 06 11:24:57 laptop libvirtd[1542]: Skip interrupt, 1 140499747788544 sep 06 11:24:57 laptop libvirtd[1542]: OBJECT_REF: obj=0x7fc878000c90 sep 06 11:24:57 laptop libvirtd[1542]: OBJECT_REF: obj=0x7fc878000c90 sep 06 11:24:57 laptop libvirtd[1542]: server=0x7fc8a60ddd60 client=0x7fc8a60e8bb0 msg=0x7fc8a60e6970 rerr=0x7fc89a32cd40 args=0x7fc8880160a0 ret=0x7fc888016030 sep 06 11:24:57 laptop libvirtd[1542]: priv=0x7fc8a60ea3a0 conn=(nil) sep 06 11:24:57 laptop libvirtd[1542]: name=lxc:/// sep 06 11:24:57 laptop libvirtd[1542]: Cannot recv data: Connection reset by peer sep 06 11:24:57 laptop libvirtd[1542]: internal error: guest failed to start: 2013-09-06 11:24:57.088+0000: 5794: debug : virFileC
Rootfs after failed creation looks like this: busybox ├── [offlineh users ] busybox ├── [offlineh users ] busybox-static_1.17.1-8_amd64.deb ├── [offlineh users ] .oldroot ├── [offlineh users ] proc └── [offlineh users ] sys
I have debugging enabled, at least LIBVIRT_DEBUG is set to 1 and i get much more messages. If there's any my granular debug please let me know.
PS: I forgot to mention my version of libvirt is 1.1.2
OK, I get it, Maybe you need this patch
1583dfda7c4e5ad71efe0615c06e5676528d8203 LXC: Don't mount securityfs when user namespace enabled
Thanks
It seems to be working now, what I needed was libvirt built with libcap support and also securityfs patch. Thanks! On Mon, Sep 9, 2013 at 1:08 PM, Jaka Hudoklin <jakahudoklin@gmail.com>wrote:
I applied your patch, but no success. What bothers me is that connection gets reseted. By the way, i'm using systemd, with process started in forking mode and as daemon. Could this cause any problems?
This is my libvirtd.conf, if it helps anything: unix_sock_group = "libvirtd" unix_sock_rw_perms = "0770" auth_unix_ro = "none" auth_unix_rw = "none"
Can you please tell me easy to setup distro with systemd that user namespaces work, so i can compare.
Thanks!
On Mon, Sep 9, 2013 at 3:08 AM, Gao feng <gaofeng@cn.fujitsu.com> wrote:
On 09/06/2013 07:32 PM, Jaka Hudoklin wrote:
Hello!
Okay i tried again with only staticly linked busybox: offlinehacker:~/ $ /home/offlinehacker/busybox/busybox BusyBox v1.17.1 (Debian 1:1.17.1-8) multi-call binary. Copyright (C) 1998-2009 Erik Andersen, Rob Landley, Denys Vlasenko and others. Licensed under GPLv2. See source distribution for full notice. ....
Again my id: uid=499(offlinehacker) gid=100(users) groups=100(users),1(wheel),57(networkmanager)
My rootfs tree(/home/offlinehacker/busybox): busybox ├── [offlineh users ] busybox └── [offlineh users ] busybox-static_1.17.1-8_amd64.deb
It works just fine as root and these folders gets created: busybox ├── [offlineh users ] busybox ├── [offlineh users ] busybox-static_1.17.1-8_amd64.deb ├── [root root ] dev ├── [root root ] .oldroot ├── [root root ] proc └── [root root ] sys
When i start it with idmap with clean rootfs(dev proc sys and .oldroot deleted) i get this error, and it is a little bit different now: error: Failed to create domain from helloworld.xml error: internal error: guest failed to start: 2013-09-06 11:24:57.088+0000: 5794: debug : virFileC
And log is pretty similar: sep 06 11:24:56 laptop libvirtd[1542]: EVENT_POLL_UPDATE_HANDLE: watch=241 events=1 sep 06 11:24:57 laptop libvirtd[1542]: Skip interrupt, 1 140499747788544 sep 06 11:24:57 laptop libvirtd[1542]: OBJECT_REF: obj=0x7fc878000c90 sep 06 11:24:57 laptop libvirtd[1542]: OBJECT_REF: obj=0x7fc878000c90 sep 06 11:24:57 laptop libvirtd[1542]: server=0x7fc8a60ddd60 client=0x7fc8a60e8bb0 msg=0x7fc8a60e6970 rerr=0x7fc89a32cd40 args=0x7fc8880160a0 ret=0x7fc888016030 sep 06 11:24:57 laptop libvirtd[1542]: priv=0x7fc8a60ea3a0 conn=(nil) sep 06 11:24:57 laptop libvirtd[1542]: name=lxc:/// sep 06 11:24:57 laptop libvirtd[1542]: Cannot recv data: Connection reset by peer sep 06 11:24:57 laptop libvirtd[1542]: internal error: guest failed to start: 2013-09-06 11:24:57.088+0000: 5794: debug : virFileC
Rootfs after failed creation looks like this: busybox ├── [offlineh users ] busybox ├── [offlineh users ] busybox-static_1.17.1-8_amd64.deb ├── [offlineh users ] .oldroot ├── [offlineh users ] proc └── [offlineh users ] sys
I have debugging enabled, at least LIBVIRT_DEBUG is set to 1 and i get much more messages. If there's any my granular debug please let me know.
PS: I forgot to mention my version of libvirt is 1.1.2
OK, I get it, Maybe you need this patch
1583dfda7c4e5ad71efe0615c06e5676528d8203 LXC: Don't mount securityfs when user namespace enabled
Thanks
On 09/09/2013 10:26 PM, Jaka Hudoklin wrote:
It seems to be working now, what I needed was libvirt built with libcap support and also securityfs patch. Thanks!
good news, I guess you haven't fully enabled the debug message. You should modify your lxc.conf and set log_with_libvirtd = 1 too. I trust this will give you more clearer error message rather than the incorrect "connection gets reseted" message. Thanks
participants (2)
-
Gao feng -
Jaka Hudoklin