3:02 a.m.
hello,
i am trying to use libvirt with LXC, and having some issues...
everything seems ok when using lxc-* tools. host box is Arch Linux.
ultimately i'm getting this error in the domain log:
.....
05:45:31.353: debug : lxcControllerMain:380 : EPOLLHUP from fd 7
i was getting this last night in #virt:
.....
20:49:57.545: error : lxcFdForward:230 : read of fd 7 failed: Input/output error
libvir: Linux Container error : read of fd 7 failed: Input/output error
instead of the former, but i can seem to reproduce now. i might have
replaced 0.7.4 version with git version... i dont remember. right now
the build is direct from master, and i am getting EPOLLHUP. didnt see
anything interesting in virsh or libvirtd debug output, but i can
paste/bin the whole thing or whatever else if need be.
-----VERSIONS/SOURCES
(loadmodconfig make target + below)
kernel: Linux PHS-001 2.6.32-custom #1 SMP PREEMPT Sun Dec 6 05:29:37
CST 2009 i686 Intel(R) Xeon(TM) CPU 3.20GHz GenuineIntel GNU/Linux
CONFIG_DEVPTS_MULTIPLE_INSTANCES=y
CONFIG_GROUP_SCHED=y
CONFIG_FAIR_GROUP_SCHED=y
CONFIG_RT_GROUP_SCHED=y
CONFIG_CGROUP_SCHED=y
CONFIG_CGROUPS=y
CONFIG_CGROUP_NS=y
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CPUSETS=y
CONFIG_PROC_PID_CPUSET=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_RESOURCE_COUNTERS=y
CONFIG_CGROUP_MEM_RES_CTLR=y
CONFIG_CGROUP_MEM_RES_CTLR_SWAP=y
CONFIG_MM_OWNER=y
CONFIG_NAMESPACES=y
CONFIG_UTS_NS=y
CONFIG_IPC_NS=y
CONFIG_USER_NS=y
CONFIG_PID_NS=y
CONFIG_NET_NS=y
CONFIG_NET_CLS_CGROUP=y
CONFIG_SECURITY_FILE_CAPABILITIES=y
(built from master)
libvirt: http://aur.archlinux.org/packages/libvirt-git/libvirt-git/PKGBUILD
--with-capng
--with-network
--with-lxc
--without-xen
--without-xen-inotify
--without-qemu
--without-uml
--without-openvz
--without-vbox
--without-one
--without-esx
libcap-ng: http://aur.archlinux.org/packages/libcap-ng/libcap-ng/PKGBUILD
container:
http://www.stgraber.org/2009/11/06/lxc-containers-or-extremely-fast-virtu...
http://www.stgraber.org/download/lxc-ubuntu-8.04-i386.tar.gz
-----CONFIGURATION
# host ptmx symlinked to pts/ptmx
[root@PHS-001 vps]# ls -l /dev/ptmx
lrwxrwxrwx 1 root root 8 2009-12-08 04:40 /dev/ptmx -> pts/ptmx
# guest ptmx symlinked to pts/ptmx
[root@PHS-001 vps]# ls -l /vps/dom/ubuntu/root/dev/ptmx
lrwxrwxrwx 1 root root 8 2009-12-06 04:11
/vps/dom/ubuntu/root/dev/ptmx -> pts/ptmx
# host devpts mounted newinstance
[root@PHS-001 vps]# mount | grep devpts
none on /dev/pts type devpts (rw,newinstance)
# single cgroup mount on host
[root@PHS-001 vps]# mount | grep cgroup
none on /cgroup type cgroup (rw)
# libvirt config
[root@PHS-001 vps]# grep -v -e '^#\|^ *$' /etc/libvirt/libvirtd.conf
auth_unix_ro = "none"
auth_unix_rw = "none"
# guest config...
# i have tried the legacy <console type='pty' /> with same results
# i dont fully understand this, why do i need to specify a host/source pty?
[root@PHS-001 vps]# cat /vps/def/exec/sys/ubuntu.xml
<domain type='lxc'>
<name>ubuntu</name>
<memory>500000</memory>
<os>
<type arch='i686'>exe</type>
<init>/sbin/init</init>
</os>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<filesystem type='mount'>
<source dir='/vps/dom/ubuntu/root'/>
<target dir='/'/>
</filesystem>
<console type='pty'>
<source path='/dev/pts/4'/>
<target port='0'/>
</console>
</devices>
</domain>
-----PROCEDURE
[root@PHS-001 vps]# LIBVIRT_DEBUG=1 libvirtd --daemon
05:42:46.121: debug : virInitialize:278 : register drivers
.....
[root@PHS-001 vps]# virsh -c lxc:/// define /vps/def/exec/sys/ubuntu.xml
Domain ubuntu defined from /vps/def/exec/sys/ubuntu.xml
[root@PHS-001 vps]# virsh -c lxc:/// define /vps/def/exec/sys/ubuntu.xml
Domain ubuntu defined from /vps/def/exec/sys/ubuntu.xml
[root@PHS-001 vps]# virsh -c lxc:/// start ubuntu
Domain ubuntu started
[root@PHS-001 vps]# virsh -c lxc:/// console ubuntu
error: Unable to get domain status
error: internal error Unable to get cgroup for ubuntu
-----LOGS
# /var/log/libvirt/lxc/ubuntu.log
05:45:31.105: debug : lxcControllerRun:540 : Setting up private /dev/pts
05:45:31.125: debug : lxcControllerRun:566 : Mouting 'devpts' on
/vps/dom/ubuntu/root/dev/pts
05:45:31.125: debug : lxcControllerRun:581 : Opening tty on private
/vps/dom/ubuntu/root/dev/pts/ptmx
05:45:31.125: debug : virCgroupNew:492 : New group /
05:45:31.125: debug : virCgroupDetect:230 : Detected mount/mapping
0:cpu at /cgroup in /1534
05:45:31.125: debug : virCgroupDetect:230 : Detected mount/mapping
1:cpuacct at /cgroup in /1534
05:45:31.125: debug : virCgroupDetect:230 : Detected mount/mapping
2:cpuset at /cgroup in /1534
05:45:31.125: debug : virCgroupDetect:230 : Detected mount/mapping
3:memory at /cgroup in /1534
05:45:31.125: debug : virCgroupDetect:230 : Detected mount/mapping
4:devices at /cgroup in /1534
05:45:31.125: debug : virCgroupDetect:230 : Detected mount/mapping
5:freezer at /cgroup in /1534
05:45:31.125: debug : virCgroupNew:492 : New group /libvirt
05:45:31.125: debug : virCgroupDetect:230 : Detected mount/mapping
0:cpu at /cgroup in /1534
05:45:31.125: debug : virCgroupDetect:230 : Detected mount/mapping
1:cpuacct at /cgroup in /1534
05:45:31.125: debug : virCgroupDetect:230 : Detected mount/mapping
2:cpuset at /cgroup in /1534
05:45:31.125: debug : virCgroupDetect:230 : Detected mount/mapping
3:memory at /cgroup in /1534
05:45:31.125: debug : virCgroupDetect:230 : Detected mount/mapping
4:devices at /cgroup in /1534
05:45:31.125: debug : virCgroupDetect:230 : Detected mount/mapping
5:freezer at /cgroup in /1534
05:45:31.125: debug : virCgroupMakeGroup:450 : Make group /libvirt
05:45:31.125: debug : virCgroupMakeGroup:462 : Make controller
/cgroup/1534/libvirt/
05:45:31.125: debug : virCgroupCpuSetInherit:416 : Setting up
inheritance / -> /libvirt
05:45:31.126: debug : virCgroupGetValueStr:305 : Get value
/cgroup/1534/cpuset.cpus
05:45:31.126: debug : virCgroupCpuSetInherit:429 : Inherit cpuset.cpus = 0-3
05:45:31.126: debug : virCgroupSetValueStr:275 : Set value
/cgroup/1534/libvirt/cpuset.cpus
05:45:31.126: debug : virCgroupGetValueStr:305 : Get value
/cgroup/1534/cpuset.mems
05:45:31.126: debug : virCgroupCpuSetInherit:429 : Inherit cpuset.mems = 0
05:45:31.126: debug : virCgroupSetValueStr:275 : Set value
/cgroup/1534/libvirt/cpuset.mems
05:45:31.126: debug : virCgroupMakeGroup:462 : Make controller
/cgroup/1534/libvirt/
05:45:31.126: debug : virCgroupMakeGroup:462 : Make controller
/cgroup/1534/libvirt/
05:45:31.126: debug : virCgroupMakeGroup:462 : Make controller
/cgroup/1534/libvirt/
05:45:31.126: debug : virCgroupMakeGroup:462 : Make controller
/cgroup/1534/libvirt/
05:45:31.126: debug : virCgroupMakeGroup:462 : Make controller
/cgroup/1534/libvirt/
05:45:31.126: debug : virCgroupNew:492 : New group /libvirt/lxc
05:45:31.126: debug : virCgroupDetect:230 : Detected mount/mapping
0:cpu at /cgroup in /1534
05:45:31.126: debug : virCgroupDetect:230 : Detected mount/mapping
1:cpuacct at /cgroup in /1534
05:45:31.126: debug : virCgroupDetect:230 : Detected mount/mapping
2:cpuset at /cgroup in /1534
05:45:31.126: debug : virCgroupDetect:230 : Detected mount/mapping
3:memory at /cgroup in /1534
05:45:31.126: debug : virCgroupDetect:230 : Detected mount/mapping
4:devices at /cgroup in /1534
05:45:31.126: debug : virCgroupDetect:230 : Detected mount/mapping
5:freezer at /cgroup in /1534
05:45:31.126: debug : virCgroupMakeGroup:450 : Make group /libvirt/lxc
05:45:31.126: debug : virCgroupMakeGroup:462 : Make controller
/cgroup/1534/libvirt/lxc/
05:45:31.211: debug : lxcContainerStart:832 : clone() returned, 1536
05:45:31.353: debug : lxcControllerMain:380 : EPOLLHUP from fd 7
-----END
i've tried endless variations of the <devices> section, but i just
cant seem to get anything going with libvirt/LXC once i move the root
to a new location for a system container. same container works fine
with lxc-* tools (although i manually make a bridge). any help is
appreciated,
C Anthony Risinger