12:30 a.m.
On Fri, Nov 9, 2012 at 12:09 AM, unicell <unicell(a)gmail.com> wrote:
Hi,
I'm trying to manage LXC instances through OpenStack, which use libvirt as
a virtualization driver layer. After launching LXC instance, I simply could
not attach to the console.
virsh # list
Id Name State
----------------------------------
14366 instance-00000078 running
virsh # console 14366
Connected to domain instance-00000078
Escape character is ^]
And it keeps stuck here.
Initially I thought it could be a cgroup device config issue, and tried to
"echo ‘c 5:1 rwm'
>/cgroup/devices/libvirt/lxc/instance-00000078/devices.allow“, to add
/dev/console access right to the container. But it does not work, console
stilll not working.
Before the cgroup tweaking, devices.list are as follows
[root@localhost libvirt]# cat
/cgroup/devices/libvirt/lxc/instance-0000007a/devices.list
c 1:3 rwm
c 1:5 rwm
c 1:7 rwm
c 1:8 rwm
c 1:9 rwm
c 5:0 rwm
c 5:2 rwm
c 136:* rwm
I'm not quite sure what kind of issue could possibly cause this console
access issue. Cause I'm now using two different guest OS templates on two
different host OSes. Following are some symptoms I observed.
* (g) for container guest OS
* (h) for host OS
* NOTE: Ubuntu guest os are generated by lxc-create -t ubuntu, which is an
Ubuntu 12.04 based lxc template.
CentOS guest os are handcrafted by myself, taking reference
of http://wiki.1tux.org/wiki/Lxc/Installation/Guest/Centos/6
1. Ubuntu (g) launched by lxc-start on Ubuntu 12.04 host, console works
2. Ubuntu (g) launched by OpenStack / libvirt on Ubuntu 12.04 host,
console works
3. Ubuntu (g) launched by OpenStack / libvirt on CentOS 6.3 host,
console NOT works
4. CentOS (g) launched by lxc-start on Ubuntu 12.04 host, console works
5. CentOS (g) launched by OpenStack / libvirt on Ubuntu12.04 host,
console NOT works
6. CentOS (g) launched by OpenStack / libvirt on CentOS 6.3 host,
console NOT works
So somehow I feel it could be something related to LXC template rootfs,
but Host OS do make a difference (2 vs. 3). lxc-start / libvirt also makes
a difference (4 vs. 5), but that might be caused by cgroup setting.
Could someone shed me some clues to further digging this issue? Thanks!
--
Qiu Yu
For the "Ubuntu (g) launched by OpenStack / libvirt on CentOS 6.3 host", I
also tried following command and console still not working.
echo "a *.* rwm"
/cgroup/devices/libvirt/lxc/instance-00000078/devices.allow
More information attached.
[root@localhost ~]# ls -l
/home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty*
crw-rw-rw- 1 root tty 5, 0 Oct 23 20:01
/home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty
crw------- 1 root tty 4, 0 Oct 23 20:01
/home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty0
crw------- 1 root tty 4, 1 Oct 23 20:01
/home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty1
crw------- 1 root tty 4, 2 Oct 23 20:01
/home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty2
crw------- 1 root tty 4, 3 Oct 23 20:01
/home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty3
crw------- 1 root tty 4, 4 Oct 23 20:01
/home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty4
crw------- 1 root tty 4, 5 Oct 23 20:01
/home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty5
crw------- 1 root tty 4, 6 Oct 23 20:01
/home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty6
crw------- 1 root tty 4, 7 Oct 23 20:01
/home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty7
crw------- 1 root tty 4, 8 Oct 23 20:01
/home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty8
crw------- 1 root tty 4, 9 Oct 23 20:01
/home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty9
[root@localhost ~]# cat
/cgroup/devices/libvirt/lxc/instance-00000078/devices.list
c 1:3 rwm
c 1:5 rwm
c 1:7 rwm
c 1:8 rwm
c 1:9 rwm
c 5:0 rwm
c 5:2 rwm
c 136:* rwm
c 5:1 rwm
c 5:* rwm
a *:* rwm
[root@localhost ~]# cat /etc/libvirt/lxc/instance-00000078.xml
<!--
WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
OVERWRITTEN AND LOST. Changes to this xml configuration should be made
using:
virsh edit instance-00000078
or other application using the libvirt API.
-->
<domain type='lxc'>
<name>instance-00000078</name>
<uuid>55f225d1-e49c-4a51-9c17-0a5dde6d1917</uuid>
<memory>524288</memory>
<currentMemory>524288</currentMemory>
<vcpu>2</vcpu>
<os>
<type arch='x86_64'>exe</type>
<init>/sbin/init</init>
<cmdline>console=ttyS0</cmdline>
</os>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/libexec/libvirt_lxc</emulator>
<filesystem type='mount' accessmode='passthrough'>
<source
dir='/home/stack/nova_state/instances/instance-00000078/rootfs'/>
<target dir='/'/>
</filesystem>
<interface type='bridge'>
<mac address='fa:16:3e:08:4f:b9'/>
<source bridge='br100'/>
<filterref filter='nova-instance-instance-00000078-fa163e084fb9'>
<parameter name='DHCPSERVER' value='10.48.253.1'/>
<parameter name='IP' value='10.48.253.2'/>
<parameter name='PROJMASK' value='255.255.255.0'/>
<parameter name='PROJNET' value='10.48.253.0'/>
</filterref>
</interface>
<console type='pty'>
<target type='lxc' port='0'/>
</console>
</devices>
</domain>
--
Qiu Yu