On Fri, Nov 9, 2012 at 12:09 AM, unicell <span dir="ltr">&lt;<a href="mailto:unicell@gmail.com" target="_blank">unicell@gmail.com</a>&gt;</span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">

Hi,<div><br></div><div>I&#39;m trying to manage LXC instances through OpenStack, which use libvirt as a virtualization driver layer. After launching LXC instance, I simply could not attach to the console.</div><div><br></div>


<blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div>virsh # list</div></div>
<div><div> Id Name                 State</div></div><div><div>----------------------------------</div></div><div><div>14366 instance-00000078    running</div></div><div><div><br></div></div><div><div>virsh # console 14366</div>



</div><div><div>Connected to domain instance-00000078</div></div><div><div>Escape character is ^]</div></div></blockquote><div><br></div><div>And it keeps stuck here.</div><div><br></div><div>Initially I thought it could be a cgroup device config issue, and tried to &quot;echo ‘c 5:1 rwm&#39; &gt;/cgroup/devices/libvirt/lxc/instance-00000078/devices.allow“, to add /dev/console access right to the container. But it does not work, console stilll not working.</div>


<div><br></div><div>Before the cgroup tweaking, devices.list are as follows</div><div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>[root@localhost libvirt]# cat /cgroup/devices/libvirt/lxc/instance-0000007a/devices.list</div>


<div><br></div><div><div>c 1:3 rwm</div></div><div><div>c 1:5 rwm</div></div><div><div>c 1:7 rwm</div></div><div><div>c 1:8 rwm</div></div><div><div>c 1:9 rwm</div></div><div><div>c 5:0 rwm</div></div><div><div>c 5:2 rwm</div>


</div><div><div>c 136:* rwm</div></div></blockquote>
<div><br></div><div>I&#39;m not quite sure what kind of issue could possibly cause this console access issue. Cause I&#39;m now using two different guest OS templates on two different host OSes. Following are some symptoms I observed.</div>


<div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><br></div><div>* (g) for container guest OS</div><div>* (h) for host OS</div><div>* NOTE: Ubuntu guest os are generated by lxc-create -t ubuntu, which is an Ubuntu 12.04 based lxc template.</div>


<div>             CentOS guest os are handcrafted by myself, taking reference of <a href="http://wiki.1tux.org/wiki/Lxc/Installation/Guest/Centos/6" target="_blank">http://wiki.1tux.org/wiki/Lxc/Installation/Guest/Centos/6</a></div>

<div>
<ol><li>Ubuntu (g) launched by lxc-start on Ubuntu 12.04 host, console works</li><li>Ubuntu (g) launched by OpenStack / libvirt on Ubuntu 12.04 host, console works</li><li>Ubuntu (g) launched by OpenStack / libvirt on CentOS 6.3 host, console NOT works</li>


<li>CentOS (g) launched by lxc-start on Ubuntu 12.04 host, console works</li><li>CentOS (g) launched by OpenStack / libvirt on Ubuntu12.04 host, console NOT works</li><li>CentOS (g) launched by OpenStack / libvirt on CentOS 6.3 host, console NOT works</li>


</ol></div></blockquote></div><div>So somehow I feel it could be something related to LXC template rootfs, but Host OS do make a difference (2 vs. 3). lxc-start / libvirt also makes a difference (4 vs. 5), but that might be caused by cgroup setting.</div>


<div><br></div><div>Could someone shed me some clues to further digging this issue? Thanks!  </div><div><br></div><div>--<br>Qiu Yu<br>
</div>
</blockquote></div><br><div>For the &quot;Ubuntu (g) launched by OpenStack / libvirt on CentOS 6.3 host&quot;, I also tried following command and console still not working. </div><div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px">

<div>echo &quot;a *.* rwm&quot; &gt;/cgroup/devices/libvirt/lxc/instance-00000078/devices.allow</div></blockquote><div><br></div><div>More information attached.</div><div><br></div><div><div>[root@localhost ~]# ls -l /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty*</div>

<div>crw-rw-rw- 1 root tty 5, 0 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty</div><div>crw------- 1 root tty 4, 0 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty0</div>

<div>crw------- 1 root tty 4, 1 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty1</div><div>crw------- 1 root tty 4, 2 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty2</div>

<div>crw------- 1 root tty 4, 3 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty3</div><div>crw------- 1 root tty 4, 4 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty4</div>

<div>crw------- 1 root tty 4, 5 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty5</div><div>crw------- 1 root tty 4, 6 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty6</div>

<div>crw------- 1 root tty 4, 7 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty7</div><div>crw------- 1 root tty 4, 8 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty8</div>

<div>crw------- 1 root tty 4, 9 Oct 23 20:01 /home/stack/nova_state/instances/instance-00000078/rootfs/dev/tty9</div><div><br></div><div><br></div><div><div>[root@localhost ~]# cat /cgroup/devices/libvirt/lxc/instance-00000078/devices.list </div>

<div>c 1:3 rwm</div><div>c 1:5 rwm</div><div>c 1:7 rwm</div><div>c 1:8 rwm</div><div>c 1:9 rwm</div><div>c 5:0 rwm</div><div>c 5:2 rwm</div><div>c 136:* rwm</div><div>c 5:1 rwm</div><div>c 5:* rwm</div><div>a *:* rwm</div>

</div><div><br></div><div><br></div><div><div>[root@localhost ~]# cat /etc/libvirt/lxc/instance-00000078.xml </div><div>&lt;!--</div><div>WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE </div><div>

OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:</div><div>  virsh edit instance-00000078</div><div>or other application using the libvirt API.</div><div>--&gt;</div><div><br></div><div>&lt;domain type=&#39;lxc&#39;&gt;</div>

<div>  &lt;name&gt;instance-00000078&lt;/name&gt;</div><div>  &lt;uuid&gt;55f225d1-e49c-4a51-9c17-0a5dde6d1917&lt;/uuid&gt;</div><div>  &lt;memory&gt;524288&lt;/memory&gt;</div><div>  &lt;currentMemory&gt;524288&lt;/currentMemory&gt;</div>

<div>  &lt;vcpu&gt;2&lt;/vcpu&gt;</div><div>  &lt;os&gt;</div><div>    &lt;type arch=&#39;x86_64&#39;&gt;exe&lt;/type&gt;</div><div>    &lt;init&gt;/sbin/init&lt;/init&gt;</div><div>    &lt;cmdline&gt;console=ttyS0&lt;/cmdline&gt;</div>

<div>  &lt;/os&gt;</div><div>  &lt;clock offset=&#39;utc&#39;/&gt;</div><div>  &lt;on_poweroff&gt;destroy&lt;/on_poweroff&gt;</div><div>  &lt;on_reboot&gt;restart&lt;/on_reboot&gt;</div><div>  &lt;on_crash&gt;destroy&lt;/on_crash&gt;</div>

<div>  &lt;devices&gt;</div><div>    &lt;emulator&gt;/usr/libexec/libvirt_lxc&lt;/emulator&gt;</div><div>    &lt;filesystem type=&#39;mount&#39; accessmode=&#39;passthrough&#39;&gt;</div><div>      &lt;source dir=&#39;/home/stack/nova_state/instances/instance-00000078/rootfs&#39;/&gt;</div>

<div>      &lt;target dir=&#39;/&#39;/&gt;</div><div>    &lt;/filesystem&gt;</div><div>    &lt;interface type=&#39;bridge&#39;&gt;</div><div>      &lt;mac address=&#39;fa:16:3e:08:4f:b9&#39;/&gt;</div><div>      &lt;source bridge=&#39;br100&#39;/&gt;</div>

<div>      &lt;filterref filter=&#39;nova-instance-instance-00000078-fa163e084fb9&#39;&gt;</div><div>        &lt;parameter name=&#39;DHCPSERVER&#39; value=&#39;10.48.253.1&#39;/&gt;</div><div>        &lt;parameter name=&#39;IP&#39; value=&#39;10.48.253.2&#39;/&gt;</div>

<div>        &lt;parameter name=&#39;PROJMASK&#39; value=&#39;255.255.255.0&#39;/&gt;</div><div>        &lt;parameter name=&#39;PROJNET&#39; value=&#39;10.48.253.0&#39;/&gt;</div><div>      &lt;/filterref&gt;</div><div>    &lt;/interface&gt;</div>

<div>    &lt;console type=&#39;pty&#39;&gt;</div><div>      &lt;target type=&#39;lxc&#39; port=&#39;0&#39;/&gt;</div><div>    &lt;/console&gt;</div><div>  &lt;/devices&gt;</div><div>&lt;/domain&gt;</div></div><div><br></div>

<div><br></div>--<br>Qiu Yu<br></div>