[libvirt] Libvirt + LXC terminals
Hello all, I am looking for a way to start a container with libvirt and bind multiple consoles to it. My container is running busybox, the filesystem created using the LXC package (0.8.0). My libvirt version is 0.10.1. Configuration of the filesystem: - in the container rootfs, /dev contains 2 devices - /dev/console and /dev/tty1 - the inittab file looks like this: ::sysinit:/etc/init.d/rcS tty1::respawn:/bin/getty -L tty1 115200 vt100 console::askfirst:/bin/sh - it will link the /bin/sh process to /dev/console and the /bin/getty process to /dev/tty - when starting the container with lxc-start, each of these processes gets his own local /dev/pts device: 25609 ? Ss 0:00 lxc-start -n foo -d 25612 ? Ss 0:00 \_ init 25684 ? Ss 0:00 \_ /bin/syslogd 25686 pts/3 Ss+ 0:00 \_ /bin/getty -L tty1 115200 vt100 25687 pts/4 Ss+ 0:00 \_ init Here is the container.xml file I use to run with libvirt (you may ignore the console devices, since I don't know how much sense they make right now): <domain type='lxc'> <name>foo</name> <memory>500000</memory> <os> <type>exe</type> <init>/bin/init</init> </os> <vcpu>1</vcpu> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/libexec/libvirt_lxc</emulator> <filesystem type="mount"> <source dir="/usr/var/lib/lxc/foo/rootfs" /> <target dir="/" /> </filesystem> <interface type="bridge"> <source bridge="br0" /> <name>eth0</name> <ip address="10.171.73.128" netmask="255.255.255.0" /> </interface> <console type='pty' tty='/dev/pts/3'> <source path='/dev/pts/3' /> <target path='/dev/console' /> </console> <serial type='pty' tty='/dev/pts/4'> <source path='/dev/pts/4' /> <target path='/dev/tty1' /> </serial> </devices> </domain> When starting the container with virsh start foo, the firt init process gets a local /dev/pts device, and it is inherited by its children: [root@everest][~]# ps axf 25858 ? Ss 0:00 /usr/libexec/libvirt_lxc --name foo --console 20 --security=selinux --handshake 23 --background --veth veth1 25859 pts/0 Ss+ 0:00 \_ init 25870 ? Ss 0:00 \_ /bin/syslogd 25872 ? Ss 0:00 \_ /bin/getty -L tty1 115200 vt100 25873 ? Ss 0:00 \_ init [root@everest][~]# ls -l /proc/25872/fd total 0 lrwx------. 1 root root 64 Mar 14 07:09 0 -> /dev/pts/0 lrwx------. 1 root root 64 Mar 14 07:09 1 -> /dev/pts/0 lrwx------. 1 root root 64 Mar 14 07:09 2 -> /dev/pts/0 [root@everest][~]# ls -l /proc/25873/fd total 0 lrwx------. 1 root root 64 Mar 14 07:09 0 -> /dev/pts/0 lrwx------. 1 root root 64 Mar 14 07:09 1 -> /dev/pts/0 lrwx------. 1 root root 64 Mar 14 07:09 2 -> /dev/pts/0 What I tried to do in the .xml, and I would need, is a way to bind a local /dev/pts devices to child processes in the container. Explicitly: - bind /dev/pts/3 to <container_rootfs>/dev/console - bind /dev/pts/4 to <container_rootfs>/dev/tty1 I've found some documentation at [1], hence my trials in the presented .xml. Is my scenario possible / supported? Thank you very much, Bogdan P. [1] http://libvirt.org/formatdomain.html#elementsConsole
On Thu, Mar 14, 2013 at 11:10:31AM +0000, Purcareata Bogdan-B43198 wrote:
Hello all,
I am looking for a way to start a container with libvirt and bind multiple consoles to it. My container is running busybox, the filesystem created using the LXC package (0.8.0). My libvirt version is 0.10.1.
Configuration of the filesystem: - in the container rootfs, /dev contains 2 devices - /dev/console and /dev/tty1
Ok, there's no need to put anything in /dev. Libvirt will mount a tmpfs on /dev and a devpts on /dev/pts create all the devices nodes itself.
- the inittab file looks like this: ::sysinit:/etc/init.d/rcS tty1::respawn:/bin/getty -L tty1 115200 vt100 console::askfirst:/bin/sh - it will link the /bin/sh process to /dev/console and the /bin/getty process to /dev/tty - when starting the container with lxc-start, each of these processes gets his own local /dev/pts device:
I'm not sure how sf.net LXC tools setup consoles, but what you have won't work with Libvirt LXC. The primary console is setup as /dev/console and /dev/tty1, so your inittab here is putting both processes on the same console device. The second console is /dev/tty2 with libvirt.
I've found some documentation at [1], hence my trials in the presented .xml. Is my scenario possible / supported?
Here is the XML I have just used to test the scenario you want: <domain type='lxc'> <name>busyinit</name> <uuid>dda7b948-0846-1759-2873-0f375df7d7eb</uuid> <memory unit='KiB'>32768</memory> <currentMemory unit='KiB'>32768</currentMemory> <vcpu placement='static'>1</vcpu> <os> <type arch='x86_64'>exe</type> <init>/init</init> </os> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/libexec/libvirt_lxc</emulator> <filesystem type='mount' accessmode='passthrough'> <source dir='/root/containers/root-busybox'/> <target dir='/'/> </filesystem> <console type='pty'> <target type='serial' port='0'/> </console> <console type='pty'> <target type='serial' port='1'/> </console> <console type='pty'> <target type='serial' port='2'/> </console> <console type='pty'> <target type='serial' port='3'/> </console> </devices> <seclabel type='static' model='selinux' relabel='no'> <label>system_u:system_r:svirt_t:s0:c724,c995</label> </seclabel> </domain> And I have this inittab # cat /root/containers/root-busybox/etc/inittab ::sysinit:/etc/init.d/rcS tty1::askfirst:/bin/sh tty2::respawn:/bin/getty -L tty2 115200 vt100 tty3::respawn:/bin/getty -L tty3 115200 vt100 tty4::respawn:/bin/getty -L tty4 115200 vt100 Now if you start this 'virsh -c lxc:/// start busyinit' Then # virsh -c lxc:/// console busyinit Connected to domain busyinit Escape character is ^] Please press Enter to activate this console. /bin/sh: can't access tty; job control turned off # ... will connect you to the first console, which is tty1 and your /bin/sh To connect to the 2nd, 3rd, etc consoles you can use # virsh -c lxc:/// console --devname console1 busyinit Connected to domain busyinit Escape character is ^] t500wlan.home.berrange.com login: Use 'virsh -c lxc:/// dumpxml busyinit' to see what console device names libvirt gave to each - for LXC they'll be named 'console0', 'console1', 'console2', etc. NB the tty paths in the XML are the host side TTY names, which are different from the guest side names. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Thank you very much for the detailed help! Your setup worked flawlessly. Best regards, Bogdan P.
-----Original Message----- From: Daniel P. Berrange [mailto:berrange@redhat.com] Sent: Thursday, March 14, 2013 2:31 PM To: Purcareata Bogdan-B43198 Cc: libvirt-list@redhat.com Subject: Re: [libvirt] Libvirt + LXC terminals
On Thu, Mar 14, 2013 at 11:10:31AM +0000, Purcareata Bogdan-B43198 wrote:
Hello all,
I am looking for a way to start a container with libvirt and bind multiple consoles to it. My container is running busybox, the filesystem created using the LXC package (0.8.0). My libvirt version is 0.10.1.
Configuration of the filesystem: - in the container rootfs, /dev contains 2 devices - /dev/console and /dev/tty1
Ok, there's no need to put anything in /dev. Libvirt will mount a tmpfs on /dev and a devpts on /dev/pts create all the devices nodes itself.
- the inittab file looks like this: ::sysinit:/etc/init.d/rcS tty1::respawn:/bin/getty -L tty1 115200 vt100 console::askfirst:/bin/sh - it will link the /bin/sh process to /dev/console and the /bin/getty process to /dev/tty - when starting the container with lxc-start, each of these processes gets his own local /dev/pts device:
I'm not sure how sf.net LXC tools setup consoles, but what you have won't work with Libvirt LXC. The primary console is setup as /dev/console and /dev/tty1, so your inittab here is putting both processes on the same console device. The second console is /dev/tty2 with libvirt.
I've found some documentation at [1], hence my trials in the presented .xml. Is my scenario possible / supported?
Here is the XML I have just used to test the scenario you want:
<domain type='lxc'> <name>busyinit</name> <uuid>dda7b948-0846-1759-2873-0f375df7d7eb</uuid> <memory unit='KiB'>32768</memory> <currentMemory unit='KiB'>32768</currentMemory> <vcpu placement='static'>1</vcpu> <os> <type arch='x86_64'>exe</type> <init>/init</init> </os> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/libexec/libvirt_lxc</emulator> <filesystem type='mount' accessmode='passthrough'> <source dir='/root/containers/root-busybox'/> <target dir='/'/> </filesystem> <console type='pty'> <target type='serial' port='0'/> </console> <console type='pty'> <target type='serial' port='1'/> </console> <console type='pty'> <target type='serial' port='2'/> </console> <console type='pty'> <target type='serial' port='3'/> </console> </devices> <seclabel type='static' model='selinux' relabel='no'> <label>system_u:system_r:svirt_t:s0:c724,c995</label> </seclabel> </domain>
And I have this inittab
# cat /root/containers/root-busybox/etc/inittab ::sysinit:/etc/init.d/rcS tty1::askfirst:/bin/sh tty2::respawn:/bin/getty -L tty2 115200 vt100 tty3::respawn:/bin/getty -L tty3 115200 vt100 tty4::respawn:/bin/getty -L tty4 115200 vt100
Now if you start this 'virsh -c lxc:/// start busyinit'
Then
# virsh -c lxc:/// console busyinit Connected to domain busyinit Escape character is ^]
Please press Enter to activate this console. /bin/sh: can't access tty; job control turned off # ...
will connect you to the first console, which is tty1 and your /bin/sh
To connect to the 2nd, 3rd, etc consoles you can use
# virsh -c lxc:/// console --devname console1 busyinit Connected to domain busyinit Escape character is ^]
t500wlan.home.berrange.com login:
Use 'virsh -c lxc:/// dumpxml busyinit' to see what console device names libvirt gave to each - for LXC they'll be named 'console0', 'console1', 'console2', etc. NB the tty paths in the XML are the host side TTY names, which are different from the guest side names.
Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
participants (2)
-
Daniel P. Berrange -
Purcareata Bogdan-B43198