Hi All, On F12 i was playing with libvirt support for container. I am seeing two issue over here: 1> Not able to create a container when start cgconfig services. 2> When i stop cgconfig, then able to create a container, but seeing few error messages to libvirtd daemon. 3> After creation of container through libvirt, i am not able to touch any file or directory. Attached: vm1.xml , through which i created container. I referred following link: http://libvirt.org/drvlxc.html http://openvz.org/pipermail/devel/2008-September/014314.html Detailed Bug Report: When i create a container without starting "cgconfig" service, i am able to create but getting some error message in libvirtd daemon: [root@mx3950 ~]# libvirtd 14:47:24.556: warning : qemudStartup:565 : Unable to create cgroup for driver: No such device or address 14:47:24.647: warning : lxcStartup:1462 : Unable to create cgroup for driver: No such device or address libvir: Linux Container error : failed to connect to client socket: No such file or directory libvir: Linux Container error : failed to connect to client socket: No such file or directory libvir: Linux Container error : failed to connect to client socket: No such file or directory But when i start cgconfig service, then i am not able to open console for my container. Below is the detailed steps which i executed. [root@mx3950 ~]# /etc/init.d/cgconfig stop Stopping cgconfig service: [ OK ] [root@mx3950 ~]# virsh -c lxc:/// define vm1.xml error: Failed to define domain from vm1.xml error: operation failed: domain 'vm1' is already defined with uuid ef558ab3-54ea-5b95-205d-8c0096ae9a4f [root@mx3950 ~]# virsh -c lxc:/// start vm1 Domain vm1 started [root@mx3950 ~]# virsh -c lxc:/// console vm1 Connected to domain vm1 Escape character is ^] sh-4.0# ps -ef UID PID PPID C STIME TTY TIME CMD root 1 0 0 15:23 pts/4 00:00:00 /bin/sh root 4 1 0 15:23 pts/4 00:00:00 ps -ef sh-4.0# touch rishi touch: cannot touch `rishi': Permission denied sh-4.0# exit exit [root@mx3950 ~]# /etc/init.d/cgconfig start Starting cgconfig service: [ OK ] [root@mx3950 ~]# virsh -c lxc:/// start vm1 Domain vm1 started [root@mx3950 ~]# virsh -c lxc:/// console vm1 Connected to domain vm1 Escape character is ^] 15:24:12.869: error : vshRunConsole:77 : unable to open tty /dev/pts/3: No such file or directory [root@mx3950 ~]# Package List: [root@mx3950 ~]# rpm -qa | grep -e libcgroup -e libvirt libcgroup-0.34-2.fc12.x86_64 libvirt-0.7.1-1.fc12.x86_64 libvirt-client-0.7.1-1.fc12.x86_64 libvirt-python-0.7.1-1.fc12.x86_64 [root@mx3950 ~]# uname -a Linux mx3950.in.ibm.com 2.6.31-14.fc12.x86_64 #1 SMP Tue Sep 15 03:48:57 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux ------------------------------------------------------------------------ -- Libvir-list mailing list Libvir-list(a)redhat.com https://www.redhat.com/mailman/listinfo/libvir-list

R> can you help me here ? Probably not, I'm not familiar with cgconfig nor the latest cgroup support in libvirt. R> 14:47:24.556: warning : qemudStartup:565 : Unable to create cgroup R> for driver: No such device or address My guess is that this means you don't have the cgroup filesystem mounted, or it's mounted in a way that prevents libvirt from doing its work. IIRC, there are things you can do to the system that require a reboot to undo. If cgconfig is setting up the cgroups in such a way, you might need to reboot between stopping cgconfig and trying libvirt by itself.

R> But when i start cgconfig service, then i am not able to open R> console for my container. Again, knowing nothing of cgconfig, I can only speculate that it's doing something to prevent accessing the console pty.

I would push on the libcgroup folks to help figure out what it's doing (either by default or with your config) to prevent that and then go from there.

On Wed, Oct 07, 2009 at 08:12:02PM +0530, Dhaval Giani wrote: > On Wed, Oct 07, 2009 at 06:57:48AM -0700, Dan Smith wrote: > > R> can you help me here ? > > > > Probably not, I'm not familiar with cgconfig nor the latest cgroup > > support in libvirt. > > > > R> 14:47:24.556: warning : qemudStartup:565 : Unable to create cgroup > > R> for driver: No such device or address > > > > My guess is that this means you don't have the cgroup filesystem > > mounted, or it's mounted in a way that prevents libvirt from doing its > > work. IIRC, there are things you can do to the system that require a > > reboot to undo. If cgconfig is setting up the cgroups in such a way, > > you might need to reboot between stopping cgconfig and trying libvirt > > by itself. > > > > Or the cgroups are being setup in funny fashion. Rishi, what does your > config file look like? > > > R> But when i start cgconfig service, then i am not able to open > > R> console for my container. > > > > Again, knowing nothing of cgconfig, I can only speculate that it's > > doing something to prevent accessing the console pty. > > > > cgconfig does nothing apart from setup the cgroup filesystem as > mentioned in the configuration files. > > > I would push on the libcgroup folks to help figure out what it's doing > > (either by default or with your config) to prevent that and then go > > from there. > > > > Would appreciate any help from the libvirt folks who have worked on the > cgroup support in libvirt. The most likely suspect is that the cgroups controllers were mounted or unmounted after libvirtd started. A restart of libvirtd ought to resolve the problem if this is indeed the case

Hi daniel, Sorry for late response, was busy with some other high priority task. Daniel P. Berrange wrote: > On Wed, Oct 07, 2009 at 08:12:02PM +0530, Dhaval Giani wrote: > > The most likely suspect is that the cgroups controllers were mounted or > unmounted after libvirtd started. A restart of libvirtd ought to resolve > the problem if this is indeed the case > >

Actually not , restarting libvirt daemon did not help me. I had controllers mounted as per /proc/mount . And i am really seeing an issue after starting cgconfig services on my system. Shell :1 virsh # start vm1 Domain vm1 started virsh # list --all Id Name State ---------------------------------- - vm1 shut off virsh # Shell 2: [root@mls41 ~]# service libvirtd restart Stopping libvirtd daemon: [ OK ] Starting libvirtd daemon: [ OK ] [root@mls41 ~]# cat /proc/mounts | grep cgroup cgroup /cgroup cgroup rw,relatime,cpuacct,cpu 0 0 [root@mls41 ~]# Let me know if you need some more info. -Rishi > Daniel; > > ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ Libcg-devel mailing list Libcg-devel(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libcg-devel

Rishikesh wrote:

Cgroup config file & controllers mount shows like this : [root@mhs21a ~]# cat /proc/mounts | grep cgroup cgroup /cgroups/cpu cgroup rw,relatime,cpu 0 0 cgroup /cgroups/cpuacct cgroup rw,relatime,cpuacct 0 0 cgroup /cgroups/memory cgroup rw,relatime,memory 0 0

> virsh # start vm1 > error: Failed to start domain vm1 > error: Failed to read pid file /var/run/libvirt/lxc/vm1.pid: No such file > or directory

On Wed, Oct 14, 2009 at 06:07:26AM +0900, Ryota Ozaki wrote: > Hi Rishikesh, > > On Tue, Oct 13, 2009 at 3:09 PM, Rishikesh <risrajak(a)linux.vnet.ibm.com&gt; wrote: > >> Rishikesh wrote: >> >> Cgroup config file & controllers mount shows like this : >> [root@mhs21a ~]# cat /proc/mounts | grep cgroup >> cgroup /cgroups/cpu cgroup rw,relatime,cpu 0 0 >> cgroup /cgroups/cpuacct cgroup rw,relatime,cpuacct 0 0 >> cgroup /cgroups/memory cgroup rw,relatime,memory 0 0 >> > AFAIW, libvirt requires "device" subsystem of cgroups as well. > Without it, an assistant subprocess of libvirtd will die soon. > > So rishi, could add a devices line as well to the cgconfig configuratoin file, and test it out?

Hi Rishikesh, On Tue, Oct 13, 2009 at 3:09 PM, Rishikesh <risrajak(a)linux.vnet.ibm.com&gt; wrote: > Rishikesh wrote: > Cgroup config file & controllers mount shows like this : > [root@mhs21a ~]# cat /proc/mounts | grep cgroup > cgroup /cgroups/cpu cgroup rw,relatime,cpu 0 0 > cgroup /cgroups/cpuacct cgroup rw,relatime,cpuacct 0 0 > cgroup /cgroups/memory cgroup rw,relatime,memory 0 0 AFAIW, libvirt requires "device" subsystem of cgroups as well. Without it, an assistant subprocess of libvirtd will die soon.

Dhaval Giani wrote: > On Wed, Oct 07, 2009 at 06:57:48AM -0700, Dan Smith wrote: > >> R> can you help me here ? >> >> Probably not, I'm not familiar with cgconfig nor the latest cgroup >> support in libvirt. >> >> R> 14:47:24.556: warning : qemudStartup:565 : Unable to create cgroup >> R> for driver: No such device or address >> >> My guess is that this means you don't have the cgroup filesystem >> mounted, or it's mounted in a way that prevents libvirt from doing its >> work. IIRC, there are things you can do to the system that require a >> reboot to undo. If cgconfig is setting up the cgroups in such a way, >> you might need to reboot between stopping cgconfig and trying libvirt >> by itself. >> >> > > Or the cgroups are being setup in funny fashion. Rishi, what does your > config file look like? > I am attaching the config file.

group default { perm { admin { uid = root; git = root; } task { uid = root; gid = root; } } cpu { cpu.shares = 1024; cpu.rt_runtime_us = 920000; } } group test { perm { admin { uid = test; gid = test; } task { uid = test; gid = test; } } cpu { cpu.shares = 4096; cpu.rt_runtime_us = 30000; } } mount { cpu = /cgroup; cpuacct = /cgroup; }

I'm assuming that 'cgconfig' mounts or unmounts cgroups controllers. The libvirt daemon only checks where they are all mounted when it initially starts up. So it won't be at all happy if you mount/unmount cgroups while it is running. If you restarts libvirtd after this it would pick up the changes