[libvirt] failed to mount cgroup
Hi, I tried to run libvirt-1.0.2 with LXC as below, but it failed to mount cgroup. 1). mount /cgroup # mkdir /cgroup # mount -t cgroup cgroup /cgroup # vi vm1.xml <domain type='lxc'> <name>vm1</name> <memory>32768</memory> <os> <type>exe</type> <init>/bin/sh</init> </os> <vcpu>1</vcpu> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/libexec/libvirt_lxc</emulator> <console type='pty' tty='/dev/pts/3'> <source path='/dev/pts/3'/> <target port='0'/> </console> </devices> </domain> # virsh -c lxc:/// define vm1.xml # virsh -c lxc:/// start vm1 error: Failed to start domain vm1 error: internal error guest failed to start: 2013-03-05 02:13:03.639+0000: 2403: info : libvirt version: 1.0.2 2013-03-05 02:13:03.639+0000: 2403: warning : lxcCapsInit:73 : Failed to get host power management capabilities PATH=/bin:/sbin TERM=linux container=lxc-libvirt container_uuid=d50aec75-4566-62b3-ed81-adf3e8705efe LIBVIRT_LXC_UUID=d50aec75-4566-62b3-ed81-adf3e8705efe LIBVIRT_LXC_NAME=vm1 /bin/sh 2013-03-05 02:13:03.779+0000: 1: error : lxcContainerIdentifyCGroups:1775 : Unable to read directory : No such file or directory 2013-03-05 02:13:03.779+0000: 2404: error : virLXCControllerRun:1468 : error receiving signal from container: Input/output error # ps aux | grep libvirtd root 2314 0.1 0.4 108320 4876 ? Sl 02:47 0:00 libvirtd -d root 2383 0.0 0.0 4116 452 ttyS0 S+ 02:52 0:00 grep libvirtd # cat /proc/2314/status Name: libvirtd State: S (sleeping) Tgid: 2314 Pid: 2314 PPid: 1 TracerPid: 0 Uid: 0 0 0 0 Gid: 0 0 0 0 FDSize: 32 Groups: 0 VmPeak: 112420 kB VmSize: 108320 kB VmLck: 0 kB VmHWM: 4884 kB VmRSS: 4876 kB VmData: 91540 kB VmStk: 136 kB VmExe: 348 kB VmLib: 13092 kB VmPTE: 96 kB VmSwap: 0 kB Threads: 11 SigQ: 0/7760 SigPnd: 0000000000000000 ShdPnd: 0000000000000000 SigBlk: 0000000000000000 SigIgn: 0000000000001000 SigCgt: 0000000180004cef CapInh: 0000000000000000 CapPrm: ffffffffffffffff CapEff: ffffffffffffffff CapBnd: ffffffffffffffff Cpus_allowed: 3 Cpus_allowed_list: 0-1 Mems_allowed: 1 Mems_allowed_list: 0 voluntary_ctxt_switches: 59 nonvoluntary_ctxt_switches: 24 # cat /proc/2314/cgroup 1:freezer,devices,memory,cpuacct,cpuset:/ # mount | grep cgroup cgroup on /cgroup type cgroup (rw,relatime,freezer,devices,memory,cpuacct,cpuset) # vi /var/log/libvirt/lxc/vm1.log 2013-03-05 02:48:39.218+0000: starting up PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin LIBVIRT_DEBUG=3 LIBVIRT_LOG_OUTPUTS=3:stderr /usr/libexec/libvirt_lxc --name vm1 --console 17 --security=none --handshake 20 --background 2013-03-05 02:48:39.229+0000: 2365: info : libvirt version: 1.0.2 2013-03-05 02:48:39.229+0000: 2365: warning : lxcCapsInit:73 : Failed to get host power management capabilities PATH=/bin:/sbin TERM=linux container=lxc-libvirt container_uuid=523492fb-36c8-3662-c61e-91697d5ad0ab LIBVIRT_LXC_UUID=523492fb-36c8-3662-c61e-91697d5ad0ab LIBVIRT_LXC_NAME=vm1 /bin/sh 2013-03-05 02:48:39.299+0000: 1: error : lxcContainerIdentifyCGroups:1775 : Unable to read directory : No such file or directory 2013-03-05 02:48:39.300+0000: 2366: error : virLXCControllerRun:1468 : error receiving signal from container: Input/output error But if mount cgroup on another poinit, the error message is different. 2). mount /dev/cgroups # mkdir /dev/cgroups # mount -t cgroup cgroup /dev/cgroups # virsh -c lxc:/// start vm1 # vi /var/log/libvirt/lxc/vm1.log 2013-03-05 03:15:49.660+0000: starting up PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin LIBVIRT_DEBUG=3 LIBVIRT_LOG_OUTPUTS=3:stderr /usr/libexec/libvirt_lxc --name vm1 --console 2013-03-05 03:15:49.666+0000: 2430: info : libvirt version: 1.0.2 2013-03-05 03:15:49.666+0000: 2430: warning : lxcCapsInit:73 : Failed to get host power management capabilities PATH=/bin:/sbin TERM=linux container=lxc-libvirt container_uuid=518d6735-42d5-90b0-3780-d41a651385fc LIBVIRT_LXC_UUID=518d6735-42d5-90b0-3780-d41a651385fc LI 2013-03-05 03:15:49.749+0000: 1: error : lxcContainerMountCGroups:1892 : Failed to mount cgroup on '/dev/cgroups': No such file or directory 2013-03-05 03:15:49.749+0000: 2431: error : virLXCControllerRun:1468 : error receiving signal from container: Input/output error Best Regards, Olivia
On 2013/03/05 14:43, Yin Olivia-R63875 wrote:
Hi,
I tried to run libvirt-1.0.2 with LXC as below, but it failed to mount cgroup.
You should change your cgroup configuration as below cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event) Libvirt lxc doesn't support configure cgroup as you did below now.
On Tue, Mar 05, 2013 at 05:29:02PM +0800, Gao feng wrote:
On 2013/03/05 14:43, Yin Olivia-R63875 wrote:
Hi,
I tried to run libvirt-1.0.2 with LXC as below, but it failed to mount cgroup.
You should change your cgroup configuration as below cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
Libvirt lxc doesn't support configure cgroup as you did below now.
Why? IIRC old systems mount cgroup at /cgroup by default. This means libvirt lxc has backward compatibility on supporting cgroup? And, in the case can libvirt lxc give a user-friendly message to user about what user can do?
On 2013/03/05 17:48, Hu Tao wrote:
On Tue, Mar 05, 2013 at 05:29:02PM +0800, Gao feng wrote:
On 2013/03/05 14:43, Yin Olivia-R63875 wrote:
Hi,
I tried to run libvirt-1.0.2 with LXC as below, but it failed to mount cgroup.
You should change your cgroup configuration as below cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
Libvirt lxc doesn't support configure cgroup as you did below now.
Why? IIRC old systems mount cgroup at /cgroup by default. This means libvirt lxc has backward compatibility on supporting cgroup?
YES Libvirt can successfully create cgroup for lxc doamin, the problem is we trying to mount cgroup in container and in special case we failed.
And, in the case can libvirt lxc give a user-friendly message to user about what user can do?
Yes, maybe this is the best solution.
On Tue, Mar 5, 2013 at 5:48 PM, Hu Tao <hutao@cn.fujitsu.com> wrote:
On Tue, Mar 05, 2013 at 05:29:02PM +0800, Gao feng wrote:
On 2013/03/05 14:43, Yin Olivia-R63875 wrote:
Hi,
I tried to run libvirt-1.0.2 with LXC as below, but it failed to mount cgroup.
You should change your cgroup configuration as below cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
Libvirt lxc doesn't support configure cgroup as you did below now.
Why? IIRC old systems mount cgroup at /cgroup by default. This means libvirt lxc has backward compatibility on supporting cgroup?
I think it is due to the upstream kernel change, which is merged in 2.6.36 version. Discussion thread here for your reference. http://thread.gmane.org/gmane.linux.kernel/1013171 -- Qiu Yu
There's no cgroug config file in rootfs on my embedded PPC platform, I could mount cgroups manually. Should it be hierarchy directory mounted? Or could also looks like: # Mount -t cgroup cgroup /sys/fs/cgroup # mount rootfs on / type rootfs (rw) /dev/root on / type ext2 (rw,relatime) proc on /proc type proc (rw,relatime) sysfs on /sys type sysfs (rw,relatime) none on /dev type tmpfs (rw,relatime,mode=755) devpts on /dev/pts type devpts (rw,relatime,gid=5,mode=620,ptmxmode=000) tmpfs on /var/volatile type tmpfs (rw,relatime) tmpfs on /media/ram type tmpfs (rw,relatime) cgroup on /sys/fs/cgroup type cgroup (rw,relatime,freezer,devices,memory,cpuacct,cpuset) But it also failed to mount cgroup on /sys/fs/cgroup # virsh -c lxc:/// start vm1 error: Failed to start domain vm1 error: internal error guest failed to start: 2013-03-07 08:31:02.418+0000: 2387: info : libvirt version: 1.0.3 2013-03-07 08:31:02.418+0000: 2387: warning : lxcCapsInit:73 : Failed to get host power management capabilities PATH=/bin:/sbin TERM=linux container=lxc-libvirt container_uuid=8b8492e0-5d90-441c-8822-c04b8e72bda3 LIBVIRT_LXC_UUID=8b8492e0-5d90-441c-8822-c04b8e72bda3 LIBVIRT_LXC_NAME=vm1 /bin/sh 2013-03-07 08:31:02.522+0000: 1: error : lxcContainerMountCGroups:1892 : Failed to mount cgroup on '/sys/fs/cgroup': No such file or directory 2013-03-07 08:31:02.522+0000: 2388: error : virLXCControllerRun:1468 : error receiving signal from container: Input/output error Best Regards, Olivia
-----Original Message----- From: Gao feng [mailto:gaofeng@cn.fujitsu.com] Sent: Tuesday, March 05, 2013 5:29 PM To: Yin Olivia-R63875 Cc: libvir-list@redhat.com Subject: Re: [libvirt] failed to mount cgroup
On 2013/03/05 14:43, Yin Olivia-R63875 wrote:
Hi,
I tried to run libvirt-1.0.2 with LXC as below, but it failed to mount cgroup.
You should change your cgroup configuration as below cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
Libvirt lxc doesn't support configure cgroup as you did below now.
On 2013/03/07 16:45, Yin Olivia-R63875 wrote:
cgroup on /sys/fs/cgroup type cgroup (rw,relatime,freezer,devices,memory,cpuacct,cpuset)
If you prefer to mount these subsystems together. you should try below steps: mkdir /sys/fs/cgroup/freezer,devices,memory,cpuacct,cpuset mount -t cgroup -ofreezer,devices,memory,cpuacct,cpuset cgroup /sys/fs/cgroup/freezer,devices,memory,cpuacct,cpuset Key is the name of mount point should equal to the cgroup subsystems.
Thanks, It works now. Best Regards, Olivia
-----Original Message----- From: Gao feng [mailto:gaofeng@cn.fujitsu.com] Sent: Thursday, March 07, 2013 4:58 PM To: Yin Olivia-R63875 Cc: libvir-list@redhat.com Subject: Re: [libvirt] failed to mount cgroup
On 2013/03/07 16:45, Yin Olivia-R63875 wrote:
cgroup on /sys/fs/cgroup type cgroup (rw,relatime,freezer,devices,memory,cpuacct,cpuset)
If you prefer to mount these subsystems together. you should try below steps:
mkdir /sys/fs/cgroup/freezer,devices,memory,cpuacct,cpuset
mount -t cgroup -ofreezer,devices,memory,cpuacct,cpuset cgroup /sys/fs/cgroup/freezer,devices,memory,cpuacct,cpuset
Key is the name of mount point should equal to the cgroup subsystems.
Exactly, the mount point could also be /cgroup. The difference is the mount type. Scenario 1 could work with both libvirt-0.10.x and higher version. Scenario 2 could work with libvirt-0.10.x but failed with higher version. What's limitation with 'mount -t cgroup cgroup /cgroup'? 1. # mount -t tmpfs cgroup /cgroup # mkdir /cgroup/{freezer,devices,memory,cpuacct,cpuset} # mount -t cgroup -ofreezer cgroup /cgroup/freezer # mount -t cgroup -odevices cgroup /cgroup/devices # mount -t cgroup -omemory cgroup /cgroup/memory # mount -t cgroup -ocpuacct cgroup /cgroup/cpuacct # mount -t cgroup -ocpuset cgroup /cgroup/cpuset # cat /proc/mounts rootfs / rootfs rw 0 0 /dev/root / ext2 rw,relatime 0 0 proc /proc proc rw,relatime 0 0 sysfs /sys sysfs rw,relatime 0 0 none /dev tmpfs rw,relatime,mode=755 0 0 devpts /dev/pts devpts rw,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /var/volatile tmpfs rw,relatime 0 0 tmpfs /media/ram tmpfs rw,relatime 0 0 cgroup /cgroup tmpfs rw,relatime 0 0 cgroup /cgroup/freezer cgroup rw,relatime,freezer 0 0 cgroup /cgroup/devices cgroup rw,relatime,devices 0 0 cgroup /cgroup/memory cgroup rw,relatime,memory 0 0 cgroup /cgroup/cpuacct cgroup rw,relatime,cpuacct 0 0 cgroup /cgroup/cpuset cgroup rw,relatime,cpuset 0 0 # mount rootfs on / type rootfs (rw) /dev/root on / type ext2 (rw,relatime) proc on /proc type proc (rw,relatime) sysfs on /sys type sysfs (rw,relatime) none on /dev type tmpfs (rw,relatime,mode=755) devpts on /dev/pts type devpts (rw,relatime,gid=5,mode=620,ptmxmode=000) tmpfs on /var/volatile type tmpfs (rw,relatime) tmpfs on /media/ram type tmpfs (rw,relatime) cgroup on /cgroup type tmpfs (rw,relatime) cgroup on /cgroup/freezer type cgroup (rw,relatime,freezer) cgroup on /cgroup/devices type cgroup (rw,relatime,devices) cgroup on /cgroup/memory type cgroup (rw,relatime,memory) cgroup on /cgroup/cpuacct type cgroup (rw,relatime,cpuacct) cgroup on /cgroup/cpuset type cgroup (rw,relatime,cpuset) # cat /proc/cgroups #subsys_name hierarchy num_cgroups enabled cpuset 6 1 1 cpuacct 5 1 1 memory 4 1 1 devices 3 1 1 freezer 2 1 1 2. # mkdir /cgroup # mount -t cgroup cgroup /cgroup # cat /proc/mounts rootfs / rootfs rw 0 0 /dev/root / ext2 rw,relatime 0 0 proc /proc proc rw,relatime 0 0 sysfs /sys sysfs rw,relatime 0 0 none /dev tmpfs rw,relatime,mode=755 0 0 devpts /dev/pts devpts rw,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /var/volatile tmpfs rw,relatime 0 0 tmpfs /media/ram tmpfs rw,relatime 0 0 cgroup /cgroup cgroup rw,relatime,freezer,devices,memory,cpuacct,cpuset 0 0 # mount rootfs on / type rootfs (rw) /dev/root on / type ext2 (rw,relatime) proc on /proc type proc (rw,relatime) sysfs on /sys type sysfs (rw,relatime) none on /dev type tmpfs (rw,relatime,mode=755) devpts on /dev/pts type devpts (rw,relatime,gid=5,mode=620,ptmxmode=000) tmpfs on /var/volatile type tmpfs (rw,relatime) tmpfs on /media/ram type tmpfs (rw,relatime) cgroup on /cgroup type cgroup (rw,relatime,freezer,devices,memory,cpuacct,cpuset) # cat /proc/cgroups #subsys_name hierarchy num_cgroups enabled cpuset 1 1 1 cpuacct 1 1 1 memory 1 1 1 devices 1 1 1 freezer 1 1 1 Best Regards, Olivia
-----Original Message----- From: Gao feng [mailto:gaofeng@cn.fujitsu.com] Sent: Thursday, March 07, 2013 4:58 PM To: Yin Olivia-R63875 Cc: libvir-list@redhat.com Subject: Re: [libvirt] failed to mount cgroup
On 2013/03/07 16:45, Yin Olivia-R63875 wrote:
cgroup on /sys/fs/cgroup type cgroup (rw,relatime,freezer,devices,memory,cpuacct,cpuset)
If you prefer to mount these subsystems together. you should try below steps:
mkdir /sys/fs/cgroup/freezer,devices,memory,cpuacct,cpuset
mount -t cgroup -ofreezer,devices,memory,cpuacct,cpuset cgroup /sys/fs/cgroup/freezer,devices,memory,cpuacct,cpuset
Key is the name of mount point should equal to the cgroup subsystems.
participants (4)
-
Gao feng -
Hu Tao -
unicell -
Yin Olivia-R63875