8:23 a.m.
Hi all
I have CentOS Linux release 7.0.1406, libvirt 1.2.7 installed.
Just after create and start inside LXC container present cgroups.
Example for memory:
[root@ce7-t1 /]# ls -la /sys/fs/cgroup/memory/
total 0
drwxr-xr-x 2 root root 0 Sep 15 17:14 .
drwxr-xr-x 12 root root 280 Sep 15 17:14 ..
-rw-r--r-- 1 root root 0 Sep 15 17:14 cgroup.clone_children
--w--w--w- 1 root root 0 Sep 15 17:14 cgroup.event_control
-rw-r--r-- 1 root root 0 Sep 15 17:15 cgroup.procs
-rw-r--r-- 1 root root 0 Sep 15 17:14 memory.failcnt
--w------- 1 root root 0 Sep 15 17:14 memory.force_empty
-rw-r--r-- 1 root root 0 Sep 15 17:14 memory.kmem.failcnt
-rw-r--r-- 1 root root 0 Sep 15 17:14 memory.kmem.limit_in_bytes
-rw-r--r-- 1 root root 0 Sep 15 17:14 memory.kmem.max_usage_in_bytes
-r--r--r-- 1 root root 0 Sep 15 17:14 memory.kmem.slabinfo
-rw-r--r-- 1 root root 0 Sep 15 17:14 memory.kmem.tcp.failcnt
-rw-r--r-- 1 root root 0 Sep 15 17:14 memory.kmem.tcp.limit_in_bytes
-rw-r--r-- 1 root root 0 Sep 15 17:14 memory.kmem.tcp.max_usage_in_bytes
-r--r--r-- 1 root root 0 Sep 15 17:14 memory.kmem.tcp.usage_in_bytes
-r--r--r-- 1 root root 0 Sep 15 17:14 memory.kmem.usage_in_bytes
-rw-r--r-- 1 root root 0 Sep 15 17:14 memory.limit_in_bytes
-rw-r--r-- 1 root root 0 Sep 15 17:14 memory.max_usage_in_bytes
-rw-r--r-- 1 root root 0 Sep 15 17:14 memory.memsw.failcnt
-rw-r--r-- 1 root root 0 Sep 15 17:14 memory.memsw.limit_in_bytes
-rw-r--r-- 1 root root 0 Sep 15 17:14 memory.memsw.max_usage_in_bytes
-r--r--r-- 1 root root 0 Sep 15 17:14 memory.memsw.usage_in_bytes
-rw-r--r-- 1 root root 0 Sep 15 17:14 memory.move_charge_at_immigrate
-r--r--r-- 1 root root 0 Sep 15 17:14 memory.numa_stat
-rw-r--r-- 1 root root 0 Sep 15 17:14 memory.oom_control
---------- 1 root root 0 Sep 15 17:14 memory.pressure_level
-rw-r--r-- 1 root root 0 Sep 15 17:14 memory.soft_limit_in_bytes
-r--r--r-- 1 root root 0 Sep 15 17:14 memory.stat
-rw-r--r-- 1 root root 0 Sep 15 17:14 memory.swappiness
-r--r--r-- 1 root root 0 Sep 15 17:14 memory.usage_in_bytes
-rw-r--r-- 1 root root 0 Sep 15 17:14 memory.use_hierarchy
-rw-r--r-- 1 root root 0 Sep 15 17:14 notify_on_release
-rw-r--r-- 1 root root 0 Sep 15 17:14 tasks
Command "free" inside LXC showed almost normal values:
[root@ce7-t1 /]# free
total used free shared buffers cached
Mem: 1048576 32972 1015604 4473848 0 -4445364
-/+ buffers/cache: 4478336 -3429760
Swap: 1048576 0 1048576
(some problem with negative values)
After unpredictable time passed (1-5 day ?), cgroups inside LXC
magicaly removed. "free" in such containers show 2^53-1 as maximum
values:
[root@puppet01 /]# free
total used free shared buffers cached
Mem: 9007199254740991 591180 9007199254149811 0
0 267924
-/+ buffers/cache: 323256 9007199254417735
Swap: 0 0 0
And no more any cgroups presented at least in memory category:
[root@puppet01 /]# ls -la /sys/fs/cgroup/memory/
total 0
b.r.
Maxim Kozin
9:18 a.m.
HI all
After unpredictable time passed (1-5 day ?), cgroups inside LXC
magicaly removed.
virsh dumpxml config look like this:
<domain type='lxc' id='3566'>
<name>puppet</name>
<uuid>6d49b280-5686-4e3c-b048-1b5d362fb137</uuid>
<memory unit='KiB'>8388608</memory>
<currentMemory unit='KiB'>8388608</currentMemory>
<memtune>
<hard_limit unit='KiB'>8388608</hard_limit>
<soft_limit unit='KiB'>8388608</soft_limit>
<swap_hard_limit unit='KiB'>9437184</swap_hard_limit>
</memtune>
<vcpu placement='static'>2</vcpu>
<resource>
<partition>/machine</partition>
</resource>
<os>
<type arch='x86_64'>exe</type>
<init>/sbin/init</init>
</os>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>/usr/libexec/libvirt_lxc</emulator>
<filesystem type='block' accessmode='passthrough'>
<source dev='/dev/data/puppet'/>
<target dir='/'/>
</filesystem>
<interface type='direct'>
<mac address='02:00:00:25:a7:56'/>
<source dev='br502' mode='bridge'/>
<model type='virtio'/>
</interface>
<console type='pty' tty='/dev/pts/1'>
<source path='/dev/pts/1'/>
<target type='lxc' port='0'/>
<alias name='console0'/>
</console>
</devices>
<seclabel type='none'/>
</domain>
Problem go away after restart LXC container. But occurred again after
some time passed.
For containers with broken cgroups virsh memtune fails with error:
[root@]# virsh memtune --hard-limit 8G puppet --live --config
error: Unable to change memory parameters
error: unable to set memory hard_limit tunable: Operation not permitted
b.r.
Maxim Kozin
8:41 a.m.
>After unpredictable time passed (1-5 day ?), cgroups inside LXC
>magicaly removed.
I use next workaround for such containers:
1) restore subgroup in cgroup hierarchy :
[root@]# mkdir /sys/fs/cgroup/memory/machine.slice/machine-lxc\\x2dpuppet.scope
2) put value in limits:
[root@]# echo 8589934592 >
/sys/fs/cgroup/memory/machine.slice/machine-lxc\\x2dpuppet.scope/memory.limit_in_bytes
[root@]# echo 9663676416 >
/sys/fs/cgroup/memory/machine.slice/machine-lxc\\x2dphosgene7.infra.scope/memory.memsw.limit_in_bytes
[root@]# echo 8589934592 >
/sys/fs/cgroup/memory/machine.slice/machine-lxc\\x2dpuppet.scope/memory.kmem.limit_in_bytes
[root@]# echo 8589934592 >
/sys/fs/cgroup/memory/machine.slice/machine-lxc\\x2dpuppet.scope/memory.kmem.tcp.limit_in_bytes
[root@]# echo 8589934592 >
/sys/fs/cgroup/memory/machine.slice/machine-lxc\\x2dpuppet.scope/memory.soft_limit_in_bytes
3) check that inside container limits still unset:
[root@puppet01 /]# free
total used free shared buffers cached
Mem: 9007199254740991 1591360 9007199253149631 0
0 913112
-/+ buffers/cache: 678248 9007199254062743
Swap: 0 0 0
4) get libvirtd pid of such container:
[root@]# ps ax | grep puppet | grep libvirt
3142 ? Ssl 0:17 /usr/libexec/libvirt_lxc --name puppet
--console 21 --security=none --handshake 27 --background --veth
macvlan0
Moreover, you need get pid of all process existed in LXC. We would test
in ssh sessions, because get pid of sshd run in our container.
[root@]# pstree -ap 3142 | grep ssh
| `-sshd,5527
5) Assign pid with cgroups:
[root@]# cgclassify -g
memory:machine.slice/machine-lxc\\x2dpuppet.scope 3142 3143 5527
(I put pid 3143 in list - it's init inside LXC)
6) check again in container - start new ssh session:
[root@puppet01 /]# free
total used free shared buffers cached
Mem: 8388608 392 8388216 0 0 0
-/+ buffers/cache: 392 8388216
Swap: 1048576 0 1048576
7) compile and run loadmem:
#include <stdlib.h>
int main() {
int *p;
while(1) {
int inc=1024*1024*sizeof(char);
p=(int*) calloc(1,inc);
if(!p) break;
}
}
After 8G allocated process loadmem would be killed
b.r.
Maxim Kozin
3721
days inactive
3724
days old
2 comments
1 participants
participants (1)
-
mxs kolo