On Wed, Apr 19, 2017 at 09:03:55AM -0600, Jim Fehlig wrote:
I recently received a report of a libvirt+Xen installation reaching the pids cgroup controller TasksMax limit
kernel: [71282.213347] cgroup: fork rejected by pids controller in /system.slice/libvirtd.service
The default setting of TaskMax is 512 on this system
# cat /sys/fs/cgroup/pids/system.slice/libvirtd.service/pids.max 512
Depending on domain type and configuration, I've noticed between 5-7 tasks (IO threads, qemu process, etc.) are created when starting a domain. It doesn't take too many domains before the 512 limit is reached.
LimitNOFILE was recently changed by commit 27cd763500 to support 4096 domains. Following similar logic, would it be ok to increase TasksMax to 32768? That would accommodate 4096 domains with 8 tasks each. TasksMax also supports the special value of "infinity", but that seems a bit aggressive to me.
KVM uses 1 thread per vCPU, so 8 pids is probably somewhat on the low side if considering high vcpu counts. I think we could go for something like 128k pids which allows for closer to 32 vcpus per guest Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|