[libvirt-users] can't establish more than 1000 connections with virsh
Hi: I'm trying to connect with libvirtd with virsh from a remote PC,but only can establish 1000 connections. If try more connections,prompt: "error: failed to connect to the hypervisor error: Failed to open file '/etc/libvirt/libvirt.conf': Too many open files" I tried in another PC,also get the same prompt. Anybody know why? Thanks! environment: libvirt:libvirt-2.0.0-10.el7 OS:centos7.3
On Thu, May 25, 2017 at 06:20:51PM +0800, dw wrote:
Hi:
I'm trying to connect with libvirtd with virsh from a remote PC,but only can establish 1000 connections.
If try more connections,prompt:
"error: failed to connect to the hypervisor error: Failed to open file '/etc/libvirt/libvirt.conf': Too many open files"
I tried in another PC,also get the same prompt.
Anybody know why?
There's a limit of 1024 open file handles by default on Linux. You would have to raise that limit in /etc/security/limits.d/ for your user. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
在 2017年05月25日 18:37, Daniel P. Berrange 写道:
On Thu, May 25, 2017 at 06:20:51PM +0800, dw wrote:
Hi:
I'm trying to connect with libvirtd with virsh from a remote PC,but only can establish 1000 connections.
If try more connections,prompt:
"error: failed to connect to the hypervisor error: Failed to open file '/etc/libvirt/libvirt.conf': Too many open files"
I tried in another PC,also get the same prompt.
Anybody know why? There's a limit of 1024 open file handles by default on Linux. You would have to raise that limit in /etc/security/limits.d/ for your user.
Regards, Daniel
In /etc/security/limits.d/,there is only 20-nproc.conf. I modified /etc/security/limits.conf,what I added is: "* soft nofile 81920 * hard nofile 81920" I also modified /etc/pam.d/login,that I added is: "session required pam_limits.so" The output of "ulimit -a" is: core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 385898 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 81920 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 385898 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited What I did is right?
On Thu, May 25, 2017 at 06:51:40PM +0800, dw wrote:
在 2017年05月25日 18:37, Daniel P. Berrange 写道:
On Thu, May 25, 2017 at 06:20:51PM +0800, dw wrote:
Hi:
I'm trying to connect with libvirtd with virsh from a remote PC,but only can establish 1000 connections.
If try more connections,prompt:
"error: failed to connect to the hypervisor error: Failed to open file '/etc/libvirt/libvirt.conf': Too many open files"
I tried in another PC,also get the same prompt.
Anybody know why? There's a limit of 1024 open file handles by default on Linux. You would have to raise that limit in /etc/security/limits.d/ for your user.
In /etc/security/limits.d/,there is only 20-nproc.conf.
You can create any files you want in that sub-directory - they just use the same syntax as limits.conf. It means that if RPM upgrades limits.conf it doesn't interfere with your changes in limits.d/custom.conf
I modified /etc/security/limits.conf,what I added is:
"* soft nofile 81920 * hard nofile 81920"
I also modified /etc/pam.d/login,that I added is:
"session required pam_limits.so"
I don't think that's needed - its already activated via pam.d/system-auth which is included from login.
The output of "ulimit -a" is:
core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 385898 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 81920 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 385898 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
What I did is right?
That at least shows the new limit is active Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
在 2017年05月25日 18:54, Daniel P. Berrange 写道:
在 2017年05月25日 18:37, Daniel P. Berrange 写道:
On Thu, May 25, 2017 at 06:20:51PM +0800, dw wrote:
Hi:
I'm trying to connect with libvirtd with virsh from a remote PC,but only can establish 1000 connections.
If try more connections,prompt:
"error: failed to connect to the hypervisor error: Failed to open file '/etc/libvirt/libvirt.conf': Too many open files"
I tried in another PC,also get the same prompt.
Anybody know why? There's a limit of 1024 open file handles by default on Linux. You would have to raise that limit in /etc/security/limits.d/ for your user.
In /etc/security/limits.d/,there is only 20-nproc.conf. You can create any files you want in that sub-directory - they just use the same syntax as limits.conf. It means that if RPM upgrades
On Thu, May 25, 2017 at 06:51:40PM +0800, dw wrote: limits.conf it doesn't interfere with your changes in limits.d/custom.conf
I modified /etc/security/limits.conf,what I added is:
"* soft nofile 81920 * hard nofile 81920"
I also modified /etc/pam.d/login,that I added is:
"session required pam_limits.so" I don't think that's needed - its already activated via pam.d/system-auth which is included from login.
The output of "ulimit -a" is:
core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 385898 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 81920 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 385898 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
What I did is right? That at least shows the new limit is active
Regards, Daniel
I restored /etc/security/limits.conf and /etc/pam.d/login. And added file /etc/security/limits.d/21-nofile.conf, the content is * soft nofile 10240 * hard nofile 10240" The output of "ulimit -a" now is: core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 385898 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 10240 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 385898 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited The related config of /etc/libvirt/libvirtd.conf is: max_clients = 5000 max_queued_clients = 2000 min_workers = 200 max_workers = 2000 max_requests = 5000 max_client_requests = 2000 Now I still get the error: error: failed to connect to the hypervisor error: Failed to open file '/etc/libvirt/libvirt.conf': Too many open files
在 2017年05月26日 09:42, dw 写道:
在 2017年05月25日 18:54, Daniel P. Berrange 写道:
在 2017年05月25日 18:37, Daniel P. Berrange 写道:
On Thu, May 25, 2017 at 06:20:51PM +0800, dw wrote:
Hi:
I'm trying to connect with libvirtd with virsh from a remote PC,but only can establish 1000 connections.
If try more connections,prompt:
"error: failed to connect to the hypervisor error: Failed to open file '/etc/libvirt/libvirt.conf': Too many open files"
I tried in another PC,also get the same prompt.
Anybody know why? There's a limit of 1024 open file handles by default on Linux. You would have to raise that limit in /etc/security/limits.d/ for your user.
In /etc/security/limits.d/,there is only 20-nproc.conf. You can create any files you want in that sub-directory - they just use the same syntax as limits.conf. It means that if RPM upgrades
On Thu, May 25, 2017 at 06:51:40PM +0800, dw wrote: limits.conf it doesn't interfere with your changes in limits.d/custom.conf
I modified /etc/security/limits.conf,what I added is:
"* soft nofile 81920 * hard nofile 81920"
I also modified /etc/pam.d/login,that I added is:
"session required pam_limits.so" I don't think that's needed - its already activated via pam.d/system-auth which is included from login.
The output of "ulimit -a" is:
core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 385898 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 81920 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 385898 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
What I did is right? That at least shows the new limit is active
Regards, Daniel
I restored /etc/security/limits.conf and /etc/pam.d/login.
And added file /etc/security/limits.d/21-nofile.conf, the content is
* soft nofile 10240 * hard nofile 10240"
The output of "ulimit -a" now is:
core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 385898 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 10240 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 385898 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
The related config of /etc/libvirt/libvirtd.conf is:
max_clients = 5000 max_queued_clients = 2000 min_workers = 200 max_workers = 2000 max_requests = 5000 max_client_requests = 2000
Now I still get the error:
error: failed to connect to the hypervisor error: Failed to open file '/etc/libvirt/libvirt.conf': Too many open files
I tried twice,and the result is: virsh -c qemu+tcp://remote_pc/system error: failed to connect to the hypervisor error: Failed to open file '/etc/libvirt/libvirt.conf': Too many open files virsh -c qemu+ssh://remote_pc/system error: failed to connect to the hypervisor error: Failed to open file '/proc/4931/stat': Too many open files
在 2017年05月26日 09:48, dw 写道:
在 2017年05月26日 09:42, dw 写道:
在 2017年05月25日 18:54, Daniel P. Berrange 写道:
在 2017年05月25日 18:37, Daniel P. Berrange 写道:
On Thu, May 25, 2017 at 06:20:51PM +0800, dw wrote:
Hi:
I'm trying to connect with libvirtd with virsh from a remote PC,but only can establish 1000 connections.
If try more connections,prompt:
"error: failed to connect to the hypervisor error: Failed to open file '/etc/libvirt/libvirt.conf': Too many open files"
I tried in another PC,also get the same prompt.
Anybody know why? There's a limit of 1024 open file handles by default on Linux. You would have to raise that limit in /etc/security/limits.d/ for your user.
In /etc/security/limits.d/,there is only 20-nproc.conf. You can create any files you want in that sub-directory - they just use the same syntax as limits.conf. It means that if RPM upgrades
On Thu, May 25, 2017 at 06:51:40PM +0800, dw wrote: limits.conf it doesn't interfere with your changes in limits.d/custom.conf
I modified /etc/security/limits.conf,what I added is:
"* soft nofile 81920 * hard nofile 81920"
I also modified /etc/pam.d/login,that I added is:
"session required pam_limits.so" I don't think that's needed - its already activated via pam.d/system-auth which is included from login.
The output of "ulimit -a" is:
core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 385898 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 81920 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 385898 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
What I did is right? That at least shows the new limit is active
Regards, Daniel
I restored /etc/security/limits.conf and /etc/pam.d/login.
And added file /etc/security/limits.d/21-nofile.conf, the content is
* soft nofile 10240 * hard nofile 10240"
The output of "ulimit -a" now is:
core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 385898 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 10240 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 385898 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
The related config of /etc/libvirt/libvirtd.conf is:
max_clients = 5000 max_queued_clients = 2000 min_workers = 200 max_workers = 2000 max_requests = 5000 max_client_requests = 2000
Now I still get the error:
error: failed to connect to the hypervisor error: Failed to open file '/etc/libvirt/libvirt.conf': Too many open files
I tried twice,and the result is:
virsh -c qemu+tcp://remote_pc/system error: failed to connect to the hypervisor error: Failed to open file '/etc/libvirt/libvirt.conf': Too many open files
virsh -c qemu+ssh://remote_pc/system error: failed to connect to the hypervisor error: Failed to open file '/proc/4931/stat': Too many open files
Hi Daniel: I have solved the problem. Not anything above need to do in my PC, the only need to do is: Create /etc/systemd/system/libvirtd.service.d/openfiles.conf, and write the following two lines in it: [Service] LimitNOFILE=2048 then run "service libvirtd restart",and if needed,run "systemctl daemon-reload" If need more connections, increase LimitNOFILE, and also need to modify /etc/libvirt/libvirtd.conf. Thank you for your help! Regards, dw
participants (2)
-
Daniel P. Berrange -
dw