[libvirt] [PATCH] Raise default limit on number of processes for qemu user
The daemon/libvirtd.limits file (which is supposed to be copied to /etc/security/limits.d/libvirtd.conf) is generated based on --qemu-user option passed at configure time. The file is intentionally not installed by make install since installing it on distributions with higher or no limit on number of process could actually result in lowering the limit. Packagers may choose whether to install the file or not. It is installed by libvirt.spec for RPM based distributions. --- configure.ac | 3 ++- daemon/libvirtd.limits.in | 6 ++++++ libvirt.spec.in | 7 +++++++ src/qemu/qemu.conf | 4 +++- 4 files changed, 18 insertions(+), 2 deletions(-) create mode 100644 daemon/libvirtd.limits.in diff --git a/configure.ac b/configure.ac index a58ee4e..d8002b4 100644 --- a/configure.ac +++ b/configure.ac @@ -2237,6 +2237,7 @@ AC_ARG_WITH([qemu-group], [QEMU_GROUP=root]) AC_DEFINE_UNQUOTED([QEMU_USER], ["$QEMU_USER"], [QEMU user account]) AC_DEFINE_UNQUOTED([QEMU_GROUP], ["$QEMU_GROUP"], [QEMU group account]) +AC_SUBST([QEMU_USER]) AC_ARG_WITH([macvtap], @@ -2344,7 +2345,7 @@ AC_OUTPUT(Makefile src/Makefile include/Makefile docs/Makefile \ po/Makefile.in \ include/libvirt/Makefile include/libvirt/libvirt.h \ python/Makefile python/tests/Makefile \ - daemon/Makefile \ + daemon/Makefile daemon/libvirtd.limits \ tools/Makefile \ tests/Makefile \ examples/apparmor/Makefile \ diff --git a/daemon/libvirtd.limits.in b/daemon/libvirtd.limits.in new file mode 100644 index 0000000..7b580ae --- /dev/null +++ b/daemon/libvirtd.limits.in @@ -0,0 +1,6 @@ +# /etc/security/limits.d/libvirtd.conf +# +# Increase default limit for number of processes owned by qemu-user +# Also see @sysconfdir@/libvirt/qemu.conf where qemu-user can be overridden + +@QEMU_USER@ soft nproc 10000 diff --git a/libvirt.spec.in b/libvirt.spec.in index 45a8fe0..02be928 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -705,6 +705,12 @@ mv $RPM_BUILD_ROOT%{_datadir}/doc/libvirt-%{version}/html \ rm -rf $RPM_BUILD_ROOT%{_sysconfdir}/libvirt/qemu.conf rm -rf $RPM_BUILD_ROOT%{_sysconfdir}/logrotate.d/libvirtd.qemu %endif + +%if %{with_libvirtd} && %{with_qemu} +install -d -m 0755 $RPM_BUILD_ROOT%{_sysconfdir}/security/limits.d/ +cp daemon/libvirtd.limits \ + $RPM_BUILD_ROOT%{_sysconfdir}/security/limits.d/libvirtd.conf +%endif %if ! %{with_lxc} rm -rf $RPM_BUILD_ROOT%{_sysconfdir}/libvirt/lxc.conf rm -rf $RPM_BUILD_ROOT%{_sysconfdir}/logrotate.d/libvirtd.lxc @@ -869,6 +875,7 @@ fi %if %{with_qemu} %config(noreplace) %{_sysconfdir}/libvirt/qemu.conf %config(noreplace) %{_sysconfdir}/logrotate.d/libvirtd.qemu +%config(noreplace) %{_sysconfdir}/security/limits.d/libvirtd.conf %endif %if %{with_lxc} %config(noreplace) %{_sysconfdir}/libvirt/lxc.conf diff --git a/src/qemu/qemu.conf b/src/qemu/qemu.conf index 8c6b996..52c6ac7 100644 --- a/src/qemu/qemu.conf +++ b/src/qemu/qemu.conf @@ -139,7 +139,9 @@ # security_driver = "selinux" -# The user ID for QEMU processes run by the system instance. +# The user ID for QEMU processes run by the system instance. If you +# change this setting, /etc/security/limits.d/libvirtd.conf (if +# installed) needs to be changed to match this user ID. #user = "root" # The group ID for QEMU processes run by the system instance. -- 1.7.4.1
On Wed, Mar 09, 2011 at 02:20:09PM +0100, Jiri Denemark wrote:
The daemon/libvirtd.limits file (which is supposed to be copied to /etc/security/limits.d/libvirtd.conf) is generated based on --qemu-user option passed at configure time.
The file is intentionally not installed by make install since installing it on distributions with higher or no limit on number of process could actually result in lowering the limit. Packagers may choose whether to install the file or not. It is installed by libvirt.spec for RPM based distributions. --- configure.ac | 3 ++- daemon/libvirtd.limits.in | 6 ++++++ libvirt.spec.in | 7 +++++++ src/qemu/qemu.conf | 4 +++- 4 files changed, 18 insertions(+), 2 deletions(-) create mode 100644 daemon/libvirtd.limits.in
diff --git a/configure.ac b/configure.ac index a58ee4e..d8002b4 100644 --- a/configure.ac +++ b/configure.ac @@ -2237,6 +2237,7 @@ AC_ARG_WITH([qemu-group], [QEMU_GROUP=root]) AC_DEFINE_UNQUOTED([QEMU_USER], ["$QEMU_USER"], [QEMU user account]) AC_DEFINE_UNQUOTED([QEMU_GROUP], ["$QEMU_GROUP"], [QEMU group account]) +AC_SUBST([QEMU_USER])
AC_ARG_WITH([macvtap], @@ -2344,7 +2345,7 @@ AC_OUTPUT(Makefile src/Makefile include/Makefile docs/Makefile \ po/Makefile.in \ include/libvirt/Makefile include/libvirt/libvirt.h \ python/Makefile python/tests/Makefile \ - daemon/Makefile \ + daemon/Makefile daemon/libvirtd.limits \ tools/Makefile \ tests/Makefile \ examples/apparmor/Makefile \ diff --git a/daemon/libvirtd.limits.in b/daemon/libvirtd.limits.in new file mode 100644 index 0000000..7b580ae --- /dev/null +++ b/daemon/libvirtd.limits.in @@ -0,0 +1,6 @@ +# /etc/security/limits.d/libvirtd.conf +# +# Increase default limit for number of processes owned by qemu-user +# Also see @sysconfdir@/libvirt/qemu.conf where qemu-user can be overridden + +@QEMU_USER@ soft nproc 10000 diff --git a/libvirt.spec.in b/libvirt.spec.in index 45a8fe0..02be928 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -705,6 +705,12 @@ mv $RPM_BUILD_ROOT%{_datadir}/doc/libvirt-%{version}/html \ rm -rf $RPM_BUILD_ROOT%{_sysconfdir}/libvirt/qemu.conf rm -rf $RPM_BUILD_ROOT%{_sysconfdir}/logrotate.d/libvirtd.qemu %endif + +%if %{with_libvirtd} && %{with_qemu} +install -d -m 0755 $RPM_BUILD_ROOT%{_sysconfdir}/security/limits.d/ +cp daemon/libvirtd.limits \ + $RPM_BUILD_ROOT%{_sysconfdir}/security/limits.d/libvirtd.conf +%endif %if ! %{with_lxc} rm -rf $RPM_BUILD_ROOT%{_sysconfdir}/libvirt/lxc.conf rm -rf $RPM_BUILD_ROOT%{_sysconfdir}/logrotate.d/libvirtd.lxc @@ -869,6 +875,7 @@ fi %if %{with_qemu} %config(noreplace) %{_sysconfdir}/libvirt/qemu.conf %config(noreplace) %{_sysconfdir}/logrotate.d/libvirtd.qemu +%config(noreplace) %{_sysconfdir}/security/limits.d/libvirtd.conf %endif %if %{with_lxc} %config(noreplace) %{_sysconfdir}/libvirt/lxc.conf diff --git a/src/qemu/qemu.conf b/src/qemu/qemu.conf index 8c6b996..52c6ac7 100644 --- a/src/qemu/qemu.conf +++ b/src/qemu/qemu.conf @@ -139,7 +139,9 @@ # security_driver = "selinux"
-# The user ID for QEMU processes run by the system instance. +# The user ID for QEMU processes run by the system instance. If you +# change this setting, /etc/security/limits.d/libvirtd.conf (if +# installed) needs to be changed to match this user ID. #user = "root"
# The group ID for QEMU processes run by the system instance. --
ACK, sounds fine by me. Out of curiosity 10000 processes is quite a lot, but maybe we can reach this with fake QEMU, Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
On Wed, Mar 09, 2011 at 02:20:09PM +0100, Jiri Denemark wrote:
The daemon/libvirtd.limits file (which is supposed to be copied to /etc/security/limits.d/libvirtd.conf) is generated based on --qemu-user option passed at configure time.
The file is intentionally not installed by make install since installing it on distributions with higher or no limit on number of process could actually result in lowering the limit. Packagers may choose whether to install the file or not. It is installed by libvirt.spec for RPM based distributions. --- configure.ac | 3 ++- daemon/libvirtd.limits.in | 6 ++++++ libvirt.spec.in | 7 +++++++ src/qemu/qemu.conf | 4 +++- 4 files changed, 18 insertions(+), 2 deletions(-) create mode 100644 daemon/libvirtd.limits.in
diff --git a/configure.ac b/configure.ac index a58ee4e..d8002b4 100644 --- a/configure.ac +++ b/configure.ac @@ -2237,6 +2237,7 @@ AC_ARG_WITH([qemu-group], [QEMU_GROUP=root]) AC_DEFINE_UNQUOTED([QEMU_USER], ["$QEMU_USER"], [QEMU user account]) AC_DEFINE_UNQUOTED([QEMU_GROUP], ["$QEMU_GROUP"], [QEMU group account]) +AC_SUBST([QEMU_USER])
AC_ARG_WITH([macvtap], @@ -2344,7 +2345,7 @@ AC_OUTPUT(Makefile src/Makefile include/Makefile docs/Makefile \ po/Makefile.in \ include/libvirt/Makefile include/libvirt/libvirt.h \ python/Makefile python/tests/Makefile \ - daemon/Makefile \ + daemon/Makefile daemon/libvirtd.limits \ tools/Makefile \ tests/Makefile \ examples/apparmor/Makefile \ diff --git a/daemon/libvirtd.limits.in b/daemon/libvirtd.limits.in new file mode 100644 index 0000000..7b580ae --- /dev/null +++ b/daemon/libvirtd.limits.in @@ -0,0 +1,6 @@ +# /etc/security/limits.d/libvirtd.conf +# +# Increase default limit for number of processes owned by qemu-user +# Also see @sysconfdir@/libvirt/qemu.conf where qemu-user can be overridden + +@QEMU_USER@ soft nproc 10000 diff --git a/libvirt.spec.in b/libvirt.spec.in index 45a8fe0..02be928 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -705,6 +705,12 @@ mv $RPM_BUILD_ROOT%{_datadir}/doc/libvirt-%{version}/html \ rm -rf $RPM_BUILD_ROOT%{_sysconfdir}/libvirt/qemu.conf rm -rf $RPM_BUILD_ROOT%{_sysconfdir}/logrotate.d/libvirtd.qemu %endif + +%if %{with_libvirtd} && %{with_qemu} +install -d -m 0755 $RPM_BUILD_ROOT%{_sysconfdir}/security/limits.d/ +cp daemon/libvirtd.limits \ + $RPM_BUILD_ROOT%{_sysconfdir}/security/limits.d/libvirtd.conf +%endif %if ! %{with_lxc} rm -rf $RPM_BUILD_ROOT%{_sysconfdir}/libvirt/lxc.conf rm -rf $RPM_BUILD_ROOT%{_sysconfdir}/logrotate.d/libvirtd.lxc @@ -869,6 +875,7 @@ fi %if %{with_qemu} %config(noreplace) %{_sysconfdir}/libvirt/qemu.conf %config(noreplace) %{_sysconfdir}/logrotate.d/libvirtd.qemu +%config(noreplace) %{_sysconfdir}/security/limits.d/libvirtd.conf %endif %if %{with_lxc} %config(noreplace) %{_sysconfdir}/libvirt/lxc.conf diff --git a/src/qemu/qemu.conf b/src/qemu/qemu.conf index 8c6b996..52c6ac7 100644 --- a/src/qemu/qemu.conf +++ b/src/qemu/qemu.conf @@ -139,7 +139,9 @@ # security_driver = "selinux"
-# The user ID for QEMU processes run by the system instance. +# The user ID for QEMU processes run by the system instance. If you +# change this setting, /etc/security/limits.d/libvirtd.conf (if +# installed) needs to be changed to match this user ID. #user = "root"
Hmm, did you actually test this setup to make sure it works as we expect ? I have this nasty feeling in the back of my mind that the files under /etc/security/limits.d/ are only processed by PAM modules. Since PAM isn't at all involved when libvirt changes UID to 'qemu' to launch QEMU, how does QEMU actually see the increased limit being set ? Something needs to be calling the setrlimit() systemcall for the QEMU process when it is launched and I don't see what is yet ? Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Thu, Mar 10, 2011 at 11:33:24AM +0000, Daniel P. Berrange wrote:
On Wed, Mar 09, 2011 at 02:20:09PM +0100, Jiri Denemark wrote:
The daemon/libvirtd.limits file (which is supposed to be copied to /etc/security/limits.d/libvirtd.conf) is generated based on --qemu-user option passed at configure time.
The file is intentionally not installed by make install since installing it on distributions with higher or no limit on number of process could actually result in lowering the limit. Packagers may choose whether to install the file or not. It is installed by libvirt.spec for RPM based distributions.
[snip]
Hmm, did you actually test this setup to make sure it works as we expect ? I have this nasty feeling in the back of my mind that the files under /etc/security/limits.d/ are only processed by PAM modules. Since PAM isn't at all involved when libvirt changes UID to 'qemu' to launch QEMU, how does QEMU actually see the increased limit being set ?
Well I've tested this now and confirm that putting a file into /etc/security/limits.d for the 'qemu' user, has absolutely no effect on QEMU as launched by libvirtd.
Something needs to be calling the setrlimit() systemcall for the QEMU process when it is launched and I don't see what is yet ?
Because we don't use PAM, QEMU is just inheriting the limits from libvirtd. For added fun, the limits that libvirtd sees typically differ depending on whether libvirtd was started from a root login shell, or from init. So AFAICT, we have to use setrlimit() if we want to control this. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
participants (3)
-
Daniel P. Berrange -
Daniel Veillard -
Jiri Denemark