[libvirt] [PATCH] enable parallel builds

I'm not sure if this is the best way to do this, but it seems to work. ---- Enable parallel compilation of the repository when running the autobuild script and/or via rpmbuild. --- autobuild.sh | 15 ++++++++++++++- libvirt.spec.in | 2 +- 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/autobuild.sh b/autobuild.sh index 7ae5d1e..1a5413b 100755 --- a/autobuild.sh +++ b/autobuild.sh @@ -21,7 +21,20 @@ rm -rf coverage --with-lxc \ --with-xen-proxy -make +# from the gdc spec file +# if RPM_BUILD_NCPUS unset, set it +if [ -z "$RPM_BUILD_NCPUS" ] ; then + if [ -x /usr/bin/getconf ] ; then + RPM_BUILD_NCPUS=$(/usr/bin/getconf _NPROCESSORS_ONLN) + if [ $RPM_BUILD_NCPUS -eq 0 ]; then + RPM_BUILD_NCPUS=1 + fi + else + RPM_BUILD_NCPUS=1 + fi +fi + +make -j$RPM_BUILD_NCPUS make install set -o pipefail diff --git a/libvirt.spec.in b/libvirt.spec.in index d37c0e0..cd8e937 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -169,7 +169,7 @@ of recent versions of Linux (and other OSes). --with-init-script=redhat \ --with-qemud-pid-file=%{_localstatedir}/run/libvirt_qemud.pid \ --with-remote-file=%{_localstatedir}/run/libvirtd.pid -make +make %{?_smp_mflags} %install rm -fr %{buildroot}

On Fri, Aug 22, 2008 at 06:36:23PM +1000, James Morris wrote:
I'm not sure if this is the best way to do this, but it seems to work.
Sure, gets my vote. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

James Morris <jmorris@namei.org> wrote:
I'm not sure if this is the best way to do this, but it seems to work.
----
Enable parallel compilation of the repository when running the autobuild script and/or via rpmbuild.
---
autobuild.sh | 15 ++++++++++++++- libvirt.spec.in | 2 +- 2 files changed, 15 insertions(+), 2 deletions(-)
diff --git a/autobuild.sh b/autobuild.sh index 7ae5d1e..1a5413b 100755 --- a/autobuild.sh +++ b/autobuild.sh @@ -21,7 +21,20 @@ rm -rf coverage --with-lxc \ --with-xen-proxy
-make +# from the gdc spec file +# if RPM_BUILD_NCPUS unset, set it +if [ -z "$RPM_BUILD_NCPUS" ] ; then + if [ -x /usr/bin/getconf ] ; then + RPM_BUILD_NCPUS=$(/usr/bin/getconf _NPROCESSORS_ONLN) + if [ $RPM_BUILD_NCPUS -eq 0 ]; then + RPM_BUILD_NCPUS=1 + fi + else + RPM_BUILD_NCPUS=1 + fi +fi
Good idea. I'll be happy to commit it with something like the following in place of the above: # If the MAKEFLAGS envvar does not yet include a -j option, # add -jN where N depends on the number of processors. case $MAKEFLAGS in *-j*) ;; *) n=$(getconf _NPROCESSORS_ONLN 2> /dev/null) test "$n" -gt 0 || n=1 n=`expr $n + 1` MAKEFLAGS="$MAKEFLAGS -j$n" export MAKEFLAGS ;; esac Then you don't have to change the make invocation below, and it won't interfere if someone has already set MAKEFLAGS. Also, not using an absolute path to getconf lets that program work even also when it's installed in a different location. Finally, I prefer to use N_CPUS+1 as the -j option, here. (personally, I use 2*$N_CPUS+1, but that's probably too aggressive) Ok?
+make -j$RPM_BUILD_NCPUS make install
set -o pipefail
Here's the patch I tested: diff --git a/autobuild.sh b/autobuild.sh index ce12692..cb6101f 100755 --- a/autobuild.sh +++ b/autobuild.sh @@ -21,6 +21,18 @@ rm -rf coverage --with-lxc \ --with-xen-proxy +# If the MAKEFLAGS envvar does not yet include a -j option, +# add -jN where N depends on the number of processors. +case $MAKEFLAGS in + *-j*) ;; + *) n=$(getconf _NPROCESSORS_ONLN 2> /dev/null) + test "$n" -gt 0 || n=1 + n=`expr $n + 1` + MAKEFLAGS="$MAKEFLAGS -j$n" + export MAKEFLAGS + ;; +esac + make make install diff --git a/libvirt.spec.in b/libvirt.spec.in index 4aff4a5..7c151ef 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -157,7 +157,7 @@ of recent versions of Linux (and other OSes). --with-init-script=redhat \ --with-qemud-pid-file=%{_localstatedir}/run/libvirt_qemud.pid \ --with-remote-file=%{_localstatedir}/run/libvirtd.pid -make +make %{?_smp_mflags} %install rm -fr %{buildroot}

Jim Meyering <jim@meyering.net> wrote:
James Morris <jmorris@namei.org> wrote:
I'm not sure if this is the best way to do this, but it seems to work.
Enable parallel compilation of the repository when running the autobuild script and/or via rpmbuild. ... diff --git a/autobuild.sh b/autobuild.sh ... Good idea. I'll be happy to commit it with something like the following in place of the above:
# If the MAKEFLAGS envvar does not yet include a -j option, # add -jN where N depends on the number of processors. case $MAKEFLAGS in *-j*) ;; *) n=$(getconf _NPROCESSORS_ONLN 2> /dev/null) test "$n" -gt 0 || n=1 n=`expr $n + 1` MAKEFLAGS="$MAKEFLAGS -j$n" export MAKEFLAGS ;; esac
Then you don't have to change the make invocation below, and it won't interfere if someone has already set MAKEFLAGS. Also, not using an absolute path to getconf lets that program work even also when it's installed in a different location.
Finally, I prefer to use N_CPUS+1 as the -j option, here. (personally, I use 2*$N_CPUS+1, but that's probably too aggressive)
Ok?
I'm interpreting non-response as agreement, so have just committed my proposed change: http://article.gmane.org/gmane.comp.emulators.libvirt/8215

On Thu, Aug 28, 2008 at 11:13:03AM +0200, Jim Meyering wrote:
Jim Meyering <jim@meyering.net> wrote:
James Morris <jmorris@namei.org> wrote:
I'm not sure if this is the best way to do this, but it seems to work.
Enable parallel compilation of the repository when running the autobuild script and/or via rpmbuild. ... diff --git a/autobuild.sh b/autobuild.sh ... Good idea. I'll be happy to commit it with something like the following in place of the above:
# If the MAKEFLAGS envvar does not yet include a -j option, # add -jN where N depends on the number of processors. case $MAKEFLAGS in *-j*) ;; *) n=$(getconf _NPROCESSORS_ONLN 2> /dev/null) test "$n" -gt 0 || n=1 n=`expr $n + 1` MAKEFLAGS="$MAKEFLAGS -j$n" export MAKEFLAGS ;; esac
Then you don't have to change the make invocation below, and it won't interfere if someone has already set MAKEFLAGS. Also, not using an absolute path to getconf lets that program work even also when it's installed in a different location.
Finally, I prefer to use N_CPUS+1 as the -j option, here. (personally, I use 2*$N_CPUS+1, but that's probably too aggressive)
Ok?
I'm interpreting non-response as agreement,
Sorry, thought I had replied to your mail already, but obviously forgot. It was fine by me. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
participants (3)
-
Daniel P. Berrange
-
James Morris
-
Jim Meyering