[libvirt] [PATCH] build: honor autogen.sh --no-git
Based on a report by Chandrashekar Shastri, at https://bugzilla.redhat.com/show_bug.cgi?id=979360 On systems where git cannot access the outside world, a developer can instead arrange to get a copy of gnulib at the right commit via side channels (such as NFS share drives), set GNULIB_SRCDIR, then use ./autogen.sh --no-git. In this setup, we will now avoid direct use of git. Of course, this means no automatic gnulib updates when libvirt.git updates its submodule, but it is expected that any developer in such a situation is already prepared to deal with the fallout. * .gnulib: Update to latest, for bootstrap. * bootstrap: Synchronize from gnulib. * autogen.sh (no_git): Avoid git when requested. * cfg.mk (_update_required): Skip automatic rerun of bootstrap if we can't use git. Signed-off-by: Eric Blake <eblake@redhat.com> --- This does not qualify as a build-breaker, and although it touches gnulib, it is complex enough that I'm not quite willing to push it under my carte-blanche rule for gnulib updates outside of freezes. So, if anyone is willing to review, great; if not, I'll probably still push on Monday, as it did pass my testing. * .gnulib f40e61e...b72ff2a (1):
bootstrap: honor --no-git
.gnulib | 2 +- autogen.sh | 10 ++++++++-- bootstrap | 21 ++++++++++++++------- cfg.mk | 1 + 4 files changed, 24 insertions(+), 10 deletions(-) diff --git a/.gnulib b/.gnulib index f40e61e..b72ff2a 160000 --- a/.gnulib +++ b/.gnulib @@ -1 +1 @@ -Subproject commit f40e61ea0c4940b027aade7dd48948aa93f133a4 +Subproject commit b72ff2a45efde544c406804186d37a3254728571 diff --git a/autogen.sh b/autogen.sh index d4957f1..42e5608 100755 --- a/autogen.sh +++ b/autogen.sh @@ -49,6 +49,10 @@ fi # we rerun bootstrap to pull in those diffs. bootstrap_hash() { + if test "$no_git"; then + echo no-git + return + fi git submodule status | sed 's/^[ +-]//;s/ .*//' git hash-object bootstrap.conf git ls-tree -d HEAD gnulib/local | awk '{print $3}' @@ -62,7 +66,9 @@ bootstrap_hash() # Only run bootstrap from a git checkout, never from a tarball. if test -d .git; then curr_status=.git-module-status t= - if test -d .gnulib; then + if test "$no_git"; then + t=no-git + elif test -d .gnulib; then t=$(bootstrap_hash; git diff .gnulib) fi case $t:${CLEAN_SUBMODULE+set} in @@ -78,7 +84,7 @@ if test -d .git; then # good, it's up to date, all we need is autoreconf autoreconf -if else - if test ${CLEAN_SUBMODULE+set}; then + if test -z "$no_git" && test ${CLEAN_SUBMODULE+set}; then echo cleaning up submodules... git submodule foreach 'git clean -dfqx && git reset --hard' fi diff --git a/bootstrap b/bootstrap index 0cbea66..9c52204 100755 --- a/bootstrap +++ b/bootstrap @@ -1,6 +1,6 @@ #! /bin/sh # Print a version string. -scriptversion=2013-05-08.20; # UTC +scriptversion=2013-07-03.20; # UTC # Bootstrap this package from checked-out sources. @@ -551,7 +551,7 @@ fi echo "$0: Bootstrapping from checked-out $package sources..." # See if we can use gnulib's git-merge-changelog merge driver. -if test -d .git && (git --version) >/dev/null 2>/dev/null ; then +if $use_git && test -d .git && (git --version) >/dev/null 2>/dev/null ; then if git config merge.merge-changelog.driver >/dev/null ; then : elif (git-merge-changelog --version) >/dev/null 2>/dev/null ; then @@ -574,13 +574,17 @@ git_modules_config () { test -f .gitmodules && git config --file .gitmodules "$@" } -gnulib_path=$(git_modules_config submodule.gnulib.path) -test -z "$gnulib_path" && gnulib_path=gnulib +if $use_git; then + gnulib_path=$(git_modules_config submodule.gnulib.path) + test -z "$gnulib_path" && gnulib_path=gnulib +fi -# Get gnulib files. +# Get gnulib files. Populate $GNULIB_SRCDIR, possibly updating a +# submodule, for use in the rest of the script. case ${GNULIB_SRCDIR--} in -) + # Note that $use_git is necessarily true in this case. if git_modules_config submodule.gnulib.url >/dev/null; then echo "$0: getting gnulib files..." git submodule init || exit $? @@ -601,8 +605,8 @@ case ${GNULIB_SRCDIR--} in GNULIB_SRCDIR=$gnulib_path ;; *) - # Use GNULIB_SRCDIR as a reference. - if test -d "$GNULIB_SRCDIR"/.git && \ + # Use GNULIB_SRCDIR directly or as a reference. + if $use_git && test -d "$GNULIB_SRCDIR"/.git && \ git_modules_config submodule.gnulib.url >/dev/null; then echo "$0: getting gnulib files..." if git submodule -h|grep -- --reference > /dev/null; then @@ -628,6 +632,9 @@ case ${GNULIB_SRCDIR--} in ;; esac +# $GNULIB_SRCDIR now points to the version of gnulib to use, and +# we no longer need to use git or $gnulib_path below here. + if $bootstrap_sync; then cmp -s "$0" "$GNULIB_SRCDIR/build-aux/bootstrap" || { echo "$0: updating bootstrap and restarting..." diff --git a/cfg.mk b/cfg.mk index bbe84b3..147e788 100644 --- a/cfg.mk +++ b/cfg.mk @@ -828,6 +828,7 @@ ifeq (0,$(MAKELEVEL)) test -d .git || { echo 0; exit; }; \ test -f po/Makevars || { echo 1; exit; }; \ test -f AUTHORS || { echo 1; exit; }; \ + test "no-git" = "$$(cat $(_curr_status))" && { echo 0; exit; }; \ actual=$$(git submodule status | $(_submodule_hash); \ git hash-object bootstrap.conf; \ git ls-tree -d HEAD gnulib/local | awk '{print $$3}'; \ -- 1.8.1.4
On Wed, Jul 03, 2013 at 03:07:51PM -0600, Eric Blake wrote:
Based on a report by Chandrashekar Shastri, at https://bugzilla.redhat.com/show_bug.cgi?id=979360
On systems where git cannot access the outside world, a developer can instead arrange to get a copy of gnulib at the right commit via side channels (such as NFS share drives), set GNULIB_SRCDIR, then use ./autogen.sh --no-git. In this setup, we will now avoid direct use of git. Of course, this means no automatic gnulib updates when libvirt.git updates its submodule, but it is expected that any developer in such a situation is already prepared to deal with the fallout.
Could you also expand http://libvirt.org/compiling.html#building to describe how to build from GIT, but without needing direct net access. It has come up periodically & we don't have any docs about this aspect afaik. ACK to changes Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 07/04/2013 03:32 AM, Daniel P. Berrange wrote:
On Wed, Jul 03, 2013 at 03:07:51PM -0600, Eric Blake wrote:
Based on a report by Chandrashekar Shastri, at https://bugzilla.redhat.com/show_bug.cgi?id=979360
On systems where git cannot access the outside world, a developer can instead arrange to get a copy of gnulib at the right commit via side channels (such as NFS share drives), set GNULIB_SRCDIR, then use ./autogen.sh --no-git. In this setup, we will now avoid direct use of git. Of course, this means no automatic gnulib updates when libvirt.git updates its submodule, but it is expected that any developer in such a situation is already prepared to deal with the fallout.
Could you also expand
http://libvirt.org/compiling.html#building
to describe how to build from GIT, but without needing direct net access. It has come up periodically & we don't have any docs about this aspect afaik.
ACK to changes
How does this work as a followup patch? (Actually, I may just squash it in with the gnulib patch and do it all at once...) Hmm, this would be the first time the auto-generated HACKING file refers to an in-tree html file (rather than an external project page), and I'm not sure I like how the formatting turned out. Any suggestions on how to make that look better? diff --git i/HACKING w/HACKING index a310faa..37e102c 100644 --- i/HACKING +++ w/HACKING @@ -36,7 +36,10 @@ developer is: git checkout -t origin -b workbranch Hack, committing any changes along the way -Then, when you want to post your patches: +More hints on compiling can be found + + here + compiling.html. When you want to post your patches: git pull --rebase (fix any conflicts) diff --git i/docs/compiling.html.in w/docs/compiling.html.in index 66d2925..215fdf1 100644 --- i/docs/compiling.html.in +++ w/docs/compiling.html.in @@ -65,8 +65,36 @@ checkout it is necessary to generate the configure script and Makefile.in templates using the <code>autogen.sh</code> command. By default when the <code>configure</code> script is run from within a GIT checkout, it - will turn on -Werror for builds. This can be disabled with --disable-werror, - but this is not recommended. To build & install libvirt to your home + will turn on -Werror for builds. This can be disabled with + --disable-werror, but this is not recommended. + </p> + <p> + Libvirt takes advantage of + the <a href="http://www.gnu.org/software/gnulib/">gnulib</a> + project to provide portability to a number of platforms. This + is normally done dynamically via a git submodule in + the <code>.gnulib</code> subdirectory, which is auto-updated as + needed when you do incremental builds. Setting the environment + variable <code>GNULIB_SRCDIR</code> to a local directory + containing a git checkout of gnulib will let you reduce local + disk space requirements and network download time, regardless of + which actual commit you have in that reference directory. + </p> + <p> + However, if you are developing on a platform where git is not + available, or are behind a firewall that does not allow for git + to easily obtain the gnulib submodule, it is possible to instead + use a static mode of operation where you are then responsible + for updating the git submodule yourself. In this mode, you must + track the exact gnulib commit needed by libvirt (usually not the + latest gnulib.git) via alternative means, such as a shared NFS + drive or manual download, and run this any time libvirt.git + updates the commit stored in the .gnulib submodule:</p> + <pre> + $ GNULIB_SRCDIR=/path/to/gnulib ./autogen.sh --no-git + </pre> + + <p>To build & install libvirt to your home directory the following commands can be run: </p> diff --git i/docs/hacking.html.in w/docs/hacking.html.in index 904b846..78d4df6 100644 --- i/docs/hacking.html.in +++ w/docs/hacking.html.in @@ -35,7 +35,9 @@ git checkout -t origin -b workbranch Hack, committing any changes along the way </pre> - <p>Then, when you want to post your patches:</p> + <p>More hints on compiling can be + found <a href="compiling.html">here</a>. When you want to + post your patches:</p> <pre> git pull --rebase (fix any conflicts) -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Mon, Jul 08, 2013 at 07:44:18AM -0600, Eric Blake wrote:
On 07/04/2013 03:32 AM, Daniel P. Berrange wrote:
On Wed, Jul 03, 2013 at 03:07:51PM -0600, Eric Blake wrote:
Based on a report by Chandrashekar Shastri, at https://bugzilla.redhat.com/show_bug.cgi?id=979360
On systems where git cannot access the outside world, a developer can instead arrange to get a copy of gnulib at the right commit via side channels (such as NFS share drives), set GNULIB_SRCDIR, then use ./autogen.sh --no-git. In this setup, we will now avoid direct use of git. Of course, this means no automatic gnulib updates when libvirt.git updates its submodule, but it is expected that any developer in such a situation is already prepared to deal with the fallout.
Could you also expand
http://libvirt.org/compiling.html#building
to describe how to build from GIT, but without needing direct net access. It has come up periodically & we don't have any docs about this aspect afaik.
ACK to changes
How does this work as a followup patch? (Actually, I may just squash it in with the gnulib patch and do it all at once...)
Hmm, this would be the first time the auto-generated HACKING file refers to an in-tree html file (rather than an external project page), and I'm not sure I like how the formatting turned out. Any suggestions on how to make that look better?
diff --git i/HACKING w/HACKING index a310faa..37e102c 100644 --- i/HACKING +++ w/HACKING @@ -36,7 +36,10 @@ developer is: git checkout -t origin -b workbranch Hack, committing any changes along the way
-Then, when you want to post your patches: +More hints on compiling can be found + + here + compiling.html. When you want to post your patches:
git pull --rebase (fix any conflicts) diff --git i/docs/compiling.html.in w/docs/compiling.html.in index 66d2925..215fdf1 100644 --- i/docs/compiling.html.in +++ w/docs/compiling.html.in @@ -65,8 +65,36 @@ checkout it is necessary to generate the configure script and Makefile.in templates using the <code>autogen.sh</code> command. By default when the <code>configure</code> script is run from within a GIT checkout, it - will turn on -Werror for builds. This can be disabled with --disable-werror, - but this is not recommended. To build & install libvirt to your home + will turn on -Werror for builds. This can be disabled with + --disable-werror, but this is not recommended. + </p> + <p> + Libvirt takes advantage of + the <a href="http://www.gnu.org/software/gnulib/">gnulib</a> + project to provide portability to a number of platforms. This + is normally done dynamically via a git submodule in + the <code>.gnulib</code> subdirectory, which is auto-updated as + needed when you do incremental builds. Setting the environment + variable <code>GNULIB_SRCDIR</code> to a local directory + containing a git checkout of gnulib will let you reduce local + disk space requirements and network download time, regardless of + which actual commit you have in that reference directory. + </p> + <p> + However, if you are developing on a platform where git is not + available, or are behind a firewall that does not allow for git + to easily obtain the gnulib submodule, it is possible to instead + use a static mode of operation where you are then responsible + for updating the git submodule yourself. In this mode, you must + track the exact gnulib commit needed by libvirt (usually not the + latest gnulib.git) via alternative means, such as a shared NFS + drive or manual download, and run this any time libvirt.git + updates the commit stored in the .gnulib submodule:</p> + <pre> + $ GNULIB_SRCDIR=/path/to/gnulib ./autogen.sh --no-git + </pre> + + <p>To build & install libvirt to your home directory the following commands can be run: </p>
diff --git i/docs/hacking.html.in w/docs/hacking.html.in index 904b846..78d4df6 100644 --- i/docs/hacking.html.in +++ w/docs/hacking.html.in @@ -35,7 +35,9 @@ git checkout -t origin -b workbranch Hack, committing any changes along the way </pre> - <p>Then, when you want to post your patches:</p> + <p>More hints on compiling can be + found <a href="compiling.html">here</a>. When you want to + post your patches:</p> <pre> git pull --rebase (fix any conflicts)
ACK Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 07/08/2013 07:47 AM, Daniel P. Berrange wrote:
Could you also expand
http://libvirt.org/compiling.html#building
to describe how to build from GIT, but without needing direct net access. It has come up periodically & we don't have any docs about this aspect afaik.
ACK to changes
How does this work as a followup patch? (Actually, I may just squash it in with the gnulib patch and do it all at once...)
I'm still inclined to squash the two patches together.
Hmm, this would be the first time the auto-generated HACKING file refers to an in-tree html file (rather than an external project page), and I'm not sure I like how the formatting turned out. Any suggestions on how to make that look better?
I figured it out; see https://www.redhat.com/archives/libvir-list/2013-July/msg00446.html which I plan to apply first, once it gets reviewed.
diff --git i/HACKING w/HACKING index a310faa..37e102c 100644 --- i/HACKING +++ w/HACKING @@ -36,7 +36,10 @@ developer is: git checkout -t origin -b workbranch Hack, committing any changes along the way
-Then, when you want to post your patches: +More hints on compiling can be found + + here + compiling.html. When you want to post your patches:
With the xslt pre-req patch for <a> handling, this section of the generated text-only HACKING file is now more legible as: More hints on compiling can be found here <compiling.html>. When...
+++ w/docs/hacking.html.in @@ -35,7 +35,9 @@ git checkout -t origin -b workbranch Hack, committing any changes along the way </pre> - <p>Then, when you want to post your patches:</p> + <p>More hints on compiling can be + found <a href="compiling.html">here</a>. When you want to + post your patches:</p> <pre> git pull --rebase (fix any conflicts)
ACK
Thanks for the review; I'll push soon. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On 07/08/2013 07:47 AM, Daniel P. Berrange wrote:
Could you also expand
http://libvirt.org/compiling.html#building
to describe how to build from GIT, but without needing direct net access. It has come up periodically & we don't have any docs about this aspect afaik.
ACK to changes
- <p>Then, when you want to post your patches:</p> + <p>More hints on compiling can be + found <a href="compiling.html">here</a>. When you want to + post your patches:</p> <pre> git pull --rebase (fix any conflicts)
ACK
Thanks; pushed. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
participants (2)
-
Daniel P. Berrange -
Eric Blake