Re: [libvirt-glib PATCH] gitlab: introduce CI jobs testing git master & distro libvirt
On Wed, 2020-06-17 at 14:35 +0100, Daniel P. Berrangé wrote:
On Wed, Jun 17, 2020 at 12:54:56PM +0200, Andrea Bolognani wrote:
> On Thu, 2020-06-11 at 17:42 +0100, Daniel P. Berrangé wrote:
> > +.script_variables: &script_variables |
> > + export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)"
> > + export VROOT="$SCRATCH_DIR/vroot"
> > + export CCACHE_BASEDIR="$(pwd)"
> > + export CCACHE_DIR="$CCACHE_BASEDIR/ccache"
> > + export CCACHE_MAXSIZE="500M"
> > + export PATH="$CCACHE_WRAPPERSDIR:$VROOT/bin:$PATH"
> > + export SCRATCH_DIR="/tmp/scratch"
> > + export PKG_CONFIG_PATH="$VROOT/lib/pkgconfig"
>
> You need to define $SCRATCH_DIR before $VROOT, otherwise the latter
> will get the wrong value.
>
> I also just realized that the way we set $CCACHE_BASEDIR might not
> work for the libvirt build that we perform as a prerequisite, and
There's no problem - $(pwd) expands at the time the variable is
defined.
The problem with $CCACHE_BASEDIR being set to $PWD is that the build
that we do for libvirt happens *outside* of that directory, which I
think messes up with the path rewriting and might result in libvirt
builds not taking proper advantage of the cache.
It's all a bit confusing, to be honest, so I could have gotten it
wrong, but IIUC the idea is that if you have multiple built trees
for the same project you'd be able to do something like
$ cd ~/build/tree/one
$ export CCACHE_BASEDIR=$PWD
$ make
$ cd ~/build/tree/two
$ export CCACHE_BASEDIR=$PWD
$ make
and, thanks to $CCACHE_BASEDIR, ccache will be able to figure out
that some of the files are perfect matches and will thus achieve
better hit rates.
This is not our case for a few reasons:
* we only build each project once per container;
* we set $CCACHE_BASEDIR once instead of once per build directory;
* we build libvirt outside of $CCACHE_BASEDIR.
So basically our current use of $CCACHE_BASEDIR is likely useless if
not even somewhat harmful, and we should just get rid of it.
> since we use the same paths across builds anyway there
doesn't seem
> to be a point in setting it. So I suggest we have
>
> export SCRATCH_DIR="/tmp/scratch"
> export VROOT="$SCRATCH_DIR/vroot"
> export CCACHE_DIR="$SCRATCH_DIR/ccache"
The ccache dir has to be a subdir of the source checkout for
gitlab to cache it between jobs.
Right! I think you told me about this already, but then I forgot
about it O:-)
Keeping that in mind, we should have
export CCACHE_DIR="$PWD/ccache"
then.
> > + script:
> > + - mkdir build
> > + - cd build
> > + - ../autogen.sh --prefix="$VROOT"
> > + - $MAKE install
> > + - $MAKE dist
>
> Do we want distcheck here, or is the combination of dist plus
> building the RPM package (which effectively runs check inside the
> generated tarball) good enough?
I think its good enough. Also I want to switch to meson for glib
asap.
Okay.
Be aware of the fact that, with the switch to Meson, we're going to
need something like
https://gitlab.com/libosinfo/libosinfo/-/blob/master/.gitlab-ci.yml#L6-13
because Meson, unlike autotools, is properly multiarch-aware.
In fact, a better solution would probably be to bake the multilib
string into the Dockerfile directly, similarly to what we already do
for other multiarch-related information, instead of figuring it out
dynamically at build time. I haven't had the time to look into
implementing that yet, unfortunatley, but maybe we can get that in
before libvirt-glib moves off autotools.
--
Andrea Bolognani / Red Hat / Virtualization