Re: [libvirt PATCH v3 10/12] gitlab: introduce use of ccache for speeding up rebuilds
On Thu, 2020-03-26 at 14:05 +0000, Daniel P. Berrangé wrote:
On Thu, Mar 26, 2020 at 02:50:48PM +0100, Andrea Bolognani wrote:
> On Thu, 2020-03-26 at 12:35 +0000, Daniel P. Berrangé wrote:
> > .native_build_default_job_template: &native_build_default_job_definition
> > stage: native_build
> > + cache:
> > + paths:
> > + - ccache/
> > + key: "$CI_JOB_NAME"
> > + before_script:
> > + - mkdir -p ccache
> > + - export CC="ccache gcc"
> > + - export CCACHE_BASEDIR=${PWD}
> > + - export CCACHE_DIR=${PWD}/ccache
>
> Having to set this up at the job level is kinda gross, and
> specifically the
>
> export CC="ccache gcc"
>
> trick is 1) not going to work for FreeBSD and 2) going to break
> Go builds.
That's easy enough to fix for FreeBSD - we just set CC=clang vs CC=gcc
variable for the jobs at the top level, and then this code can change
to CC="ccache $CC".
We don't have any Go code in libvirt but what's the issue with ccache
in that case ?
Yeah, we were hitting it with libvirt-go{,-xml} so it's not really
relevant in the immediate, but it's still something that we're going
to have to deal with at some point.
The problem, IIRC, was that cgo will just break when $CC contains
more than a single word, so the link farm approach works fine but
the CC="ccache $CC" approach doesn't.
> Ultimately I think we need to take a cue from what lcitool does
when
> configuring VMs and generate a simple environment file that is baked
> into images and can be sourced from jobs with a single line.
I much prefer to have the job configuration all in the gitlab config file
rather than split between the gitlab config and the images, as it lets you
understand the full setup.
I agree in theory, but 1) that specific ship has sailed when we
started adding stuff like $LANG, $ABI and $CONFIGURE_OPTS in the
container image's environment and 2) doing it at the .gitlab-ci.yml
level will result in duplicating a lot of the logic that we already
have in lcitool.
We could however make the container images setup a link farm for
ccache
as we did with the VM images. So then the job just needs to set the
$PATH to point to the link farm bin dir, instead of $CC.
Yeah, in the long run we definitely want to build the link farm.
--
Andrea Bolognani / Red Hat / Virtualization