Re: [libvirt PATCH 4/5] gitlab: add several native CI jobs
On Fri, Mar 20, 2020 at 03:31:24PM +0000, Daniel P. Berrangé wrote:
On Fri, Mar 20, 2020 at 04:18:58PM +0100, Erik Skultety wrote:
> On Fri, Mar 20, 2020 at 02:59:36PM +0000, Daniel P. Berrangé wrote:
> > On Fri, Mar 20, 2020 at 03:52:15PM +0100, Erik Skultety wrote:
> > > On Tue, Mar 10, 2020 at 10:09:44AM +0000, Daniel P. Berrangé wrote:
> > > > With GitLab CI aiming to replace Jenkins and Travis for CI purposes,
we
> > > > need to expand the coverage to include native builds. This patch
adds
> > > > all the jobs currently run in Travis. Compared to Jenkins we
obviously
> > > > miss the FreeBSD jobs, but also Debian 10 and Fedora 30, but we gain
the
> > > > Ubuntu 1804 job as a substitute for Debian.
> > > >
> > > > Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
> > > > ---
> > > > .gitlab-ci.yml | 41 +++++++++++++++++++++++++++++++++++++++++
> > > > 1 file changed, 41 insertions(+)
> > > >
> > > > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > > > index e28ec584ea..3e15d08d17 100644
> > > > --- a/.gitlab-ci.yml
> > > > +++ b/.gitlab-ci.yml
> > > > @@ -1,7 +1,39 @@
> > > > stages:
> > > > - website
> > > > + - native_build
> > > > - cross_build
> > > >
> > > > +
> > > > +.native_build_job_template: &native_build_job_definition
> > > > + stage: native_build
> > > > + script:
> > > > + - mkdir build
> > > > + - cd build
> > > > + - ../autogen.sh $CONFIGURE_OPTS || (cat config.log &&
exit 1)
> > > > + - make -j $(getconf _NPROCESSORS_ONLN) syntax-check
> > > > + - make -j $(getconf _NPROCESSORS_ONLN) distcheck
> > >
> > > I think ^this should more closely follow what we have in the lcitool
playbooks,
> > > e.g. start with:
> > > - rm -rf build
> >
> > The source tree is already pristine because this is always executed in
> > a fresh container environment, so there's nothing that will need deleting.
>
> Right, but my point was that if e.g. we introduce a FreeBSD builder, we'd want
> to reference the same job template in which case the directory will exist.
I've not looked in great detail about how gitlab runners work, but something
on the runner needs to be responsible for checking out the git repo before
any of this gitlab-ci.yml scrpit runs.
IMHO that can ensure it is "git clean -xdf" so that its state matches what
we see with the built-in shared runners build environment.
This is achieved by the GIT_CLEAN_FLAGS which, if unspecified, defaults to
-ffdx, so we're covered there.
One more thing that crossed my mind and may speedup the builds a bit - some
time ago I proposed a patch against libvirt-jenkins-ci to enable shallow
cloning which was rejected because of compelling reasons. However, for gitlab
purposes, we can't really do any debugging on the shared runners, so I believe
we may want to apply shallow cloning there, especially since libvirt is quite a
large repo. Gitlab claims [1] the default strategy to be 'fetch' if the
worktree can be found on the disk (this is when used with the 'docker' executor
which should be the case with shared runners). What I couldn't find is how long
are these worktrees cached which could therefore be a factor and we might want
to favour the shallow clones instead.
[1] https://docs.gitlab.com/ee/ci/large_repositories/#git-strategy