Re: [libvirt PATCH v2 6/9] gitlab: add x86_64 native CI jobs
On Tue, 2020-03-24 at 18:21 +0000, Daniel P. Berrangé wrote:
On Tue, Mar 24, 2020 at 06:47:01PM +0100, Andrea Bolognani wrote:
> I don't get the rationale behind the split.
>
> Right now we're not using merge requests, but we're limiting the
> number of jobs for the merge request case; at the same time, we say
> that once we switch to a MR-based workflow, we're going to run the
> extra jobs on each merge request as well. So what does the
> distinction buy us?
With this split today, if I push to my private fork, then the
reduced set of jobs run. This gives quick turnaround for developers
developing patches.
When it gets reviewed & pushed to master, the full set run post
merge.
In the future, when we switch to merge requests, we'll change
it so that the full set run when the merge request is opened,
instad of post-merge.
What is run for developers private branches will remain the
same
Okay, I understand the rationale now, and I can see that we were
arguing for very similar approaches the entire time - it's just that
I could not see that. Thanks for taking the time to spell it out for
me :)
> * pick a selection of jobs that includes both native and
cross
> build, across various distros of different vintage, such that
> they can all run without waiting on shared runners. This can be
> used by developers as a reasonably quick (~20 minutes) smoke
> test while working on a feature;
"without waiting on shared runners" is undefined, as whether you
wait & how long, is dependant on many things outside our control.
As notedin the cover letter though, this minimal set of native
+ cross build jobs gives about a 35 min turn around based on
load I see today. I think that's acceptably fast.
Given the intentions, I think the current split can be improved
further: having cross_build as a separate stage from native_build
means that the former has to wait before the latter is done to even
start, and that makes wait times longer.
I suggest we instead have the following stages:
artifacts:
- website
- potfile
smoke:
- armv7l-debian-sid
- mingw32-fedora-30
- mingw64-fedora-30
- s390x-debian-10
- x64-centos-7
- x64-debian-10
- x64-fedora-30
- x64-fedora-rawhide
- x64-opensuse-151
- x64-ubuntu-1604
full_coverage:
- x64-centos-8
- x64-debian-9
- x64-debian-sid
- x64-fedora-31
- x64-ubuntu-1804
Aside from the name change, this maintains the same job split, except
for merging the native_build and cross_build stages together. When I
tested this split, the artifacts + smoke builds only took ~20 minutes
thanks to the increased parallelism; even if for load balancing
reasons the smoke builds were not able to start all at the same time,
I reckon this setup would still fare better than forcing the second
batch to start only after the first one has completed.
Another thing that I would like to point out is that, while it makes
sense that we'd only have a small number of cross-builds happening in
the smoke test phase, when it comes to post-merge (and later merge
request) time we don't really need to limit ourselves as much, and we
could easily leave most of the existing cross-build jobs enabled as
part of the full_coverage stage.
--
Andrea Bolognani / Red Hat / Virtualization