On Wed, Mar 02, 2022 at 09:43:24 +0000, Daniel P. Berrangé wrote:
On Wed, Mar 02, 2022 at 08:42:30AM +0100, Erik Skultety wrote:
> ...
Ultimately when we switch to using merge requests, the integration tests
should be run as a gating job, triggered from the merge train when the
code gets applied to git, so that we prevent regressions actually making
it into git master at all.
Post-merge integration testing always exhibits the problem that people will
consider it somebody else's problem to fix the regression. So effectively
whoever creates the integration testing system ends up burdened with the
job of investigating failures and finding someone to poke to fix it. With
it run pre-merge then whoever wants to get their code merged needs to
investigate the problems. Now sometimes the problems with of course be
with the integration test system itself, not the submitters code, but
this is OK because it leads to situation where the job of maintaining
the integration tests are more equitably spread across all involved and
builds a mindset that functional / integration testing is a critical
part of delivering code, which is something we've lacked for too long
in libvirt.
This is all fine as long as there's a way to explicitly merge something
even if CI fails. I don't like waiting for a fix in something completely
unrelated. For example, a MR with translations or an important bug fix
should not be blocked by a bug in CI or an infrastructure issue. So
having a way (restricted, of course) to merge even if pipeline failed
would solve this issue.
Jirka