On Wed, Mar 02, 2022 at 09:43:24 +0000, Daniel P. Berrangé wrote:
On Wed, Mar 02, 2022 at 08:42:30AM +0100, Erik Skultety wrote:
... Ultimately when we switch to using merge requests, the integration tests should be run as a gating job, triggered from the merge train when the code gets applied to git, so that we prevent regressions actually making it into git master at all.
Post-merge integration testing always exhibits the problem that people will consider it somebody else's problem to fix the regression. So effectively whoever creates the integration testing system ends up burdened with the job of investigating failures and finding someone to poke to fix it. With it run pre-merge then whoever wants to get their code merged needs to investigate the problems. Now sometimes the problems with of course be with the integration test system itself, not the submitters code, but this is OK because it leads to situation where the job of maintaining the integration tests are more equitably spread across all involved and builds a mindset that functional / integration testing is a critical part of delivering code, which is something we've lacked for too long in libvirt.
This is all fine as long as there's a way to explicitly merge something even if CI fails. I don't like waiting for a fix in something completely unrelated. For example, a MR with translations or an important bug fix should not be blocked by a bug in CI or an infrastructure issue. So having a way (restricted, of course) to merge even if pipeline failed would solve this issue. Jirka