Re: [libvirt PATCH 1/2] ci: Enable Cirrus CI integration

On Mon, Jun 08, 2020 at 07:14:20PM +0200, Andrea Bolognani wrote: > > > +freebsd_11_task: > > > + install_script: > > > + - pkg install -y > > > + augeas > > > + autoconf > > > + automake > [...] > > > > For the dockerfiles, we're auto-generating using lcitool. > > > > IIUC, we should have sufficient info avialable in lcitol that > > we can wrote a command to generate this entire file, including > > the build_script commands. > > > > In fact eventually we could try to get to a point where we > > auto-generate the .gitlab-ci.yml from lcitool. > > I mention this in the commit message. The goal is absolutely to > generate these files using lcitool, I just haven't invested time > into doing that yet because I wanted to make sure the overall > approach was considered acceptable first. If we only consider macOS for a minute, then I think this change is a no-brainer. It is clearly better than Travis because it presents all the CI results in the same dashboard, and doesn't require contributors to manually push to github themselves. There is no viable alternative for macOS, as we're never going to provide any VMs for macOS ourselves due to license issues around hardware you have to run on. The versions of macOS/XCode in Cirrus are different from Travis, but that doesn't bother me, given the clear benefits of the workflow. So the question becomes... Given that we should use Cirrus for macOS no matter what, is there a strong enough reason to not use this for FreeBSD too. The main downside I see of using Cirrus CI is that both Cirrus & GitHub are a closed source service. This is the same situation as Travis. I'm willing to ignore that for macOS as there is no way in which we can ever provide a macOS CI runner ourselves. ie the choice for macOS is "no CI at all" vs "CI using closed source" and the latter clearly wins. For FreeBSD things are a different though, as with Erik's work we do have the ability to provide FreeBSD runners and avoid any dep on either GitHub or Cirrus. We'll need Erik's work regardless for runners to do integration testing with, though admittedly our focus there will likely be on Linux runners for integration testing, not FreeBSD. The real benefit of Cirrus CI for FreeBSD is that it lets contributors get FreeBSD CI without having to bring their own VM to the party. They'll be using Cirrus CI for macOS anyway, so it seems silly to tell them they should setup Cirrus for macOS, but then not let them use it for FreeBSD and go through hoops for custom runners. I think this means we simply have to allow FreeBSD on Cirrus for forks, once we have this for macOS.

The only variation I can see here is to write the gitlab-ci.yml such that forks do their pre-merge CI against Cirrus, but the master repo does its post-merge CI again our custom runner. I do strongly prefer open source infra wherever possible, but I'm not convinced the FreeBSD CI is a case where its worth the hassle of maintaining FreeBSD runners.... at least not unless we get to a point where we need runners for integration testing on FreeBSD which is not any time soon.