Re: [libvirt PATCH 4/4] gitlab-ci: Introduce a new test 'integration' pipeline stage

On Wed, Mar 02, 2022 at 06:27:04AM -0800, Andrea Bolognani wrote: > I don't think we can expect integration tests to be merged at the > same time as a feature when new APIs are involved. If tests are > written in Python, then the Python bindings need to introduce support > for the new API before the test can exist, and that can't happen > until the feature has been merged. Again, that is a logical conclusion which brings us to an unrelated process question: How do we change the contribution process so that the contribution of a feature doesn't end with it being merged to the C library, IOW we'd ideally want to have a test introduced with every feature,but given that we'd need the bindings first to actually do that, but we can't have a binding unless the C counterpart is already merged, how do we keep the contributors motivated enough? (Note that it's food for thought, it's only tangential to this effort)

> If the feature or bug fix doesn't require new APIs to be introduced > this is of course not an issue. Most changes should fall into this > bucket. > > So overall I still think using existing artifacts would be the better > approach, at least initially. We can always change things later if we > find that we've outgrown it. So, given that https://gitlab.com/libvirt/libvirt-perl/-/merge_requests/55 was already merged we should not get to a situation where no artifacts would be available because gitlab documents that even if artifacts expired they won't be deleted until new artifacts become available, I think we can depend on the latest available artifacts without building them. I'll refresh the patch accordingly and test.

> I think it's not unreasonable that when Red Hat, or any other entity, > provides hardware access to the project there will be some strings > attached. This is already the case for GitLab and Cirrus CI, for > example. > > I could easily see the instance of libvirt-gitlab-executor running on > hardware owned and operated by Red Hat returning a failure if a job > submitted to it comes with DISTRO=debian-11. libvirt-gitlab-executor is supposed to be system owner agnostic, I'd even like to make the project part of the libvirt gitlab namespace umbrella so that anyone can use it to prepare their machine to be plugged into and used for libvirt upstream integration testing.

Therefore, I don't think the project is the right place for such checks, those should IMO live solely in the gitlab-ci.yml configuration.