Re: GSoC 2022 CI project idea proposal

On Thu, Feb 24, 2022 at 06:49:29AM -0800, Andrea Bolognani wrote: > I'm not entirely convinced that requiring the use of lcitool for this > task is necessarily the best idea though. Ideally, it should be > possible for developers to reproduce and debug CI failures locally > without having to clone a second repository. It's fine to require > lcitool for tasks that are performed by maintainers, but casual > contributors should find all they need in libvirt.git itself. Unless they also want to test with VMs as well in which case lcitool is highly recommended. The thing is that right now ci/helper wraps the Makefile. Let's say you (or someone else) replaces it with a python equivalent, I don't think that the number of contributors suddenly starting to use it will differ much compared to the situation right now, but maybe I'm wrong :). My motivation to add the support to lcitool is simple - right now there's an RFC on adopting some infrastructure in order to run TCK-based functional tests as part of GitLab. That's just the first stage, ultimately we want developers to be able to run the same test suite locally without the need to wait for gitlab and since the current design of the above is using nested virt, it is an achievable goal. There was also a suggestion to adopt the Avocado framework for a new test framework which would be largely based on TCK [1]. Now, let's get to the point I'm trying to make :) - my expectation/vision is that both the remote and local functional test executions will revolve around lcitool and so eventually with the adoption of *some* framework it is going to become a requirement that libvirt developers do perform functional testing before submitting a patch series. With that said, it makes more sense in my mind to focus on adding the container runtime support to lcitool in order to move towards the bigger goal and have everything consolidated in a single place.

> Another thing that has been bothering me is that neither 'ci/helper > build' nor 'lcitool build' will actually perform the exact same build > steps as the corresponding CI job, making them only mildly effective > as debugging tools for CI failures. And of course each of these build > recipes is maintained separately, so we have three similar but not > quite identical scripts floating around. Yes, but on its own, I think that is a problem which can be solved separately.