On Fri, Feb 25, 2022 at 03:02:49AM -0800, Andrea Bolognani wrote:
On Fri, Feb 25, 2022 at 11:31:39AM +0100, Erik Skultety wrote:
On Thu, Feb 24, 2022 at 06:49:29AM -0800, Andrea Bolognani wrote:
I'm not entirely convinced that requiring the use of lcitool for this task is necessarily the best idea though. Ideally, it should be possible for developers to reproduce and debug CI failures locally without having to clone a second repository. It's fine to require lcitool for tasks that are performed by maintainers, but casual contributors should find all they need in libvirt.git itself.
Unless they also want to test with VMs as well in which case lcitool is highly recommended. The thing is that right now ci/helper wraps the Makefile. Let's say you (or someone else) replaces it with a python equivalent, I don't think that the number of contributors suddenly starting to use it will differ much compared to the situation right now, but maybe I'm wrong :). My motivation to add the support to lcitool is simple - right now there's an RFC on adopting some infrastructure in order to run TCK-based functional tests as part of GitLab. That's just the first stage, ultimately we want developers to be able to run the same test suite locally without the need to wait for gitlab and since the current design of the above is using nested virt, it is an achievable goal. There was also a suggestion to adopt the Avocado framework for a new test framework which would be largely based on TCK [1]. Now, let's get to the point I'm trying to make :) - my expectation/vision is that both the remote and local functional test executions will revolve around lcitool and so eventually with the adoption of *some* framework it is going to become a requirement that libvirt developers do perform functional testing before submitting a patch series. With that said, it makes more sense in my mind to focus on adding the container runtime support to lcitool in order to move towards the bigger goal and have everything consolidated in a single place.
Alright, that's a pretty solid argument. And even if we ultimately decide that we don't want to require using lcitool, once we have a project-agnostic Python implementation of this logic it would be reasonably straightforward to turn it into a standalone tool similar to the current ci/helper, so we have an exit strategy.
Another thing that has been bothering me is that neither 'ci/helper build' nor 'lcitool build' will actually perform the exact same build steps as the corresponding CI job, making them only mildly effective as debugging tools for CI failures. And of course each of these build recipes is maintained separately, so we have three similar but not quite identical scripts floating around.
Yes, but on its own, I think that is a problem which can be solved separately.
Agreed that the two tasks are not necessarily depending on each other, but there's significant overlap. I think it makes sense to rationalize how the build steps for a project are defined and maintained as part of adding "build in a local container" support to lcitool.
Do you have any high-level concerns about the ci/build approach I vaguely described? The finer details are of course far from being set in stone, but I think the overall idea is solid and we should aim for it being implemented as we evolve our CI tooling.
No, I think that was a solid proposal, I would probably think of along those lines as well had I ever come to that idea myself :). Having each repo define their own build script which can be consumed both during local test executions and copied to the Dockerfile for a gitlab job to consume makes complete sense. Top it off with something like 'lcitool container --script <path_to_build_script>' and I think we have a solid ground for future work. Erik