On Thu, 2019-09-05 at 12:42 +0100, Daniel P. Berrangé wrote: Reducing the number of languages used by the project and facilitating the introduction of more languages seem to be contrasting goals. Accordingly, I would leave out the last part of the sentence. Instead of esplicitly singling out 2.7 and 3.x, I would just say that for both C and Python there it is required to support all platforms listed in "platforms.html". s/contributions./contributions:/ -- Andrea Bolognani / Red Hat / Virtualization

On Thu, 2019-09-05 at 15:42 +0100, Daniel P. Berrangé wrote: I think the fact that we want to add more languages only makes reducing the number of languages more pressing than it would be otherwise, but reducing the cognitive load for contributors is a worthy goal in its own right and alone would be enough to justify standardizing on Python in my opinion. So I'd still prefer it if you dropped that last part of the sentence, but I won't insist further if you're adamant that you want to keep it. Alright. If you fix this one: Reviewed-by: Andrea Bolognani <abologna(a)redhat.com&gt; -- Andrea Bolognani / Red Hat / Virtualization

On Thu, Sep 05, 2019 at 12:30:27PM -0400, Laine Stump wrote: Generally speaking it *should* be used as an argument against adding new languages. Given current state of the world I'd only make exceptions for Go or Rust, because they are the only credible options for replacing use of C as an efficient systems programming language. I get what you're saying though. Lemme think about it some more. Mostly what I was struggling with was trying not to promise a bunch of ponies that we've not yet delivered on. The HACKING file should really reflect what current committed best practice is. THings would be much clearer if I could set out a general "vision" or "roadmap" for language usage, where I could talk about where we want to get to in 5 years hence, despite it not being the case today. I believe they are run by the Red Hat virt QE team, as I ave got bug reports against the Perl libvirt bindings, where the reproducer is a TCK script. The TCK really covered three distinct things - framework for controlling test execution - library modules/helper APIs to facilitate test creation - the large set of indivdual tests There is no code binding between the tests and the framework. They communicate exclusively through the TAP protocol. In that way we can throw out the current framework and drop in a different one quite easily - the new framework merely needs to be able to consume the TAP data format. For any framework which does not natively support the TAP format, it is quite easy to write an adapter / shim that would convert to the desired data format. You might loose some finese / fine grained details in the logs/progress reporting but functionally the tests should work fine. As a point of reference the GLib testing harness has recently switched from using their own custom output format to using the TAP output format. So if we adopt GLib in the libvirt.git and use it in our unit tests those would end up using TAP. GLib provides a shim for automake that allows it to consume the results from GLib based units tests. QEMU's makefiles recently switched to using TAP for its unit and (some of) its functional tests too. IOW, I'm perfectly happy to throw away the existing TCK framework impl if someone proposes a better alternative. As for the individual TCK tests themselves, there's two distinct questions - Should they be re-written in a different language - Should they continue to use TAP output format If we are not going to rewrite the tests, then I think they should continue to use TAP output format. It is the easiest thing to use when writing tests in Perl. If we are going to rewrite the tests, then the tests should ideally use the output format that best suits the framework invoking them. IOW, if the framework still uses TAP, we should continue to use TAP. If not, then don't. There is an adapter for python that can convert its native unittest output format into TAP format. I'm pretty ambivalent on rewriting the existing tests. The language of existing tests only matters if you expect people to be modifying them. Assuming a decent support library it should be no harder to define a new test, than to extend an existing test. IOW, if we want to write TCK tests in Python, we don't really need to rewrite existing ones. We instead need to write a Python variant of the helper APIs, so that you can trivially write new Python tests which operate in the same manner as the Perl tests. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On 9/5/19 1:57 PM, Andrea Bolognani wrote: That *definitely* needs more cowbell!

On 9/23/19 7:52 AM, Daniel P. Berrangé wrote: Sorry for jumping in late, but I also run TCK regularly and agree that it does find real bugs on occasion. I've submitted several patches over the years fixing bugs found by TCK, and a sprinkling of patches to TCK itself. I admit the perl requirement has deterred writing new tests. Regards, Jim

On Tue, Oct 01, 2019 at 10:27:33AM +0100, Daniel P. Berrangé wrote: So, currently I'm working on fixing the last bits on TCK failures to enable running it upstream on ci.centos.org and continue from there. I think there was a thread somewhere (may have been an internal one), where we also considered the usage of plain avocado as the execution and harness engine as it not only provides TAP support, but also other results reporting formats as well as having support for test matrices by using a multiplexer. Is that idea out of question, as I'm willing to (and already spending time on) to make progress in this area. Erik