Re: [libvirt] [PATCH v3 01/12] tests: Add some tests for PCI controller options

On 02/21/2018 09:14 AM, Andrea Bolognani wrote: > The input configurations set all existing options for all PCI > controllers, even those that are not valid for the controller. > As we implement validation for PCI controller options, we expect > these test to start failing. A noble cause, but since multiple options are being tested for multiple controllers in the same file, once you have all the proper checks in place the tests won't actually be verifying all of the negative tests - only the first failure will be noticed - if one of the others is missed, it won't "fail extra hard" or anything.

Although I hate to explode the number of tests, I think if you want to have proper negative testing for every options that doesn't belong on a particular controller, then you'll need a separate test case for each combination of option and controller model. And since that would make for a *lot* of test cases if we tried to extrapolate to all other options for all other elements, I don't know that it's worth going down that rabbit hole.