Re: [libvirt] [test-API] RFC: Stabilization of libvirt-test-API

On 03/29/2012 08:14 AM, Martin Kletzander wrote: Forgive me if I'm suggesting things that already exist - I haven't had time to go through all of the test-API code, but want to make sure this gets brought up sooner rather than later. Two things that I think are very important for a general purpose test setup are: 1) The ability to have multiple physical machines, network devices of various types (bridges, standard NICs, sr-iov NICs) in the test harness available via standard names (implying, of course, that the address/capabilities (and even the existence/non-existence) of at least one other machine be configurable in a global config file referenced by every test). 2) The ability to mark some tests as requiring certain standard objects from the global configuration (e.g. the second machine, a bridge interface, sr-iov NICs) and to either skip the test when some required object is missing, or fail the test (the test itself would be marked either OPTIONAL or REQUIRED). This would allow us to have, for example, full testing of networking capabilities present in the test suite, but without raising an entry barrier for "random Joe" who only has a single machine, one NIC, etc., but wants to run the test suite. When Joe ran the tests, each test that required multiple hosts (or sr-iov NICs or whatever esoteric piece of hardware/driver) and was designated as an "OPTIONAL" test, would be semi-silently skipped, but someone with a full complement of hardware who demanded a thorough and complete test could set a switch and guarantee that every single test would be run (or a failure saying, e.g. "object REMOTE_HOST required by this test is missing in config", or something like that). Is there currently an allowance for these two items?