Re: [libvirt] [PATCH 19/20] qemuagenttest: Test arbitrary qemu commands and timeouting of commands

On 07/30/2013 12:52 PM, Daniel P. Berrange wrote: >> Hmm, I wonder if it's worth adding some sort of escape hatch, maybe if >> 'VIR_TEST_FAST' is defined to non-empty in the environment, then we skip >> this test; developers that don't like the long wait can then export that >> variable as 1, whereas the spec file can ensure it is 0. That could be >> a followup patch, though, and it might be worth getting more feedback >> than just mine on whether the new long-running test needs tweaking to >> allow developers to avoid waiting, while still avoiding bit-rotting of >> the test at release time. > > IMHO we don't want any of the tests doing multi-second timeouts by > default. IOW, rather than VIR_TEST_FAST, we should have a > VIR_TEST_ALLOW_SLEEP=1, and ensure that libvirt.spec.in sets that > when doing 'make check' and also make sure that autobuild.sh sets > it. So all automated builds fully exercise the tests, but day to > day usage isn't delayed For that matter, 'make check' for day-to-day usage should be able to skip the gnulib subdirectory - the results in gnulib/tests will only change if you upgrade the gnulib submodule, glibc, or some other core component, which is not what we change on a day-to-day basis when hacking gnulib, but is also something an autobuilder should be running always. I'll see if I can hack something up to speed up 'make check' for normal users on the gnulib front, which we can then extend into skipping Peter's new test.

GNU coreutils calls its variable RUN_EXPENSIVE_TESTS, defaulting to no, but set to yes in autobuilders. Sounds like the best type of naming (maybe VIR_TEST_EXPENSIVE, to keep it in the VIR_ namespace). Anyone else want to chime in with a bikeshed color?