Re: [PATCH] Increase timeout for tests and syntax-check

On 1/29/21 1:45 PM, Pavel Hrdina wrote: > On Fri, Jan 29, 2021 at 09:30:24AM -0300, Daniel Henrique Barboza wrote: > > > > > > On 1/27/21 2:59 PM, Michal Privoznik wrote: > > > Since we've switched to meson our tests run with a timeout (meson > > > uses 30 seconds as the default). However, not every machine that > > > builds libvirt is fast enough to run every test under 30 seconds > > > (each test binary has its own timeout, but still). For instance > > > when building a package for distro on a farm that's under load. > > > Or on a generally slow ARM hardware. While each developer can > > > tune their command line for building by adding > > > --timeout-multiplier=10, this is hard to do for aforementioned > > > build farms. > > > > > > It's time to admit that not everybody has the latest, top shelf > > > CPU and increase the timeout. > > > > > > Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com&gt; > > > --- > > > > This sure will help these build farms environments, but what about the cases > > where an actual timeout means that there is something wrong with the code? > > E.g. commits 46d88d8dba56 and 2ba0b7497ce7 were only possible because I was > > seeing tests timing out in Power hosts when the 30 sec timeout was being > > enforced. > > > > A 120 second default timeout for the majority of the test cases is a long time. > > virschematest in this laptop I use takes 2.5 sec to complete. If I do something > > wrong in the code and now the test is now 4 times slower (10 sec) I will not be > > able to detect it (I'll need to start keeping track or something). You'll have to > > run the test suit on your RasPi 2B to see that something went wrong because the > > timeout is better tuned to your RasPI than this laptop, but then the code is already > > upstream. > > > > And the tests will get more complex and will naturally take longer to complete. > > Eventually this timeout might no be enough. Increase the timeout again? > > > > Meson 0.57 has support for disabling test timeout entirely with --timeout-multiplier=0, > > disabling test timeout entirely. Can't we bump the meson version to 0.57 and > > then tell the distros to use timeout-multiplier=0? That will solve the problems > > for them, I keep the short timeouts for development, and we won't need to keep > > bumping the test timeout once every 2 years or something because the s390x > > TCG enviroment in Fedora COPR is timing out again. > > Agreed. The timeout is useful and we can always find a machine where the > current timeout will not be good enough. I don't think it is a good idea > to increase the timeout like this for only few specific use-cases. So we agree that timeouts are evil :-) Back in the autotools days we did not

use any timeout at all and I don't remember that causing any trouble. But maybe it was so painful that I blocked it in my mind :-) Michal