Re: [PATCH] tests: Record negative tests as pass instead of expected-fail

meson's "test()" function provides a "should_fail: bool" argument that checks for a command to exit with a non-zero exit code instead of the usual zero exit code to signal success. If the program under test does so, it is recorded as "EXPECTEDFAIL" instead of "OK". While there is an argument to be made that the program under test failed as expected, the test in itself is successful and should be recorded as such.

Before: $ meson test ... 151/300 libvirt:bin / libvirtd fail with missing config EXPECTEDFAIL 0.03s exit status 1

... Ok: 299 Expected Fail: 1 Fail: 0 Unexpected Pass: 0 Skipped: 0 Timeout: 0 After: $ meson test ... 151/300 libvirt:bin / libvirtd fail with missing config OK 0.03s ... Ok: 300 Expected Fail: 0 Fail: 0 Unexpected Pass: 0 Skipped: 0 Timeout: 0 Signed-off-by: Tim Wiederhake <twiederh(a)redhat.com&gt; --- tests/expected-fail | 13 +++++++++++++ tests/meson.build | 5 ++--- 2 files changed, 15 insertions(+), 3 deletions(-) create mode 100755 tests/expected-fail diff --git a/tests/expected-fail b/tests/expected-fail new file mode 100755 index 0000000000..85738c95dc --- /dev/null +++ b/tests/expected-fail @@ -0,0 +1,13 @@ +#!/bin/sh + +# meson's "test()" function provides a "should_fail: bool" argument that +# checks for a command to exit with a non-zero exit code instead of the usual +# zero exit code to signal success. If the program under test does so, it is +# recorded as "EXPECTEDFAIL" instead of "OK". While there is an argument to be +# made that the program under test failed as expected, the test in itself is +# successful and should be recorded as such. + +if "$@" +then + exit 1 +fi