Re: [libvirt] [Bug] make check fails when LIBVIRT_DEBUG is defined
[moderator note: the mail server balked at 1.3 megabytes of attachments, so I've stripped them in my reply] On 12/31/1969 05:00 PM, wrote:
Hello everybody,
I set up a clean libvirt repo with: git clone ./autogen.sh make -j5
and then executed tests with various values of LIBVIRT_DEBUG using the following command: LIBVIRT_DEBUG=0…5 VIR_TEST_DEBUG=1 make check
The logs are in this mail's attachments.
With LIBVIRT_DEBUG=0 (or =5 or probably any invalid value), the logs say: "Ignoring invalid log level setting” but still, 10 tests fail due to various reasons.
With LIBVIRT_DEBUG=1, there is a slight difference. The number of tests that fail is also 10, but here, commandtest fails instead of eventtest.
With LIBVIRT_DEBUG=2, only 9 tests fail, because commandtest passes.
With LIBVIRT_DEBUG=3, only 1 test fails: qemuargv2xmltest. The reason is the same as in the previous tests.
With LIBVIRT_DEBUG=4, it’s just qemuargv2xmltest that is failing, but for a different reason than before: "qemuParseCommandLineString should have logged a warning”.
Perhaps it’s okay that these tests fail, but still there's no harm in letting people know.
Thank you rbogorodskiy for guidance on the mailing list :)
Have a nice day, Tomasz
I don't know that anyone ever expected the testsuite to work with LIBVIRT_DEBUG set to anything (because turning on debug can change output of libvirt itself). Things are different with VIR_TEST_DEBUG, which explicitly exists to change the verbosity of the testsuite itself, but not affect libvirt. If the testsuite passes without LIBVIRT_DEBUG set, then that's the solution :) But if you still get failures with LIBVIRT_DEBUG removed from the environment on your particular platform, then it is worth investigating how we can fix those failures (it may be a weakness in the testsuite, or an actual libvirt bug). -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Sat, Jun 25, 2016 at 04:32:21PM -0600, Eric Blake wrote:
[moderator note: the mail server balked at 1.3 megabytes of attachments, so I've stripped them in my reply]
On 12/31/1969 05:00 PM, wrote:
Hello everybody,
I set up a clean libvirt repo with: git clone ./autogen.sh make -j5
and then executed tests with various values of LIBVIRT_DEBUG using the following command: LIBVIRT_DEBUG=0…5 VIR_TEST_DEBUG=1 make check
The logs are in this mail's attachments.
With LIBVIRT_DEBUG=0 (or =5 or probably any invalid value), the logs say: "Ignoring invalid log level setting” but still, 10 tests fail due to various reasons.
With LIBVIRT_DEBUG=1, there is a slight difference. The number of tests that fail is also 10, but here, commandtest fails instead of eventtest.
With LIBVIRT_DEBUG=2, only 9 tests fail, because commandtest passes.
With LIBVIRT_DEBUG=3, only 1 test fails: qemuargv2xmltest. The reason is the same as in the previous tests.
With LIBVIRT_DEBUG=4, it’s just qemuargv2xmltest that is failing, but for a different reason than before: "qemuParseCommandLineString should have logged a warning”.
Perhaps it’s okay that these tests fail, but still there's no harm in letting people know.
Thank you rbogorodskiy for guidance on the mailing list :)
Have a nice day, Tomasz
I don't know that anyone ever expected the testsuite to work with LIBVIRT_DEBUG set to anything (because turning on debug can change output of libvirt itself). Things are different with VIR_TEST_DEBUG,
I'd certainly expect the test suite to pass 100% no matter what the debug level is set to. Debug logging should not have a functional effect on our code, besides outputting the log messages.
which explicitly exists to change the verbosity of the testsuite itself, but not affect libvirt. If the testsuite passes without LIBVIRT_DEBUG set, then that's the solution :) But if you still get failures with LIBVIRT_DEBUG removed from the environment on your particular platform, then it is worth investigating how we can fix those failures (it may be a weakness in the testsuite, or an actual libvirt bug).
Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 27.06.2016 10:05, Daniel P. Berrange wrote:
On Sat, Jun 25, 2016 at 04:32:21PM -0600, Eric Blake wrote:
[moderator note: the mail server balked at 1.3 megabytes of attachments, so I've stripped them in my reply]
On 12/31/1969 05:00 PM, wrote:
Hello everybody,
I set up a clean libvirt repo with: git clone ./autogen.sh make -j5
and then executed tests with various values of LIBVIRT_DEBUG using the following command: LIBVIRT_DEBUG=0…5 VIR_TEST_DEBUG=1 make check
The logs are in this mail's attachments.
With LIBVIRT_DEBUG=0 (or =5 or probably any invalid value), the logs say: "Ignoring invalid log level setting” but still, 10 tests fail due to various reasons.
With LIBVIRT_DEBUG=1, there is a slight difference. The number of tests that fail is also 10, but here, commandtest fails instead of eventtest.
With LIBVIRT_DEBUG=2, only 9 tests fail, because commandtest passes.
With LIBVIRT_DEBUG=3, only 1 test fails: qemuargv2xmltest. The reason is the same as in the previous tests.
With LIBVIRT_DEBUG=4, it’s just qemuargv2xmltest that is failing, but for a different reason than before: "qemuParseCommandLineString should have logged a warning”.
Perhaps it’s okay that these tests fail, but still there's no harm in letting people know.
Thank you rbogorodskiy for guidance on the mailing list :)
Have a nice day, Tomasz
I don't know that anyone ever expected the testsuite to work with LIBVIRT_DEBUG set to anything (because turning on debug can change output of libvirt itself). Things are different with VIR_TEST_DEBUG,
I'd certainly expect the test suite to pass 100% no matter what the debug level is set to. Debug logging should not have a functional effect on our code, besides outputting the log messages.
That's not 100% true. If we enable the debug messages, those are printed to stdout. Now we have some tests that expect certain output or some exact message to be logged. If we enable debug logs, the outputs are poisoned and don't match the expected values. I'd say this is a weakness of our test suite, not actual libvirt code. Michal
On Mon, Jun 27, 2016 at 01:23:46PM +0200, Michal Privoznik wrote:
On 27.06.2016 10:05, Daniel P. Berrange wrote:
On Sat, Jun 25, 2016 at 04:32:21PM -0600, Eric Blake wrote:
[moderator note: the mail server balked at 1.3 megabytes of attachments, so I've stripped them in my reply]
On 12/31/1969 05:00 PM, wrote:
Hello everybody,
I set up a clean libvirt repo with: git clone ./autogen.sh make -j5
and then executed tests with various values of LIBVIRT_DEBUG using the following command: LIBVIRT_DEBUG=0…5 VIR_TEST_DEBUG=1 make check
The logs are in this mail's attachments.
With LIBVIRT_DEBUG=0 (or =5 or probably any invalid value), the logs say: "Ignoring invalid log level setting” but still, 10 tests fail due to various reasons.
With LIBVIRT_DEBUG=1, there is a slight difference. The number of tests that fail is also 10, but here, commandtest fails instead of eventtest.
With LIBVIRT_DEBUG=2, only 9 tests fail, because commandtest passes.
With LIBVIRT_DEBUG=3, only 1 test fails: qemuargv2xmltest. The reason is the same as in the previous tests.
With LIBVIRT_DEBUG=4, it’s just qemuargv2xmltest that is failing, but for a different reason than before: "qemuParseCommandLineString should have logged a warning”.
Perhaps it’s okay that these tests fail, but still there's no harm in letting people know.
Thank you rbogorodskiy for guidance on the mailing list :)
Have a nice day, Tomasz
I don't know that anyone ever expected the testsuite to work with LIBVIRT_DEBUG set to anything (because turning on debug can change output of libvirt itself). Things are different with VIR_TEST_DEBUG,
I'd certainly expect the test suite to pass 100% no matter what the debug level is set to. Debug logging should not have a functional effect on our code, besides outputting the log messages.
That's not 100% true. If we enable the debug messages, those are printed to stdout. Now we have some tests that expect certain output or some exact message to be logged. If we enable debug logs, the outputs are poisoned and don't match the expected values. I'd say this is a weakness of our test suite, not actual libvirt code.
Hmm, yes, unit tests really should not be expecting precise output on stderr/stdout. Any that do are broken by design IMHO. At most they should be doing a sub-string match if they want to look for something and certainly not fail if extra output appears. Not least because I'll often set LIBVIRT_DEBUG in order to debug the unit tests and would not expect that to have side effects on them. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Mon, Jun 27, 2016 at 01:23:46PM +0200, Michal Privoznik wrote:
On 27.06.2016 10:05, Daniel P. Berrange wrote:
On Sat, Jun 25, 2016 at 04:32:21PM -0600, Eric Blake wrote:
[moderator note: the mail server balked at 1.3 megabytes of attachments, so I've stripped them in my reply]
On 12/31/1969 05:00 PM, wrote:
Hello everybody,
I set up a clean libvirt repo with: git clone ./autogen.sh make -j5
and then executed tests with various values of LIBVIRT_DEBUG using the following command: LIBVIRT_DEBUG=0…5 VIR_TEST_DEBUG=1 make check
The logs are in this mail's attachments.
With LIBVIRT_DEBUG=0 (or =5 or probably any invalid value), the logs say: "Ignoring invalid log level setting” but still, 10 tests fail due to various reasons.
With LIBVIRT_DEBUG=1, there is a slight difference. The number of tests that fail is also 10, but here, commandtest fails instead of eventtest.
With LIBVIRT_DEBUG=2, only 9 tests fail, because commandtest passes.
With LIBVIRT_DEBUG=3, only 1 test fails: qemuargv2xmltest. The reason is the same as in the previous tests.
With LIBVIRT_DEBUG=4, it’s just qemuargv2xmltest that is failing, but for a different reason than before: "qemuParseCommandLineString should have logged a warning”.
Perhaps it’s okay that these tests fail, but still there's no harm in letting people know.
Thank you rbogorodskiy for guidance on the mailing list :)
Have a nice day, Tomasz
I don't know that anyone ever expected the testsuite to work with LIBVIRT_DEBUG set to anything (because turning on debug can change output of libvirt itself). Things are different with VIR_TEST_DEBUG,
I'd certainly expect the test suite to pass 100% no matter what the debug level is set to. Debug logging should not have a functional effect on our code, besides outputting the log messages.
That's not 100% true. If we enable the debug messages, those are printed to stdout. Now we have some tests that expect certain output or some exact message to be logged. If we enable debug logs, the outputs are poisoned and don't match the expected values. I'd say this is a weakness of our test suite, not actual libvirt code.
I'd expect that not to work, OTOH I don't see why we couldn't make it work. Some tests must check the output, of course, but we can prioritize the tests' logging choice above users'. Having said that I must mention that I wouldn't put fixing this above Fixing the argv2xml is pretty easy (although during that I've found bunch of old things that could use a good scrub as well) but the rest I haven't looked at. Martin
Michal
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
participants (4)
-
Daniel P. Berrange -
Eric Blake -
Martin Kletzander -
Michal Privoznik