[libvirt] encountered failed test cases with latest checkout of libvirt
Hi all, This email content might be duplicated with other thread posted in other places that I do not know yet where to find or search (please let me know if it is true). I am working on a small bug of virsh domxml-to-native. Before I made any changes to the code in order to fix the bug, I 'make test'-ed and 'valgrind test'-ed the code, encountered four failures in 'make test' and 19 failures in 'make -C tests valgrind'. What I did today was the following: git checkout master git pull ./autogen.sh --prefix=$(pwd)/build make make install # So far so good, no errors ; however make check VIR_TEST_EXPENSIVE=1 # generated the following test-suite.log ========================================= libvirt 3.2.0: tests/test-suite.log ========================================= # TOTAL: 115 # PASS: 111 # SKIP: 0 # XFAIL: 0 # FAIL: 4 # XPASS: 0 # ERROR: 0 .. contents:: :depth: 2 FAIL: virfirewalltest ===================== TEST: virfirewalltest !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 40 FAIL FAIL virfirewalltest (exit status: 1) FAIL: networkxml2firewalltest ============================= TEST: networkxml2firewalltest 0 FAIL FAIL networkxml2firewalltest (exit status: 1) FAIL: nwfilterebiptablestest ============================ TEST: nwfilterebiptablestest 0 FAIL FAIL nwfilterebiptablestest (exit status: 1) FAIL: nwfilterxml2firewalltest ============================== TEST: nwfilterxml2firewalltest 0 FAIL FAIL nwfilterxml2firewalltest (exit status: 1) make syntax-check VIR_TEST_EXTENSIVE=1 # I did not paste the result here and I do not understand what it means. # available upon request Finally, make -C tests valgrind # generated file was attached. It seems most of the valgrind failures were false positives, I can only tell by comparing the log file with the examples in the HACKING page. But I do not have the knowledge to determine. Could anyone tell me if I am missing something? Having those test failures is a problem or not? Which ones can be safely ignored? Is there any easier way to avoid the false positives? I am concerned because I also need to run the test when I make changes to the code. On the other hand, I would also like to learn the right way to do the testing or whatever it takes to make things right. I am running linux using vmware Workstation Pro 12 on Windows 10. Linux version 4.4.55-1-MANJARO (builduser@manjaro) (gcc version 6.3.1 20170306 (GCC) ) #1 SMP PREEMPT Dan
Hi all,
This email content might be duplicated with other thread posted in other places that I do not know yet where to find or search (please let me know if it is true). I am working on a small bug of virsh domxml-to-native. Before I made any changes to the code in order to fix the bug, I 'make test'-ed and 'valgrind test'-ed the code, encountered four failures in 'make test' and 19 failures in 'make -C tests valgrind'. What I did today was the following:
git checkout master git pull ./autogen.sh --prefix=$(pwd)/build make make install # So far so good, no errors ; however
make check VIR_TEST_EXPENSIVE=1 # generated the following test-suite.log ========================================= libvirt 3.2.0: tests/test-suite.log =========================================
# TOTAL: 115 # PASS: 111 # SKIP: 0 # XFAIL: 0 # FAIL: 4 # XPASS: 0 # ERROR: 0
.. contents:: :depth: 2
FAIL: virfirewalltest =====================
TEST: virfirewalltest !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 40 FAIL FAIL virfirewalltest (exit status: 1)
FAIL: networkxml2firewalltest =============================
TEST: networkxml2firewalltest 0 FAIL FAIL networkxml2firewalltest (exit status: 1)
FAIL: nwfilterebiptablestest ============================
TEST: nwfilterebiptablestest 0 FAIL FAIL nwfilterebiptablestest (exit status: 1)
FAIL: nwfilterxml2firewalltest ==============================
TEST: nwfilterxml2firewalltest 0 FAIL FAIL nwfilterxml2firewalltest (exit status: 1)
make syntax-check VIR_TEST_EXTENSIVE=1 # I did not paste the result here and I do not understand what it means. # available upon request
Finally, make -C tests valgrind # generated file was attached.
It seems most of the valgrind failures were false positives, I can only tell by comparing the log file with the examples in the HACKING page. But I do not have the knowledge to determine.
Could anyone tell me if I am missing something? Having those test failures is a problem or not? Which ones can be safely ignored? Is there any easier way to avoid the false positives? I am concerned because I also need to run the test when I make changes to the code. On the other hand, I would also like to learn the right way to do the testing or whatever it takes to make things right.
I am running linux using vmware Workstation Pro 12 on Windows 10. Linux version 4.4.55-1-MANJARO (builduser@manjaro) (gcc version 6.3.1 20170306 (GCC) ) #1 SMP PREEMPT
Dan
On ubuntu16 and ubuntu14, ebtables were pre-installed. But on Archlinux and Debian, ebtables have to be installed explicitly in order to
On Wed, Mar 29, 2017 at 1:01 AM, D L <srwx4096@gmail.com> wrote: pass all test cases in 'make check'. I found out this after I enabled debugging checking $ make check VIR_TEST_DEBUG=2 VIR_TEST_EXPENSIVE=1 For the valgrind testing with 'make -C tests valgrind', the errors were *mostly* due to "FATAL: can't open suppressions file "./.valgrind.supp". I am using valgrind 3.12.0. Dan
On Wed, Mar 29, 2017 at 03:01:06AM -0400, Dan wrote:
For the valgrind testing with 'make -C tests valgrind', the errors were *mostly* due to "FATAL: can't open suppressions file "./.valgrind.supp". I am using valgrind 3.12.0.
Yeah, we reference the suppression file relatively, you need to: cd tests make valgrind instead the -C parameter.
Dan
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On 03/29/2017 09:01 AM, Dan wrote:
On Wed, Mar 29, 2017 at 1:01 AM, D L <srwx4096@gmail.com> wrote:
Hi all,
This email content might be duplicated with other thread posted in other places that I do not know yet where to find or search (please let me know if it is true). I am working on a small bug of virsh domxml-to-native. Before I made any changes to the code in order to fix the bug, I 'make test'-ed and 'valgrind test'-ed the code, encountered four failures in 'make test' and 19 failures in 'make -C tests valgrind'. What I did today was the following:
git checkout master git pull ./autogen.sh --prefix=$(pwd)/build make make install # So far so good, no errors ; however
make check VIR_TEST_EXPENSIVE=1 # generated the following test-suite.log ========================================= libvirt 3.2.0: tests/test-suite.log =========================================
# TOTAL: 115 # PASS: 111 # SKIP: 0 # XFAIL: 0 # FAIL: 4 # XPASS: 0 # ERROR: 0
.. contents:: :depth: 2
FAIL: virfirewalltest =====================
TEST: virfirewalltest !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 40 FAIL FAIL virfirewalltest (exit status: 1)
FAIL: networkxml2firewalltest =============================
TEST: networkxml2firewalltest 0 FAIL FAIL networkxml2firewalltest (exit status: 1)
FAIL: nwfilterebiptablestest ============================
TEST: nwfilterebiptablestest 0 FAIL FAIL nwfilterebiptablestest (exit status: 1)
FAIL: nwfilterxml2firewalltest ==============================
TEST: nwfilterxml2firewalltest 0 FAIL FAIL nwfilterxml2firewalltest (exit status: 1)
make syntax-check VIR_TEST_EXTENSIVE=1 # I did not paste the result here and I do not understand what it means. # available upon request
Finally, make -C tests valgrind # generated file was attached.
It seems most of the valgrind failures were false positives, I can only tell by comparing the log file with the examples in the HACKING page. But I do not have the knowledge to determine.
Could anyone tell me if I am missing something? Having those test failures is a problem or not? Which ones can be safely ignored? Is there any easier way to avoid the false positives? I am concerned because I also need to run the test when I make changes to the code. On the other hand, I would also like to learn the right way to do the testing or whatever it takes to make things right.
I am running linux using vmware Workstation Pro 12 on Windows 10. Linux version 4.4.55-1-MANJARO (builduser@manjaro) (gcc version 6.3.1 20170306 (GCC) ) #1 SMP PREEMPT
Dan
On ubuntu16 and ubuntu14, ebtables were pre-installed. But on Archlinux and Debian, ebtables have to be installed explicitly in order to pass all test cases in 'make check'. I found out this after I enabled debugging checking $ make check VIR_TEST_DEBUG=2 VIR_TEST_EXPENSIVE=1
Ah, very well you. I'm not sure what to do here. I mean, we should definitely skip networkxml2firewalltest, nwfilterebiptablestest and nwfilterxml2firewalltest if EBTABLES is undefined. But the first one is slightly more tricky as it can use DBUS when available. Well, we can skip it it neither of DBUS and EBTABLES is present. This can be your first patch to libvirt ;-)
For the valgrind testing with 'make -C tests valgrind', the errors were *mostly* due to "FATAL: can't open suppressions file "./.valgrind.supp". I am using valgrind 3.12.0.
That's weird, the file should be there. What is a problem though is our mocking. Due to some weird glibc-ness, valgrind cannot handle libraries that have some unresolved symbols. For instance, if library L that is loaded from binary B provides symbols L1 and L2 but rely on symbol B1, valgrind (in fact glibc) fails to load the binary even though if run without valgrind everything works just fine. I've looked into this the other day and went from our sources to valgrind ones and from there to glibc where I got lost. For instance, virpcitest fails because virpcimock uses virAlloc, virFree and other internal APIs even though it's not linking with libvirt_util. Frankly, I've no idea how to fix this. Perhaps Eric have a bright idea? BTW: linking the mock with libvirt_util fails in exactly the opposite case - when not running under valgrind. Michal
On 03/29/2017 04:47 AM, Michal Privoznik wrote:
For the valgrind testing with 'make -C tests valgrind', the errors were *mostly* due to "FATAL: can't open suppressions file "./.valgrind.supp". I am using valgrind 3.12.0.
I suspect it may be a simple tweak to makefile to supply an absolute reference instead of relative to .valgrind.supp so that 'make -C' would find the right file, for this situation.
That's weird, the file should be there. What is a problem though is our mocking. Due to some weird glibc-ness, valgrind cannot handle libraries that have some unresolved symbols. For instance, if library L that is loaded from binary B provides symbols L1 and L2 but rely on symbol B1, valgrind (in fact glibc) fails to load the binary even though if run without valgrind everything works just fine.
I've looked into this the other day and went from our sources to valgrind ones and from there to glibc where I got lost. For instance, virpcitest fails because virpcimock uses virAlloc, virFree and other internal APIs even though it's not linking with libvirt_util.
Frankly, I've no idea how to fix this. Perhaps Eric have a bright idea?
Sorry, no bright ideas here. A strict link ordering (so that no library relies on unresolved symbols) is a good idea for mingw, so it's worth shooting for anyways, and may clean up the valgrind issues, but I'm not sure how easy or hard it would be to rearrange link orders to get to that point.
BTW: linking the mock with libvirt_util fails in exactly the opposite case - when not running under valgrind.
Michal
-- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
participants (5)
-
D L -
Dan
-
Eric Blake -
Martin Kletzander -
Michal Privoznik