On Fri, Jul 05, 2019 at 09:47:14AM +0100, Daniel P. Berrangé wrote:
On Thu, Jul 04, 2019 at 12:15:25PM +0200, Sahid Orentino Ferdjaoui wrote:
The open_auth test cases were randomly failling. It seems that because tests are executed in parallel if a sasl connection is open when an other happen in same time from the same libvirt connection an error happens: "Failed to start SASL negotiation: -4 (SASL(-4): no mechanism available: No worthy mechs found)".
This doesn't really make sense. Libvirt itself is fully multi-threaded so it should be fine to be opening connections in parallel.
If there's a non-deterministic failure happening this suggests a race condition bug somewhere, which shouldn't just be ignored by changing the tests to avoid hitting the race,
You are completely right I did lot of tests and I was not able to find the root cause. I was only able to reproduce the problem when I'm using the test framework and even with that if I'm using test-threads=1, no problems, or if I'm executing only one of them several times, no problems. Even, running 2 of example/auth.rs in parallel lot of time does not cause any issues. That is the leak which sometime happens: ==20734== ==20734== HEAP SUMMARY: ==20734== in use at exit: 165,843 bytes in 852 blocks ==20734== total heap usage: 4,141 allocs, 3,289 frees, 1,923,649 bytes allocated ==20734== ==20734== 544 bytes in 1 blocks are definitely lost in loss record 103 of 121 ==20734== at 0x4C2CE8E: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==20734== by 0xA780E5E: _plug_buf_alloc (in /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25) ==20734== by 0xA77A53D: ??? (in /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25) ==20734== by 0xA77F017: ??? (in /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25) ==20734== by 0xA77F8B0: ??? (in /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25) ==20734== by 0x7DF85EF: sasl_client_step (in /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25) ==20734== by 0x7DF896D: sasl_client_start (in /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25) ==20734== by 0x4FF0756: virNetSASLSessionClientStart (in /usr/lib/libvirt.so.0.1002.2) ==20734== by 0x4FD1AD4: ??? (in /usr/lib/libvirt.so.0.1002.2) ==20734== by 0x4FD27E5: ??? (in /usr/lib/libvirt.so.0.1002.2) ==20734== by 0x4F6C4E5: ??? (in /usr/lib/libvirt.so.0.1002.2) ==20734== by 0x4F6F00C: virConnectOpenAuth (in /usr/lib/libvirt.so.0.1002.2) ==20734== ==20734== LEAK SUMMARY: ==20734== definitely lost: 544 bytes in 1 blocks ==20734== indirectly lost: 0 bytes in 0 blocks ==20734== possibly lost: 0 bytes in 0 blocks ==20734== still reachable: 165,299 bytes in 851 blocks ==20734== suppressed: 0 bytes in 0 blocks ==20734== Reachable blocks (those to which a pointer was found) are not shown. ==20734== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==20734== ==20734== For counts of detected and suppressed errors, rerun with: -v ==20734== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) I also have no clues why the heap is not completely freed. I will try to investigate a bit more. Thanks, s.
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@canonical.com> --- tests/integration_qemu.rs | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)
diff --git a/tests/integration_qemu.rs b/tests/integration_qemu.rs index 49e07c4..7db43d5 100644 --- a/tests/integration_qemu.rs +++ b/tests/integration_qemu.rs @@ -89,6 +89,16 @@ fn test_create_storage_pool_and_vols() { #[test] #[ignore] fn test_connection_with_auth() { + // Rust is excecuting tests in parallel (threads), if a sasl + // connection is open when an other happen in same time from the + // same libvirt connection an error happens: "Failed to start SASL + // negotiation: -4 (SASL(-4): no mechanism available: No worthy + // mechs found)". + connection_with_auth_ok(); + connection_with_auth_wrong(); +} + +fn connection_with_auth_ok() { fn callback(creds: &mut Vec<ConnectCredential>) { for cred in creds { match cred.typed { @@ -118,10 +128,7 @@ fn test_connection_with_auth() { } }
- -#[test] -#[ignore] -fn test_connection_with_auth_wrong() { +fn connection_with_auth_wrong() { fn callback(creds: &mut Vec<ConnectCredential>) { for cred in creds { match cred.typed { -- 2.20.1
Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|