On Fri, Jul 05, 2019 at 09:47:14AM +0100, Daniel P. Berrangé wrote:
On Thu, Jul 04, 2019 at 12:15:25PM +0200, Sahid Orentino Ferdjaoui
wrote:
> The open_auth test cases were randomly failling. It seems that because
> tests are executed in parallel if a sasl connection is open when an
> other happen in same time from the same libvirt connection an error
> happens: "Failed to start SASL negotiation: -4 (SASL(-4): no mechanism
> available: No worthy mechs found)".
This doesn't really make sense. Libvirt itself is fully multi-threaded
so it should be fine to be opening connections in parallel.
If there's a non-deterministic failure happening this suggests a race
condition bug somewhere, which shouldn't just be ignored by changing
the tests to avoid hitting the race,
You are completely right I did lot of tests and I was not able to find
the root cause. I was only able to reproduce the problem when I'm
using the test framework and even with that if I'm using
test-threads=1, no problems, or if I'm executing only one of them
several times, no problems. Even, running 2 of example/auth.rs in
parallel lot of time does not cause any issues.
That is the leak which sometime happens:
==20734==
==20734== HEAP SUMMARY:
==20734== in use at exit: 165,843 bytes in 852 blocks
==20734== total heap usage: 4,141 allocs, 3,289 frees, 1,923,649 bytes allocated
==20734==
==20734== 544 bytes in 1 blocks are definitely lost in loss record 103 of 121
==20734== at 0x4C2CE8E: realloc (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==20734== by 0xA780E5E: _plug_buf_alloc (in
/usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25)
==20734== by 0xA77A53D: ??? (in
/usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25)
==20734== by 0xA77F017: ??? (in
/usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25)
==20734== by 0xA77F8B0: ??? (in
/usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25)
==20734== by 0x7DF85EF: sasl_client_step (in
/usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25)
==20734== by 0x7DF896D: sasl_client_start (in
/usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25)
==20734== by 0x4FF0756: virNetSASLSessionClientStart (in /usr/lib/libvirt.so.0.1002.2)
==20734== by 0x4FD1AD4: ??? (in /usr/lib/libvirt.so.0.1002.2)
==20734== by 0x4FD27E5: ??? (in /usr/lib/libvirt.so.0.1002.2)
==20734== by 0x4F6C4E5: ??? (in /usr/lib/libvirt.so.0.1002.2)
==20734== by 0x4F6F00C: virConnectOpenAuth (in /usr/lib/libvirt.so.0.1002.2)
==20734==
==20734== LEAK SUMMARY:
==20734== definitely lost: 544 bytes in 1 blocks
==20734== indirectly lost: 0 bytes in 0 blocks
==20734== possibly lost: 0 bytes in 0 blocks
==20734== still reachable: 165,299 bytes in 851 blocks
==20734== suppressed: 0 bytes in 0 blocks
==20734== Reachable blocks (those to which a pointer was found) are not shown.
==20734== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==20734==
==20734== For counts of detected and suppressed errors, rerun with: -v
==20734== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
I also have no clues why the heap is not completely freed. I will try
to investigate a bit more.
Thanks,
s.
>
> Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui(a)canonical.com>
> ---
> tests/integration_qemu.rs | 15 +++++++++++----
> 1 file changed, 11 insertions(+), 4 deletions(-)
>
> diff --git a/tests/integration_qemu.rs b/tests/integration_qemu.rs
> index 49e07c4..7db43d5 100644
> --- a/tests/integration_qemu.rs
> +++ b/tests/integration_qemu.rs
> @@ -89,6 +89,16 @@ fn test_create_storage_pool_and_vols() {
> #[test]
> #[ignore]
> fn test_connection_with_auth() {
> + // Rust is excecuting tests in parallel (threads), if a sasl
> + // connection is open when an other happen in same time from the
> + // same libvirt connection an error happens: "Failed to start SASL
> + // negotiation: -4 (SASL(-4): no mechanism available: No worthy
> + // mechs found)".
> + connection_with_auth_ok();
> + connection_with_auth_wrong();
> +}
> +
> +fn connection_with_auth_ok() {
> fn callback(creds: &mut Vec<ConnectCredential>) {
> for cred in creds {
> match cred.typed {
> @@ -118,10 +128,7 @@ fn test_connection_with_auth() {
> }
> }
>
> -
> -#[test]
> -#[ignore]
> -fn test_connection_with_auth_wrong() {
> +fn connection_with_auth_wrong() {
> fn callback(creds: &mut Vec<ConnectCredential>) {
> for cred in creds {
> match cred.typed {
> --
> 2.20.1
>
Regards,
Daniel
--
|:
https://berrange.com -o-
https://www.flickr.com/photos/dberrange :|
|:
https://libvirt.org -o-
https://fstop138.berrange.com :|
|:
https://entangle-photo.org -o-
https://www.instagram.com/dberrange :|