12:20 p.m.
Hello.
We faced a very strange leak while using libvirt library in long-running server
application for cluster orchestration.
Leak does not directly related to libvirt code and exposed only on specific build options
(?) and/or system environment (?).
Here are the key points:
1. Libvirt client leaks memory while making (RPC) calls to a server. I mean that RSS
memory usage shown by ps, top, etc. indefinitely grows (plz check an attachment ps.log).
Test app attached.
2. Leak detected on Debian and Ubuntu and absent on Mac OS and Gentoo, so it's exactly
an environment or build problem.
3. Valgrind does not see the leak. From valgrind's point of view from start to finish
application constantly consumes 110kb of memory (while ps shows multiple megabytes) and
does not contain any leaks.
4. Logging activity of virMalloc, virRealloc, etc. functions does not show anything: as
expected, all allocated memory is correctly freed, so it's definitely not a bug of
code (we tested that before recognized that problem is distrib/platform specific).
Some useful logs and test code attached. I think digging build options and legoing system
libraries will help us to beat the problem but it would be nice if someone already had a
working solution. Thank you for help.
2:41 p.m.
Hello.
We did some progress in our search, here is there news:
1. Leak is NOT platform dependent, we tested different versions on different platforms and
made a wrong conclusion.
2. Leak originated somewhere in changes between versions 1.0.1 and v 1.0.2.
15.02.2013, в 22:20, Igor Lukyanov <igor(a)lukyanov.org> написал(а):
Hello.
We faced a very strange leak while using libvirt library in long-running server
application for cluster orchestration.
Leak does not directly related to libvirt code and exposed only on specific build options
(?) and/or system environment (?).
Here are the key points:
1. Libvirt client leaks memory while making (RPC) calls to a server. I mean that RSS
memory usage shown by ps, top, etc. indefinitely grows (plz check an attachment ps.log).
Test app attached.
2. Leak detected on Debian and Ubuntu and absent on Mac OS and Gentoo, so it's
exactly an environment or build problem.
3. Valgrind does not see the leak. From valgrind's point of view from start to finish
application constantly consumes 110kb of memory (while ps shows multiple megabytes) and
does not contain any leaks.
4. Logging activity of virMalloc, virRealloc, etc. functions does not show anything: as
expected, all allocated memory is correctly freed, so it's definitely not a bug of
code (we tested that before recognized that problem is distrib/platform specific).
Some useful logs and test code attached. I think digging build options and legoing system
libraries will help us to beat the problem but it would be nice if someone already had a
working solution. Thank you for help.
<leak.c><ps.log><valgrind.log>
8:20 a.m.
New subject: [libvirt] [PATCH] Fix memory leak in virNetClientIOWriteMessage
Commit 18937c3ae0990b4417a43aa07a2c35aaf8cb6ec2 introduced the
memory leak when client->msg.fds is copied to thecall->msg
and then never freed.
---
src/rpc/virnetclient.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/rpc/virnetclient.c b/src/rpc/virnetclient.c
index 4efa578..bfa1624 100644
--- a/src/rpc/virnetclient.c
+++ b/src/rpc/virnetclient.c
@@ -1182,6 +1182,7 @@ virNetClientIOWriteMessage(virNetClientPtr client,
}
thecall->msg->donefds = 0;
thecall->msg->bufferOffset = thecall->msg->bufferLength = 0;
+ VIR_FREE(thecall->msg->fds);
VIR_FREE(thecall->msg->buffer);
if (thecall->expectReply)
thecall->mode = VIR_NET_CLIENT_MODE_WAIT_RX;
--
1.7.10.4
5:01 a.m.
New subject: [libvirt] [PATCH] Fix memory leak in virNetClientIOWriteMessage
Commit 18937c3ae0990b4417a43aa07a2c35aaf8cb6ec2 introduced the
memory leak when client->msg.fds is moved to thecall->msg
and then never freed.
---
Hello,
Please ignore previous patch with incorrent formatting.
src/rpc/virnetclient.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/rpc/virnetclient.c b/src/rpc/virnetclient.c
index 4efa578..bfa1624 100644
--- a/src/rpc/virnetclient.c
+++ b/src/rpc/virnetclient.c
@@ -1182,6 +1182,7 @@ virNetClientIOWriteMessage(virNetClientPtr client,
}
thecall->msg->donefds = 0;
thecall->msg->bufferOffset = thecall->msg->bufferLength = 0;
+ VIR_FREE(thecall->msg->fds);
VIR_FREE(thecall->msg->buffer);
if (thecall->expectReply)
thecall->mode = VIR_NET_CLIENT_MODE_WAIT_RX;
--
1.7.10.4
11:01 a.m.
New subject: [libvirt] [PATCH] Fix memory leak in virNetClientIOWriteMessage
On 18.02.2013 12:01, Sergey Fionov wrote:
Commit 18937c3ae0990b4417a43aa07a2c35aaf8cb6ec2 introduced the
memory leak when client->msg.fds is moved to thecall->msg
and then never freed.
---
Hello,
Please ignore previous patch with incorrent formatting.
src/rpc/virnetclient.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/rpc/virnetclient.c b/src/rpc/virnetclient.c
index 4efa578..bfa1624 100644
--- a/src/rpc/virnetclient.c
+++ b/src/rpc/virnetclient.c
@@ -1182,6 +1182,7 @@ virNetClientIOWriteMessage(virNetClientPtr client,
}
thecall->msg->donefds = 0;
thecall->msg->bufferOffset = thecall->msg->bufferLength = 0;
+ VIR_FREE(thecall->msg->fds);
VIR_FREE(thecall->msg->buffer);
if (thecall->expectReply)
thecall->mode = VIR_NET_CLIENT_MODE_WAIT_RX;
ACKed and pushed. Fortunately, the commit that introduced this is not
present in any of our stable branches, so we don't need to backport it.
Michal
4:31 a.m.
On 15.02.2013 19:20, Igor Lukyanov wrote:
Hello.
We faced a very strange leak while using libvirt library in long-running server
application for cluster orchestration.
Leak does not directly related to libvirt code and exposed only on specific build options
(?) and/or system environment (?).
Here are the key points:
1. Libvirt client leaks memory while making (RPC) calls to a server. I mean that RSS
memory usage shown by ps, top, etc. indefinitely grows (plz check an attachment ps.log).
Test app attached.
2. Leak detected on Debian and Ubuntu and absent on Mac OS and Gentoo, so it's
exactly an environment or build problem.
3. Valgrind does not see the leak. From valgrind's point of view from start to finish
application constantly consumes 110kb of memory (while ps shows multiple megabytes) and
does not contain any leaks.
4. Logging activity of virMalloc, virRealloc, etc. functions does not show anything: as
expected, all allocated memory is correctly freed, so it's definitely not a bug of
code (we tested that before recognized that problem is distrib/platform specific).
Some useful logs and test code attached. I think digging build options and legoing system
libraries will help us to beat the problem but it would be nice if someone already had a
working solution. Thank you for help.
--
libvir-list mailing list
libvir-list(a)redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
I don't think there's any leak. I mean, not the true one. From your
attached valgrind output, and from my own try, I see the only libvirt
leak is the one by virInitialize() where init of underlying libraries
(like gnutls) takes place. Unfortunately, we cannot de-init them (e.g.
by introducing virDeinit API) since the mgmt application can still want
to use the libraries. For more info see discussion in:
http://www.redhat.com/archives/libvir-list/2011-August/msg01127.html
Michal
4293
days inactive
4298
days old
5 comments
3 participants
participants (3)
-
Igor Lukyanov -
Michal Privoznik -
Sergey Fionov