[libvirt] [PATCH] Don't crash if a connection closes early

https://bugzilla.redhat.com/show_bug.cgi?id=1047577 When a client closes its connection to libvirtd early during virConnectOpen, more specifically just after making REMOTE_PROC_CONNECT_SUPPORTS_FEATURE call to check if VIR_DRV_FEATURE_PROGRAM_KEEPALIVE is supported without even waiting for the result, libvirtd may crash due to a race in keep-alive initialization. Once receiving the REMOTE_PROC_CONNECT_SUPPORTS_FEATURE call, the daemon's event loop delegates it to a worker thread. In case the event loop detects EOF on the connection and calls virNetServerClientClose before the worker thread starts to handle REMOTE_PROC_CONNECT_SUPPORTS_FEATURE call, client->keepalive will be disposed by the time virNetServerClientStartKeepAlive gets called from remoteDispatchConnectSupportsFeature. Because the flow is common for both authenticated and read-only connections, even unprivileged clients may cause the daemon to crash. To void the crash, virNetServerClientStartKeepAlive needs to check if the connection is still open before starting keep-alive protocol. Every libvirt release since 0.9.8 is affected by this bug. Signed-off-by: Jiri Denemark <jdenemar@redhat.com> --- src/rpc/virnetserverclient.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/src/rpc/virnetserverclient.c b/src/rpc/virnetserverclient.c index 8aebeb0..7830b7f 100644 --- a/src/rpc/virnetserverclient.c +++ b/src/rpc/virnetserverclient.c @@ -1539,9 +1539,22 @@ cleanup: int virNetServerClientStartKeepAlive(virNetServerClientPtr client) { - int ret; + int ret = -1; + virObjectLock(client); + + /* The connection might have been closed before we got here and thus the + * keepalive object could have been removed too. + */ + if (!client->sock) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("connection not open")); + goto cleanup; + } + ret = virKeepAliveStart(client->keepalive, 0, 0); + +cleanup: virObjectUnlock(client); return ret; } -- 1.8.5.2

On 01/10/2014 12:19 PM, Jiri Denemark wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1047577
When a client closes its connection to libvirtd early during virConnectOpen, more specifically just after making REMOTE_PROC_CONNECT_SUPPORTS_FEATURE call to check if VIR_DRV_FEATURE_PROGRAM_KEEPALIVE is supported without even waiting for the result, libvirtd may crash due to a race in keep-alive initialization. Once receiving the REMOTE_PROC_CONNECT_SUPPORTS_FEATURE call, the daemon's event loop delegates it to a worker thread. In case the event loop detects EOF on the connection and calls virNetServerClientClose before the worker thread starts to handle REMOTE_PROC_CONNECT_SUPPORTS_FEATURE call, client->keepalive will be disposed by the time virNetServerClientStartKeepAlive gets called from remoteDispatchConnectSupportsFeature. Because the flow is common for both authenticated and read-only connections, even unprivileged clients may cause the daemon to crash.
To void the crash, virNetServerClientStartKeepAlive needs to check if
s/void/avoid/
the connection is still open before starting keep-alive protocol.
Every libvirt release since 0.9.8 is affected by this bug.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com> --- src/rpc/virnetserverclient.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
ACK. Definitely worth having in 1.2.1. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org

On Fri, Jan 10, 2014 at 13:27:49 -0700, Eric Blake wrote:
On 01/10/2014 12:19 PM, Jiri Denemark wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1047577
When a client closes its connection to libvirtd early during virConnectOpen, more specifically just after making REMOTE_PROC_CONNECT_SUPPORTS_FEATURE call to check if VIR_DRV_FEATURE_PROGRAM_KEEPALIVE is supported without even waiting for the result, libvirtd may crash due to a race in keep-alive initialization. Once receiving the REMOTE_PROC_CONNECT_SUPPORTS_FEATURE call, the daemon's event loop delegates it to a worker thread. In case the event loop detects EOF on the connection and calls virNetServerClientClose before the worker thread starts to handle REMOTE_PROC_CONNECT_SUPPORTS_FEATURE call, client->keepalive will be disposed by the time virNetServerClientStartKeepAlive gets called from remoteDispatchConnectSupportsFeature. Because the flow is common for both authenticated and read-only connections, even unprivileged clients may cause the daemon to crash.
To void the crash, virNetServerClientStartKeepAlive needs to check if
s/void/avoid/
the connection is still open before starting keep-alive protocol.
Every libvirt release since 0.9.8 is affected by this bug.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com> --- src/rpc/virnetserverclient.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
ACK. Definitely worth having in 1.2.1.
Pushed, thanks. Jirka

On Fri, Jan 10, 2014 at 13:27:49 -0700, Eric Blake wrote:
On 01/10/2014 12:19 PM, Jiri Denemark wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1047577
When a client closes its connection to libvirtd early during virConnectOpen, more specifically just after making REMOTE_PROC_CONNECT_SUPPORTS_FEATURE call to check if VIR_DRV_FEATURE_PROGRAM_KEEPALIVE is supported without even waiting for the result, libvirtd may crash due to a race in keep-alive initialization. Once receiving the REMOTE_PROC_CONNECT_SUPPORTS_FEATURE call, the daemon's event loop delegates it to a worker thread. In case the event loop detects EOF on the connection and calls virNetServerClientClose before the worker thread starts to handle REMOTE_PROC_CONNECT_SUPPORTS_FEATURE call, client->keepalive will be disposed by the time virNetServerClientStartKeepAlive gets called from remoteDispatchConnectSupportsFeature. Because the flow is common for both authenticated and read-only connections, even unprivileged clients may cause the daemon to crash.
To void the crash, virNetServerClientStartKeepAlive needs to check if
s/void/avoid/
the connection is still open before starting keep-alive protocol.
Every libvirt release since 0.9.8 is affected by this bug.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com> --- src/rpc/virnetserverclient.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
ACK. Definitely worth having in 1.2.1.
Oops, except that my reproducer which used sleep() to be reliable caused I missed that virNetServerClientClose unlocks and relocks the client object after removing client->keepalive but before removing client->sock. Thus it is still possible to call virKeepAliveStart with a NULL object with this patch. A oneliner fix is coming... Jirka
participants (2)
-
Eric Blake
-
Jiri Denemark