18 minutes and 44 seconds | ||
Jim Fehlig | 33c6eb9 Changeset → | |
libvirtd: fix potential deadlock when reloading It is possible to deadlock libvirtd when concurrently starting a domain and restarting the daemon. Threads involved in the deadlock are Thread 4 (Thread 0x7fc13b53e700 (LWP 64084)): /lib64/libpthread.so.0 at util/virthread.c:154 at qemu/qemu_monitor.c:1083 cmd=0x7fc110017700, scm_fd=-1, reply=0x7fc13b53d318) at qemu/qemu_monitor_json.c:305 cmd=0x7fc110017700, reply=0x7fc13b53d318) at qemu/qemu_monitor_json.c:335 at qemu/qemu_monitor_json.c:1298 at qemu/qemu_monitor.c:1697 vm=0x7fc110003d00, asyncJob=QEMU_ASYNC_JOB_START) at qemu/qemu_process.c:1763 vm=0x7fc110003d00, asyncJob=6, logCtxt=0x7fc1100089c0) at qemu/qemu_process.c:1835 vm=0x7fc110003d00, asyncJob=6, logCtxt=0x7fc1100089c0) at qemu/qemu_process.c:2180 driver=0x7fc12004e1e0, vm=0x7fc110003d00, asyncJob=QEMU_ASYNC_JOB_START, incoming=0x0, snapshot=0x0, vmop=VIR_NETDEV_VPORT_PROFILE_OP_CREATE, flags=17) at qemu/qemu_process.c:6111 driver=0x7fc12004e1e0, vm=0x7fc110003d00, updatedCPU=0x0, asyncJob=QEMU_ASYNC_JOB_START, migrateFrom=0x0, migrateFd=-1, migratePath=0x0, snapshot=0x0, vmop=VIR_NETDEV_VPORT_PROFILE_OP_CREATE, flags=17) at qemu/qemu_process.c:6334 xml=0x7fc110000ed0 "<!--\nWARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE\nOVERWRITTEN AND LOST. Changes to this xml configuration should be made using:\n virsh edit testvv\nor other applicati"..., flags=0) at qemu/qemu_driver.c:1776 ... Thread 1 (Thread 0x7fc143c66880 (LWP 64081)): /lib64/libpthread.so.0 at util/virthread.c:122 conf/nwfilter_conf.c:159 sig=0x7ffe0a831e30, opaque=0x0) at remote/remote_daemon.c:724 opaque=0x558c5328b230) at rpc/virnetdaemon.c:654 at util/vireventpoll.c:508 rpc/virnetdaemon.c:858 remote/remote_daemon.c:1496 (gdb) thr 1 [Switching to thread 1 (Thread 0x7fc143c66880 (LWP 64081))] /lib64/libpthread.so.0 (gdb) f 1 at util/virthread.c:122 122 pthread_rwlock_wrlock(&m->lock); (gdb) p updateLock $1 = {lock = {__data = {__lock = 0, __nr_readers = 1, __readers_wakeup = 0, __writer_wakeup = 0, __nr_readers_queued = 0, __nr_writers_queued = 1, __writer = 0, __shared = 0, __rwelision = 0 '\000', __pad1 = "\000\000\000\000\000\000", __pad2 = 0, __flags = 0}, __size = "\000\000\000\000\001", '\000' <repeats 15 times>, "\001", '\000' <repeats 34 times>, __align = 4294967296}} Reloading of the nwfilter driver is stuck waiting for a write lock, which already has a reader (from qemuDomainCreateXML) in the critical section. Since the reload occurs in the context of the main event loop thread, libvirtd becomes deadlocked. The deadlock can be avoided by offloading the reload work to a thread. Signed-off-by: Jim Fehlig <jfehlig@suse.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> |