2:09 a.m.
Hi. I'm testing libvirt 3.5.0 and have many issues like this:
Jul 14 09:47:25 cn25 libvirtd[18820]: 2017-07-14 06:47:25.598+0000:
18841: error : qemuDomainObjBeginJobInternal:3872 : Timed out during
operation: cannot acquire state change lock (held by
remoteDispatchConnectGetAllDomainStats)
Jul 14 09:47:25 cn25 libvirtd[18820]: 2017-07-14 06:47:25.598+0000:
18841: warning : qemuDomainObjBeginJobInternal:3860 : Cannot start job
(query, none) for domain 147565; current job is (query, none) owned by
(18828 remoteDispatchConnectGetAllDomainStats, 0 <null>) for (53670s,
0s)
Jul 14 09:46:55 cn25 libvirtd[18820]: 2017-07-14 06:46:55.576+0000:
18836: error : virNetDevTapInterfaceStats:751 : internal error:
/proc/net/dev: Interface not found
Jul 14 09:46:55 cn25 libvirtd[18820]: 2017-07-14 06:46:55.542+0000:
18832: warning : qemuGetProcessInfo:1428 : cannot parse process status
data
Jul 14 09:46:31 cn25 libvirtd[18820]: 2017-07-14 06:46:31.050+0000:
18838: warning : qemuGetProcessInfo:1428 : cannot parse process status
data
Domain in status in shutdown
as i see tap device already removed from the system, so
virNetDevTapInterfaceStats says right info.
My question - why lock now released and why only libvirt restart can
helps in this case
--
Vasiliy Tolstov,
e-mail: v.tolstov(a)selfip.ru
2:27 a.m.
On Fri, Jul 14, 2017 at 10:09:00AM +0300, Vasiliy Tolstov wrote:
Hi. I'm testing libvirt 3.5.0 and have many issues like this:
Jul 14 09:47:25 cn25 libvirtd[18820]: 2017-07-14 06:47:25.598+0000:
18841: error : qemuDomainObjBeginJobInternal:3872 : Timed out during
operation: cannot acquire state change lock (held by
remoteDispatchConnectGetAllDomainStats)
Jul 14 09:47:25 cn25 libvirtd[18820]: 2017-07-14 06:47:25.598+0000:
18841: warning : qemuDomainObjBeginJobInternal:3860 : Cannot start job
(query, none) for domain 147565; current job is (query, none) owned by
(18828 remoteDispatchConnectGetAllDomainStats, 0 <null>) for (53670s,
0s)
Jul 14 09:46:55 cn25 libvirtd[18820]: 2017-07-14 06:46:55.576+0000:
18836: error : virNetDevTapInterfaceStats:751 : internal error:
/proc/net/dev: Interface not found
Jul 14 09:46:55 cn25 libvirtd[18820]: 2017-07-14 06:46:55.542+0000:
18832: warning : qemuGetProcessInfo:1428 : cannot parse process status
data
Jul 14 09:46:31 cn25 libvirtd[18820]: 2017-07-14 06:46:31.050+0000:
18838: warning : qemuGetProcessInfo:1428 : cannot parse process status
data
Domain in status in shutdown
as i see tap device already removed from the system, so
virNetDevTapInterfaceStats says right info.
My question - why lock now released and why only libvirt restart can
helps in this case
What's be stacktrace of libvirtd in this case?
--
Vasiliy Tolstov,
e-mail: v.tolstov(a)selfip.ru
--
libvir-list mailing list
libvir-list(a)redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
3:53 a.m.
2017-07-14 10:27 GMT+03:00 Martin Kletzander <mkletzan(a)redhat.com>:
On Fri, Jul 14, 2017 at 10:09:00AM +0300, Vasiliy Tolstov wrote:
>
> Hi. I'm testing libvirt 3.5.0 and have many issues like this:
>
> Jul 14 09:47:25 cn25 libvirtd[18820]: 2017-07-14 06:47:25.598+0000:
> 18841: error : qemuDomainObjBeginJobInternal:3872 : Timed out during
> operation: cannot acquire state change lock (held by
> remoteDispatchConnectGetAllDomainStats)
> Jul 14 09:47:25 cn25 libvirtd[18820]: 2017-07-14 06:47:25.598+0000:
> 18841: warning : qemuDomainObjBeginJobInternal:3860 : Cannot start job
> (query, none) for domain 147565; current job is (query, none) owned by
> (18828 remoteDispatchConnectGetAllDomainStats, 0 <null>) for (53670s,
> 0s)
> Jul 14 09:46:55 cn25 libvirtd[18820]: 2017-07-14 06:46:55.576+0000:
> 18836: error : virNetDevTapInterfaceStats:751 : internal error:
> /proc/net/dev: Interface not found
> Jul 14 09:46:55 cn25 libvirtd[18820]: 2017-07-14 06:46:55.542+0000:
> 18832: warning : qemuGetProcessInfo:1428 : cannot parse process status
> data
> Jul 14 09:46:31 cn25 libvirtd[18820]: 2017-07-14 06:46:31.050+0000:
> 18838: warning : qemuGetProcessInfo:1428 : cannot parse process status
> data
>
> Domain in status in shutdown
> as i see tap device already removed from the system, so
> virNetDevTapInterfaceStats says right info.
>
> My question - why lock now released and why only libvirt restart can
> helps in this case
What's be stacktrace of libvirtd in this case?
How can i get it when new lock appears?
--
Vasiliy Tolstov,
e-mail: v.tolstov(a)selfip.ru
4:07 a.m.
On Fri, Jul 14, 2017 at 11:53:15AM +0300, Vasiliy Tolstov wrote:
2017-07-14 10:27 GMT+03:00 Martin Kletzander
<mkletzan(a)redhat.com>:
> On Fri, Jul 14, 2017 at 10:09:00AM +0300, Vasiliy Tolstov wrote:
>>
>> Hi. I'm testing libvirt 3.5.0 and have many issues like this:
>>
>> Jul 14 09:47:25 cn25 libvirtd[18820]: 2017-07-14 06:47:25.598+0000:
>> 18841: error : qemuDomainObjBeginJobInternal:3872 : Timed out during
>> operation: cannot acquire state change lock (held by
>> remoteDispatchConnectGetAllDomainStats)
>> Jul 14 09:47:25 cn25 libvirtd[18820]: 2017-07-14 06:47:25.598+0000:
>> 18841: warning : qemuDomainObjBeginJobInternal:3860 : Cannot start job
>> (query, none) for domain 147565; current job is (query, none) owned by
>> (18828 remoteDispatchConnectGetAllDomainStats, 0 <null>) for (53670s,
>> 0s)
>> Jul 14 09:46:55 cn25 libvirtd[18820]: 2017-07-14 06:46:55.576+0000:
>> 18836: error : virNetDevTapInterfaceStats:751 : internal error:
>> /proc/net/dev: Interface not found
>> Jul 14 09:46:55 cn25 libvirtd[18820]: 2017-07-14 06:46:55.542+0000:
>> 18832: warning : qemuGetProcessInfo:1428 : cannot parse process status
>> data
>> Jul 14 09:46:31 cn25 libvirtd[18820]: 2017-07-14 06:46:31.050+0000:
>> 18838: warning : qemuGetProcessInfo:1428 : cannot parse process status
>> data
>>
>> Domain in status in shutdown
>> as i see tap device already removed from the system, so
>> virNetDevTapInterfaceStats says right info.
>>
>> My question - why lock now released and why only libvirt restart can
>> helps in this case
>
>
> What's be stacktrace of libvirtd in this case?
>
How can i get it when new lock appears?
gdb -batch -p $(pgrep libvirtd) -ex 't a a bt'
--
Vasiliy Tolstov,
e-mail: v.tolstov(a)selfip.ru
4:18 a.m.
2017-07-14 12:07 GMT+03:00 Martin Kletzander <mkletzan(a)redhat.com>:
>> What's be stacktrace of libvirtd in this case?
>>
>
>
> How can i get it when new lock appears?
>
gdb -batch -p $(pgrep libvirtd) -ex 't a a bt'
I'm try, does i need to have debugging symbols on this host ?
--
Vasiliy Tolstov,
e-mail: v.tolstov(a)selfip.ru
4:21 a.m.
On Fri, Jul 14, 2017 at 12:18:06PM +0300, Vasiliy Tolstov wrote:
2017-07-14 12:07 GMT+03:00 Martin Kletzander
<mkletzan(a)redhat.com>:
>>> What's be stacktrace of libvirtd in this case?
>>>
>>
>>
>> How can i get it when new lock appears?
>>
>
> gdb -batch -p $(pgrep libvirtd) -ex 't a a bt'
>
>
I'm try, does i need to have debugging symbols on this host ?
That helps a lot
--
Vasiliy Tolstov,
e-mail: v.tolstov(a)selfip.ru
7:15 a.m.
2017-07-14 12:21 GMT+03:00 Martin Kletzander <mkletzan(a)redhat.com>:
On Fri, Jul 14, 2017 at 12:18:06PM +0300, Vasiliy Tolstov wrote:
>
> 2017-07-14 12:07 GMT+03:00 Martin Kletzander <mkletzan(a)redhat.com>:
>
>>>> What's be stacktrace of libvirtd in this case?
>>>>
>>>
>>>
>>> How can i get it when new lock appears?
>>>
>>
>> gdb -batch -p $(pgrep libvirtd) -ex 't a a bt'
>>
>>
>
> I'm try, does i need to have debugging symbols on this host ?
>
That helps a lot
I can't now install dbg symbols but can get backtraces:
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007fe2d463eaed in poll () from /lib/x86_64-linux-gnu/libc.so.6
Thread 47 (Thread 0x7fe2967fc700 (LWP 23281)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 46 (Thread 0x7fe2b2f99700 (LWP 18710)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 45 (Thread 0x7fe2b379a700 (LWP 18709)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 44 (Thread 0x7fe2b3f9b700 (LWP 18708)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 43 (Thread 0x7fe2b479c700 (LWP 18707)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 42 (Thread 0x7fe2b4f9d700 (LWP 18706)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 41 (Thread 0x7fe2bc7b2700 (LWP 18705)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 40 (Thread 0x7fe2bcfb3700 (LWP 18704)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 39 (Thread 0x7fe2bd7b4700 (LWP 18703)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 38 (Thread 0x7fe2bdfb5700 (LWP 18702)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 37 (Thread 0x7fe2be7b6700 (LWP 18701)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 36 (Thread 0x7fe2befb7700 (LWP 18700)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 35 (Thread 0x7fe2bf7b8700 (LWP 18699)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 34 (Thread 0x7fe2bffb9700 (LWP 18698)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 33 (Thread 0x7fe2c07ba700 (LWP 18697)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 32 (Thread 0x7fe2c0fbb700 (LWP 18696)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 31 (Thread 0x7fe2c17bc700 (LWP 18695)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 30 (Thread 0x7fe2c1fbd700 (LWP 18694)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 29 (Thread 0x7fe2c27be700 (LWP 18693)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 28 (Thread 0x7fe2c2fbf700 (LWP 18692)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 27 (Thread 0x7fe2c37c0700 (LWP 18691)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 26 (Thread 0x7fe2c3fc1700 (LWP 18690)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 25 (Thread 0x7fe2c47c2700 (LWP 18689)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 24 (Thread 0x7fe2c4fc3700 (LWP 18688)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 23 (Thread 0x7fe2c57c4700 (LWP 18687)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 22 (Thread 0x7fe2c5fc5700 (LWP 18686)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 21 (Thread 0x7fe2c67c6700 (LWP 18685)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 20 (Thread 0x7fe2c6fc7700 (LWP 18684)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 19 (Thread 0x7fe2c77c8700 (LWP 18683)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 18 (Thread 0x7fe2c7fc9700 (LWP 18682)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 17 (Thread 0x7fe2c87ca700 (LWP 18681)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 16 (Thread 0x7fe2c8fcb700 (LWP 18680)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 15 (Thread 0x7fe2c97cc700 (LWP 18679)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 14 (Thread 0x7fe2c9fcd700 (LWP 18678)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 13 (Thread 0x7fe2ca7ce700 (LWP 18677)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 12 (Thread 0x7fe2cafcf700 (LWP 18676)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 11 (Thread 0x7fe2cb7d0700 (LWP 18675)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 10 (Thread 0x7fe2cbfd1700 (LWP 18674)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 9 (Thread 0x7fe2cc7d2700 (LWP 18673)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 8 (Thread 0x7fe2ccfd3700 (LWP 18672)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 7 (Thread 0x7fe2cd7d4700 (LWP 18671)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 6 (Thread 0x7fe2cdfd5700 (LWP 18670)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2b5973d5b in qemuMonitorSend () from
/usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
#3 0x00007fe2b5987ab5 in ?? () from
/usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
#4 0x00007fe2b5989ff1 in qemuMonitorJSONGetMemoryStats () from
/usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
#5 0x00007fe2b599adb7 in ?? () from
/usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
#6 0x00007fe2b599af7a in ?? () from
/usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
#7 0x00007fe2b599b54d in ?? () from
/usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
#8 0x00007fe2d826e0d5 in virConnectGetAllDomainStats () from
/usr/lib/libvirt.so.0
#9 0x00005556cea0ec70 in ?? ()
#10 0x00007fe2d82ca0d9 in virNetServerProgramDispatch () from
/usr/lib/libvirt.so.0
#11 0x00005556cea3fbe8 in ?? ()
#12 0x00007fe2d81a7598 in ?? () from /usr/lib/libvirt.so.0
#13 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#14 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#15 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 5 (Thread 0x7fe2ce7d6700 (LWP 18669)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 4 (Thread 0x7fe2cefd7700 (LWP 18668)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 3 (Thread 0x7fe2cf7d8700 (LWP 18667)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 2 (Thread 0x7fe2cffd9700 (LWP 18666)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 1 (Thread 0x7fe2d8a198c0 (LWP 18664)):
#0 0x00007fe2d463eaed in poll () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007fe2d81514d7 in virEventPollRunOnce () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81500b1 in virEventRunDefaultImpl () from /usr/lib/libvirt.so.0
#3 0x00007fe2d82c446d in virNetDaemonRun () from /usr/lib/libvirt.so.0
#4 0x00005556cea06282 in ?? ()
#5 0x00007fe2d4580b45 in __libc_start_main () from
/lib/x86_64-linux-gnu/libc.so.6
#6 0x00005556cea06c37 in ?? ()
--
Vasiliy Tolstov,
e-mail: v.tolstov(a)selfip.ru
6:31 a.m.
Another lock with libvirt 3.5.0
root@cn12:~# gdb -batch -p $(pgrep libvirtd) -ex 't a a bt'
warning: Unable to find libthread_db matching inferior's thread
library, thread debugging will not be available.
0x00007f95b5d4bd3d in internal_fallocate (fd=-1177453080,
offset=4294967295, len=121) at ../sysdeps/posix/posix_fallocate.c:88
88 ../sysdeps/posix/posix_fallocate.c: No such file or directory.
Thread 1 (process 16629):
#0 0x00007f95b5d4bd3d in internal_fallocate (fd=-1177453080,
offset=4294967295, len=121) at ../sysdeps/posix/posix_fallocate.c:88
#1 0x00007f95bc0e4530 in ?? ()
#2 0x00007f95bc0e4c40 in ?? ()
#3 0x00007f95bc0e4bd0 in ?? ()
#4 0x00007f95bc0e4b60 in ?? ()
#5 0x00007f95bc0e44e0 in ?? ()
#6 0x0000000000000000 in ?? ()
root@cn12:~#
2017-07-26 15:15 GMT+03:00 Vasiliy Tolstov <v.tolstov(a)selfip.ru>:
2017-07-14 12:21 GMT+03:00 Martin Kletzander
<mkletzan(a)redhat.com>:
> On Fri, Jul 14, 2017 at 12:18:06PM +0300, Vasiliy Tolstov wrote:
>>
>> 2017-07-14 12:07 GMT+03:00 Martin Kletzander <mkletzan(a)redhat.com>:
>>
>>>>> What's be stacktrace of libvirtd in this case?
>>>>>
>>>>
>>>>
>>>> How can i get it when new lock appears?
>>>>
>>>
>>> gdb -batch -p $(pgrep libvirtd) -ex 't a a bt'
>>>
>>>
>>
>> I'm try, does i need to have debugging symbols on this host ?
>>
>
> That helps a lot
>
I can't now install dbg symbols but can get backtraces:
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007fe2d463eaed in poll () from /lib/x86_64-linux-gnu/libc.so.6
Thread 47 (Thread 0x7fe2967fc700 (LWP 23281)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 46 (Thread 0x7fe2b2f99700 (LWP 18710)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 45 (Thread 0x7fe2b379a700 (LWP 18709)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 44 (Thread 0x7fe2b3f9b700 (LWP 18708)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 43 (Thread 0x7fe2b479c700 (LWP 18707)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 42 (Thread 0x7fe2b4f9d700 (LWP 18706)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 41 (Thread 0x7fe2bc7b2700 (LWP 18705)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 40 (Thread 0x7fe2bcfb3700 (LWP 18704)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 39 (Thread 0x7fe2bd7b4700 (LWP 18703)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 38 (Thread 0x7fe2bdfb5700 (LWP 18702)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 37 (Thread 0x7fe2be7b6700 (LWP 18701)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 36 (Thread 0x7fe2befb7700 (LWP 18700)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 35 (Thread 0x7fe2bf7b8700 (LWP 18699)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 34 (Thread 0x7fe2bffb9700 (LWP 18698)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 33 (Thread 0x7fe2c07ba700 (LWP 18697)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 32 (Thread 0x7fe2c0fbb700 (LWP 18696)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 31 (Thread 0x7fe2c17bc700 (LWP 18695)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 30 (Thread 0x7fe2c1fbd700 (LWP 18694)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 29 (Thread 0x7fe2c27be700 (LWP 18693)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 28 (Thread 0x7fe2c2fbf700 (LWP 18692)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 27 (Thread 0x7fe2c37c0700 (LWP 18691)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 26 (Thread 0x7fe2c3fc1700 (LWP 18690)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 25 (Thread 0x7fe2c47c2700 (LWP 18689)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 24 (Thread 0x7fe2c4fc3700 (LWP 18688)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 23 (Thread 0x7fe2c57c4700 (LWP 18687)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 22 (Thread 0x7fe2c5fc5700 (LWP 18686)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a7664 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 21 (Thread 0x7fe2c67c6700 (LWP 18685)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 20 (Thread 0x7fe2c6fc7700 (LWP 18684)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 19 (Thread 0x7fe2c77c8700 (LWP 18683)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 18 (Thread 0x7fe2c7fc9700 (LWP 18682)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 17 (Thread 0x7fe2c87ca700 (LWP 18681)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 16 (Thread 0x7fe2c8fcb700 (LWP 18680)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 15 (Thread 0x7fe2c97cc700 (LWP 18679)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 14 (Thread 0x7fe2c9fcd700 (LWP 18678)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 13 (Thread 0x7fe2ca7ce700 (LWP 18677)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 12 (Thread 0x7fe2cafcf700 (LWP 18676)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 11 (Thread 0x7fe2cb7d0700 (LWP 18675)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 10 (Thread 0x7fe2cbfd1700 (LWP 18674)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 9 (Thread 0x7fe2cc7d2700 (LWP 18673)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 8 (Thread 0x7fe2ccfd3700 (LWP 18672)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 7 (Thread 0x7fe2cd7d4700 (LWP 18671)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 6 (Thread 0x7fe2cdfd5700 (LWP 18670)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2b5973d5b in qemuMonitorSend () from
/usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
#3 0x00007fe2b5987ab5 in ?? () from
/usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
#4 0x00007fe2b5989ff1 in qemuMonitorJSONGetMemoryStats () from
/usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
#5 0x00007fe2b599adb7 in ?? () from
/usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
#6 0x00007fe2b599af7a in ?? () from
/usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
#7 0x00007fe2b599b54d in ?? () from
/usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
#8 0x00007fe2d826e0d5 in virConnectGetAllDomainStats () from
/usr/lib/libvirt.so.0
#9 0x00005556cea0ec70 in ?? ()
#10 0x00007fe2d82ca0d9 in virNetServerProgramDispatch () from
/usr/lib/libvirt.so.0
#11 0x00005556cea3fbe8 in ?? ()
#12 0x00007fe2d81a7598 in ?? () from /usr/lib/libvirt.so.0
#13 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#14 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#15 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 5 (Thread 0x7fe2ce7d6700 (LWP 18669)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 4 (Thread 0x7fe2cefd7700 (LWP 18668)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 3 (Thread 0x7fe2cf7d8700 (LWP 18667)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 2 (Thread 0x7fe2cffd9700 (LWP 18666)):
#0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81a76a3 in ?? () from /usr/lib/libvirt.so.0
#3 0x00007fe2d81a6968 in ?? () from /usr/lib/libvirt.so.0
#4 0x00007fe2d4912064 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007fe2d464762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 1 (Thread 0x7fe2d8a198c0 (LWP 18664)):
#0 0x00007fe2d463eaed in poll () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007fe2d81514d7 in virEventPollRunOnce () from /usr/lib/libvirt.so.0
#2 0x00007fe2d81500b1 in virEventRunDefaultImpl () from /usr/lib/libvirt.so.0
#3 0x00007fe2d82c446d in virNetDaemonRun () from /usr/lib/libvirt.so.0
#4 0x00005556cea06282 in ?? ()
#5 0x00007fe2d4580b45 in __libc_start_main () from
/lib/x86_64-linux-gnu/libc.so.6
#6 0x00005556cea06c37 in ?? ()
--
Vasiliy Tolstov,
e-mail: v.tolstov(a)selfip.ru
--
Vasiliy Tolstov,
e-mail: v.tolstov(a)selfip.ru
1:43 a.m.
On Thu, Aug 24, 2017 at 02:31:05PM +0300, Vasiliy Tolstov wrote:
Another lock with libvirt 3.5.0
root@cn12:~# gdb -batch -p $(pgrep libvirtd) -ex 't a a bt'
warning: Unable to find libthread_db matching inferior's thread
library, thread debugging will not be available.
0x00007f95b5d4bd3d in internal_fallocate (fd=-1177453080,
offset=4294967295, len=121) at ../sysdeps/posix/posix_fallocate.c:88
88 ../sysdeps/posix/posix_fallocate.c: No such file or directory.
I think you have some problem with your gdb/libthread or something. It
showed only one thread.
Thread 1 (process 16629):
#0 0x00007f95b5d4bd3d in internal_fallocate (fd=-1177453080,
offset=4294967295, len=121) at ../sysdeps/posix/posix_fallocate.c:88
#1 0x00007f95bc0e4530 in ?? ()
#2 0x00007f95bc0e4c40 in ?? ()
#3 0x00007f95bc0e4bd0 in ?? ()
#4 0x00007f95bc0e4b60 in ?? ()
#5 0x00007f95bc0e44e0 in ?? ()
#6 0x0000000000000000 in ?? ()
root@cn12:~#
This is not helpful at all. What do you think I can gather from this?
Please, install debugging symbols or run libvirt built with debugging
info directly from git.
2017-07-26 15:15 GMT+03:00 Vasiliy Tolstov
<v.tolstov(a)selfip.ru>:
Everything above was pointless until here:
> Thread 6 (Thread 0x7fe2cdfd5700 (LWP 18670)):
> #0 0x00007fe2d491604f in pthread_cond_wait@(a)GLIBC_2.3.2 () from
> /lib/x86_64-linux-gnu/libpthread.so.0
> #1 0x00007fe2d81a6bc6 in virCondWait () from /usr/lib/libvirt.so.0
We are waiting...
> #2 0x00007fe2b5973d5b in qemuMonitorSend () from
for a reply from qemu monitor...
> /usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
> #3 0x00007fe2b5987ab5 in ?? () from
> /usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
> #4 0x00007fe2b5989ff1 in qemuMonitorJSONGetMemoryStats () from
after asked about memory statistics...
> /usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
> #5 0x00007fe2b599adb7 in ?? () from
> /usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
> #6 0x00007fe2b599af7a in ?? () from
> /usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
> #7 0x00007fe2b599b54d in ?? () from
> /usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
> #8 0x00007fe2d826e0d5 in virConnectGetAllDomainStats () from
using virConnectGetAllDomainStats().
That's the absolute maximum I can gather from this one.
The rest was pointless as well.
2646
days inactive
2688
days old
8 comments
2 participants
participants (2)
-
Martin Kletzander -
Vasiliy Tolstov