Re: [libvirt-users] libvirtd hangs

Wednesday, 31 January 2018

Ján Tomko писал 2018-01-25 16:04:
...
 On Thu, Jan 25, 2018 at 11:17:54AM +0700, Artem Likhachev wrote:
> Ján Tomko wrote 2018-01-18 14:27:
>> On Thu, Jan 18, 2018 at 11:30:16AM +0700, Artem Likhachev wrote:
> 
>>> # gdb -p 5786
>>> (gdb) thread apply all bt
>>> 

 [...]

>>> Thread 13 (Thread 0x7f41189d3700 (LWP 5792)):
>>> ---Type <return> to continue, or q <return> to quit---
>>> 
>> 
>> You omitted the backtrace of the other threads.
>> 
>> Jan
> 
> Please, specify what do you mean? What should I do? I'm not familiar
> with gdb at good level.
> 

 When gdb asks:
 '---Type <return> to continue, or q <return> to quit---'
 keep pressing Enter until it displays the backtrace of all the threads,
 not just the ones that fit on the first screen.

 Jan

 !DSPAM:5a699db46321070757529! 
Sorry, here is is:

(gdb) thread apply all bt

Thread 17 (Thread 0x7f360fc20700 (LWP 993)):
#0  pthread_cond_wait@(a)GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f361eeee2e6 in virCondWait (c=c@entry=0x7f3621ae9bb8, 
m=m@entry=0x7f3621ae9b90) at util/virthread.c:154
#2  0x00007f361eeeeda3 in virThreadPoolWorker 
(opaque=opaque@entry=0x7f3621adeab0) at util/virthreadpool.c:124
#3  0x00007f361eeee078 in virThreadHelper (data=<optimized out>) at 
util/virthread.c:206
#4  0x00007f361c2f7dc5 in start_thread (arg=0x7f360fc20700) at 
pthread_create.c:308
#5  0x00007f361c02673d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 16 (Thread 0x7f360f41f700 (LWP 994)):
#0  pthread_cond_wait@(a)GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f361eeee2e6 in virCondWait (c=c@entry=0x7f3621ae9bb8, 
m=m@entry=0x7f3621ae9b90) at util/virthread.c:154
#2  0x00007f361eeeeda3 in virThreadPoolWorker 
(opaque=opaque@entry=0x7f3621adea00) at util/virthreadpool.c:124
#3  0x00007f361eeee078 in virThreadHelper (data=<optimized out>) at 
util/virthread.c:206
#4  0x00007f361c2f7dc5 in start_thread (arg=0x7f360f41f700) at 
pthread_create.c:308
#5  0x00007f361c02673d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 15 (Thread 0x7f360ec1e700 (LWP 995)):
#0  pthread_cond_wait@(a)GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f361eeee2e6 in virCondWait (c=c@entry=0x7f3621ae9bb8, 
m=m@entry=0x7f3621ae9b90) at util/virthread.c:154
#2  0x00007f361eeeeda3 in virThreadPoolWorker 
(opaque=opaque@entry=0x7f3621ade950) at util/virthreadpool.c:124
#3  0x00007f361eeee078 in virThreadHelper (data=<optimized out>) at 
util/virthread.c:206
#4  0x00007f361c2f7dc5 in start_thread (arg=0x7f360ec1e700) at 
pthread_create.c:308
#5  0x00007f361c02673d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 14 (Thread 0x7f360e41d700 (LWP 996)):
#0  pthread_cond_wait@(a)GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f361eeee2e6 in virCondWait (c=c@entry=0x7f3621ae9bb8, 
m=m@entry=0x7f3621ae9b90) at util/virthread.c:154
#2  0x00007f361eeeeda3 in virThreadPoolWorker 
(opaque=opaque@entry=0x7f3621ade8a0) at util/virthreadpool.c:124
#3  0x00007f361eeee078 in virThreadHelper (data=<optimized out>) at 
util/virthread.c:206
#4  0x00007f361c2f7dc5 in start_thread (arg=0x7f360e41d700) at 
pthread_create.c:308
#5  0x00007f361c02673d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 13 (Thread 0x7f360dc1c700 (LWP 997)):
---Type <return> to continue, or q <return> to quit---
#0  pthread_cond_wait@(a)GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f361eeee2e6 in virCondWait (c=c@entry=0x7f3621ae9bb8, 
m=m@entry=0x7f3621ae9b90) at util/virthread.c:154
#2  0x00007f361eeeeda3 in virThreadPoolWorker 
(opaque=opaque@entry=0x7f3621adea00) at util/virthreadpool.c:124
#3  0x00007f361eeee078 in virThreadHelper (data=<optimized out>) at 
util/virthread.c:206
#4  0x00007f361c2f7dc5 in start_thread (arg=0x7f360dc1c700) at 
pthread_create.c:308
#5  0x00007f361c02673d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 12 (Thread 0x7f360d41b700 (LWP 998)):
#0  pthread_cond_wait@(a)GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f361eeee2e6 in virCondWait (c=c@entry=0x7f3621ae9c58, 
m=m@entry=0x7f3621ae9b90) at util/virthread.c:154
#2  0x00007f361eeeed3b in virThreadPoolWorker 
(opaque=opaque@entry=0x7f3621ade8a0) at util/virthreadpool.c:124
#3  0x00007f361eeee078 in virThreadHelper (data=<optimized out>) at 
util/virthread.c:206
#4  0x00007f361c2f7dc5 in start_thread (arg=0x7f360d41b700) at 
pthread_create.c:308
#5  0x00007f361c02673d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 11 (Thread 0x7f360cc1a700 (LWP 999)):
#0  pthread_cond_wait@(a)GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f361eeee2e6 in virCondWait (c=c@entry=0x7f3621ae9c58, 
m=m@entry=0x7f3621ae9b90) at util/virthread.c:154
#2  0x00007f361eeeed3b in virThreadPoolWorker 
(opaque=opaque@entry=0x7f3621ade950) at util/virthreadpool.c:124
#3  0x00007f361eeee078 in virThreadHelper (data=<optimized out>) at 
util/virthread.c:206
#4  0x00007f361c2f7dc5 in start_thread (arg=0x7f360cc1a700) at 
pthread_create.c:308
#5  0x00007f361c02673d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 10 (Thread 0x7f360c419700 (LWP 1000)):
#0  pthread_cond_wait@(a)GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f361eeee2e6 in virCondWait (c=c@entry=0x7f3621ae9c58, 
m=m@entry=0x7f3621ae9b90) at util/virthread.c:154
#2  0x00007f361eeeed3b in virThreadPoolWorker 
(opaque=opaque@entry=0x7f3621adea00) at util/virthreadpool.c:124
#3  0x00007f361eeee078 in virThreadHelper (data=<optimized out>) at 
util/virthread.c:206
#4  0x00007f361c2f7dc5 in start_thread (arg=0x7f360c419700) at 
pthread_create.c:308
#5  0x00007f361c02673d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 9 (Thread 0x7f360bc18700 (LWP 1001)):
#0  pthread_cond_wait@(a)GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f361eeee2e6 in virCondWait (c=c@entry=0x7f3621ae9c58, 
m=m@entry=0x7f3621ae9b90) at util/virthread.c:154
---Type <return> to continue, or q <return> to quit---
#2  0x00007f361eeeed3b in virThreadPoolWorker 
(opaque=opaque@entry=0x7f3621ade8a0) at util/virthreadpool.c:124
#3  0x00007f361eeee078 in virThreadHelper (data=<optimized out>) at 
util/virthread.c:206
#4  0x00007f361c2f7dc5 in start_thread (arg=0x7f360bc18700) at 
pthread_create.c:308
#5  0x00007f361c02673d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 8 (Thread 0x7f360b417700 (LWP 1002)):
#0  pthread_cond_wait@(a)GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f361eeee2e6 in virCondWait (c=c@entry=0x7f3621ae9c58, 
m=m@entry=0x7f3621ae9b90) at util/virthread.c:154
#2  0x00007f361eeeed3b in virThreadPoolWorker 
(opaque=opaque@entry=0x7f3621adea00) at util/virthreadpool.c:124
#3  0x00007f361eeee078 in virThreadHelper (data=<optimized out>) at 
util/virthread.c:206
#4  0x00007f361c2f7dc5 in start_thread (arg=0x7f360b417700) at 
pthread_create.c:308
#5  0x00007f361c02673d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 7 (Thread 0x7f360136f700 (LWP 1003)):
#0  pthread_cond_wait@(a)GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f361eeee2e6 in virCondWait (c=c@entry=0x7f3621b01f68, 
m=m@entry=0x7f3621b01f40) at util/virthread.c:154
#2  0x00007f361eeeeda3 in virThreadPoolWorker 
(opaque=opaque@entry=0x7f3621add6c0) at util/virthreadpool.c:124
#3  0x00007f361eeee078 in virThreadHelper (data=<optimized out>) at 
util/virthread.c:206
#4  0x00007f361c2f7dc5 in start_thread (arg=0x7f360136f700) at 
pthread_create.c:308
#5  0x00007f361c02673d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 6 (Thread 0x7f3600b6e700 (LWP 1004)):
#0  pthread_cond_wait@(a)GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f361eeee2e6 in virCondWait (c=c@entry=0x7f3621b01f68, 
m=m@entry=0x7f3621b01f40) at util/virthread.c:154
#2  0x00007f361eeeeda3 in virThreadPoolWorker 
(opaque=opaque@entry=0x7f3621add610) at util/virthreadpool.c:124
#3  0x00007f361eeee078 in virThreadHelper (data=<optimized out>) at 
util/virthread.c:206
#4  0x00007f361c2f7dc5 in start_thread (arg=0x7f3600b6e700) at 
pthread_create.c:308
#5  0x00007f361c02673d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 5 (Thread 0x7f360036d700 (LWP 1005)):
#0  pthread_cond_wait@(a)GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f361eeee2e6 in virCondWait (c=c@entry=0x7f3621b01f68, 
m=m@entry=0x7f3621b01f40) at util/virthread.c:154
#2  0x00007f361eeeeda3 in virThreadPoolWorker 
(opaque=opaque@entry=0x7f3621add6c0) at util/virthreadpool.c:124
#3  0x00007f361eeee078 in virThreadHelper (data=<optimized out>) at 
util/virthread.c:206
---Type <return> to continue, or q <return> to quit---
#4  0x00007f361c2f7dc5 in start_thread (arg=0x7f360036d700) at 
pthread_create.c:308
#5  0x00007f361c02673d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 4 (Thread 0x7f35ffb6c700 (LWP 1007)):
#0  pthread_cond_wait@(a)GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f361eeee2e6 in virCondWait (c=c@entry=0x7f3621b01f68, 
m=m@entry=0x7f3621b01f40) at util/virthread.c:154
#2  0x00007f361eeeeda3 in virThreadPoolWorker 
(opaque=opaque@entry=0x7f3621add610) at util/virthreadpool.c:124
#3  0x00007f361eeee078 in virThreadHelper (data=<optimized out>) at 
util/virthread.c:206
#4  0x00007f361c2f7dc5 in start_thread (arg=0x7f35ffb6c700) at 
pthread_create.c:308
#5  0x00007f361c02673d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 3 (Thread 0x7f35ff36b700 (LWP 1010)):
#0  pthread_cond_wait@(a)GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f361eeee2e6 in virCondWait (c=c@entry=0x7f3621b01f68, 
m=m@entry=0x7f3621b01f40) at util/virthread.c:154
#2  0x00007f361eeeeda3 in virThreadPoolWorker 
(opaque=opaque@entry=0x7f3621add6c0) at util/virthreadpool.c:124
#3  0x00007f361eeee078 in virThreadHelper (data=<optimized out>) at 
util/virthread.c:206
#4  0x00007f361c2f7dc5 in start_thread (arg=0x7f35ff36b700) at 
pthread_create.c:308
#5  0x00007f361c02673d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 2 (Thread 0x7f35abfff700 (LWP 13299)):
#0  pthread_cond_wait@(a)GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f361eeee2e6 in virCondWait (c=c@entry=0x7f35f82d3d68, 
m=m@entry=0x7f35f82d3d40) at util/virthread.c:154
#2  0x00007f361eeeeda3 in virThreadPoolWorker 
(opaque=opaque@entry=0x7f3621ad8890) at util/virthreadpool.c:124
#3  0x00007f361eeee078 in virThreadHelper (data=<optimized out>) at 
util/virthread.c:206
#4  0x00007f361c2f7dc5 in start_thread (arg=0x7f35abfff700) at 
pthread_create.c:308
#5  0x00007f361c02673d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 1 (Thread 0x7f361fb94880 (LWP 850)):
#0  0x00007f361c01bdfd in poll () at 
../sysdeps/unix/syscall-template.S:81
#1  0x00007f361ee97f96 in poll (__timeout=5000, __nfds=39, 
__fds=<optimized out>) at /usr/include/bits/poll2.h:46
#2  virEventPollRunOnce () at util/vireventpoll.c:641
#3  0x00007f361ee96a72 in virEventRunDefaultImpl () at 
util/virevent.c:314
#4  0x00007f361f0053ad in virNetDaemonRun (dmn=dmn@entry=0x7f3621aeb770) 
at rpc/virnetdaemon.c:818
#5  0x00007f361fbf2e0c in main (argc=<optimized out>, argv=<optimized 
out>) at libvirtd.c:1541
(gdb)
(gdb)

-- 
Artem Likhachev

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Re: [libvirt-users] libvirtd hangs