Re: [libvirt] increase number of libvirt threads by starting tansient guest doamin - is it a bug?
Hello, sorry for my later answer. so, after i started libvirtd (no vm´s running) and attach gdb i get the following threads (gdb) info thread Id Target Id Frame 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 * 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 0x00007f190677d7cd in poll () at ../sysdeps/unix/syscall-template.S:81 if i restore an persistent domain, i see the following in gdb: Detaching after fork from child process 20880. Detaching after fork from child process 20882. [New Thread 0x7f190893d700 (LWP 20883)] Detaching after fork from child process 20890. Detaching after fork from child process 20906. (gdb) info thread Id Target Id Frame 12 Thread 0x7f190893d700 (LWP 20883) "libvirtd" 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 * 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 0x00007f190677d7cd in poll () at ../sysdeps/unix/syscall-template.S:81 if i now destroy this vm i get the following: (gdb) info thread Id Target Id Frame 12 Thread 0x7f190893d700 (LWP 20883) "libvirtd" 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 * 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 0x00007f190677d7cd in poll () at ../sysdeps/unix/syscall-template.S:81 attaching thread Id 12 and bt: (gdb) thread 12 [Switching to thread 12 (Thread 0x7f190893d700 (LWP 20883))] #0 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 81 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS) (gdb) bt #0 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007f18f8c415be in read (__nbytes=16, __buf=0x7f18d4000c50, __fd=26) at /usr/include/bits/unistd.h:44 #2 read_all (fd=26, data=data@entry =0x7f18d4000c50, len=len@entry =16, nonblocking=nonblocking@entry =0) at xs.c:374 #3 0x00007f18f8c41675 in read_message (h=h@entry =0x7f18e8001070, nonblocking=nonblocking@entry =0) at xs.c:1139 #4 0x00007f18f8c41e90 in read_thread (arg=0x7f18e8001070) at xs.c:1211 #5 0x00007f1906e5ff35 in start_thread (arg=0x7f190893d700) at pthread_create.c:309 #6 0x00007f1906787c3d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 and then bt full: (gdb) bt full #0 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 No locals. #1 0x00007f18f8c415be in read (__nbytes=16, __buf=0x7f18d4000c50, __fd=26) at /usr/include/bits/unistd.h:44 No locals. #2 read_all (fd=26, data=data@entry =0x7f18d4000c50, len=len@entry =16, nonblocking=nonblocking@entry =0) at xs.c:374 done = <optimized out> #3 0x00007f18f8c41675 in read_message (h=h@entry =0x7f18e8001070, nonblocking=nonblocking@entry =0) at xs.c:1139 __clframe = {__cancel_routine = <optimized out>, __cancel_arg = 0x7f18d4000c40, __do_it = 1, __cancel_type = <optimized out>} msg = 0x7f18d4000c40 body = 0x0 saved_errno = 0 ret = -1 #4 0x00007f18f8c41e90 in read_thread (arg=0x7f18e8001070) at xs.c:1211 h = 0x7f18e8001070 fd = <optimized out> #5 0x00007f1906e5ff35 in start_thread (arg=0x7f190893d700) at pthread_create.c:309 __res = <optimized out> pd = 0x7f190893d700 now = <optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139745494816512, 9092725460686644452, 1, 0, 139745494817216, 139745494816512, -9214988784542636828, -9214975709026802460}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <optimized out> pagesize_m1 = <optimized out> sp = <optimized out> freesize = <optimized out> #6 0x00007f1906787c3d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 No locals. now the same for a transient domain, also after start of libvirtd (no vm running): (gdb) info thread Id Target Id Frame 11 Thread 0x7fe5cf507700 (LWP 21097) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 10 Thread 0x7fe5ced06700 (LWP 21098) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 9 Thread 0x7fe5ce505700 (LWP 21099) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 8 Thread 0x7fe5cdd04700 (LWP 21100) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 7 Thread 0x7fe5cd503700 (LWP 21101) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 6 Thread 0x7fe5ccd02700 (LWP 21102) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 5 Thread 0x7fe5cc501700 (LWP 21103) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 4 Thread 0x7fe5cbd00700 (LWP 21104) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 3 Thread 0x7fe5cb4ff700 (LWP 21105) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 2 Thread 0x7fe5cacfe700 (LWP 21106) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 * 1 Thread 0x7fe5d8ee8840 (LWP 21096) "libvirtd" 0x00007fe5d6d367cd in poll () at ../sysdeps/unix/syscall-template.S:81 now restore the transient domain from a statefile without define them before: Continuing. Detaching after fork from child process 21276. Detaching after fork from child process 21278. [New Thread 0x7fe5d8ef6700 (LWP 21279)] Detaching after fork from child process 21285. Detaching after fork from child process 21302. (gdb) info thread Id Target Id Frame 12 Thread 0x7fe5d8ef6700 (LWP 21279) "libvirtd" 0x00007fe5d741f84d in read () at ../sysdeps/unix/syscall-template.S:81 11 Thread 0x7fe5cf507700 (LWP 21097) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 10 Thread 0x7fe5ced06700 (LWP 21098) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 9 Thread 0x7fe5ce505700 (LWP 21099) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 8 Thread 0x7fe5cdd04700 (LWP 21100) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 7 Thread 0x7fe5cd503700 (LWP 21101) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 6 Thread 0x7fe5ccd02700 (LWP 21102) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 5 Thread 0x7fe5cc501700 (LWP 21103) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 4 Thread 0x7fe5cbd00700 (LWP 21104) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 3 Thread 0x7fe5cb4ff700 (LWP 21105) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 2 Thread 0x7fe5cacfe700 (LWP 21106) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 * 1 Thread 0x7fe5d8ee8840 (LWP 21096) "libvirtd" 0x00007fe5d6d367cd in poll () at ../sysdeps/unix/syscall-template.S:81 destroy these vm: (gdb) info thread Id Target Id Frame 12 Thread 0x7fe5d8ef6700 (LWP 21279) "libvirtd" 0x00007fe5d741f84d in read () at ../sysdeps/unix/syscall-template.S:81 11 Thread 0x7fe5cf507700 (LWP 21097) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 10 Thread 0x7fe5ced06700 (LWP 21098) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 9 Thread 0x7fe5ce505700 (LWP 21099) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 8 Thread 0x7fe5cdd04700 (LWP 21100) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 7 Thread 0x7fe5cd503700 (LWP 21101) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 6 Thread 0x7fe5ccd02700 (LWP 21102) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 5 Thread 0x7fe5cc501700 (LWP 21103) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 4 Thread 0x7fe5cbd00700 (LWP 21104) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 3 Thread 0x7fe5cb4ff700 (LWP 21105) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 2 Thread 0x7fe5cacfe700 (LWP 21106) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 * 1 Thread 0x7fe5d8ee8840 (LWP 21096) "libvirtd" 0x00007fe5d6d367cd in poll () at ../sysdeps/unix/syscall-template.S:81 and again restore vm: Continuing. Detaching after fork from child process 21347. Detaching after fork from child process 21349. [New Thread 0x7fe5d8dff700 (LWP 21350)] Detaching after fork from child process 21353. Detaching after fork from child process 21373. (gdb) info thread Id Target Id Frame 13 Thread 0x7fe5d8dff700 (LWP 21350) "libvirtd" 0x00007fe5d741f84d in read () at ../sysdeps/unix/syscall-template.S:81 12 Thread 0x7fe5d8ef6700 (LWP 21279) "libvirtd" 0x00007fe5d741f84d in read () at ../sysdeps/unix/syscall-template.S:81 11 Thread 0x7fe5cf507700 (LWP 21097) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 10 Thread 0x7fe5ced06700 (LWP 21098) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 9 Thread 0x7fe5ce505700 (LWP 21099) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 8 Thread 0x7fe5cdd04700 (LWP 21100) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 7 Thread 0x7fe5cd503700 (LWP 21101) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 6 Thread 0x7fe5ccd02700 (LWP 21102) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 5 Thread 0x7fe5cc501700 (LWP 21103) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 4 Thread 0x7fe5cbd00700 (LWP 21104) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 3 Thread 0x7fe5cb4ff700 (LWP 21105) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 2 Thread 0x7fe5cacfe700 (LWP 21106) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 * 1 Thread 0x7fe5d8ee8840 (LWP 21096) "libvirtd" 0x00007fe5d6d367cd in poll () at ../sysdeps/unix/syscall-template.S:81 and again, destroy vm: (gdb) info thread Id Target Id Frame 13 Thread 0x7fe5d8dff700 (LWP 21350) "libvirtd" 0x00007fe5d741f84d in read () at ../sysdeps/unix/syscall-template.S:81 12 Thread 0x7fe5d8ef6700 (LWP 21279) "libvirtd" 0x00007fe5d741f84d in read () at ../sysdeps/unix/syscall-template.S:81 11 Thread 0x7fe5cf507700 (LWP 21097) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 10 Thread 0x7fe5ced06700 (LWP 21098) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 9 Thread 0x7fe5ce505700 (LWP 21099) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 8 Thread 0x7fe5cdd04700 (LWP 21100) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 7 Thread 0x7fe5cd503700 (LWP 21101) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 6 Thread 0x7fe5ccd02700 (LWP 21102) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 5 Thread 0x7fe5cc501700 (LWP 21103) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 4 Thread 0x7fe5cbd00700 (LWP 21104) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 3 Thread 0x7fe5cb4ff700 (LWP 21105) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 2 Thread 0x7fe5cacfe700 (LWP 21106) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 * 1 Thread 0x7fe5d8ee8840 (LWP 21096) "libvirtd" 0x00007fe5d6d367cd in poll () at ../sysdeps/unix/syscall-template.S:81 attach thread 13 and bt full: (gdb) thread 13 [Switching to thread 13 (Thread 0x7fe5d8dff700 (LWP 21350))] #0 0x00007fe5d741f84d in read () at ../sysdeps/unix/syscall-template.S:81 81 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS) (gdb) bt full #0 0x00007fe5d741f84d in read () at ../sysdeps/unix/syscall-template.S:81 No locals. #1 0x00007fe5c91fa5be in read (__nbytes=16, __buf=0x7fe5a4000be0, __fd=37) at /usr/include/bits/unistd.h:44 No locals. #2 read_all (fd=37, data=data@entry =0x7fe5a4000be0, len=len@entry =16, nonblocking=nonblocking@entry =0) at xs.c:374 done = <optimized out> #3 0x00007fe5c91fa675 in read_message (h=h@entry =0x7fe5b80019b0, nonblocking=nonblocking@entry =0) at xs.c:1139 __clframe = {__cancel_routine = <optimized out>, __cancel_arg = 0x7fe5a4000bd0, __do_it = 1, __cancel_type = <optimized out>} msg = 0x7fe5a4000bd0 body = 0x0 saved_errno = 0 ret = -1 #4 0x00007fe5c91fae90 in read_thread (arg=0x7fe5b80019b0) at xs.c:1211 h = 0x7fe5b80019b0 fd = <optimized out> #5 0x00007fe5d7418f35 in start_thread (arg=0x7fe5d8dff700) at pthread_create.c:309 __res = <optimized out> pd = 0x7fe5d8dff700 now = <optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140625162794752, 9202584519348287509, 1, 0, 140625162795456, 140625162794752, -9188024510920296427, -9188034420488439787}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <optimized out> pagesize_m1 = <optimized out> sp = <optimized out> freesize = <optimized out> #6 0x00007fe5d6d40c3d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 No locals. i seen nothing that give me a hint while the new threads a still open. all the best max ----------------ursprüngliche Nachricht----------------- Von: "Michal Privoznik" mprivozn@redhat.com An: ustermann78@web.de, libvirt-users@redhat.com Datum: Fri, 26 Sep 2014 15:22:01 +0200 -------------------------------------------------
[Disabling cross-post]
On 26.09.2014 12:21, ustermann78@web.de wrote:
Hi Michal, thank you for your answer. so if i understand that correctly, no matter if i shutdown or destroy the domain and no matter if it is a transient or an persistent vm, the thread should disappear, right?
in my case they still exist, also an hour after i destroy the domain (and don´t start any new one).
i use libvirt-1.1.35 on fedora core 20, for information.
Well, can you attach gdb and see what is the new thread doing?
Michal
--
On 01.10.2014 10:31, web2 wrote:
Hello,
sorry for my later answer.
so, after i started libvirtd (no vm´s running) and attach gdb i get the following threads
(gdb) info thread Id Target Id Frame 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 * 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 0x00007f190677d7cd in poll () at ../sysdeps/unix/syscall-template.S:81
if i restore an persistent domain, i see the following in gdb:
Detaching after fork from child process 20880. Detaching after fork from child process 20882. [New Thread 0x7f190893d700 (LWP 20883)] Detaching after fork from child process 20890. Detaching after fork from child process 20906.
(gdb) info thread Id Target Id Frame 12 Thread 0x7f190893d700 (LWP 20883) "libvirtd" 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 * 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 0x00007f190677d7cd in poll () at ../sysdeps/unix/syscall-template.S:81
if i now destroy this vm i get the following:
(gdb) info thread Id Target Id Frame 12 Thread 0x7f190893d700 (LWP 20883) "libvirtd" 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 * 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 0x00007f190677d7cd in poll () at ../sysdeps/unix/syscall-template.S:81
attaching thread Id 12 and bt: (gdb) thread 12 [Switching to thread 12 (Thread 0x7f190893d700 (LWP 20883))] #0 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 81 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS) (gdb) bt #0 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007f18f8c415be in read (__nbytes=16, __buf=0x7f18d4000c50, __fd=26) at /usr/include/bits/unistd.h:44 #2 read_all (fd=26, data=data@entry =0x7f18d4000c50, len=len@entry =16, nonblocking=nonblocking@entry =0) at xs.c:374 #3 0x00007f18f8c41675 in read_message (h=h@entry =0x7f18e8001070, nonblocking=nonblocking@entry =0) at xs.c:1139 #4 0x00007f18f8c41e90 in read_thread (arg=0x7f18e8001070) at xs.c:1211 #5 0x00007f1906e5ff35 in start_thread (arg=0x7f190893d700) at pthread_create.c:309 #6 0x00007f1906787c3d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
and then bt full:
(gdb) bt full #0 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 No locals. #1 0x00007f18f8c415be in read (__nbytes=16, __buf=0x7f18d4000c50, __fd=26) at /usr/include/bits/unistd.h:44 No locals. #2 read_all (fd=26, data=data@entry =0x7f18d4000c50, len=len@entry =16, nonblocking=nonblocking@entry =0) at xs.c:374 done = <optimized out> #3 0x00007f18f8c41675 in read_message (h=h@entry =0x7f18e8001070, nonblocking=nonblocking@entry =0) at xs.c:1139 __clframe = {__cancel_routine = <optimized out>, __cancel_arg = 0x7f18d4000c40, __do_it = 1, __cancel_type = <optimized out>} msg = 0x7f18d4000c40 body = 0x0 saved_errno = 0 ret = -1 #4 0x00007f18f8c41e90 in read_thread (arg=0x7f18e8001070) at xs.c:1211 h = 0x7f18e8001070 fd = <optimized out>
This stack doesn't come from libvirt. I suspect it comes from a library that libvirtd is linked with. Maybe libxl? What's the connection uri you're using?
#5 0x00007f1906e5ff35 in start_thread (arg=0x7f190893d700) at pthread_create.c:309 __res = <optimized out> pd = 0x7f190893d700 now = <optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139745494816512, 9092725460686644452, 1, 0, 139745494817216, 139745494816512, -9214988784542636828, -9214975709026802460}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <optimized out> pagesize_m1 = <optimized out> sp = <optimized out> freesize = <optimized out> #6 0x00007f1906787c3d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 No locals.
Michal
Hi ----------------ursprüngliche Nachricht----------------- Von: "Michal Privoznik" mprivozn@redhat.com An: "web2" ustermann78@web.de , "libvirt-users redhat.com" , "libvirt-list redhat.com" Datum: Wed, 01 Oct 2014 18:12:45 +0200 -------------------------------------------------
On 01.10.2014 10:31, web2 wrote:
Hello,
sorry for my later answer.
so, after i started libvirtd (no vm´s running) and attach gdb i get the following threads
(gdb) info thread Id Target Id Frame 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 * 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 0x00007f190677d7cd in poll () at ../sysdeps/unix/syscall-template.S:81
if i restore an persistent domain, i see the following in gdb:
Detaching after fork from child process 20880. Detaching after fork from child process 20882. [New Thread 0x7f190893d700 (LWP 20883)] Detaching after fork from child process 20890. Detaching after fork from child process 20906.
(gdb) info thread Id Target Id Frame 12 Thread 0x7f190893d700 (LWP 20883) "libvirtd" 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 * 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 0x00007f190677d7cd in poll () at ../sysdeps/unix/syscall-template.S:81
if i now destroy this vm i get the following:
(gdb) info thread Id Target Id Frame 12 Thread 0x7f190893d700 (LWP 20883) "libvirtd" 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 * 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 0x00007f190677d7cd in poll () at ../sysdeps/unix/syscall-template.S:81
attaching thread Id 12 and bt: (gdb) thread 12 [Switching to thread 12 (Thread 0x7f190893d700 (LWP 20883))] #0 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 81 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS) (gdb) bt #0 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007f18f8c415be in read (__nbytes=16, __buf=0x7f18d4000c50, __fd=26) at /usr/include/bits/unistd.h:44 #2 read_all (fd=26, data=data@entry =0x7f18d4000c50, len=len@entry =16, nonblocking=nonblocking@entry =0) at xs.c:374 #3 0x00007f18f8c41675 in read_message (h=h@entry =0x7f18e8001070, nonblocking=nonblocking@entry =0) at xs.c:1139 #4 0x00007f18f8c41e90 in read_thread (arg=0x7f18e8001070) at xs.c:1211 #5 0x00007f1906e5ff35 in start_thread (arg=0x7f190893d700) at pthread_create.c:309 #6 0x00007f1906787c3d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
and then bt full:
(gdb) bt full #0 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 No locals. #1 0x00007f18f8c415be in read (__nbytes=16, __buf=0x7f18d4000c50, __fd=26) at /usr/include/bits/unistd.h:44 No locals. #2 read_all (fd=26, data=data@entry =0x7f18d4000c50, len=len@entry =16, nonblocking=nonblocking@entry =0) at xs.c:374 done = <optimized out> #3 0x00007f18f8c41675 in read_message (h=h@entry =0x7f18e8001070, nonblocking=nonblocking@entry =0) at xs.c:1139 __clframe = {__cancel_routine = <optimized out>, __cancel_arg = 0x7f18d4000c40, __do_it = 1, __cancel_type = <optimized out>} msg = 0x7f18d4000c40 body = 0x0 saved_errno = 0 ret = -1 #4 0x00007f18f8c41e90 in read_thread (arg=0x7f18e8001070) at xs.c:1211 h = 0x7f18e8001070 fd = <optimized out>
This stack doesn't come from libvirt. I suspect it comes from a library that libvirtd is linked with. Maybe libxl? What's the connection uri you're using?
i use xen:///. as i said early, i do tests with xen xm´s. xend is disabled, so i use the libxl. all the best max
#5 0x00007f1906e5ff35 in start_thread (arg=0x7f190893d700) at pthread_create.c:309 __res = <optimized out> pd = 0x7f190893d700 now = <optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139745494816512, 9092725460686644452, 1, 0, 139745494817216, 139745494816512, -9214988784542636828, -9214975709026802460}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <optimized out> pagesize_m1 = <optimized out> sp = <optimized out> freesize = <optimized out> #6 0x00007f1906787c3d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 No locals.
Michal
--
web2 wrote:
Hi
----------------ursprüngliche Nachricht----------------- Von: "Michal Privoznik" mprivozn@redhat.com An: "web2" ustermann78@web.de , "libvirt-users redhat.com" , "libvirt-list redhat.com" Datum: Wed, 01 Oct 2014 18:12:45 +0200 -------------------------------------------------
On 01.10.2014 10:31, web2 wrote:
Hello,
sorry for my later answer.
so, after i started libvirtd (no vm´s running) and attach gdb i get the following threads
(gdb) info thread Id Target Id Frame 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 * 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 0x00007f190677d7cd in poll () at ../sysdeps/unix/syscall-template.S:81
if i restore an persistent domain, i see the following in gdb:
Detaching after fork from child process 20880. Detaching after fork from child process 20882. [New Thread 0x7f190893d700 (LWP 20883)] Detaching after fork from child process 20890. Detaching after fork from child process 20906.
(gdb) info thread Id Target Id Frame 12 Thread 0x7f190893d700 (LWP 20883) "libvirtd" 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 * 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 0x00007f190677d7cd in poll () at ../sysdeps/unix/syscall-template.S:81
if i now destroy this vm i get the following:
(gdb) info thread Id Target Id Frame 12 Thread 0x7f190893d700 (LWP 20883) "libvirtd" 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 * 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 0x00007f190677d7cd in poll () at ../sysdeps/unix/syscall-template.S:81
attaching thread Id 12 and bt: (gdb) thread 12 [Switching to thread 12 (Thread 0x7f190893d700 (LWP 20883))] #0 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 81 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS) (gdb) bt #0 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007f18f8c415be in read (__nbytes=16, __buf=0x7f18d4000c50, __fd=26) at /usr/include/bits/unistd.h:44 #2 read_all (fd=26, data=data@entry =0x7f18d4000c50, len=len@entry =16, nonblocking=nonblocking@entry =0) at xs.c:374 #3 0x00007f18f8c41675 in read_message (h=h@entry =0x7f18e8001070, nonblocking=nonblocking@entry =0) at xs.c:1139 #4 0x00007f18f8c41e90 in read_thread (arg=0x7f18e8001070) at xs.c:1211 #5 0x00007f1906e5ff35 in start_thread (arg=0x7f190893d700) at pthread_create.c:309 #6 0x00007f1906787c3d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
and then bt full:
(gdb) bt full #0 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 No locals. #1 0x00007f18f8c415be in read (__nbytes=16, __buf=0x7f18d4000c50, __fd=26) at /usr/include/bits/unistd.h:44 No locals. #2 read_all (fd=26, data=data@entry =0x7f18d4000c50, len=len@entry =16, nonblocking=nonblocking@entry =0) at xs.c:374 done = <optimized out> #3 0x00007f18f8c41675 in read_message (h=h@entry =0x7f18e8001070, nonblocking=nonblocking@entry =0) at xs.c:1139 __clframe = {__cancel_routine = <optimized out>, __cancel_arg = 0x7f18d4000c40, __do_it = 1, __cancel_type = <optimized out>} msg = 0x7f18d4000c40 body = 0x0 saved_errno = 0 ret = -1 #4 0x00007f18f8c41e90 in read_thread (arg=0x7f18e8001070) at xs.c:1211 h = 0x7f18e8001070 fd = <optimized out>
This stack doesn't come from libvirt. I suspect it comes from a library that libvirtd is linked with.
Yep, a thread created by libxl.
Maybe libxl? What's the connection uri you're using?
i use xen:///. as i said early, i do tests with xen xm´s. xend is disabled, so i use the libxl.
Earlier you mentioned using libvirt 1.1.3 for your Xen hosts. Have you tried the freshly released 1.2.9? I didn't see the problem with libvirt git master and Xen 4.4.1. Regards, Jim
----------------ursprüngliche Nachricht----------------- Von: "Jim Fehlig" jfehlig@suse.com An: "web2" ustermann78@web.de Kopie: "Michal Privoznik" mprivozn@redhat.com , "libvirt-users redhat.com" , "libvirt-list redhat.com" Datum: Wed, 01 Oct 2014 21:38:05 -0600 -------------------------------------------------
web2 wrote:
Hi
----------------ursprüngliche Nachricht----------------- Von: "Michal Privoznik" mprivozn@redhat.com An: "web2" ustermann78@web.de , "libvirt-users redhat.com" , "libvirt-list redhat.com" Datum: Wed, 01 Oct 2014 18:12:45 +0200 -------------------------------------------------
On 01.10.2014 10:31, web2 wrote:
Hello,
sorry for my later answer.
so, after i started libvirtd (no vm´s running) and attach gdb i get the following threads
(gdb) info thread Id Target Id Frame 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
* 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 0x00007f190677d7cd in poll () at ../sysdeps/unix/syscall-template.S:81
if i restore an persistent domain, i see the following in gdb:
Detaching after fork from child process 20880. Detaching after fork from child process 20882. [New Thread 0x7f190893d700 (LWP 20883)] Detaching after fork from child process 20890. Detaching after fork from child process 20906.
(gdb) info thread Id Target Id Frame 12 Thread 0x7f190893d700 (LWP 20883) "libvirtd" 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
* 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 0x00007f190677d7cd in poll () at ../sysdeps/unix/syscall-template.S:81
if i now destroy this vm i get the following:
(gdb) info thread Id Target Id Frame 12 Thread 0x7f190893d700 (LWP 20883) "libvirtd" 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
* 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 0x00007f190677d7cd in poll () at ../sysdeps/unix/syscall-template.S:81
attaching thread Id 12 and bt: (gdb) thread 12 [Switching to thread 12 (Thread 0x7f190893d700 (LWP 20883))] #0 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 81 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS) (gdb) bt #0 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007f18f8c415be in read (__nbytes=16, __buf=0x7f18d4000c50, __fd=26) at /usr/include/bits/unistd.h:44 #2 read_all (fd=26, data=data@entry =0x7f18d4000c50, len=len@entry =16, nonblocking=nonblocking@entry =0) at xs.c:374 #3 0x00007f18f8c41675 in read_message (h=h@entry =0x7f18e8001070, nonblocking=nonblocking@entry =0) at xs.c:1139 #4 0x00007f18f8c41e90 in read_thread (arg=0x7f18e8001070) at xs.c:1211 #5 0x00007f1906e5ff35 in start_thread (arg=0x7f190893d700) at pthread_create.c:309 #6 0x00007f1906787c3d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
and then bt full:
(gdb) bt full #0 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 No locals. #1 0x00007f18f8c415be in read (__nbytes=16, __buf=0x7f18d4000c50, __fd=26) at /usr/include/bits/unistd.h:44 No locals. #2 read_all (fd=26, data=data@entry =0x7f18d4000c50, len=len@entry =16, nonblocking=nonblocking@entry =0) at xs.c:374 done = <optimized out> #3 0x00007f18f8c41675 in read_message (h=h@entry =0x7f18e8001070, nonblocking=nonblocking@entry =0) at xs.c:1139 __clframe = {__cancel_routine = <optimized out>, __cancel_arg = 0x7f18d4000c40, __do_it = 1, __cancel_type = <optimized out>} msg = 0x7f18d4000c40 body = 0x0 saved_errno = 0 ret = -1 #4 0x00007f18f8c41e90 in read_thread (arg=0x7f18e8001070) at xs.c:1211 h = 0x7f18e8001070 fd = <optimized out>
This stack doesn't come from libvirt. I suspect it comes from a library that libvirtd is linked with.
Yep, a thread created by libxl.
Maybe libxl? What's the connection uri you're using?
i use xen:///. as i said early, i do tests with xen xm´s. xend is disabled, so i use the libxl.
Earlier you mentioned using libvirt 1.1.3 for your Xen hosts. Have you tried the freshly released 1.2.9? I didn't see the problem with libvirt git master and Xen 4.4.1.
I use libvirt 1.1.3.5 and Xen 4.3.3 for the tests. I will try a newer version of Xen and then a newer version of libvirt. thanks max
Regards, Jim
--
Hi, i have found the time and tested Xen-4.4.1 with libvirt-1.2.6 and the problem with the threads are disappered. But now i see after every restore of an xen-vm (windows-xp hvm) an openfile-handle (via lsof -p `pgrep libvirtd`) to an file in /var/lib/xen with the name qemu-resume.<vm-id> and what me real make confusion is that the entry is marked as deleted.If i look in the directory, the file is really no more their, but libvirt have still have an handle to that none existing file!?! I know that "Xen" create such a file before they startet the vm, and i also know that xen delete that file , but i don´t understand while it is listed under the openfiles of libvirtd after the vm is started/destroyed??? all the best max ----------------ursprüngliche Nachricht----------------- Von: "web2" ustermann78@web.de An: "Jim Fehlig" jfehlig@suse.com , "web2" ustermann78@web.de Kopie: "Michal Privoznik" mprivozn@redhat.com , "libvirt-users redhat.com" , "libvirt-list redhat.com" Datum: Thu, 2 Oct 2014 07:49:03 +0000 -------------------------------------------------
----------------ursprüngliche Nachricht----------------- Von: "Jim Fehlig" jfehlig@suse.com An: "web2" ustermann78@web.de Kopie: "Michal Privoznik" mprivozn@redhat.com , "libvirt-users redhat.com"
, "libvirt-list redhat.com" Datum: Wed, 01 Oct 2014 21:38:05 -0600 -------------------------------------------------
web2 wrote:
Hi
----------------ursprüngliche Nachricht----------------- Von: "Michal Privoznik" mprivozn@redhat.com An: "web2" ustermann78@web.de , "libvirt-users redhat.com" , "libvirt-list redhat.com" Datum: Wed, 01 Oct 2014 18:12:45 +0200 -------------------------------------------------
On 01.10.2014 10:31, web2 wrote:
Hello,
sorry for my later answer.
so, after i started libvirtd (no vm´s running) and attach gdb i get the following threads
(gdb) info thread Id Target Id Frame 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
* 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 0x00007f190677d7cd in poll () at ../sysdeps/unix/syscall-template.S:81
if i restore an persistent domain, i see the following in gdb:
Detaching after fork from child process 20880. Detaching after fork from child process 20882. [New Thread 0x7f190893d700 (LWP 20883)] Detaching after fork from child process 20890. Detaching after fork from child process 20906.
(gdb) info thread Id Target Id Frame 12 Thread 0x7f190893d700 (LWP 20883) "libvirtd" 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
* 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 0x00007f190677d7cd in poll () at ../sysdeps/unix/syscall-template.S:81
if i now destroy this vm i get the following:
(gdb) info thread Id Target Id Frame 12 Thread 0x7f190893d700 (LWP 20883) "libvirtd" 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 11 Thread 0x7f18fef4e700 (LWP 20695) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
10 Thread 0x7f18fe74d700 (LWP 20696) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
9 Thread 0x7f18fdf4c700 (LWP 20697) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
8 Thread 0x7f18fd74b700 (LWP 20698) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
7 Thread 0x7f18fcf4a700 (LWP 20699) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
6 Thread 0x7f18fc749700 (LWP 20700) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
5 Thread 0x7f18fbf48700 (LWP 20701) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
4 Thread 0x7f18fb747700 (LWP 20702) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
3 Thread 0x7f18faf46700 (LWP 20703) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
2 Thread 0x7f18fa745700 (LWP 20704) "libvirtd" pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:1 85
* 1 Thread 0x7f190892f840 (LWP 20694) "libvirtd" 0x00007f190677d7cd in poll () at ../sysdeps/unix/syscall-template.S:81
attaching thread Id 12 and bt: (gdb) thread 12 [Switching to thread 12 (Thread 0x7f190893d700 (LWP 20883))] #0 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 81 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS) (gdb) bt #0 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 #1 0x00007f18f8c415be in read (__nbytes=16, __buf=0x7f18d4000c50, __fd=26) at /usr/include/bits/unistd.h:44 #2 read_all (fd=26, data=data@entry =0x7f18d4000c50, len=len@entry =16, nonblocking=nonblocking@entry =0) at xs.c:374 #3 0x00007f18f8c41675 in read_message (h=h@entry =0x7f18e8001070, nonblocking=nonblocking@entry =0) at xs.c:1139 #4 0x00007f18f8c41e90 in read_thread (arg=0x7f18e8001070) at xs.c:1211 #5 0x00007f1906e5ff35 in start_thread (arg=0x7f190893d700) at
pthread_create.c:309 #6 0x00007f1906787c3d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
and then bt full:
(gdb) bt full #0 0x00007f1906e6684d in read () at ../sysdeps/unix/syscall-template.S:81 No locals. #1 0x00007f18f8c415be in read (__nbytes=16, __buf=0x7f18d4000c50, __fd=26) at /usr/include/bits/unistd.h:44 No locals. #2 read_all (fd=26, data=data@entry =0x7f18d4000c50, len=len@entry =16, nonblocking=nonblocking@entry =0) at xs.c:374 done = <optimized out> #3 0x00007f18f8c41675 in read_message (h=h@entry =0x7f18e8001070, nonblocking=nonblocking@entry =0) at xs.c:1139 __clframe = {__cancel_routine = <optimized out>, __cancel_arg =
0x7f18d4000c40, __do_it = 1, __cancel_type = <optimized out>} msg = 0x7f18d4000c40 body = 0x0 saved_errno = 0 ret = -1 #4 0x00007f18f8c41e90 in read_thread (arg=0x7f18e8001070) at xs.c:1211 h = 0x7f18e8001070 fd = <optimized out>
This stack doesn't come from libvirt. I suspect it comes from a library
that libvirtd is linked with.
Yep, a thread created by libxl.
Maybe libxl? What's the connection uri you're using?
i use xen:///. as i said early, i do tests with xen xm´s. xend is disabled, so i use the libxl.
Earlier you mentioned using libvirt 1.1.3 for your Xen hosts. Have you tried the freshly released 1.2.9? I didn't see the problem with libvirt git master and Xen 4.4.1.
I use libvirt 1.1.3.5 and Xen 4.3.3 for the tests. I will try a newer version of Xen and then a newer version of libvirt.
thanks max
Regards, Jim
--
participants (3)
-
Jim Fehlig -
Michal Privoznik -
web2