On 03/08/2018 05:54 AM, llilulu wrote:

Hi My libvirt version is 3.4.0,host system is centos 7.4 ,kernel is 3.10.0-693.el7.x86_64 , when I shutdown domain in virtual system, My program call virDomainMemoryStats, My program blocked in this api. the call stack is

#0 0x00007ff242d78a3d in poll () from /lib64/libc.so.6 #1 0x00007ff243755ce8 in virNetClientIOEventLoop () from /lib64/libvirt.so.0 #2 0x00007ff24375654b in virNetClientSendInternal () from /lib64/libvirt.so.0#3 0x00007ff2437579b3 in virNetClientSendWithReply () from /lib64/libvirt.so.0 #4 0x00007ff2437581c2 in virNetClientProgramCall () from /lib64/libvirt.so.0#5 0x00007ff24372d392 in callFull.isra.3 () from /lib64/libvirt.so.0 #6 0x00007ff24373b8a1 in remoteDomainMemoryStats () from /lib64/libvirt.so.0 #7 0x00007ff2436f14e2 in virDomainMemoryStats () from /lib64/libvirt.so.0 #8 0x00007ff22ee975c2 in VM::getMemoryStats (this=0x7ff1d4064ae0,

This is a client stack trace and it only shows that client called virDomainMemoryStats(). Can you get daemon stack trace?

Begin I think is my program problem, But when I use virsh domstats cmd , The cmd block too. In my program, I set timeout event for virConnectPtr. But my program blocked all the time. I find my system log, I find some libvirtd error: libvirtd: 2018-03-08 04:25:13.144+0000: 1289: error : virKeepAliveTimerInternal:143 : internal error: connection closed due to keepalive timeout Mar 7 23:33:05 localhost libvirtd: 2018-03-08 04:33:05.738+0000: 1289: error : virNetSocketReadWire:1808 : End of file while reading data: Input/output error Mar 7 23:33:14 localhost libvirtd: 2018-03-08 04:33:14.253+0000: 1289: error : virNetSocketReadWire:1808 : End of file while reading data: Input/output error Mar 7 23:33:16 localhost libvirtd: 2018-03-08 04:33:16.962+0000: 1289: error : virNetSocketReadWire:1808 : End of file while reading data: Input/output error Mar 7 23:33:25 localhost libvirtd: 2018-03-08 04:33:25.879+0000: 1300: warning : qemuDomainObjBeginJobInternal:3847 : Cannot start job (query, none) for domain d7bb3005-4fa4-4c3a-8802-ab55464d074b; current job is (query, none) owned by (1299 remoteDispatchDomainMemoryStats, 0 <null>) for (3680s, 0s) Mar 7 23:33:25 localhost libvirtd: 2018-03-08 04:33:25.879+0000: 1300: error : qemuDomainObjBeginJobInternal:3859 : Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainMemoryStats) Mar 7 23:33:25 localhost libvirtd: 2018-03-08 04:33:25.879+0000: 1300: warning : qemuGetProcessInfo:1428 : cannot parse process status data Mar 7 23:33:25 localhost libvirtd: 2018-03-08 04:33:25.880+0000: 1300: error : virNetDevTapInterfaceStats:751 : internal error: /proc/net/dev: Interface not found Mar 7 23:33:41 localhost libvirtd: 2018-03-08 04:33:41.826+0000: 1298: warning : qemuDomainObjBeginJobInternal:3847 : Cannot start job (query, none) for domain d7bb3005-4fa4-4c3a-8802-ab55464d074b; current job is (query, none) owned by (1299 remoteDispatchDomainMemoryStats, 0 <null>) for (3696s, 0s)Mar 7 23:33:41 localhost libvirtd: 2018-03-08 04:33:41.826+0000: 1298: error : qemuDomainObjBeginJobInternal:3859 : Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainMemoryStats)

Is this problem a bug or I can't statistics some specifi status domain?

This is very likely a bug that has been fixed. There were some fixes concerning statistics fetching. Please try to reproduce with the latest git HEAD. Michal