On 29.05.2015 16:38, cloudbuglist wrote:

Hi dear! Today I came across a problem, the following is a log and software versions. soft version: os:centos 6.5 libvirt:0.10.29 qemu:0.12.415

2015-05-28 03:28:58.001+0000: 20919: error : qemuDomainObjBeginJobInternal:843 : Timed out during operation: cannot acquire state change lock 2015-05-28 03:30:37.000+0000: 20918: warning : qemuDomainObjBeginJobInternal:839 : Cannot start job (query, none) for domain i-2-437-VM; current job is (query, none) owned by (20917, 0) 2015-05-28 03:30:37.000+0000: 20918: error : qemuDomainObjBeginJobInternal:843 : Timed out during operation: cannot acquire state change lock 2015-05-28 03:31:01.000+0000: 20919: warning : qemuDomainObjBeginJobInternal:839 : Cannot start job (query, none) for domain i-2-437-VM; current job is (query, none) owned by (20917, 0) 2015-05-28 03:31:01.000+0000: 20919: error : qemuDomainObjBeginJobInternal:843 : Timed out during operation: cannot acquire state change lock 2015-05-28 03:32:40.000+0000: 20921: warning : qemuDomainObjBeginJobInternal:839 : Cannot start job (query, none) for domain i-2-437-VM; current job is (query, none) owned by (20917, 0) 2015-05-28 03:32:40.000+0000: 20921: error : qemuDomainObjBeginJobInternal:843 : Timed out during operation: cannot acquire state change lock 2015-05-28 03:33:06.000+0000: 20918: warning : qemuDomainObjBeginJobInternal:839 : Cannot start job (query, none) for domain i-2-437-VM; current job is (query, none) owned by (20917, 0) 2015-05-28 03:33:06.000+0000: 20918: error : qemuDomainObjBeginJobInternal:843 : Timed out during operation: cannot acquire state change lock 2015-05-28 03:33:42.009+0000: 20916: error : virNetSocketReadWire:1194 : End of file while reading data: Input/output error 2015-05-28 03:34:02.000+0000: 20921: warning : qemuDomainObjBeginJobInternal:839 : Cannot start job (query, none) for domain i-2-437-VM; current job is (query, none) owned by (20917, 0) 2015-05-28 03:34:02.000+0000: 20921: error : qemuDomainObjBeginJobInternal:843 : Timed out during operation: cannot acquire state change lock 2015-05-28 03:34:19.001+0000: 20920: warning : qemuDomainObjBeginJobInternal:839 : Cannot start job (query, none) for domain i-2-437-VM; current job is (query, none) owned by (20917, 0) 2015-05-28 03:34:19.001+0000: 20920: error : qemuDomainObjBeginJobInternal:843 : Timed out during operation: cannot acquire state change lock 2015-05-28 03:34:43.384+0000: 20916: error : virNetSocketReadWire:1194 : End of file while reading data: Input/output error 2015-05-28 03:34:47.000+0000: 20918: warning : qemuDomainObjBeginJobInternal:839 : Cannot start job (query, none) for domain i-2-437-VM; current job is (query, none) owned by (20917, 0) 2015-05-28 03:34:47.000+0000: 20918: error : qemuDomainObjBeginJobInternal:843 : Timed out during operation: cannot acquire state change lock 2015-05-28 03:34:52.000+0000: 20920: warning : qemuDomainObjBeginJobInternal:839 : Cannot start job (query, none) for domain i-2-437-VM; current job is (query, none) owned by (20917, 0) 2015-05-28 03:34:52.000+0000: 20920: error : qemuDomainObjBeginJobInternal:843 : Timed out during operation: cannot acquire state change lock 2015-05-28 03:35:09.001+0000: 20921: warning : qemuDomainObjBeginJobInternal:839 : Cannot start job (query, none) for domain i-2-437-VM; current job is (query, none) owned by (20917, 0) 2015-05-28 03:35:09.001+0000: 20921: error : qemuDomainObjBeginJobInternal:843 : Timed out during operation: cannot acquire state change lock

This usually means, that the other thread (20917) has locked the domain for changes (or to be more technical, it set a job). If you attach gdb onto the daemon and query what is the other thread doing, you'll get the picture. Unfortunately, some operations on a domain can take ages to finish. The other possibility could be, that a thread has set the job and accessed an NFS backed storage which went stale. Or this can as well be a bug. Hard to tell from only this info. Also - does this happen with new libvirt? 0.10.29 is a bit old today. Michal