----- On Jun 5, 2019, at 4:49 PM, Peter Krempa pkrempa@redhat.com wrote:
On Wed, Jun 05, 2019 at 13:33:49 +0200, Lentes, Bernd wrote:
Hi Peter,
thanks for your help.
----- On Jun 5, 2019, at 9:27 AM, Peter Krempa pkrempa@redhat.com wrote:
[...]
So that's interresting. Usually assertion failure in qemu leads to calling abort() and thus the vm would have crashed. Didn't you HA solution restart it?
No. As said the VM didn't crash. It kept running.
That's interresting. I hope you manage to reproduce it then.
At any rate it would be really beneficial if you could collect debug logs for libvirtd which also contain the monitor interactions with qemu:
https://wiki.libvirt.org/page/DebugLogs
The qemu assertion failure above should ideally be reported to qemu, but if you are able to reproduce the problem with libvirtd debug logs enabled I can extract more useful info from there which the qemu project would ask you anyways.
I can't reproduce it. It seems to happen accidentally. But i can collect the logs. Do they get very large ? I can contact you the next time it happen. Is that ok for you ?
Unfortunately they do get very large if there's some monitoring gathering stats through libvirt, but it's okay to nuke them prior to attempting the block commit, or daily or so.
Please do contact me if you gather anything interresting.
Hi, it happened again. Following the log of my script it started on 8th of june at 5:59:09 (UTC+2) to blockcommit the domain. These are the related lines in libvirtd.log: =================================================== 2019-06-07 20:30:57.170+0000: 30299: error : qemuMonitorIO:719 : internal error: End of file from qemu monitor 2019-06-08 03:59:17.690+0000: 30299: error : qemuMonitorIO:719 : internal error: End of file from qemu monitor 2019-06-08 03:59:26.145+0000: 30300: warning : qemuGetProcessInfo:1461 : cannot parse process status data 2019-06-08 03:59:26.191+0000: 30303: warning : qemuGetProcessInfo:1461 : cannot parse process status data 2019-06-08 03:59:56.095+0000: 27956: warning : qemuDomainObjBeginJobInternal:4865 : Cannot start job (destroy, none) for domain severin; current job is (modify, none) owned by (13061 remoteDispatchDomainBlockJobAbort, 0 <null>) for (38s, 0s) 2019-06-08 03:59:56.095+0000: 27956: error : qemuDomainObjBeginJobInternal:4877 : Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainBlockJobAbort) 2019-06-08 03:59:56.325+0000: 13060: warning : qemuGetProcessInfo:1461 : cannot parse process status data 2019-06-08 03:59:56.372+0000: 30304: warning : qemuGetProcessInfo:1461 : cannot parse process status data 2019-06-08 04:00:26.503+0000: 13060: warning : qemuGetProcessInfo:1461 : cannot parse process status data ==================================================== Since then the script is stuck. Thanks for your help. Bernd Helmholtz Zentrum Muenchen Deutsches Forschungszentrum fuer Gesundheit und Umwelt (GmbH) Ingolstaedter Landstr. 1 85764 Neuherberg www.helmholtz-muenchen.de Aufsichtsratsvorsitzende: MinDir'in Prof. Dr. Veronika von Messling Geschaeftsfuehrung: Prof. Dr. med. Dr. h.c. Matthias Tschoep, Heinrich Bassler, Kerstin Guenther Registergericht: Amtsgericht Muenchen HRB 6466 USt-IdNr: DE 129521671