[libvirt] Migration failure related to EOF from Monitor
Hello All, I am trying to migrate a KVM guest while implementing the patches for PCI-Passthrough of SRIOV Vf's and I repeatedly see the following error: [root@c6100k cwd]# virsh migrate --live dibenchvm1 qemu+ssh://c6100l.uk.level5networks.com/system error: operation failed: migration job: unexpectedly failed I have turned on debugging in /etc/libvirt/libvirtd.conf and I see the following debug in /var/log/libvirt/libvirtd.log 10:37:26.632: 2449: error : qemuMonitorIO:584 : internal error End of file from monitor 10:37:26.632: 2449: debug : qemuMonitorIO:617 : Error on monitor internal error End of file from monitor 10:37:26.632: 2449: debug : virEventPollUpdateHandle:145 : Update handle w=21 e=12 10:37:26.632: 2449: debug : virEventPollInterruptLocked:676 : Skip interrupt, 1 -1497479264 10:37:26.632: 2449: debug : qemuMonitorIO:640 : Triggering EOF callback 10:37:26.632: 2449: debug : qemuProcessHandleMonitorEOF:125 : Received EOF on 0x7f588c003540 'dibenchvm1' 10:37:26.632: 2449: debug : qemuProcessStop:3271 : Shutting down VM 'dibenchvm1' pid=18739 migrated=0 10:37:26.632: 2449: debug : qemuMonitorClose:765 : mon=0x7f588c00c7f0 10:37:26.632: 2449: debug : virEventPollRemoveHandle:172 : Remove handle w=21 10:37:26.633: 2449: debug : virEventPollRemoveHandle:185 : mark delete 9 20 10:37:26.633: 2449: debug : virEventPollInterruptLocked:676 : Skip interrupt, 1 -1497479264 10:37:26.633: 2449: debug : qemuProcessKill:3217 : vm=dibenchvm1 pid=18739 gracefully=0 10:37:26.633: 2449: debug : qemuProcessAutoDestroyRemove:3736 : vm=dibenchvm1 uuid=a4452511-0f97-734a-dbcc-f4c66f821956 10:37:26.633: 2449: debug : virSecurityDACRestoreSecurityAllLabel:504 : Restoring security label on dibenchvm1 migrated=0 May I ask for suggestions on how I can debug the monitor and find out the reason why the qemu monitor dies? Many Thanks, Regards, Shradha Shah
On 22.02.2012 14:46, Shradha Shah wrote:
Hello All,
I am trying to migrate a KVM guest while implementing the patches for PCI-Passthrough of SRIOV Vf's and I repeatedly see the following error: [root@c6100k cwd]# virsh migrate --live dibenchvm1 qemu+ssh://c6100l.uk.level5networks.com/system error: operation failed: migration job: unexpectedly failed
I have turned on debugging in /etc/libvirt/libvirtd.conf and I see the following debug in /var/log/libvirt/libvirtd.log
10:37:26.632: 2449: error : qemuMonitorIO:584 : internal error End of file from monitor 10:37:26.632: 2449: debug : qemuMonitorIO:617 : Error on monitor internal error End of file from monitor 10:37:26.632: 2449: debug : virEventPollUpdateHandle:145 : Update handle w=21 e=12 10:37:26.632: 2449: debug : virEventPollInterruptLocked:676 : Skip interrupt, 1 -1497479264 10:37:26.632: 2449: debug : qemuMonitorIO:640 : Triggering EOF callback 10:37:26.632: 2449: debug : qemuProcessHandleMonitorEOF:125 : Received EOF on 0x7f588c003540 'dibenchvm1' 10:37:26.632: 2449: debug : qemuProcessStop:3271 : Shutting down VM 'dibenchvm1' pid=18739 migrated=0 10:37:26.632: 2449: debug : qemuMonitorClose:765 : mon=0x7f588c00c7f0 10:37:26.632: 2449: debug : virEventPollRemoveHandle:172 : Remove handle w=21 10:37:26.633: 2449: debug : virEventPollRemoveHandle:185 : mark delete 9 20 10:37:26.633: 2449: debug : virEventPollInterruptLocked:676 : Skip interrupt, 1 -1497479264 10:37:26.633: 2449: debug : qemuProcessKill:3217 : vm=dibenchvm1 pid=18739 gracefully=0 10:37:26.633: 2449: debug : qemuProcessAutoDestroyRemove:3736 : vm=dibenchvm1 uuid=a4452511-0f97-734a-dbcc-f4c66f821956 10:37:26.633: 2449: debug : virSecurityDACRestoreSecurityAllLabel:504 : Restoring security label on dibenchvm1 migrated=0
May I ask for suggestions on how I can debug the monitor and find out the reason why the qemu monitor dies?
I've seen this many times esp. when using upstream qemu. I mean in general when qemu dies during migration. Some info may be in domain log under /var/log/libvirt/qemu/<domain>.log; But if qemu dies there is not much libvirt can do to resurrect it. Michal
On 2012-2-22 21:46, Shradha Shah wrote:
Hello All,
I am trying to migrate a KVM guest while implementing the patches for PCI-Passthrough of SRIOV Vf's and I repeatedly see the following error: [root@c6100k cwd]# virsh migrate --live dibenchvm1 qemu+ssh://c6100l.uk.level5networks.com/system error: operation failed: migration job: unexpectedly failed Are you sure your ssh setting in the destination host is correct? Maybe, you can try qemu+tcp to ignore ssh setting.
I have turned on debugging in /etc/libvirt/libvirtd.conf and I see the following debug in /var/log/libvirt/libvirtd.log
10:37:26.632: 2449: error : qemuMonitorIO:584 : internal error End of file from monitor 10:37:26.632: 2449: debug : qemuMonitorIO:617 : Error on monitor internal error End of file from monitor 10:37:26.632: 2449: debug : virEventPollUpdateHandle:145 : Update handle w=21 e=12 10:37:26.632: 2449: debug : virEventPollInterruptLocked:676 : Skip interrupt, 1 -1497479264 10:37:26.632: 2449: debug : qemuMonitorIO:640 : Triggering EOF callback 10:37:26.632: 2449: debug : qemuProcessHandleMonitorEOF:125 : Received EOF on 0x7f588c003540 'dibenchvm1' 10:37:26.632: 2449: debug : qemuProcessStop:3271 : Shutting down VM 'dibenchvm1' pid=18739 migrated=0 10:37:26.632: 2449: debug : qemuMonitorClose:765 : mon=0x7f588c00c7f0 10:37:26.632: 2449: debug : virEventPollRemoveHandle:172 : Remove handle w=21 10:37:26.633: 2449: debug : virEventPollRemoveHandle:185 : mark delete 9 20 10:37:26.633: 2449: debug : virEventPollInterruptLocked:676 : Skip interrupt, 1 -1497479264 10:37:26.633: 2449: debug : qemuProcessKill:3217 : vm=dibenchvm1 pid=18739 gracefully=0 10:37:26.633: 2449: debug : qemuProcessAutoDestroyRemove:3736 : vm=dibenchvm1 uuid=a4452511-0f97-734a-dbcc-f4c66f821956 10:37:26.633: 2449: debug : virSecurityDACRestoreSecurityAllLabel:504 : Restoring security label on dibenchvm1 migrated=0
May I ask for suggestions on how I can debug the monitor and find out the reason why the qemu monitor dies?
Many Thanks, Regards, Shradha Shah
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
-- Shu Ming<shuming@linux.vnet.ibm.com> IBM China Systems and Technology Laboratory
participants (3)
-
Michal Privoznik -
Shradha Shah -
Shu Ming