[libvirt] Save & Restore Synchronization
Hi, I am using libvirt for qemu/kvm. I have following questions in terms of save and restore (Version 0.6.2): 1) As I checked the libvirt code, restore command is asynchronous. There is a macro called VIR_EXEC_NONBLOCK but there is no macro called VIR_EXEC_BLOCK. Is it possible to execute the restore in blocking mode so that I can measure the whole restore time? 2) I tried to measure the time spent to save the VM. I am executing save command and getting a timing. However when I execute the following operations in order I got an error right after script issues the restore operation (No delay between them): "save->restore->save" ... the error is: 20:19:02.478: error : internal error Unable to open monitor path /dev/pts/3 libvir: QEMU error : internal error Unable to open monitor path /dev/pts/3 20:19:02.478: error : internal error unable to start guest: char device redirected to /dev/pts/3 inet_listen: bind(ipv4,127.0.0.1,5912): Address already in use inet_listen: FAILED libvir: QEMU error : internal error unable to start guest: char device redirected to /dev/pts/3 inet_listen: bind(ipv4,127.0.0.1,5912): Address already in use inet_listen: FAILED 20:19:02.482: error : operation failed: failed to start VM libvir: QEMU error : operation failed: failed to start VM error: Failed to restore domain from /path/to/the/save/file error: operation failed: failed to start VM (You can find the detailed error report in the attachement when LIBVIRT_DEBUG flag is 1) I think that some of the cleanup is non-blocking which are cleanup of pty device and vncserver. However, I am not sure if destroying the qemu process is asynchronous?? P.S.: If I put one second delay between first save and restore, there is no error. Save and restore work fine. Thanks, Erkan Unal 20:26:07.506: debug : virInitialize:287 : register drivers 20:26:07.506: debug : virRegisterDriver:660 : registering Test as driver 0 20:26:07.506: debug : virRegisterNetworkDriver:560 : registering Test as network driver 0 20:26:07.506: debug : virRegisterStorageDriver:591 : registering Test as storage driver 0 20:26:07.506: debug : virRegisterDeviceMonitor:622 : registering Test as device driver 0 20:26:07.506: debug : virRegisterDriver:660 : registering Xen as driver 1 20:26:07.506: debug : virRegisterDriver:660 : registering OPENVZ as driver 2 20:26:07.506: debug : virRegisterDriver:660 : registering remote as driver 3 20:26:07.506: debug : virRegisterNetworkDriver:560 : registering remote as network driver 1 20:26:07.506: debug : virRegisterStorageDriver:591 : registering remote as storage driver 1 20:26:07.506: debug : virRegisterDeviceMonitor:622 : registering remote as device driver 1 20:26:07.506: debug : virConnectOpenAuth:1089 : name=qemu:///session, auth=0x7fa21cbae660, flags=0 20:26:07.506: debug : do_open:909 : name "qemu:///session" to URI components: scheme qemu opaque (null) authority (null) server (null) user (null) port 0 path /session 20:26:07.506: debug : do_open:919 : trying driver 0 (Test) ... 20:26:07.506: debug : do_open:925 : driver 0 Test returned DECLINED 20:26:07.506: debug : do_open:919 : trying driver 1 (Xen) ... 20:26:07.506: debug : do_open:925 : driver 1 Xen returned DECLINED 20:26:07.506: debug : do_open:919 : trying driver 2 (OPENVZ) ... 20:26:07.506: debug : do_open:925 : driver 2 OPENVZ returned DECLINED 20:26:07.506: debug : do_open:919 : trying driver 3 (remote) ... 20:26:07.506: debug : remoteOpen:974 : Auto-remote UNIX socket 20:26:07.506: debug : remoteOpen:992 : Auto-spawn user daemon instance 20:26:07.506: debug : doRemoteOpen:511 : proceeding with name = qemu:///session 20:26:07.507: debug : call:6473 : Doing call 66 (nil) 20:26:07.507: debug : call:6543 : We have the buck 66 0x7fa21bdb5010 0x7fa21bdb5010 20:26:07.507: debug : processCallRecvLen:6131 : Got length, now need 36 total (32 more) 20:26:07.507: debug : processCalls:6399 : Giving up the buck 66 0x7fa21bdb5010 (nil) 20:26:07.507: debug : call:6574 : All done with our call 66 (nil) 0x7fa21bdb5010 20:26:07.507: debug : call:6473 : Doing call 1 (nil) 20:26:07.507: debug : call:6543 : We have the buck 1 0xf86d80 0xf86d80 20:26:07.509: debug : processCallRecvLen:6131 : Got length, now need 28 total (24 more) 20:26:07.509: debug : processCalls:6399 : Giving up the buck 1 0xf86d80 (nil) 20:26:07.509: debug : call:6574 : All done with our call 1 (nil) 0xf86d80 20:26:07.509: debug : doRemoteOpen:822 : Adding Handler for remote events 20:26:07.509: debug : doRemoteOpen:829 : virEventAddHandle failed: No addHandleImpl defined. continuing without events. 20:26:07.509: debug : do_open:925 : driver 3 remote returned SUCCESS 20:26:07.509: debug : do_open:945 : network driver 0 Test returned DECLINED 20:26:07.509: debug : do_open:945 : network driver 1 remote returned SUCCESS 20:26:07.509: debug : do_open:967 : storage driver 0 Test returned DECLINED 20:26:07.509: debug : do_open:967 : storage driver 1 remote returned SUCCESS 20:26:07.510: debug : do_open:988 : node driver 0 Test returned DECLINED 20:26:07.510: debug : do_open:988 : node driver 1 remote returned SUCCESS 20:26:07.510: debug : virDomainRestore:2012 : conn=0xf82430, from=/path/to/save/file 20:26:07.510: debug : call:6473 : Doing call 54 (nil) 20:26:07.510: debug : call:6543 : We have the buck 54 0xf86d80 0xf86d80 20:26:07.625: error : internal error Unable to open monitor path /dev/pts/3 libvir: QEMU error : internal error Unable to open monitor path /dev/pts/3 20:26:07.625: error : internal error unable to start guest: char device redirected to /dev/pts/3 inet_listen: bind(ipv4,127.0.0.1,5912): Address already in use inet_listen: FAILED libvir: QEMU error : internal error unable to start guest: char device redirected to /dev/pts/3 inet_listen: bind(ipv4,127.0.0.1,5912): Address already in use inet_listen: FAILED 20:26:07.629: error : operation failed: failed to start VM libvir: QEMU error : operation failed: failed to start VM 20:26:07.629: debug : processCallRecvLen:6131 : Got length, now need 160 total (156 more) 20:26:07.629: debug : processCalls:6399 : Giving up the buck 54 0xf86d80 (nil) 20:26:07.629: debug : call:6574 : All done with our call 54 (nil) 0xf86d80 error: Failed to restore domain from /path/to/save/file error: operation failed: failed to start VM 20:26:07.629: debug : virConnectClose:1107 : conn=0xf82430 20:26:07.629: debug : call:6473 : Doing call 2 (nil) 20:26:07.629: debug : call:6543 : We have the buck 2 0xf86d80 0xf86d80 20:26:07.629: debug : processCallRecvLen:6131 : Got length, now need 28 total (24 more) 20:26:07.629: debug : processCalls:6399 : Giving up the buck 2 0xf86d80 (nil) 20:26:07.629: debug : call:6574 : All done with our call 2 (nil) 0xf86d80 20:26:07.629: debug : virUnrefConnect:210 : unref connection 0xf82430 1 20:26:07.629: debug : virReleaseConnect:171 : release connection 0xf82430
Erkan Unal wrote:
Hi,
I am using libvirt for qemu/kvm. I have following questions in terms of save and restore (Version 0.6.2):
1) As I checked the libvirt code, restore command is asynchronous. There is a macro called VIR_EXEC_NONBLOCK but there is no macro called VIR_EXEC_BLOCK. Is it possible to execute the restore in blocking mode so that I can measure the whole restore time?
Hm, the entire thing is *meant* to be synchronous. That being said, I'm not exactly sure how we are achieving that (and maybe we aren't achieving it at all). It used to be the case that the monitor wouldn't return until the migration was finished, but that was changed long ago. I'd have to look into it again; if it is indeed the case that it is not synchronous, then we'll probably have to change it to loop on "info migrate" and wait for the migration to complete in order to make it synchronous. That may also explain your errors below.
2) I tried to measure the time spent to save the VM. I am executing save command and getting a timing. However when I execute the following operations in order I got an error right after script issues the restore operation (No delay between them):
"save->restore->save"
... the error is:
20:19:02.478: error : internal error Unable to open monitor path /dev/pts/3 libvir: QEMU error : internal error Unable to open monitor path /dev/pts/3 20:19:02.478: error : internal error unable to start guest: char device redirected to /dev/pts/3 inet_listen: bind(ipv4,127.0.0.1,5912): Address already in use inet_listen: FAILED
libvir: QEMU error : internal error unable to start guest: char device redirected to /dev/pts/3 inet_listen: bind(ipv4,127.0.0.1,5912): Address already in use inet_listen: FAILED 20:19:02.482: error : operation failed: failed to start VM libvir: QEMU error : operation failed: failed to start VM error: Failed to restore domain from /path/to/the/save/file error: operation failed: failed to start VM
(You can find the detailed error report in the attachement when LIBVIRT_DEBUG flag is 1)
I think that some of the cleanup is non-blocking which are cleanup of pty device and vncserver. However, I am not sure if destroying the qemu process is asynchronous??
Destroying the qemu process is definitely synchronous. -- Chris Lalancette
participants (2)
-
Chris Lalancette -
Erkan Unal