Re: [libvirt] [PATCH 3/6] Introduce yet another migration version in API.

Daniel, I looked at the patch-set you sent out on the 2/9/11 [libvirt] [PATCH 0/6] Introduce a new migration protocol to QEMU driver http://www.mail-archive.com/libvir-list@redhat.com/msg33223.html What is the status of this new migration protocol? Is there any pending issue blocking its integration? I would like to propose an RFC enhancement to the migration algorithm. Here is a quick summary of the proposal/idea. - finer control on migration result - possibility of specifying what features cannot fail their initialization on the dst host during migration. Migration should not succeed if any of them fails. - optional: each one of those features should be able to provide a deinit function to cleanup resources on the dst host if migration fails. This functionality would come useful for the (NIC) set port profile feature VDP (802.1Qbg/1Qbh), but what I propose is a generic config option / API that can be used by any feature. And now the details. ---------------------------------------------- enhancement: finer control on migration result ---------------------------------------------- There are different reasons why a VM may need (or be forced) to migrate. You can classify the types of the migrations also based on different semantics. For simplicity I'll classify them into two categories, based on how important it is for the VM to migrate as fast as possible: (1) It IS important In this case, whether the VM will not be able to (temporary) make use of certain resources (for example the network) on the dst host, is not that important, because the completion of the migration is considered higher priority. A possible scenario could be a server that must migrate ASAP because of a disaster/emergency. (2) It IS NOT important I can think of a VM whose applications/servers need a network connection in order to work properly. Loosing such network connectivity as a consequence of a migration would not be acceptable (or highly undesirable). Given the case (2) above, I have a comment about the Finish step, with regards to the port profile (VDP) codepath. The call to qemuMigrationVPAssociatePortProfile in qemuMigrationFinish can fail, but its result (success or failure) does not influence the result of the migration Finish step (it was already like this in migration V2). It is therefore possible for a VM to lose its network connectivity after a (successful) migration. Given that you are in the process of adding a new migration version, I was wondering if we could add a mechanism for the src host to select behavior/policy (1) or (2). It would be one more option in the libvirt XML domain config file. The above example refers to one specific feature (network connectivity based on the use of a port profile, ie VDP), but in theory this feature could be made generic such that the same policy (1)/(2) could be used for any feature that the admin may not be willing to lose as result of a (successful) migration. The management application on the src host (virt-manager, RHEV-M, etc) can for example, in case (2), retry the migration toward the same host or try with another destination host. The port profile configuration happens in two stages: - there is a 1st call during PREPARE: openMacvtaptap | +-> (a) vpAssociatePortProfileId (PREASSOCIATE) - and a second call during FINISH: qemuMigrationVPAssociatePortProfile | +-> (b) vpAssociatePortProfileId (ASSOCIATE) Right now, both in migration V2 and V3: - (a) cannot fail (ie, if it fails, migration fails too), - (b) can fail (ie, if it fails, migration does not fail and VM loses connectivity). With the introduction of the policy (1)/(2) mentioned above, this would be the new behavior: - Case (1) above, ie fast migration --> (a) and (b) are non blocking --> Their result does not affect migration result. - Case (2) above, ie possibly slower migration --> (a) and (b) are blocking --> Their result does affect migration result. It may be not possible to handle all possible scenarios (there can be cases where dependencies between features would make it difficult to chose between (1) and (2) on a per feature basis), but that should not be a reason not to provide this policy for a subset of features (network connectivity being one). The 5-steps new migration version V3 makes it easier (compared to V2) to introduce this kind of policy. I guess we could use the cookies to carry the necessary information/acks. There are still numerous things that can go wrong. For example, if migration aborts after (a), or even after (b), there should be a way for Libvirt to undo the set port profile. Actually, there should be a way for any feature (not just VDP) that successfully initialized on the dst host to (at least try to) cleanup. For this reason, it could make sense, for each one of the features using policy (2) (ie, cannot fail during migration) to also provide a deinit routine (as part of the API that marks the feature as "can't fail"). Such routine/s can be called by libvirt to revert/rollback the initializations that successfully completed during the migration attempt. This could be done probably at the end of the Finish3 phase. If something goes wrong (including a signal that aborts migration just before Confirm3 completes), we may have a small problem. I do not know how we can handle that case without another cleanup RPC call. If you agree that this kind of functionality/policy described above has some value, I can help with its implementation and testing. BTW, would the new functionality being discussed in this 3D "RFC: virInterface change transaction API" http://www.redhat.com/archives/libvir-list/2011-April/msg00499.html be able to provide the same configuration "atomicity" (ie, rollback in case of migration failure)? My understanding is that: - Such new framework would apply to (host) network config only. Even though it may cover the VDP (port profile) use case I mentioned above, it would not apply to other features that may need some kind of rollback after a migration failure. - Would it be possible to design that new transaction API (or a sister API) such that it would come useful for the guest migration case too? If so, the deinit/rollback routine I proposed may be defined as a simple wrapper around the transaction API and let other (non networking) features use their own deinit functions. Thanks /Chris 5 be implementation