[libvirt] [PATCH] qemu: Fix domain resume after failed migration

Libvirt relies on being able to kill the destination domain and resume the source one during migration until we called "cont" on the destination. Unfortunately, QEMU automatically activates block devices at the end of migration even when it's called with -S. This wasn't a big issue in the past since the guest is not running and thus no data are written to the block devices. However, when QEMU introduced its internal block device locks, we can no longer resume the source domain once the destination domain already activated the block devices (and thus acquired all locks) unless the destination domain is killed first. Since it's impossible to synchronize the destination and the source libvirt daemons after a failed migration, QEMU introduced a new migration capability called "late-block-activat" which ensures QEMU won't activate block devices until it gets "cont". The only thing we need to do is to enable this capability whenever QEMU supports it. https://bugzilla.redhat.com/show_bug.cgi?id=1568407 QEMU commit implementing the capability: v2.12.0-952-g0f073f44df Signed-off-by: Jiri Denemark <jdenemar@redhat.com> --- src/qemu/qemu_migration_params.c | 4 ++++ src/qemu/qemu_migration_params.h | 1 + 2 files changed, 5 insertions(+) diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_params.c index 2083563d32..63d20ff5dc 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -84,6 +84,7 @@ VIR_ENUM_IMPL(qemuMigrationCapability, QEMU_MIGRATION_CAP_LAST, "postcopy-ram", "compress", "pause-before-switchover", + "late-block-activate", ); @@ -129,6 +130,9 @@ struct _qemuMigrationParamsTPMapItem { static const qemuMigrationParamsAlwaysOnItem qemuMigrationParamsAlwaysOn[] = { {QEMU_MIGRATION_CAP_PAUSE_BEFORE_SWITCHOVER, QEMU_MIGRATION_SOURCE}, + + {QEMU_MIGRATION_CAP_LATE_BLOCK_ACTIVATE, + QEMU_MIGRATION_DESTINATION}, }; /* Translation from virDomainMigrateFlags to qemuMigrationCapability. */ diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_params.h index 9a865b19f3..bee503d0f4 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -37,6 +37,7 @@ typedef enum { QEMU_MIGRATION_CAP_POSTCOPY, QEMU_MIGRATION_CAP_COMPRESS, QEMU_MIGRATION_CAP_PAUSE_BEFORE_SWITCHOVER, + QEMU_MIGRATION_CAP_LATE_BLOCK_ACTIVATE, QEMU_MIGRATION_CAP_LAST } qemuMigrationCapability; -- 2.17.1

On Mon, Jun 04, 2018 at 04:51:18PM +0200, Jiri Denemark wrote:
Libvirt relies on being able to kill the destination domain and resume the source one during migration until we called "cont" on the destination. Unfortunately, QEMU automatically activates block devices at the end of migration even when it's called with -S. This wasn't a big issue in the past since the guest is not running and thus no data are written to the block devices. However, when QEMU introduced its internal block device locks, we can no longer resume the source domain once the destination domain already activated the block devices (and thus acquired all locks) unless the destination domain is killed first.
Since it's impossible to synchronize the destination and the source libvirt daemons after a failed migration, QEMU introduced a new migration capability called "late-block-activat" which ensures QEMU
activate
won't activate block devices until it gets "cont". The only thing we need to do is to enable this capability whenever QEMU supports it.
https://bugzilla.redhat.com/show_bug.cgi?id=1568407
QEMU commit implementing the capability: v2.12.0-952-g0f073f44df
Signed-off-by: Jiri Denemark <jdenemar@redhat.com> --- src/qemu/qemu_migration_params.c | 4 ++++ src/qemu/qemu_migration_params.h | 1 + 2 files changed, 5 insertions(+)
Only five insertions? You could have at least added a test :P Reviewed-by: Ján Tomko <jtomko@redhat.com> Jano

On Mon, Jun 04, 2018 at 16:51:18 +0200, Jiri Denemark wrote:
Libvirt relies on being able to kill the destination domain and resume the source one during migration until we called "cont" on the destination. Unfortunately, QEMU automatically activates block devices at the end of migration even when it's called with -S. This wasn't a big issue in the past since the guest is not running and thus no data are written to the block devices. However, when QEMU introduced its internal block device locks, we can no longer resume the source domain once the destination domain already activated the block devices (and thus acquired all locks) unless the destination domain is killed first.
Since it's impossible to synchronize the destination and the source libvirt daemons after a failed migration, QEMU introduced a new migration capability called "late-block-activat" which ensures QEMU won't activate block devices until it gets "cont". The only thing we need to do is to enable this capability whenever QEMU supports it.
I'm wondering when this new feature should _not_ be used. I did not get the information from the qemu commit message so I've cc'd David to shed some light. If it's desired to always pass it then I'm failing to see why they've added it in the first place.

* Peter Krempa (pkrempa@redhat.com) wrote:
On Mon, Jun 04, 2018 at 16:51:18 +0200, Jiri Denemark wrote:
Libvirt relies on being able to kill the destination domain and resume the source one during migration until we called "cont" on the destination. Unfortunately, QEMU automatically activates block devices at the end of migration even when it's called with -S. This wasn't a big issue in the past since the guest is not running and thus no data are written to the block devices. However, when QEMU introduced its internal block device locks, we can no longer resume the source domain once the destination domain already activated the block devices (and thus acquired all locks) unless the destination domain is killed first.
Since it's impossible to synchronize the destination and the source libvirt daemons after a failed migration, QEMU introduced a new migration capability called "late-block-activat" which ensures QEMU won't activate block devices until it gets "cont". The only thing we need to do is to enable this capability whenever QEMU supports it.
I'm wondering when this new feature should _not_ be used. I did not get the information from the qemu commit message so I've cc'd David to shed some light.
If it's desired to always pass it then I'm failing to see why they've added it in the first place.
There was some worry that doing it by default would be a subtle API change; personally I didn't really see it as a problem, but since people were worried I made it switchable. See: https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg01300.html Dave -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
participants (4)
-
Dr. David Alan Gilbert
-
Jiri Denemark
-
Ján Tomko
-
Peter Krempa