[PATCH v4 00/10] Migration deprecated parts
On this v4: - addressed all markus comments. - rebased on latest. - improve formatting of migration.json - print block migration status when needed. - patches 7-10 are not mean to merge, they just show why we want to deprecate block migration and remove its support. - Patch 7 just drop support for -i/-b and qmp equivalents. - Patch 8 shows all the helpers and convolutions we need to have to support that -i and -d. - patch 9 drops block-incremental migration support. - patch 9 drops block migration support. Please review. Thanks, Juan. On this v3: - Rebase on top of upstream. - Changed v8.1 to 8.2 (I left the reviewed by anyways) - missing the block deprecation code, please. Please, review. Later, Juan. On this v2: - dropped -incoming <uri> deprecation Paolo came with a better solution using keyvalues. - skipped field is already ready for next pull request, so dropped. - dropped the RFC bits, nermal PATCH. - Assessed all the review comments. - Added indentation of migration.json. - Used the documentation pointer to substitute block migration. Please review. [v1] Hi this series describe the migration parts that have to be deprecated. - It is an rfc because I doubt that I did the deprecation process right. Hello Markus O:-) - skipped field: It is older than me, I have never know what it stands for. As far as I know it has always been zero. - inc/blk migrate command options. They are only used by block migration (that I deprecate on the following patch). And they are really bad. grep must_remove_block_options. - block migration. block jobs, whatever they are called this week are way more flexible. Current code works, but we broke it here and there, and really nobody has stand up to maintain it. It is quite contained and can be left there. Is anyone really using it? - old compression method. It don't work. See last try from Lukas to make a test that works reliabely. I failed with the same task years ago. It is really slow, and if compression is good for you, multifd + zlib is going to perform/compress way more. I don't know what to do with this code, really. * Remove it for this release? It don't work, and haven't work reliabely in quite a few time. * Deprecate it and remove in another couple of releases, i.e. normal deprecation. * Ideas? - -incoming <uri> if you need to set parameters (multifd cames to mind, and preempt has the same problem), you really needs to use defer. So what should we do here? This part is not urget, because management apps have a working option that are already using "defer", and the code simplifacation if we remove it is not so big. So we can leave it until 9.0 or whatever we think fit. What do you think? Later, Juan. Juan Quintela (10): migration: Improve json and formatting migration: Print block status when needed migration: migrate 'inc' command option is deprecated. migration: migrate 'blk' command option is deprecated. migration: Deprecate block migration migration: Deprecate old compression method [RFC] migration: Make -i/-b an error for hmp and qmp [RFC] migration: Remove helpers needed for -i/-b migrate options [RFC] migration: Remove support for block_incremental [RFC] migration: Remove block migration support docs/about/deprecated.rst | 25 + meson.build | 2 - qapi/migration.json | 133 ++--- include/migration/misc.h | 8 - migration/block.h | 52 -- migration/migration.h | 4 - migration/options.h | 8 - migration/block.c | 1027 -------------------------------- migration/colo.c | 1 - migration/migration-hmp-cmds.c | 40 +- migration/migration.c | 47 +- migration/options.c | 89 +-- migration/ram.c | 16 - migration/savevm.c | 5 - hmp-commands.hx | 17 +- migration/meson.build | 3 - 16 files changed, 111 insertions(+), 1366 deletions(-) delete mode 100644 migration/block.h delete mode 100644 migration/block.c -- 2.41.0
Signed-off-by: Juan Quintela <quintela@redhat.com> --- qapi/migration.json | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/qapi/migration.json b/qapi/migration.json index d7dfaa5db9..6865fea3c5 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -73,7 +73,7 @@ { 'struct': 'MigrationStats', 'data': {'transferred': 'int', 'remaining': 'int', 'total': 'int' , 'duplicate': 'int', - 'skipped': { 'type': 'int', 'features': ['deprecated'] }, + 'skipped': { 'type': 'int', 'features': [ 'deprecated' ] }, 'normal': 'int', 'normal-bytes': 'int', 'dirty-pages-rate': 'int', 'mbps': 'number', 'dirty-sync-count': 'int', @@ -440,10 +440,9 @@ # compress and xbzrle are both on, compress only takes effect in # the ram bulk stage, after that, it will be disabled and only # xbzrle takes effect, this can help to minimize migration -# traffic. The feature is disabled by default. (since 2.4 ) +# traffic. The feature is disabled by default. (since 2.4) # -# @events: generate events for each migration state change (since 2.4 -# ) +# @events: generate events for each migration state change (since 2.4) # # @auto-converge: If enabled, QEMU will automatically throttle down # the guest to speed up convergence of RAM migration. (since 1.6) -- 2.41.0
Juan Quintela <quintela@redhat.com> writes:
Signed-off-by: Juan Quintela <quintela@redhat.com> --- qapi/migration.json | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/qapi/migration.json b/qapi/migration.json index d7dfaa5db9..6865fea3c5 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -73,7 +73,7 @@ { 'struct': 'MigrationStats', 'data': {'transferred': 'int', 'remaining': 'int', 'total': 'int' , 'duplicate': 'int', - 'skipped': { 'type': 'int', 'features': ['deprecated'] }, + 'skipped': { 'type': 'int', 'features': [ 'deprecated' ] }, 'normal': 'int', 'normal-bytes': 'int', 'dirty-pages-rate': 'int', 'mbps': 'number', 'dirty-sync-count': 'int', @@ -440,10 +440,9 @@ # compress and xbzrle are both on, compress only takes effect in # the ram bulk stage, after that, it will be disabled and only # xbzrle takes effect, this can help to minimize migration -# traffic. The feature is disabled by default. (since 2.4 ) +# traffic. The feature is disabled by default. (since 2.4) # -# @events: generate events for each migration state change (since 2.4 -# ) +# @events: generate events for each migration state change (since 2.4) # # @auto-converge: If enabled, QEMU will automatically throttle down # the guest to speed up convergence of RAM migration. (since 1.6)
Reviewed-by: Markus Armbruster <armbru@redhat.com>
The new line was only printed when command options were used. When we used migration parameters and capabilities, it wasn't. Signed-off-by: Juan Quintela <quintela@redhat.com> --- migration/migration-hmp-cmds.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index c115ef2d23..e0e16afa34 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -30,6 +30,7 @@ #include "sysemu/runstate.h" #include "ui/qemu-spice.h" #include "sysemu/sysemu.h" +#include "options.h" #include "migration.h" static void migration_global_dump(Monitor *mon) @@ -682,7 +683,6 @@ void hmp_x_colo_lost_heartbeat(Monitor *mon, const QDict *qdict) typedef struct HMPMigrationStatus { QEMUTimer *timer; Monitor *mon; - bool is_block_migration; } HMPMigrationStatus; static void hmp_migrate_status_cb(void *opaque) @@ -708,7 +708,7 @@ static void hmp_migrate_status_cb(void *opaque) timer_mod(status->timer, qemu_clock_get_ms(QEMU_CLOCK_REALTIME) + 1000); } else { - if (status->is_block_migration) { + if (migrate_block()) { monitor_printf(status->mon, "\n"); } if (info->error_desc) { @@ -748,7 +748,6 @@ void hmp_migrate(Monitor *mon, const QDict *qdict) status = g_malloc0(sizeof(*status)); status->mon = mon; - status->is_block_migration = blk || inc; status->timer = timer_new_ms(QEMU_CLOCK_REALTIME, hmp_migrate_status_cb, status); timer_mod(status->timer, qemu_clock_get_ms(QEMU_CLOCK_REALTIME)); -- 2.41.0
Set the 'block_incremental' migration parameter to 'true' instead. Reviewed-by: Thomas Huth <thuth@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com> --- Improve documentation and style (thanks Markus) --- docs/about/deprecated.rst | 7 +++++++ qapi/migration.json | 8 +++++++- migration/migration.c | 6 ++++++ 3 files changed, 20 insertions(+), 1 deletion(-) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 1c4d7f36f0..1b6b2870cf 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -452,3 +452,10 @@ Migration ``skipped`` field in Migration stats has been deprecated. It hasn't been used for more than 10 years. +``inc`` migrate command option (since 8.2) +'''''''''''''''''''''''''''''''''''''''''' + +The new way to modify migration is using migration parameters. +``inc`` functionality can be achieved by setting the +``block-incremental`` migration parameter to ``true``. + diff --git a/qapi/migration.json b/qapi/migration.json index 6865fea3c5..56bbd55b87 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1492,6 +1492,11 @@ # # @resume: resume one paused migration, default "off". (since 3.0) # +# Features: +# +# @deprecated: Member @inc is deprecated. Use migration parameter +# @block-incremental instead. +# # Returns: nothing on success # # Since: 0.14 @@ -1513,7 +1518,8 @@ # <- { "return": {} } ## { 'command': 'migrate', - 'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool', + 'data': {'uri': 'str', '*blk': 'bool', + '*inc': { 'type': 'bool', 'features': [ 'deprecated' ] }, '*detach': 'bool', '*resume': 'bool' } } ## diff --git a/migration/migration.c b/migration/migration.c index 1c6c81ad49..ac4897fe0d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1601,6 +1601,12 @@ static bool migrate_prepare(MigrationState *s, bool blk, bool blk_inc, { Error *local_err = NULL; + if (blk_inc) { + warn_report("@inc/-i migrate option is deprecated, set the" + "'block-incremental' migration parameter to 'true'" + " instead."); + } + if (resume) { if (s->state != MIGRATION_STATUS_POSTCOPY_PAUSED) { error_setg(errp, "Cannot resume if there is no " -- 2.41.0
Juan Quintela <quintela@redhat.com> writes:
Set the 'block_incremental' migration parameter to 'true' instead.
Reviewed-by: Thomas Huth <thuth@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
---
Improve documentation and style (thanks Markus) --- docs/about/deprecated.rst | 7 +++++++ qapi/migration.json | 8 +++++++- migration/migration.c | 6 ++++++ 3 files changed, 20 insertions(+), 1 deletion(-)
diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 1c4d7f36f0..1b6b2870cf 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -452,3 +452,10 @@ Migration ``skipped`` field in Migration stats has been deprecated. It hasn't been used for more than 10 years.
+``inc`` migrate command option (since 8.2) +'''''''''''''''''''''''''''''''''''''''''' + +The new way to modify migration is using migration parameters. +``inc`` functionality can be achieved by setting the +``block-incremental`` migration parameter to ``true``. + diff --git a/qapi/migration.json b/qapi/migration.json index 6865fea3c5..56bbd55b87 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1492,6 +1492,11 @@ # # @resume: resume one paused migration, default "off". (since 3.0) # +# Features: +# +# @deprecated: Member @inc is deprecated. Use migration parameter +# @block-incremental instead.
This is fine now. It becomes bad advice in PATCH 05, which deprecates @block-incremental. Two solutions: 1. Change this patch to point to an alternative that will *not* be deprecated. 2. Change PATCH 05. Same end result.
+# # Returns: nothing on success # # Since: 0.14 @@ -1513,7 +1518,8 @@ # <- { "return": {} } ## { 'command': 'migrate', - 'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool', + 'data': {'uri': 'str', '*blk': 'bool', + '*inc': { 'type': 'bool', 'features': [ 'deprecated' ] }, '*detach': 'bool', '*resume': 'bool' } }
## diff --git a/migration/migration.c b/migration/migration.c index 1c6c81ad49..ac4897fe0d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1601,6 +1601,12 @@ static bool migrate_prepare(MigrationState *s, bool blk, bool blk_inc, { Error *local_err = NULL;
+ if (blk_inc) { + warn_report("@inc/-i migrate option is deprecated, set the"
This is either about QMP migrate's parameter "inc", or HMP migrate's flags -i. In the former case, we want something like "parameter 'inc' is deprecated". In the latter case, we want something like "-i is deprecated". Trying to do both in a single message results in a sub-par message. If you want to do better, you have to check in hmp_migrate(), too. Then hmp_migrate can refer to "-i", and migrate_prepare() to "parameter 'inc'". Up to you. If you decide a single message is good enough, use something like "parameter 'inc' / flag -i is deprecated".
+ "'block-incremental' migration parameter to 'true'" + " instead.");
Again, fine now, becomes bad advice in PATCH 05.
+ } + if (resume) { if (s->state != MIGRATION_STATUS_POSTCOPY_PAUSED) { error_setg(errp, "Cannot resume if there is no "
Markus Armbruster <armbru@redhat.com> wrote:
Juan Quintela <quintela@redhat.com> writes:
Set the 'block_incremental' migration parameter to 'true' instead.
Reviewed-by: Thomas Huth <thuth@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
---
Improve documentation and style (thanks Markus) --- docs/about/deprecated.rst | 7 +++++++ qapi/migration.json | 8 +++++++- migration/migration.c | 6 ++++++ 3 files changed, 20 insertions(+), 1 deletion(-)
diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 1c4d7f36f0..1b6b2870cf 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -452,3 +452,10 @@ Migration ``skipped`` field in Migration stats has been deprecated. It hasn't been used for more than 10 years.
+``inc`` migrate command option (since 8.2) +'''''''''''''''''''''''''''''''''''''''''' + +The new way to modify migration is using migration parameters. +``inc`` functionality can be achieved by setting the +``block-incremental`` migration parameter to ``true``. + diff --git a/qapi/migration.json b/qapi/migration.json index 6865fea3c5..56bbd55b87 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1492,6 +1492,11 @@ # # @resume: resume one paused migration, default "off". (since 3.0) # +# Features: +# +# @deprecated: Member @inc is deprecated. Use migration parameter +# @block-incremental instead.
This is fine now. It becomes bad advice in PATCH 05, which deprecates @block-incremental. Two solutions:
1. Change this patch to point to an alternative that will *not* be deprecated.
Ok, clearly I am not explaining myself properly O:-) History of block migration: * In the beggining there was -b and -i migrate options There was the only way to do storage of migration. * We moved to use parameters and capabilities for migration So we created @block-incremental and @block. But we didn't remove the command line options (for backward compatibility). * We were asked to modify migration so some storaged was migrated and some was not migrated during migration. But block people found that it was a good idea to allow storage migration without migrating the vm, so they created this blockdev-mirror mechanism that is shinny, funny, faster, <whatever> better. So now we have old code that basically nobody uses (the last big user was COLO, but now it can use multifd). So we want to drop it, but we don't care about a direct replacement. So, why I am interested in removing this? - @block and @block-incremental: If you don't use block migration, their existence don't bother you. They are "quite" independent of the rest of the migration code (they could be better integrated, but not big trouble here). - migrate options -i/-b: This ones hurt us each time that we need to do changing in options. Notice that we have "perfect" replacements with @block and @block-incremental, exactly the same result/semantics/... You can see the trobles in the RFC patches * [PATCH v4 07/10] [RFC] migration: Make -i/-b an error for hmp and qmp * [PATCH v4 08/10] [RFC] migration: Remove helpers needed for -i/-b migrate options So what I want, I want to remove -i/-b in the next version (9.0?). For the other, I want to remove it, but I don't care if the code is around in "deprecated" state for another couple of years if there are still people that feel that they want it. This is the reason that I put a pointer for -i/-b to @block/@block-incremental. They are "perfect" replacements. I can put here to use blockdev-mirror + NBD, but the replacement is not so direct. Does this make sense?
2. Change PATCH 05.
Same end result.
+# # Returns: nothing on success # # Since: 0.14 @@ -1513,7 +1518,8 @@ # <- { "return": {} } ## { 'command': 'migrate', - 'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool', + 'data': {'uri': 'str', '*blk': 'bool', + '*inc': { 'type': 'bool', 'features': [ 'deprecated' ] }, '*detach': 'bool', '*resume': 'bool' } }
## diff --git a/migration/migration.c b/migration/migration.c index 1c6c81ad49..ac4897fe0d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1601,6 +1601,12 @@ static bool migrate_prepare(MigrationState *s, bool blk, bool blk_inc, { Error *local_err = NULL;
+ if (blk_inc) { + warn_report("@inc/-i migrate option is deprecated, set the"
This is either about QMP migrate's parameter "inc", or HMP migrate's flags -i.
Needs to be @inc. I want about the "-i" command option in other place.
In the former case, we want something like "parameter 'inc' is deprecated".
This one.
In the latter case, we want something like "-i is deprecated".
Ok, changing.
Trying to do both in a single message results in a sub-par message. If you want to do better, you have to check in hmp_migrate(), too. Then hmp_migrate can refer to "-i", and migrate_prepare() to "parameter 'inc'". Up to you.
That was the intention, but I forgot to update this message.
If you decide a single message is good enough, use something like "parameter 'inc' / flag -i is deprecated".
Later, Juan.
Juan Quintela <quintela@redhat.com> writes:
Markus Armbruster <armbru@redhat.com> wrote:
Juan Quintela <quintela@redhat.com> writes:
Set the 'block_incremental' migration parameter to 'true' instead.
Reviewed-by: Thomas Huth <thuth@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
---
Improve documentation and style (thanks Markus) --- docs/about/deprecated.rst | 7 +++++++ qapi/migration.json | 8 +++++++- migration/migration.c | 6 ++++++ 3 files changed, 20 insertions(+), 1 deletion(-)
diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 1c4d7f36f0..1b6b2870cf 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -452,3 +452,10 @@ Migration ``skipped`` field in Migration stats has been deprecated. It hasn't been used for more than 10 years.
+``inc`` migrate command option (since 8.2) +'''''''''''''''''''''''''''''''''''''''''' + +The new way to modify migration is using migration parameters. +``inc`` functionality can be achieved by setting the +``block-incremental`` migration parameter to ``true``. + diff --git a/qapi/migration.json b/qapi/migration.json index 6865fea3c5..56bbd55b87 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1492,6 +1492,11 @@ # # @resume: resume one paused migration, default "off". (since 3.0) # +# Features: +# +# @deprecated: Member @inc is deprecated. Use migration parameter +# @block-incremental instead.
This is fine now. It becomes bad advice in PATCH 05, which deprecates @block-incremental. Two solutions:
1. Change this patch to point to an alternative that will *not* be deprecated.
Ok, clearly I am not explaining myself properly O:-)
History of block migration: * In the beggining there was -b and -i migrate options There was the only way to do storage of migration. * We moved to use parameters and capabilities for migration So we created @block-incremental and @block. But we didn't remove the command line options (for backward compatibility). * We were asked to modify migration so some storaged was migrated and some was not migrated during migration. But block people found that it was a good idea to allow storage migration without migrating the vm, so they created this blockdev-mirror mechanism that is shinny, funny, faster, <whatever> better.
So now we have old code that basically nobody uses (the last big user was COLO, but now it can use multifd). So we want to drop it, but we don't care about a direct replacement.
So, why I am interested in removing this? - @block and @block-incremental: If you don't use block migration, their existence don't bother you. They are "quite" independent of the rest of the migration code (they could be better integrated, but not big trouble here). - migrate options -i/-b: This ones hurt us each time that we need to do changing in options. Notice that we have "perfect" replacements with @block and @block-incremental, exactly the same result/semantics/... You can see the trobles in the RFC patches
* [PATCH v4 07/10] [RFC] migration: Make -i/-b an error for hmp and qmp * [PATCH v4 08/10] [RFC] migration: Remove helpers needed for -i/-b migrate options
So what I want, I want to remove -i/-b in the next version (9.0?). For the other, I want to remove it, but I don't care if the code is around in "deprecated" state for another couple of years if there are still people that feel that they want it.
This is the reason that I put a pointer for -i/-b to @block/@block-incremental. They are "perfect" replacements.
I can put here to use blockdev-mirror + NBD, but the replacement is not so direct.
Does this make sense?
I see where you're coming from. Now let's change perspective for a minute: what's the purpose of deprecating stuff? We normally deprecate with intent to remove. We don't remove right away, because we promised to first deprecate for a grace period, so users can adjust in an orderly manner. The deprecation serves as signal "you need to adjust". The documentation that comes with it should help with the adjustment. It's commonly of the form "use $alternative instead". The alternative is often a direct replacement, but not always. There could even be no replacement at all. We don't promise replacements, we promise an orderly process, so users can adjust. Sometimes, we don't have firm plans to remove, but are more like "maybe remove when gets in the way". We could soften the "you need to adjust" signal in documentation, but I doubt that's a good idea. Regardless, the need to help users adjust remains. Back to your patches. There are two separate interfaces to block migration, and both are deprecated at the end of the series: 1. Migration parameter @block-incremental & friends Not in the way, content to keep around for longer if it helps users. The deprecation documentation advises to use block-mirror with NBD instead. All good. 2. QMP migrate parameters @inc and @blk Firm intent to remove as soon as the grace period expires, because it's in the way. The deprecation documentation advises to use interface 1 instead. But that's deprecated, too! Insufficiently careful readers will miss that the replacement is deprecated, and just use it. Risks surprise when the replacement goes away, too. More careful readers will realize that this advises to use something we elsewhere advise not to use. Contradiction! Confusion ensues. On further reflection, these readers might conclude that the *combined* advice is to use block-mirror with NBD instead. This is correct. So why not tell them? Perhaps you'd like to give more nuanced advice, like "you should move to block-mirror with NBD, but if that's not practical for you, you should at least move to @block-incremental & friends, which will likely stick around for longer." That's fine. All I'm asking for is to not make things more confusing than they need to be :) [...]
Markus Armbruster <armbru@redhat.com> wrote:
Juan Quintela <quintela@redhat.com> writes:
Juan Quintela <quintela@redhat.com> writes: So what I want, I want to remove -i/-b in the next version (9.0?). For
Markus Armbruster <armbru@redhat.com> wrote: the other, I want to remove it, but I don't care if the code is around in "deprecated" state for another couple of years if there are still people that feel that they want it.
This is the reason that I put a pointer for -i/-b to @block/@block-incremental. They are "perfect" replacements.
I can put here to use blockdev-mirror + NBD, but the replacement is not so direct.
Does this make sense?
I see where you're coming from. Now let's change perspective for a minute: what's the purpose of deprecating stuff?
We normally deprecate with intent to remove.
We don't remove right away, because we promised to first deprecate for a grace period, so users can adjust in an orderly manner. The deprecation serves as signal "you need to adjust". The documentation that comes with it should help with the adjustment. It's commonly of the form "use $alternative instead". The alternative is often a direct replacement, but not always. There could even be no replacement at all. We don't promise replacements, we promise an orderly process, so users can adjust.
Sometimes, we don't have firm plans to remove, but are more like "maybe remove when gets in the way". We could soften the "you need to adjust" signal in documentation, but I doubt that's a good idea. Regardless, the need to help users adjust remains.
Back to your patches. There are two separate interfaces to block migration, and both are deprecated at the end of the series:
1. Migration parameter @block-incremental & friends
Not in the way, content to keep around for longer if it helps users.
The deprecation documentation advises to use block-mirror with NBD instead. All good.
2. QMP migrate parameters @inc and @blk
Firm intent to remove as soon as the grace period expires, because it's in the way.
The deprecation documentation advises to use interface 1 instead. But that's deprecated, too!
Insufficiently careful readers will miss that the replacement is deprecated, and just use it. Risks surprise when the replacement goes away, too.
More careful readers will realize that this advises to use something we elsewhere advise not to use. Contradiction! Confusion ensues.
On further reflection, these readers might conclude that the *combined* advice is to use block-mirror with NBD instead. This is correct.
So why not tell them?
Perhaps you'd like to give more nuanced advice, like "you should move to block-mirror with NBD, but if that's not practical for you, you should at least move to @block-incremental & friends, which will likely stick around for longer." That's fine. All I'm asking for is to not make things more confusing than they need to be :)
[...]
Telling this in deprecated.rst is enough? or you want me to put it also in the error/warn messages and qapi? Later, Juan.
Juan Quintela <quintela@redhat.com> writes:
Markus Armbruster <armbru@redhat.com> wrote:
Juan Quintela <quintela@redhat.com> writes:
Juan Quintela <quintela@redhat.com> writes: So what I want, I want to remove -i/-b in the next version (9.0?). For
Markus Armbruster <armbru@redhat.com> wrote: the other, I want to remove it, but I don't care if the code is around in "deprecated" state for another couple of years if there are still people that feel that they want it.
This is the reason that I put a pointer for -i/-b to @block/@block-incremental. They are "perfect" replacements.
I can put here to use blockdev-mirror + NBD, but the replacement is not so direct.
Does this make sense?
I see where you're coming from. Now let's change perspective for a minute: what's the purpose of deprecating stuff?
We normally deprecate with intent to remove.
We don't remove right away, because we promised to first deprecate for a grace period, so users can adjust in an orderly manner. The deprecation serves as signal "you need to adjust". The documentation that comes with it should help with the adjustment. It's commonly of the form "use $alternative instead". The alternative is often a direct replacement, but not always. There could even be no replacement at all. We don't promise replacements, we promise an orderly process, so users can adjust.
Sometimes, we don't have firm plans to remove, but are more like "maybe remove when gets in the way". We could soften the "you need to adjust" signal in documentation, but I doubt that's a good idea. Regardless, the need to help users adjust remains.
Back to your patches. There are two separate interfaces to block migration, and both are deprecated at the end of the series:
1. Migration parameter @block-incremental & friends
Not in the way, content to keep around for longer if it helps users.
The deprecation documentation advises to use block-mirror with NBD instead. All good.
2. QMP migrate parameters @inc and @blk
Firm intent to remove as soon as the grace period expires, because it's in the way.
The deprecation documentation advises to use interface 1 instead. But that's deprecated, too!
Insufficiently careful readers will miss that the replacement is deprecated, and just use it. Risks surprise when the replacement goes away, too.
More careful readers will realize that this advises to use something we elsewhere advise not to use. Contradiction! Confusion ensues.
On further reflection, these readers might conclude that the *combined* advice is to use block-mirror with NBD instead. This is correct.
So why not tell them?
Perhaps you'd like to give more nuanced advice, like "you should move to block-mirror with NBD, but if that's not practical for you, you should at least move to @block-incremental & friends, which will likely stick around for longer." That's fine. All I'm asking for is to not make things more confusing than they need to be :)
[...]
Telling this in deprecated.rst is enough? or you want me to put it also in the error/warn messages and qapi?
Let's make all of them point to blockdev-mirror with NBD. If you think mentioning @block-incremental & friends would be useful in some or all places would be useful, go ahead, I don't mind.
Set the 'block' migration capability to 'true' instead. Signed-off-by: Juan Quintela <quintela@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> --- Improve documentation and style (markus) --- docs/about/deprecated.rst | 7 +++++++ qapi/migration.json | 6 ++++-- migration/migration.c | 5 +++++ 3 files changed, 16 insertions(+), 2 deletions(-) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 1b6b2870cf..e6928b96cf 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -459,3 +459,10 @@ The new way to modify migration is using migration parameters. ``inc`` functionality can be achieved by setting the ``block-incremental`` migration parameter to ``true``. +``blk`` migrate command option (since 8.2) +'''''''''''''''''''''''''''''''''''''''''' + +The new way to modify migration is using migration parameters. +``blk`` functionality can be achieved by setting the +``block`` migration capability to ``true``. + diff --git a/qapi/migration.json b/qapi/migration.json index 56bbd55b87..64ebced761 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1495,7 +1495,8 @@ # Features: # # @deprecated: Member @inc is deprecated. Use migration parameter -# @block-incremental instead. +# @block-incremental instead. Member @blk is deprecated. Set +# migration capability 'block' to 'true' instead. # # Returns: nothing on success # @@ -1518,7 +1519,8 @@ # <- { "return": {} } ## { 'command': 'migrate', - 'data': {'uri': 'str', '*blk': 'bool', + 'data': {'uri': 'str', + '*blk': { 'type': 'bool', 'features': [ 'deprecated' ] }, '*inc': { 'type': 'bool', 'features': [ 'deprecated' ] }, '*detach': 'bool', '*resume': 'bool' } } diff --git a/migration/migration.c b/migration/migration.c index ac4897fe0d..9e4ae6b772 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1607,6 +1607,11 @@ static bool migrate_prepare(MigrationState *s, bool blk, bool blk_inc, " instead."); } + if (blk) { + warn_report("@blk/-i migrate option is deprecated, set the " + "'block' capability to 'true' instead."); + } + if (resume) { if (s->state != MIGRATION_STATUS_POSTCOPY_PAUSED) { error_setg(errp, "Cannot resume if there is no " -- 2.41.0
Juan Quintela <quintela@redhat.com> writes:
Set the 'block' migration capability to 'true' instead.
Signed-off-by: Juan Quintela <quintela@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com>
---
Improve documentation and style (markus) --- docs/about/deprecated.rst | 7 +++++++ qapi/migration.json | 6 ++++-- migration/migration.c | 5 +++++ 3 files changed, 16 insertions(+), 2 deletions(-)
diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 1b6b2870cf..e6928b96cf 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -459,3 +459,10 @@ The new way to modify migration is using migration parameters. ``inc`` functionality can be achieved by setting the ``block-incremental`` migration parameter to ``true``.
+``blk`` migrate command option (since 8.2) +'''''''''''''''''''''''''''''''''''''''''' + +The new way to modify migration is using migration parameters. +``blk`` functionality can be achieved by setting the +``block`` migration capability to ``true``. + diff --git a/qapi/migration.json b/qapi/migration.json index 56bbd55b87..64ebced761 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1495,7 +1495,8 @@ # Features: # # @deprecated: Member @inc is deprecated. Use migration parameter -# @block-incremental instead. +# @block-incremental instead. Member @blk is deprecated. Set
Two spaces between sentences for consistency.
+# migration capability 'block' to 'true' instead. # # Returns: nothing on success # @@ -1518,7 +1519,8 @@ # <- { "return": {} } ## { 'command': 'migrate', - 'data': {'uri': 'str', '*blk': 'bool', + 'data': {'uri': 'str', + '*blk': { 'type': 'bool', 'features': [ 'deprecated' ] }, '*inc': { 'type': 'bool', 'features': [ 'deprecated' ] }, '*detach': 'bool', '*resume': 'bool' } }
diff --git a/migration/migration.c b/migration/migration.c index ac4897fe0d..9e4ae6b772 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1607,6 +1607,11 @@ static bool migrate_prepare(MigrationState *s, bool blk, bool blk_inc, " instead."); }
+ if (blk) { + warn_report("@blk/-i migrate option is deprecated, set the " + "'block' capability to 'true' instead."); + } + if (resume) { if (s->state != MIGRATION_STATUS_POSTCOPY_PAUSED) { error_setg(errp, "Cannot resume if there is no "
My comments for PATCH 01 apply.
It is obsolete. It is better to use driver-mirror with NBD instead. CC: Kevin Wolf <kwolf@redhat.com> CC: Eric Blake <eblake@redhat.com> CC: Stefan Hajnoczi <stefanha@redhat.com> CC: Hanna Czenczek <hreitz@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> --- docs/about/deprecated.rst | 10 ++++++++++ qapi/migration.json | 29 ++++++++++++++++++++++++----- migration/block.c | 3 +++ migration/options.c | 9 ++++++++- 4 files changed, 45 insertions(+), 6 deletions(-) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index e6928b96cf..6490925cdc 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -466,3 +466,13 @@ The new way to modify migration is using migration parameters. ``blk`` functionality can be achieved by setting the ``block`` migration capability to ``true``. +block migration (since 8.2) +''''''''''''''''''''''''''' + +Block migration is too inflexible. It needs to migrate all block +devices or none. + +Please see "QMP invocation for live storage migration with +``blockdev-mirror`` + NBD" in docs/interop/live-block-operations.rst +for a detailed explanation. + diff --git a/qapi/migration.json b/qapi/migration.json index 64ebced761..5e545d10c0 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -269,11 +269,15 @@ # average memory load of the virtual CPU indirectly. Note that # zero means guest doesn't dirty memory. (Since 8.1) # +# Features: +# +# @deprecated: Member @disk is deprecated because block migration is. +# # Since: 0.14 ## { 'struct': 'MigrationInfo', 'data': {'*status': 'MigrationStatus', '*ram': 'MigrationStats', - '*disk': 'MigrationStats', + '*disk': { 'type': 'MigrationStats', 'features': [ 'deprecated' ] }, '*vfio': 'VfioStats', '*xbzrle-cache': 'XBZRLECacheStats', '*total-time': 'int', @@ -525,6 +529,9 @@ # # Features: # +# @deprecated: Member @block is deprecated. Use blockdev-mirror with +# NBD instead. +# # @unstable: Members @x-colo and @x-ignore-shared are experimental. # # Since: 1.2 @@ -534,7 +541,8 @@ 'compress', 'events', 'postcopy-ram', { 'name': 'x-colo', 'features': [ 'unstable' ] }, 'release-ram', - 'block', 'return-path', 'pause-before-switchover', 'multifd', + { 'name': 'block', 'features': [ 'deprecated' ] }, + 'return-path', 'pause-before-switchover', 'multifd', 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate', { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] }, 'validate-uuid', 'background-snapshot', @@ -825,6 +833,9 @@ # # Features: # +# @deprecated: Member @block-incremental is deprecated. Use +# blockdev-mirror with NBD instead. +# # @unstable: Members @x-checkpoint-delay and @x-vcpu-dirty-limit-period # are experimental. # @@ -840,7 +851,7 @@ 'tls-creds', 'tls-hostname', 'tls-authz', 'max-bandwidth', 'downtime-limit', { 'name': 'x-checkpoint-delay', 'features': [ 'unstable' ] }, - 'block-incremental', + { 'name': 'block-incremental', 'features': [ 'deprecated' ] }, 'multifd-channels', 'xbzrle-cache-size', 'max-postcopy-bandwidth', 'max-cpu-throttle', 'multifd-compression', @@ -991,6 +1002,9 @@ # # Features: # +# @deprecated: Member @block-incremental is deprecated. Use +# blockdev-mirror with NBD instead. +# # @unstable: Members @x-checkpoint-delay and @x-vcpu-dirty-limit-period # are experimental. # @@ -1019,7 +1033,8 @@ '*downtime-limit': 'uint64', '*x-checkpoint-delay': { 'type': 'uint32', 'features': [ 'unstable' ] }, - '*block-incremental': 'bool', + '*block-incremental': { 'type': 'bool', + 'features': [ 'deprecated' ] }, '*multifd-channels': 'uint8', '*xbzrle-cache-size': 'size', '*max-postcopy-bandwidth': 'size', @@ -1194,6 +1209,9 @@ # # Features: # +# @deprecated: Member @block-incremental is deprecated. Use +# blockdev-mirror with NBD instead. +# # @unstable: Members @x-checkpoint-delay and @x-vcpu-dirty-limit-period # are experimental. # @@ -1219,7 +1237,8 @@ '*downtime-limit': 'uint64', '*x-checkpoint-delay': { 'type': 'uint32', 'features': [ 'unstable' ] }, - '*block-incremental': 'bool', + '*block-incremental': { 'type': 'bool', + 'features': [ 'deprecated' ] }, '*multifd-channels': 'uint8', '*xbzrle-cache-size': 'size', '*max-postcopy-bandwidth': 'size', diff --git a/migration/block.c b/migration/block.c index 5f930870a5..7f563c70f6 100644 --- a/migration/block.c +++ b/migration/block.c @@ -729,6 +729,9 @@ static int block_save_setup(QEMUFile *f, void *opaque) trace_migration_block_save("setup", block_mig_state.submitted, block_mig_state.transferred); + warn_report("block migration is deprecated. Use blockdev-mirror with" + "NBD instead."); + qemu_mutex_lock_iothread(); ret = init_blk_migration(f); if (ret < 0) { diff --git a/migration/options.c b/migration/options.c index 6bbfd4853d..bb6005052f 100644 --- a/migration/options.c +++ b/migration/options.c @@ -12,6 +12,7 @@ */ #include "qemu/osdep.h" +#include "qemu/error-report.h" #include "exec/target_page.h" #include "qapi/clone-visitor.h" #include "qapi/error.h" @@ -464,10 +465,14 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) if (new_caps[MIGRATION_CAPABILITY_BLOCK]) { error_setg(errp, "QEMU compiled without old-style (blk/-b, inc/-i) " "block migration"); - error_append_hint(errp, "Use drive_mirror+NBD instead.\n"); + error_append_hint(errp, "Use blockdev-mirror with NBD instead.\n"); return false; } #endif + if (new_caps[MIGRATION_CAPABILITY_BLOCK]) { + warn_report("Block migration is deprecated. " + "Use blockdev-mirror with NBD instead."); + } #ifndef CONFIG_REPLICATION if (new_caps[MIGRATION_CAPABILITY_X_COLO]) { @@ -1351,6 +1356,8 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp) } if (params->has_block_incremental) { + warn_report("Block migration is deprecated. " + "Use blockdev-mirror with NBD instead."); s->parameters.block_incremental = params->block_incremental; } if (params->has_multifd_channels) { -- 2.41.0
Signed-off-by: Juan Quintela <quintela@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Acked-by: Peter Xu <peterx@redhat.com> --- docs/about/deprecated.rst | 8 ++++ qapi/migration.json | 79 +++++++++++++++++++++++++-------------- migration/options.c | 13 +++++++ 3 files changed, 72 insertions(+), 28 deletions(-) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 6490925cdc..a6da0df8e1 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -476,3 +476,11 @@ Please see "QMP invocation for live storage migration with ``blockdev-mirror`` + NBD" in docs/interop/live-block-operations.rst for a detailed explanation. +old compression method (since 8.2) +'''''''''''''''''''''''''''''''''' + +Compression method fails too much. Too many races. We are going to +remove it if nobody fixes it. For starters, migration-test +compression tests are disabled becase they fail randomly. If you need +compression, use multifd compression methods. + diff --git a/qapi/migration.json b/qapi/migration.json index 5e545d10c0..e599062f7d 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -272,6 +272,10 @@ # Features: # # @deprecated: Member @disk is deprecated because block migration is. +# Member @compression is deprecated because it is unreliable and +# untested. It is recommended to use multifd migration, which +# offers an alternative compression implementation that is +# reliable and tested. # # Since: 0.14 ## @@ -289,7 +293,7 @@ '*blocked-reasons': ['str'], '*postcopy-blocktime': 'uint32', '*postcopy-vcpu-blocktime': ['uint32'], - '*compression': 'CompressionStats', + '*compression': { 'type': 'CompressionStats', 'features': [ 'deprecated' ] }, '*socket-address': ['SocketAddress'], '*dirty-limit-throttle-time-per-round': 'uint64', '*dirty-limit-ring-full-time': 'uint64'} } @@ -530,7 +534,10 @@ # Features: # # @deprecated: Member @block is deprecated. Use blockdev-mirror with -# NBD instead. +# NBD instead. Member @compression is deprecated because it is +# unreliable and untested. It is recommended to use multifd +# migration, which offers an alternative compression +# implementation that is reliable and tested. # # @unstable: Members @x-colo and @x-ignore-shared are experimental. # @@ -538,7 +545,8 @@ ## { 'enum': 'MigrationCapability', 'data': ['xbzrle', 'rdma-pin-all', 'auto-converge', 'zero-blocks', - 'compress', 'events', 'postcopy-ram', + { 'name': 'compress', 'features': [ 'deprecated' ] }, + 'events', 'postcopy-ram', { 'name': 'x-colo', 'features': [ 'unstable' ] }, 'release-ram', { 'name': 'block', 'features': [ 'deprecated' ] }, @@ -834,7 +842,9 @@ # Features: # # @deprecated: Member @block-incremental is deprecated. Use -# blockdev-mirror with NBD instead. +# blockdev-mirror with NBD instead. Members @compress-level, +# @compress-threads, @decompress-threads and @compress-wait-thread +# are deprecated because @compression is deprecated. # # @unstable: Members @x-checkpoint-delay and @x-vcpu-dirty-limit-period # are experimental. @@ -844,8 +854,11 @@ { 'enum': 'MigrationParameter', 'data': ['announce-initial', 'announce-max', 'announce-rounds', 'announce-step', - 'compress-level', 'compress-threads', 'decompress-threads', - 'compress-wait-thread', 'throttle-trigger-threshold', + { 'name': 'compress-level', 'features': [ 'deprecated' ] }, + { 'name': 'compress-threads', 'features': [ 'deprecated' ] }, + { 'name': 'decompress-threads', 'features': [ 'deprecated' ] }, + { 'name': 'compress-wait-thread', 'features': [ 'deprecated' ] }, + 'throttle-trigger-threshold', 'cpu-throttle-initial', 'cpu-throttle-increment', 'cpu-throttle-tailslow', 'tls-creds', 'tls-hostname', 'tls-authz', 'max-bandwidth', @@ -875,16 +888,16 @@ # @announce-step: Increase in delay (in milliseconds) between # subsequent packets in the announcement (Since 4.0) # -# @compress-level: compression level +# @compress-level: compression level. # -# @compress-threads: compression thread count +# @compress-threads: compression thread count. # # @compress-wait-thread: Controls behavior when all compression # threads are currently busy. If true (default), wait for a free # compression thread to become available; otherwise, send the page -# uncompressed. (Since 3.1) +# uncompressed. (Since 3.1) # -# @decompress-threads: decompression thread count +# @decompress-threads: decompression thread count. # # @throttle-trigger-threshold: The ratio of bytes_dirty_period and # bytes_xfer_period to trigger throttling. It is expressed as @@ -1003,7 +1016,9 @@ # Features: # # @deprecated: Member @block-incremental is deprecated. Use -# blockdev-mirror with NBD instead. +# blockdev-mirror with NBD instead. Members @compress-level, +# @compress-threads, @decompress-threads and @compress-wait-thread +# are deprecated because @compression is deprecated. # # @unstable: Members @x-checkpoint-delay and @x-vcpu-dirty-limit-period # are experimental. @@ -1018,10 +1033,14 @@ '*announce-max': 'size', '*announce-rounds': 'size', '*announce-step': 'size', - '*compress-level': 'uint8', - '*compress-threads': 'uint8', - '*compress-wait-thread': 'bool', - '*decompress-threads': 'uint8', + '*compress-level': { 'type': 'uint8', + 'features': [ 'deprecated' ] }, + '*compress-threads': { 'type': 'uint8', + 'features': [ 'deprecated' ] }, + '*compress-wait-thread': { 'type': 'bool', + 'features': [ 'deprecated' ] }, + '*decompress-threads': { 'type': 'uint8', + 'features': [ 'deprecated' ] }, '*throttle-trigger-threshold': 'uint8', '*cpu-throttle-initial': 'uint8', '*cpu-throttle-increment': 'uint8', @@ -1057,7 +1076,7 @@ # Example: # # -> { "execute": "migrate-set-parameters" , -# "arguments": { "compress-level": 1 } } +# "arguments": { "multifd-channels": 5 } } # <- { "return": {} } ## { 'command': 'migrate-set-parameters', 'boxed': true, @@ -1080,16 +1099,16 @@ # @announce-step: Increase in delay (in milliseconds) between # subsequent packets in the announcement (Since 4.0) # -# @compress-level: compression level +# @compress-level: compression level. # -# @compress-threads: compression thread count +# @compress-threads: compression thread count. # # @compress-wait-thread: Controls behavior when all compression # threads are currently busy. If true (default), wait for a free # compression thread to become available; otherwise, send the page -# uncompressed. (Since 3.1) +# uncompressed. (Since 3.1) # -# @decompress-threads: decompression thread count +# @decompress-threads: decompression thread count. # # @throttle-trigger-threshold: The ratio of bytes_dirty_period and # bytes_xfer_period to trigger throttling. It is expressed as @@ -1210,7 +1229,9 @@ # Features: # # @deprecated: Member @block-incremental is deprecated. Use -# blockdev-mirror with NBD instead. +# blockdev-mirror with NBD instead. Members @compress-level, +# @compress-threads, @decompress-threads and @compress-wait-thread +# are deprecated because @compression is deprecated. # # @unstable: Members @x-checkpoint-delay and @x-vcpu-dirty-limit-period # are experimental. @@ -1222,10 +1243,14 @@ '*announce-max': 'size', '*announce-rounds': 'size', '*announce-step': 'size', - '*compress-level': 'uint8', - '*compress-threads': 'uint8', - '*compress-wait-thread': 'bool', - '*decompress-threads': 'uint8', + '*compress-level': { 'type': 'uint8', + 'features': [ 'deprecated' ] }, + '*compress-threads': { 'type': 'uint8', + 'features': [ 'deprecated' ] }, + '*compress-wait-thread': { 'type': 'bool', + 'features': [ 'deprecated' ] }, + '*decompress-threads': { 'type': 'uint8', + 'features': [ 'deprecated' ] }, '*throttle-trigger-threshold': 'uint8', '*cpu-throttle-initial': 'uint8', '*cpu-throttle-increment': 'uint8', @@ -1264,10 +1289,8 @@ # # -> { "execute": "query-migrate-parameters" } # <- { "return": { -# "decompress-threads": 2, +# "multifd-channels": 2, # "cpu-throttle-increment": 10, -# "compress-threads": 8, -# "compress-level": 1, # "cpu-throttle-initial": 20, # "max-bandwidth": 33554432, # "downtime-limit": 300 diff --git a/migration/options.c b/migration/options.c index bb6005052f..e46d847de2 100644 --- a/migration/options.c +++ b/migration/options.c @@ -474,6 +474,11 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) "Use blockdev-mirror with NBD instead."); } + if (new_caps[MIGRATION_CAPABILITY_COMPRESS]) { + warn_report("Old compression method is deprecated. " + "Use multifd compression methods instead."); + } + #ifndef CONFIG_REPLICATION if (new_caps[MIGRATION_CAPABILITY_X_COLO]) { error_setg(errp, "QEMU compiled without replication module" @@ -1290,18 +1295,26 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp) /* TODO use QAPI_CLONE() instead of duplicating it inline */ if (params->has_compress_level) { + warn_report("Old compression is deprecated. " + "Use multifd compression methods instead."); s->parameters.compress_level = params->compress_level; } if (params->has_compress_threads) { + warn_report("Old compression is deprecated. " + "Use multifd compression methods instead."); s->parameters.compress_threads = params->compress_threads; } if (params->has_compress_wait_thread) { + warn_report("Old compression is deprecated. " + "Use multifd compression methods instead."); s->parameters.compress_wait_thread = params->compress_wait_thread; } if (params->has_decompress_threads) { + warn_report("Old compression is deprecated. " + "Use multifd compression methods instead."); s->parameters.decompress_threads = params->decompress_threads; } -- 2.41.0
[DON'T MERGE] Block migration is obsolete, users should use blockdev-mirror instead. Make it one error to set them. Signed-off-by: Juan Quintela <quintela@redhat.com> --- migration/migration-hmp-cmds.c | 15 ++++++++++++++- migration/migration.c | 35 +++++++--------------------------- 2 files changed, 21 insertions(+), 29 deletions(-) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index e0e16afa34..582368926a 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -731,7 +731,20 @@ void hmp_migrate(Monitor *mon, const QDict *qdict) const char *uri = qdict_get_str(qdict, "uri"); Error *err = NULL; - qmp_migrate(uri, !!blk, blk, !!inc, inc, + if (inc) { + monitor_printf(mon, "-i migrate option is deprecated, set the" + "'block-incremental' migration parameter to 'true'" + " instead.\n"); + return; + } + + if (blk) { + monitor_printf(mon, "-b migrate option is deprecated, set the " + "'block' capability to 'true' instead.\n"); + return; + } + + qmp_migrate(uri, false, false, false, false, false, false, true, resume, &err); if (hmp_handle_error(mon, err)) { return; diff --git a/migration/migration.c b/migration/migration.c index 9e4ae6b772..50fc112e98 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1599,17 +1599,17 @@ bool migration_is_blocked(Error **errp) static bool migrate_prepare(MigrationState *s, bool blk, bool blk_inc, bool resume, Error **errp) { - Error *local_err = NULL; - if (blk_inc) { - warn_report("@inc/-i migrate option is deprecated, set the" - "'block-incremental' migration parameter to 'true'" - " instead."); + error_setg(errp, "@inc migrate option is deprecated, set the" + "'block-incremental' migration parameter to 'true'" + " instead."); + return false; } if (blk) { - warn_report("@blk/-i migrate option is deprecated, set the " - "'block' capability to 'true' instead."); + error_setg(errp, "@blk/-i migrate option is deprecated, set the " + "'block' capability to 'true' instead."); + return false; } if (resume) { @@ -1661,27 +1661,6 @@ static bool migrate_prepare(MigrationState *s, bool blk, bool blk_inc, return false; } - if (blk || blk_inc) { - if (migrate_colo()) { - error_setg(errp, "No disk migration is required in COLO mode"); - return false; - } - if (migrate_block() || migrate_block_incremental()) { - error_setg(errp, "Command options are incompatible with " - "current migration capabilities"); - return false; - } - if (!migrate_cap_set(MIGRATION_CAPABILITY_BLOCK, true, &local_err)) { - error_propagate(errp, local_err); - return false; - } - s->must_remove_block_options = true; - } - - if (blk_inc) { - migrate_set_block_incremental(true); - } - if (migrate_init(s, errp)) { return false; } -- 2.41.0
[DON'T MERGE] We were abusing capabilities and parameters to implement -i/-b. Previous patch convert that options into one error. Remove all the helpers needed to implement them. Signed-off-by: Juan Quintela <quintela@redhat.com> --- migration/migration.h | 4 ---- migration/options.h | 6 ------ migration/migration.c | 2 -- migration/options.c | 41 ----------------------------------------- 4 files changed, 53 deletions(-) diff --git a/migration/migration.h b/migration/migration.h index cd5534337c..ae53d9b26a 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -382,10 +382,6 @@ struct MigrationState { /* mutex to protect errp */ QemuMutex error_mutex; - /* Do we have to clean up -b/-i from old migrate parameters */ - /* This feature is deprecated and will be removed */ - bool must_remove_block_options; - /* * Global switch on whether we need to store the global state * during migration. diff --git a/migration/options.h b/migration/options.h index 045e2a41a2..4e8c0b9223 100644 --- a/migration/options.h +++ b/migration/options.h @@ -61,7 +61,6 @@ bool migrate_tls(void); /* capabilities helpers */ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp); -bool migrate_cap_set(int cap, bool value, Error **errp); /* parameters */ @@ -91,14 +90,9 @@ const char *migrate_tls_creds(void); const char *migrate_tls_hostname(void); uint64_t migrate_xbzrle_cache_size(void); -/* parameters setters */ - -void migrate_set_block_incremental(bool value); - /* parameters helpers */ bool migrate_params_check(MigrationParameters *params, Error **errp); void migrate_params_init(MigrationParameters *params); -void block_cleanup_parameters(void); #endif diff --git a/migration/migration.c b/migration/migration.c index 50fc112e98..ceeb2e6cb2 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1205,7 +1205,6 @@ static void migrate_fd_cleanup(MigrationState *s) error_report_err(error_copy(s->error)); } notifier_list_notify(&migration_state_notifiers, s); - block_cleanup_parameters(); yank_unregister_instance(MIGRATION_YANK_INSTANCE); } @@ -1714,7 +1713,6 @@ void qmp_migrate(const char *uri, bool has_blk, bool blk, "a valid migration protocol"); migrate_set_state(&s->state, MIGRATION_STATUS_SETUP, MIGRATION_STATUS_FAILED); - block_cleanup_parameters(); } if (local_err) { diff --git a/migration/options.c b/migration/options.c index e46d847de2..ecb34a3bbd 100644 --- a/migration/options.c +++ b/migration/options.c @@ -622,26 +622,6 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) return true; } -bool migrate_cap_set(int cap, bool value, Error **errp) -{ - MigrationState *s = migrate_get_current(); - bool new_caps[MIGRATION_CAPABILITY__MAX]; - - if (migration_is_running(s->state)) { - error_setg(errp, QERR_MIGRATION_ACTIVE); - return false; - } - - memcpy(new_caps, s->capabilities, sizeof(new_caps)); - new_caps[cap] = value; - - if (!migrate_caps_check(s->capabilities, new_caps, errp)) { - return false; - } - s->capabilities[cap] = value; - return true; -} - MigrationCapabilityStatusList *qmp_query_migrate_capabilities(Error **errp) { MigrationCapabilityStatusList *head = NULL, **tail = &head; @@ -861,29 +841,8 @@ uint64_t migrate_xbzrle_cache_size(void) return s->parameters.xbzrle_cache_size; } -/* parameter setters */ - -void migrate_set_block_incremental(bool value) -{ - MigrationState *s = migrate_get_current(); - - s->parameters.block_incremental = value; -} - /* parameters helpers */ -void block_cleanup_parameters(void) -{ - MigrationState *s = migrate_get_current(); - - if (s->must_remove_block_options) { - /* setting to false can never fail */ - migrate_cap_set(MIGRATION_CAPABILITY_BLOCK, false, &error_abort); - migrate_set_block_incremental(false); - s->must_remove_block_options = false; - } -} - AnnounceParameters *migrate_announce_params(void) { static AnnounceParameters ap; -- 2.41.0
[DON'T MERGE] Signed-off-by: Juan Quintela <quintela@redhat.com> --- docs/about/deprecated.rst | 7 ----- qapi/migration.json | 55 +++++++--------------------------- migration/options.h | 1 - migration/block.c | 2 +- migration/migration-hmp-cmds.c | 18 +---------- migration/migration.c | 13 ++------ migration/options.c | 18 ----------- hmp-commands.hx | 13 +++----- 8 files changed, 20 insertions(+), 107 deletions(-) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index a6da0df8e1..6bc41a430a 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -452,13 +452,6 @@ Migration ``skipped`` field in Migration stats has been deprecated. It hasn't been used for more than 10 years. -``inc`` migrate command option (since 8.2) -'''''''''''''''''''''''''''''''''''''''''' - -The new way to modify migration is using migration parameters. -``inc`` functionality can be achieved by setting the -``block-incremental`` migration parameter to ``true``. - ``blk`` migrate command option (since 8.2) '''''''''''''''''''''''''''''''''''''''''' diff --git a/qapi/migration.json b/qapi/migration.json index e599062f7d..545da4e257 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -779,13 +779,6 @@ # @x-checkpoint-delay: The delay time (in ms) between two COLO # checkpoints in periodic mode. (Since 2.8) # -# @block-incremental: Affects how much storage is migrated when the -# block migration capability is enabled. When false, the entire -# storage backing chain is migrated into a flattened image at the -# destination; when true, only the active qcow2 layer is migrated -# and the destination must already have access to the same backing -# chain as was used on the source. (since 2.10) -# # @multifd-channels: Number of channels used to migrate data in # parallel. This is the same number that the number of sockets # used for migration. The default value is 2 (since 4.0) @@ -841,10 +834,9 @@ # # Features: # -# @deprecated: Member @block-incremental is deprecated. Use -# blockdev-mirror with NBD instead. Members @compress-level, -# @compress-threads, @decompress-threads and @compress-wait-thread -# are deprecated because @compression is deprecated. +# @deprecated: Members @compress-level, @compress-threads, +# @decompress-threads and @compress-wait-thread are deprecated +# because @compression is deprecated. # # @unstable: Members @x-checkpoint-delay and @x-vcpu-dirty-limit-period # are experimental. @@ -864,7 +856,6 @@ 'tls-creds', 'tls-hostname', 'tls-authz', 'max-bandwidth', 'downtime-limit', { 'name': 'x-checkpoint-delay', 'features': [ 'unstable' ] }, - { 'name': 'block-incremental', 'features': [ 'deprecated' ] }, 'multifd-channels', 'xbzrle-cache-size', 'max-postcopy-bandwidth', 'max-cpu-throttle', 'multifd-compression', @@ -953,13 +944,6 @@ # @x-checkpoint-delay: the delay time between two COLO checkpoints. # (Since 2.8) # -# @block-incremental: Affects how much storage is migrated when the -# block migration capability is enabled. When false, the entire -# storage backing chain is migrated into a flattened image at the -# destination; when true, only the active qcow2 layer is migrated -# and the destination must already have access to the same backing -# chain as was used on the source. (since 2.10) -# # @multifd-channels: Number of channels used to migrate data in # parallel. This is the same number that the number of sockets # used for migration. The default value is 2 (since 4.0) @@ -1015,10 +999,9 @@ # # Features: # -# @deprecated: Member @block-incremental is deprecated. Use -# blockdev-mirror with NBD instead. Members @compress-level, -# @compress-threads, @decompress-threads and @compress-wait-thread -# are deprecated because @compression is deprecated. +# @deprecated: Members @compress-level, @compress-threads, +# @decompress-threads and @compress-wait-thread are deprecated +# because @compression is deprecated. # # @unstable: Members @x-checkpoint-delay and @x-vcpu-dirty-limit-period # are experimental. @@ -1052,8 +1035,6 @@ '*downtime-limit': 'uint64', '*x-checkpoint-delay': { 'type': 'uint32', 'features': [ 'unstable' ] }, - '*block-incremental': { 'type': 'bool', - 'features': [ 'deprecated' ] }, '*multifd-channels': 'uint8', '*xbzrle-cache-size': 'size', '*max-postcopy-bandwidth': 'size', @@ -1166,13 +1147,6 @@ # @x-checkpoint-delay: the delay time between two COLO checkpoints. # (Since 2.8) # -# @block-incremental: Affects how much storage is migrated when the -# block migration capability is enabled. When false, the entire -# storage backing chain is migrated into a flattened image at the -# destination; when true, only the active qcow2 layer is migrated -# and the destination must already have access to the same backing -# chain as was used on the source. (since 2.10) -# # @multifd-channels: Number of channels used to migrate data in # parallel. This is the same number that the number of sockets # used for migration. The default value is 2 (since 4.0) @@ -1228,10 +1202,9 @@ # # Features: # -# @deprecated: Member @block-incremental is deprecated. Use -# blockdev-mirror with NBD instead. Members @compress-level, -# @compress-threads, @decompress-threads and @compress-wait-thread -# are deprecated because @compression is deprecated. +# @deprecated: Members @compress-level, @compress-threads, +# @decompress-threads and @compress-wait-thread are deprecated +# because @compression is deprecated. # # @unstable: Members @x-checkpoint-delay and @x-vcpu-dirty-limit-period # are experimental. @@ -1262,8 +1235,6 @@ '*downtime-limit': 'uint64', '*x-checkpoint-delay': { 'type': 'uint32', 'features': [ 'unstable' ] }, - '*block-incremental': { 'type': 'bool', - 'features': [ 'deprecated' ] }, '*multifd-channels': 'uint8', '*xbzrle-cache-size': 'size', '*max-postcopy-bandwidth': 'size', @@ -1527,8 +1498,6 @@ # # @blk: do block migration (full disk copy) # -# @inc: incremental disk copy migration -# # @detach: this argument exists only for compatibility reasons and is # ignored by QEMU # @@ -1536,9 +1505,8 @@ # # Features: # -# @deprecated: Member @inc is deprecated. Use migration parameter -# @block-incremental instead. Member @blk is deprecated. Set -# migration capability 'block' to 'true' instead. +# @deprecated: Member @blk is deprecated. Set migration capability +# 'block' to 'true' instead. # # Returns: nothing on success # @@ -1563,7 +1531,6 @@ { 'command': 'migrate', 'data': {'uri': 'str', '*blk': { 'type': 'bool', 'features': [ 'deprecated' ] }, - '*inc': { 'type': 'bool', 'features': [ 'deprecated' ] }, '*detach': 'bool', '*resume': 'bool' } } ## diff --git a/migration/options.h b/migration/options.h index 4e8c0b9223..237e4c75d4 100644 --- a/migration/options.h +++ b/migration/options.h @@ -67,7 +67,6 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp); const BitmapMigrationNodeAliasList *migrate_block_bitmap_mapping(void); bool migrate_has_block_bitmap_mapping(void); -bool migrate_block_incremental(void); uint32_t migrate_checkpoint_delay(void); int migrate_compress_level(void); int migrate_compress_threads(void); diff --git a/migration/block.c b/migration/block.c index 7f563c70f6..fadf96f668 100644 --- a/migration/block.c +++ b/migration/block.c @@ -420,7 +420,7 @@ static int init_blk_migration(QEMUFile *f) bmds->bulk_completed = 0; bmds->total_sectors = sectors; bmds->completed_sectors = 0; - bmds->shared_base = migrate_block_incremental(); + bmds->shared_base = false; assert(i < num_bs); bmds_bs[i].bmds = bmds; diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 582368926a..e6f6785ad3 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -330,10 +330,6 @@ void hmp_info_migrate_parameters(Monitor *mon, const QDict *qdict) monitor_printf(mon, "%s: %u ms\n", MigrationParameter_str(MIGRATION_PARAMETER_X_CHECKPOINT_DELAY), params->x_checkpoint_delay); - assert(params->has_block_incremental); - monitor_printf(mon, "%s: %s\n", - MigrationParameter_str(MIGRATION_PARAMETER_BLOCK_INCREMENTAL), - params->block_incremental ? "on" : "off"); monitor_printf(mon, "%s: %u\n", MigrationParameter_str(MIGRATION_PARAMETER_MULTIFD_CHANNELS), params->multifd_channels); @@ -583,10 +579,6 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict) p->has_x_checkpoint_delay = true; visit_type_uint32(v, param, &p->x_checkpoint_delay, &err); break; - case MIGRATION_PARAMETER_BLOCK_INCREMENTAL: - p->has_block_incremental = true; - visit_type_bool(v, param, &p->block_incremental, &err); - break; case MIGRATION_PARAMETER_MULTIFD_CHANNELS: p->has_multifd_channels = true; visit_type_uint8(v, param, &p->multifd_channels, &err); @@ -726,25 +718,17 @@ void hmp_migrate(Monitor *mon, const QDict *qdict) { bool detach = qdict_get_try_bool(qdict, "detach", false); bool blk = qdict_get_try_bool(qdict, "blk", false); - bool inc = qdict_get_try_bool(qdict, "inc", false); bool resume = qdict_get_try_bool(qdict, "resume", false); const char *uri = qdict_get_str(qdict, "uri"); Error *err = NULL; - if (inc) { - monitor_printf(mon, "-i migrate option is deprecated, set the" - "'block-incremental' migration parameter to 'true'" - " instead.\n"); - return; - } - if (blk) { monitor_printf(mon, "-b migrate option is deprecated, set the " "'block' capability to 'true' instead.\n"); return; } - qmp_migrate(uri, false, false, false, false, + qmp_migrate(uri, false, false, false, false, true, resume, &err); if (hmp_handle_error(mon, err)) { return; diff --git a/migration/migration.c b/migration/migration.c index ceeb2e6cb2..d587c8f5b8 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1595,16 +1595,9 @@ bool migration_is_blocked(Error **errp) } /* Returns true if continue to migrate, or false if error detected */ -static bool migrate_prepare(MigrationState *s, bool blk, bool blk_inc, +static bool migrate_prepare(MigrationState *s, bool blk, bool resume, Error **errp) { - if (blk_inc) { - error_setg(errp, "@inc migrate option is deprecated, set the" - "'block-incremental' migration parameter to 'true'" - " instead."); - return false; - } - if (blk) { error_setg(errp, "@blk/-i migrate option is deprecated, set the " "'block' capability to 'true' instead."); @@ -1668,7 +1661,7 @@ static bool migrate_prepare(MigrationState *s, bool blk, bool blk_inc, } void qmp_migrate(const char *uri, bool has_blk, bool blk, - bool has_inc, bool inc, bool has_detach, bool detach, + bool has_detach, bool detach, bool has_resume, bool resume, Error **errp) { bool resume_requested; @@ -1682,7 +1675,7 @@ void qmp_migrate(const char *uri, bool has_blk, bool blk, } resume_requested = has_resume && resume; - if (!migrate_prepare(s, has_blk && blk, has_inc && inc, + if (!migrate_prepare(s, has_blk && blk, resume_requested, errp)) { /* Error detected, put into errp */ return; diff --git a/migration/options.c b/migration/options.c index ecb34a3bbd..a4defb4e63 100644 --- a/migration/options.c +++ b/migration/options.c @@ -686,13 +686,6 @@ bool migrate_has_block_bitmap_mapping(void) return s->parameters.has_block_bitmap_mapping; } -bool migrate_block_incremental(void) -{ - MigrationState *s = migrate_get_current(); - - return s->parameters.block_incremental; -} - uint32_t migrate_checkpoint_delay(void) { MigrationState *s = migrate_get_current(); @@ -890,8 +883,6 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp) params->downtime_limit = s->parameters.downtime_limit; params->has_x_checkpoint_delay = true; params->x_checkpoint_delay = s->parameters.x_checkpoint_delay; - params->has_block_incremental = true; - params->block_incremental = s->parameters.block_incremental; params->has_multifd_channels = true; params->multifd_channels = s->parameters.multifd_channels; params->has_multifd_compression = true; @@ -947,7 +938,6 @@ void migrate_params_init(MigrationParameters *params) params->has_max_bandwidth = true; params->has_downtime_limit = true; params->has_x_checkpoint_delay = true; - params->has_block_incremental = true; params->has_multifd_channels = true; params->has_multifd_compression = true; params->has_multifd_zlib_level = true; @@ -1202,9 +1192,6 @@ static void migrate_params_test_apply(MigrateSetParameters *params, dest->x_checkpoint_delay = params->x_checkpoint_delay; } - if (params->has_block_incremental) { - dest->block_incremental = params->block_incremental; - } if (params->has_multifd_channels) { dest->multifd_channels = params->multifd_channels; } @@ -1327,11 +1314,6 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp) colo_checkpoint_delay_set(); } - if (params->has_block_incremental) { - warn_report("Block migration is deprecated. " - "Use blockdev-mirror with NBD instead."); - s->parameters.block_incremental = params->block_incremental; - } if (params->has_multifd_channels) { s->parameters.multifd_channels = params->multifd_channels; } diff --git a/hmp-commands.hx b/hmp-commands.hx index 63eac22734..ce26057e85 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -920,26 +920,21 @@ ERST { .name = "migrate", - .args_type = "detach:-d,blk:-b,inc:-i,resume:-r,uri:s", - .params = "[-d] [-b] [-i] [-r] uri", + .args_type = "detach:-d,blk:-b,resume:-r,uri:s", + .params = "[-d] [-b] [-r] uri", .help = "migrate to URI (using -d to not wait for completion)" "\n\t\t\t -b for migration without shared storage with" - " full copy of disk\n\t\t\t -i for migration without " - "shared storage with incremental copy of disk " - "(base image shared between src and destination)" - "\n\t\t\t -r to resume a paused migration", + " full copy of disk\n\t\t\t -r to resume a paused migration", .cmd = hmp_migrate, }, SRST -``migrate [-d] [-b] [-i]`` *uri* +``migrate [-d] [-b]`` *uri* Migrate to *uri* (using -d to not wait for completion). ``-b`` for migration with full copy of disk - ``-i`` - for migration with incremental copy of disk (base image is shared) ERST { -- 2.41.0
[DON'T MERGE] Signed-off-by: Juan Quintela <quintela@redhat.com> --- meson.build | 2 - qapi/migration.json | 31 +- include/migration/misc.h | 8 - migration/block.h | 52 -- migration/options.h | 1 - migration/block.c | 1030 -------------------------------- migration/colo.c | 1 - migration/migration-hmp-cmds.c | 34 +- migration/migration.c | 28 +- migration/options.c | 26 - migration/ram.c | 16 - migration/savevm.c | 5 - hmp-commands.hx | 12 +- migration/meson.build | 3 - 14 files changed, 14 insertions(+), 1235 deletions(-) delete mode 100644 migration/block.h delete mode 100644 migration/block.c diff --git a/meson.build b/meson.build index bd65a111aa..2332fa3dcb 100644 --- a/meson.build +++ b/meson.build @@ -2199,7 +2199,6 @@ config_host_data.set('CONFIG_DEBUG_GRAPH_LOCK', get_option('debug_graph_lock')) config_host_data.set('CONFIG_DEBUG_MUTEX', get_option('debug_mutex')) config_host_data.set('CONFIG_DEBUG_STACK_USAGE', get_option('debug_stack_usage')) config_host_data.set('CONFIG_DEBUG_TCG', get_option('debug_tcg')) -config_host_data.set('CONFIG_LIVE_BLOCK_MIGRATION', get_option('live_block_migration').allowed()) config_host_data.set('CONFIG_QOM_CAST_DEBUG', get_option('qom_cast_debug')) config_host_data.set('CONFIG_REPLICATION', get_option('replication').allowed()) @@ -4181,7 +4180,6 @@ if have_block summary_info += {'Use block whitelist in tools': get_option('block_drv_whitelist_in_tools')} summary_info += {'VirtFS (9P) support': have_virtfs} summary_info += {'VirtFS (9P) Proxy Helper support (deprecated)': have_virtfs_proxy_helper} - summary_info += {'Live block migration': config_host_data.get('CONFIG_LIVE_BLOCK_MIGRATION')} summary_info += {'replication support': config_host_data.get('CONFIG_REPLICATION')} summary_info += {'bochs support': get_option('bochs').allowed()} summary_info += {'cloop support': get_option('cloop').allowed()} diff --git a/qapi/migration.json b/qapi/migration.json index 545da4e257..46c0e0a90e 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -202,9 +202,6 @@ # @ram: @MigrationStats containing detailed migration status, only # returned if status is 'active' or 'completed'(since 1.2) # -# @disk: @MigrationStats containing detailed disk migration status, -# only returned if status is 'active' and it is a block migration -# # @xbzrle-cache: @XBZRLECacheStats containing detailed XBZRLE # migration statistics, only returned if XBZRLE feature is on and # status is 'active' or 'completed' (since 1.2) @@ -271,17 +268,15 @@ # # Features: # -# @deprecated: Member @disk is deprecated because block migration is. -# Member @compression is deprecated because it is unreliable and -# untested. It is recommended to use multifd migration, which -# offers an alternative compression implementation that is -# reliable and tested. +# @deprecated: Member @compression is deprecated because it is +# unreliable and untested. It is recommended to use multifd +# migration, which offers an alternative compression +# implementation that is reliable and tested. # # Since: 0.14 ## { 'struct': 'MigrationInfo', 'data': {'*status': 'MigrationStatus', '*ram': 'MigrationStats', - '*disk': { 'type': 'MigrationStats', 'features': [ 'deprecated' ] }, '*vfio': 'VfioStats', '*xbzrle-cache': 'XBZRLECacheStats', '*total-time': 'int', @@ -469,11 +464,6 @@ # @release-ram: if enabled, qemu will free the migrated ram pages on # the source during postcopy-ram migration. (since 2.9) # -# @block: If enabled, QEMU will also migrate the contents of all block -# devices. Default is disabled. A possible alternative uses -# mirror jobs to a builtin NBD server on the destination, which -# offers more flexibility. (Since 2.10) -# # @return-path: If enabled, migration will use the return path even # for precopy. (since 2.10) # @@ -533,8 +523,7 @@ # # Features: # -# @deprecated: Member @block is deprecated. Use blockdev-mirror with -# NBD instead. Member @compression is deprecated because it is +# @deprecated: Member @compression is deprecated because it is # unreliable and untested. It is recommended to use multifd # migration, which offers an alternative compression # implementation that is reliable and tested. @@ -549,7 +538,6 @@ 'events', 'postcopy-ram', { 'name': 'x-colo', 'features': [ 'unstable' ] }, 'release-ram', - { 'name': 'block', 'features': [ 'deprecated' ] }, 'return-path', 'pause-before-switchover', 'multifd', 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate', { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] }, @@ -1496,8 +1484,6 @@ # # @uri: the Uniform Resource Identifier of the destination VM # -# @blk: do block migration (full disk copy) -# # @detach: this argument exists only for compatibility reasons and is # ignored by QEMU # @@ -1505,9 +1491,6 @@ # # Features: # -# @deprecated: Member @blk is deprecated. Set migration capability -# 'block' to 'true' instead. -# # Returns: nothing on success # # Since: 0.14 @@ -1529,9 +1512,7 @@ # <- { "return": {} } ## { 'command': 'migrate', - 'data': {'uri': 'str', - '*blk': { 'type': 'bool', 'features': [ 'deprecated' ] }, - '*detach': 'bool', '*resume': 'bool' } } + 'data': {'uri': 'str', '*detach': 'bool', '*resume': 'bool' } } ## # @migrate-incoming: diff --git a/include/migration/misc.h b/include/migration/misc.h index 7dcc0b5c2c..c48242d70b 100644 --- a/include/migration/misc.h +++ b/include/migration/misc.h @@ -42,14 +42,6 @@ void ram_mig_init(void); void qemu_guest_free_page_hint(void *addr, size_t len); bool migrate_ram_is_ignored(RAMBlock *block); -/* migration/block.c */ - -#ifdef CONFIG_LIVE_BLOCK_MIGRATION -void blk_mig_init(void); -#else -static inline void blk_mig_init(void) {} -#endif - AnnounceParameters *migrate_announce_params(void); /* migration/savevm.c */ diff --git a/migration/block.h b/migration/block.h deleted file mode 100644 index 3178609dbd..0000000000 --- a/migration/block.h +++ /dev/null @@ -1,52 +0,0 @@ -/* - * QEMU live block migration - * - * Copyright IBM, Corp. 2009 - * - * Authors: - * Liran Schour <lirans@il.ibm.com> - * - * This work is licensed under the terms of the GNU GPL, version 2. See - * the COPYING file in the top-level directory. - * - */ - -#ifndef MIGRATION_BLOCK_H -#define MIGRATION_BLOCK_H - -#ifdef CONFIG_LIVE_BLOCK_MIGRATION -int blk_mig_active(void); -int blk_mig_bulk_active(void); -uint64_t blk_mig_bytes_transferred(void); -uint64_t blk_mig_bytes_remaining(void); -uint64_t blk_mig_bytes_total(void); - -#else -static inline int blk_mig_active(void) -{ - return false; -} - -static inline int blk_mig_bulk_active(void) -{ - return false; -} - -static inline uint64_t blk_mig_bytes_transferred(void) -{ - return 0; -} - -static inline uint64_t blk_mig_bytes_remaining(void) -{ - return 0; -} - -static inline uint64_t blk_mig_bytes_total(void) -{ - return 0; -} -#endif /* CONFIG_LIVE_BLOCK_MIGRATION */ - -void migrate_set_block_enabled(bool value, Error **errp); -#endif /* MIGRATION_BLOCK_H */ diff --git a/migration/options.h b/migration/options.h index 237e4c75d4..645ca6b44f 100644 --- a/migration/options.h +++ b/migration/options.h @@ -25,7 +25,6 @@ extern Property migration_properties[]; bool migrate_auto_converge(void); bool migrate_background_snapshot(void); -bool migrate_block(void); bool migrate_colo(void); bool migrate_compress(void); bool migrate_dirty_bitmaps(void); diff --git a/migration/block.c b/migration/block.c deleted file mode 100644 index fadf96f668..0000000000 --- a/migration/block.c +++ /dev/null @@ -1,1030 +0,0 @@ -/* - * QEMU live block migration - * - * Copyright IBM, Corp. 2009 - * - * Authors: - * Liran Schour <lirans@il.ibm.com> - * - * This work is licensed under the terms of the GNU GPL, version 2. See - * the COPYING file in the top-level directory. - * - * Contributions after 2012-01-13 are licensed under the terms of the - * GNU GPL, version 2 or (at your option) any later version. - */ - -#include "qemu/osdep.h" -#include "qapi/error.h" -#include "qemu/error-report.h" -#include "qemu/main-loop.h" -#include "qemu/cutils.h" -#include "qemu/queue.h" -#include "block.h" -#include "block/dirty-bitmap.h" -#include "migration/misc.h" -#include "migration.h" -#include "migration-stats.h" -#include "migration/register.h" -#include "qemu-file.h" -#include "migration/vmstate.h" -#include "sysemu/block-backend.h" -#include "trace.h" -#include "options.h" - -#define BLK_MIG_BLOCK_SIZE (1ULL << 20) -#define BDRV_SECTORS_PER_DIRTY_CHUNK (BLK_MIG_BLOCK_SIZE >> BDRV_SECTOR_BITS) - -#define BLK_MIG_FLAG_DEVICE_BLOCK 0x01 -#define BLK_MIG_FLAG_EOS 0x02 -#define BLK_MIG_FLAG_PROGRESS 0x04 -#define BLK_MIG_FLAG_ZERO_BLOCK 0x08 - -#define MAX_IS_ALLOCATED_SEARCH (65536 * BDRV_SECTOR_SIZE) - -#define MAX_IO_BUFFERS 512 -#define MAX_PARALLEL_IO 16 - -typedef struct BlkMigDevState { - /* Written during setup phase. Can be read without a lock. */ - BlockBackend *blk; - char *blk_name; - int shared_base; - int64_t total_sectors; - QSIMPLEQ_ENTRY(BlkMigDevState) entry; - Error *blocker; - - /* Only used by migration thread. Does not need a lock. */ - int bulk_completed; - int64_t cur_sector; - int64_t cur_dirty; - - /* Data in the aio_bitmap is protected by block migration lock. - * Allocation and free happen during setup and cleanup respectively. - */ - unsigned long *aio_bitmap; - - /* Protected by block migration lock. */ - int64_t completed_sectors; - - /* During migration this is protected by iothread lock / AioContext. - * Allocation and free happen during setup and cleanup respectively. - */ - BdrvDirtyBitmap *dirty_bitmap; -} BlkMigDevState; - -typedef struct BlkMigBlock { - /* Only used by migration thread. */ - uint8_t *buf; - BlkMigDevState *bmds; - int64_t sector; - int nr_sectors; - QEMUIOVector qiov; - BlockAIOCB *aiocb; - - /* Protected by block migration lock. */ - int ret; - QSIMPLEQ_ENTRY(BlkMigBlock) entry; -} BlkMigBlock; - -typedef struct BlkMigState { - QSIMPLEQ_HEAD(, BlkMigDevState) bmds_list; - int64_t total_sector_sum; - bool zero_blocks; - - /* Protected by lock. */ - QSIMPLEQ_HEAD(, BlkMigBlock) blk_list; - int submitted; - int read_done; - - /* Only used by migration thread. Does not need a lock. */ - int transferred; - int prev_progress; - int bulk_completed; - - /* Lock must be taken _inside_ the iothread lock and any AioContexts. */ - QemuMutex lock; -} BlkMigState; - -static BlkMigState block_mig_state; - -static void blk_mig_lock(void) -{ - qemu_mutex_lock(&block_mig_state.lock); -} - -static void blk_mig_unlock(void) -{ - qemu_mutex_unlock(&block_mig_state.lock); -} - -/* Must run outside of the iothread lock during the bulk phase, - * or the VM will stall. - */ - -static void blk_send(QEMUFile *f, BlkMigBlock * blk) -{ - int len; - uint64_t flags = BLK_MIG_FLAG_DEVICE_BLOCK; - - if (block_mig_state.zero_blocks && - buffer_is_zero(blk->buf, BLK_MIG_BLOCK_SIZE)) { - flags |= BLK_MIG_FLAG_ZERO_BLOCK; - } - - /* sector number and flags */ - qemu_put_be64(f, (blk->sector << BDRV_SECTOR_BITS) - | flags); - - /* device name */ - len = strlen(blk->bmds->blk_name); - qemu_put_byte(f, len); - qemu_put_buffer(f, (uint8_t *) blk->bmds->blk_name, len); - - /* if a block is zero we need to flush here since the network - * bandwidth is now a lot higher than the storage device bandwidth. - * thus if we queue zero blocks we slow down the migration */ - if (flags & BLK_MIG_FLAG_ZERO_BLOCK) { - qemu_fflush(f); - return; - } - - qemu_put_buffer(f, blk->buf, BLK_MIG_BLOCK_SIZE); -} - -int blk_mig_active(void) -{ - return !QSIMPLEQ_EMPTY(&block_mig_state.bmds_list); -} - -int blk_mig_bulk_active(void) -{ - return blk_mig_active() && !block_mig_state.bulk_completed; -} - -uint64_t blk_mig_bytes_transferred(void) -{ - BlkMigDevState *bmds; - uint64_t sum = 0; - - blk_mig_lock(); - QSIMPLEQ_FOREACH(bmds, &block_mig_state.bmds_list, entry) { - sum += bmds->completed_sectors; - } - blk_mig_unlock(); - return sum << BDRV_SECTOR_BITS; -} - -uint64_t blk_mig_bytes_remaining(void) -{ - return blk_mig_bytes_total() - blk_mig_bytes_transferred(); -} - -uint64_t blk_mig_bytes_total(void) -{ - BlkMigDevState *bmds; - uint64_t sum = 0; - - QSIMPLEQ_FOREACH(bmds, &block_mig_state.bmds_list, entry) { - sum += bmds->total_sectors; - } - return sum << BDRV_SECTOR_BITS; -} - - -/* Called with migration lock held. */ - -static int bmds_aio_inflight(BlkMigDevState *bmds, int64_t sector) -{ - int64_t chunk = sector / (int64_t)BDRV_SECTORS_PER_DIRTY_CHUNK; - - if (sector < bmds->total_sectors) { - return !!(bmds->aio_bitmap[chunk / (sizeof(unsigned long) * 8)] & - (1UL << (chunk % (sizeof(unsigned long) * 8)))); - } else { - return 0; - } -} - -/* Called with migration lock held. */ - -static void bmds_set_aio_inflight(BlkMigDevState *bmds, int64_t sector_num, - int nb_sectors, int set) -{ - int64_t start, end; - unsigned long val, idx, bit; - - start = sector_num / BDRV_SECTORS_PER_DIRTY_CHUNK; - end = (sector_num + nb_sectors - 1) / BDRV_SECTORS_PER_DIRTY_CHUNK; - - for (; start <= end; start++) { - idx = start / (sizeof(unsigned long) * 8); - bit = start % (sizeof(unsigned long) * 8); - val = bmds->aio_bitmap[idx]; - if (set) { - val |= 1UL << bit; - } else { - val &= ~(1UL << bit); - } - bmds->aio_bitmap[idx] = val; - } -} - -static void alloc_aio_bitmap(BlkMigDevState *bmds) -{ - int64_t bitmap_size; - - bitmap_size = bmds->total_sectors + BDRV_SECTORS_PER_DIRTY_CHUNK * 8 - 1; - bitmap_size /= BDRV_SECTORS_PER_DIRTY_CHUNK * 8; - - bmds->aio_bitmap = g_malloc0(bitmap_size); -} - -/* Never hold migration lock when yielding to the main loop! */ - -static void blk_mig_read_cb(void *opaque, int ret) -{ - BlkMigBlock *blk = opaque; - - blk_mig_lock(); - blk->ret = ret; - - QSIMPLEQ_INSERT_TAIL(&block_mig_state.blk_list, blk, entry); - bmds_set_aio_inflight(blk->bmds, blk->sector, blk->nr_sectors, 0); - - block_mig_state.submitted--; - block_mig_state.read_done++; - assert(block_mig_state.submitted >= 0); - blk_mig_unlock(); -} - -/* Called with no lock taken. */ - -static int mig_save_device_bulk(QEMUFile *f, BlkMigDevState *bmds) -{ - int64_t total_sectors = bmds->total_sectors; - int64_t cur_sector = bmds->cur_sector; - BlockBackend *bb = bmds->blk; - BlkMigBlock *blk; - int nr_sectors; - int64_t count; - - if (bmds->shared_base) { - qemu_mutex_lock_iothread(); - aio_context_acquire(blk_get_aio_context(bb)); - /* Skip unallocated sectors; intentionally treats failure or - * partial sector as an allocated sector */ - while (cur_sector < total_sectors && - !bdrv_is_allocated(blk_bs(bb), cur_sector * BDRV_SECTOR_SIZE, - MAX_IS_ALLOCATED_SEARCH, &count)) { - if (count < BDRV_SECTOR_SIZE) { - break; - } - cur_sector += count >> BDRV_SECTOR_BITS; - } - aio_context_release(blk_get_aio_context(bb)); - qemu_mutex_unlock_iothread(); - } - - if (cur_sector >= total_sectors) { - bmds->cur_sector = bmds->completed_sectors = total_sectors; - return 1; - } - - bmds->completed_sectors = cur_sector; - - cur_sector &= ~((int64_t)BDRV_SECTORS_PER_DIRTY_CHUNK - 1); - - /* we are going to transfer a full block even if it is not allocated */ - nr_sectors = BDRV_SECTORS_PER_DIRTY_CHUNK; - - if (total_sectors - cur_sector < BDRV_SECTORS_PER_DIRTY_CHUNK) { - nr_sectors = total_sectors - cur_sector; - } - - blk = g_new(BlkMigBlock, 1); - blk->buf = g_malloc(BLK_MIG_BLOCK_SIZE); - blk->bmds = bmds; - blk->sector = cur_sector; - blk->nr_sectors = nr_sectors; - - qemu_iovec_init_buf(&blk->qiov, blk->buf, nr_sectors * BDRV_SECTOR_SIZE); - - blk_mig_lock(); - block_mig_state.submitted++; - blk_mig_unlock(); - - /* We do not know if bs is under the main thread (and thus does - * not acquire the AioContext when doing AIO) or rather under - * dataplane. Thus acquire both the iothread mutex and the - * AioContext. - * - * This is ugly and will disappear when we make bdrv_* thread-safe, - * without the need to acquire the AioContext. - */ - qemu_mutex_lock_iothread(); - aio_context_acquire(blk_get_aio_context(bmds->blk)); - bdrv_reset_dirty_bitmap(bmds->dirty_bitmap, cur_sector * BDRV_SECTOR_SIZE, - nr_sectors * BDRV_SECTOR_SIZE); - blk->aiocb = blk_aio_preadv(bb, cur_sector * BDRV_SECTOR_SIZE, &blk->qiov, - 0, blk_mig_read_cb, blk); - aio_context_release(blk_get_aio_context(bmds->blk)); - qemu_mutex_unlock_iothread(); - - bmds->cur_sector = cur_sector + nr_sectors; - return (bmds->cur_sector >= total_sectors); -} - -/* Called with iothread lock taken. */ - -static int set_dirty_tracking(void) -{ - BlkMigDevState *bmds; - int ret; - - QSIMPLEQ_FOREACH(bmds, &block_mig_state.bmds_list, entry) { - bmds->dirty_bitmap = bdrv_create_dirty_bitmap(blk_bs(bmds->blk), - BLK_MIG_BLOCK_SIZE, - NULL, NULL); - if (!bmds->dirty_bitmap) { - ret = -errno; - goto fail; - } - } - return 0; - -fail: - QSIMPLEQ_FOREACH(bmds, &block_mig_state.bmds_list, entry) { - if (bmds->dirty_bitmap) { - bdrv_release_dirty_bitmap(bmds->dirty_bitmap); - } - } - return ret; -} - -/* Called with iothread lock taken. */ - -static void unset_dirty_tracking(void) -{ - BlkMigDevState *bmds; - - QSIMPLEQ_FOREACH(bmds, &block_mig_state.bmds_list, entry) { - if (bmds->dirty_bitmap) { - bdrv_release_dirty_bitmap(bmds->dirty_bitmap); - } - } -} - -static int init_blk_migration(QEMUFile *f) -{ - BlockDriverState *bs; - BlkMigDevState *bmds; - int64_t sectors; - BdrvNextIterator it; - int i, num_bs = 0; - struct { - BlkMigDevState *bmds; - BlockDriverState *bs; - } *bmds_bs; - Error *local_err = NULL; - int ret; - - block_mig_state.submitted = 0; - block_mig_state.read_done = 0; - block_mig_state.transferred = 0; - block_mig_state.total_sector_sum = 0; - block_mig_state.prev_progress = -1; - block_mig_state.bulk_completed = 0; - block_mig_state.zero_blocks = migrate_zero_blocks(); - - for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) { - num_bs++; - } - bmds_bs = g_malloc0(num_bs * sizeof(*bmds_bs)); - - for (i = 0, bs = bdrv_first(&it); bs; bs = bdrv_next(&it), i++) { - if (bdrv_is_read_only(bs)) { - continue; - } - - sectors = bdrv_nb_sectors(bs); - if (sectors <= 0) { - ret = sectors; - bdrv_next_cleanup(&it); - goto out; - } - - bmds = g_new0(BlkMigDevState, 1); - bmds->blk = blk_new(qemu_get_aio_context(), - BLK_PERM_CONSISTENT_READ, BLK_PERM_ALL); - bmds->blk_name = g_strdup(bdrv_get_device_name(bs)); - bmds->bulk_completed = 0; - bmds->total_sectors = sectors; - bmds->completed_sectors = 0; - bmds->shared_base = false; - - assert(i < num_bs); - bmds_bs[i].bmds = bmds; - bmds_bs[i].bs = bs; - - block_mig_state.total_sector_sum += sectors; - - if (bmds->shared_base) { - trace_migration_block_init_shared(bdrv_get_device_name(bs)); - } else { - trace_migration_block_init_full(bdrv_get_device_name(bs)); - } - - QSIMPLEQ_INSERT_TAIL(&block_mig_state.bmds_list, bmds, entry); - } - - /* Can only insert new BDSes now because doing so while iterating block - * devices may end up in a deadlock (iterating the new BDSes, too). */ - for (i = 0; i < num_bs; i++) { - bmds = bmds_bs[i].bmds; - bs = bmds_bs[i].bs; - - if (bmds) { - ret = blk_insert_bs(bmds->blk, bs, &local_err); - if (ret < 0) { - error_report_err(local_err); - goto out; - } - - alloc_aio_bitmap(bmds); - error_setg(&bmds->blocker, "block device is in use by migration"); - bdrv_op_block_all(bs, bmds->blocker); - } - } - - ret = 0; -out: - g_free(bmds_bs); - return ret; -} - -/* Called with no lock taken. */ - -static int blk_mig_save_bulked_block(QEMUFile *f) -{ - int64_t completed_sector_sum = 0; - BlkMigDevState *bmds; - int progress; - int ret = 0; - - QSIMPLEQ_FOREACH(bmds, &block_mig_state.bmds_list, entry) { - if (bmds->bulk_completed == 0) { - if (mig_save_device_bulk(f, bmds) == 1) { - /* completed bulk section for this device */ - bmds->bulk_completed = 1; - } - completed_sector_sum += bmds->completed_sectors; - ret = 1; - break; - } else { - completed_sector_sum += bmds->completed_sectors; - } - } - - if (block_mig_state.total_sector_sum != 0) { - progress = completed_sector_sum * 100 / - block_mig_state.total_sector_sum; - } else { - progress = 100; - } - if (progress != block_mig_state.prev_progress) { - block_mig_state.prev_progress = progress; - qemu_put_be64(f, (progress << BDRV_SECTOR_BITS) - | BLK_MIG_FLAG_PROGRESS); - trace_migration_block_progression(progress); - } - - return ret; -} - -static void blk_mig_reset_dirty_cursor(void) -{ - BlkMigDevState *bmds; - - QSIMPLEQ_FOREACH(bmds, &block_mig_state.bmds_list, entry) { - bmds->cur_dirty = 0; - } -} - -/* Called with iothread lock and AioContext taken. */ - -static int mig_save_device_dirty(QEMUFile *f, BlkMigDevState *bmds, - int is_async) -{ - BlkMigBlock *blk; - int64_t total_sectors = bmds->total_sectors; - int64_t sector; - int nr_sectors; - int ret = -EIO; - - for (sector = bmds->cur_dirty; sector < bmds->total_sectors;) { - blk_mig_lock(); - if (bmds_aio_inflight(bmds, sector)) { - blk_mig_unlock(); - blk_drain(bmds->blk); - } else { - blk_mig_unlock(); - } - bdrv_dirty_bitmap_lock(bmds->dirty_bitmap); - if (bdrv_dirty_bitmap_get_locked(bmds->dirty_bitmap, - sector * BDRV_SECTOR_SIZE)) { - if (total_sectors - sector < BDRV_SECTORS_PER_DIRTY_CHUNK) { - nr_sectors = total_sectors - sector; - } else { - nr_sectors = BDRV_SECTORS_PER_DIRTY_CHUNK; - } - bdrv_reset_dirty_bitmap_locked(bmds->dirty_bitmap, - sector * BDRV_SECTOR_SIZE, - nr_sectors * BDRV_SECTOR_SIZE); - bdrv_dirty_bitmap_unlock(bmds->dirty_bitmap); - - blk = g_new(BlkMigBlock, 1); - blk->buf = g_malloc(BLK_MIG_BLOCK_SIZE); - blk->bmds = bmds; - blk->sector = sector; - blk->nr_sectors = nr_sectors; - - if (is_async) { - qemu_iovec_init_buf(&blk->qiov, blk->buf, - nr_sectors * BDRV_SECTOR_SIZE); - - blk->aiocb = blk_aio_preadv(bmds->blk, - sector * BDRV_SECTOR_SIZE, - &blk->qiov, 0, blk_mig_read_cb, - blk); - - blk_mig_lock(); - block_mig_state.submitted++; - bmds_set_aio_inflight(bmds, sector, nr_sectors, 1); - blk_mig_unlock(); - } else { - ret = blk_pread(bmds->blk, sector * BDRV_SECTOR_SIZE, - nr_sectors * BDRV_SECTOR_SIZE, blk->buf, 0); - if (ret < 0) { - goto error; - } - blk_send(f, blk); - - g_free(blk->buf); - g_free(blk); - } - - sector += nr_sectors; - bmds->cur_dirty = sector; - break; - } - - bdrv_dirty_bitmap_unlock(bmds->dirty_bitmap); - sector += BDRV_SECTORS_PER_DIRTY_CHUNK; - bmds->cur_dirty = sector; - } - - return (bmds->cur_dirty >= bmds->total_sectors); - -error: - trace_migration_block_save_device_dirty(sector); - g_free(blk->buf); - g_free(blk); - return ret; -} - -/* Called with iothread lock taken. - * - * return value: - * 0: too much data for max_downtime - * 1: few enough data for max_downtime -*/ -static int blk_mig_save_dirty_block(QEMUFile *f, int is_async) -{ - BlkMigDevState *bmds; - int ret = 1; - - QSIMPLEQ_FOREACH(bmds, &block_mig_state.bmds_list, entry) { - aio_context_acquire(blk_get_aio_context(bmds->blk)); - ret = mig_save_device_dirty(f, bmds, is_async); - aio_context_release(blk_get_aio_context(bmds->blk)); - if (ret <= 0) { - break; - } - } - - return ret; -} - -/* Called with no locks taken. */ - -static int flush_blks(QEMUFile *f) -{ - BlkMigBlock *blk; - int ret = 0; - - trace_migration_block_flush_blks("Enter", block_mig_state.submitted, - block_mig_state.read_done, - block_mig_state.transferred); - - blk_mig_lock(); - while ((blk = QSIMPLEQ_FIRST(&block_mig_state.blk_list)) != NULL) { - if (migration_rate_exceeded(f)) { - break; - } - if (blk->ret < 0) { - ret = blk->ret; - break; - } - - QSIMPLEQ_REMOVE_HEAD(&block_mig_state.blk_list, entry); - blk_mig_unlock(); - blk_send(f, blk); - blk_mig_lock(); - - g_free(blk->buf); - g_free(blk); - - block_mig_state.read_done--; - block_mig_state.transferred++; - assert(block_mig_state.read_done >= 0); - } - blk_mig_unlock(); - - trace_migration_block_flush_blks("Exit", block_mig_state.submitted, - block_mig_state.read_done, - block_mig_state.transferred); - return ret; -} - -/* Called with iothread lock taken. */ - -static int64_t get_remaining_dirty(void) -{ - BlkMigDevState *bmds; - int64_t dirty = 0; - - QSIMPLEQ_FOREACH(bmds, &block_mig_state.bmds_list, entry) { - aio_context_acquire(blk_get_aio_context(bmds->blk)); - dirty += bdrv_get_dirty_count(bmds->dirty_bitmap); - aio_context_release(blk_get_aio_context(bmds->blk)); - } - - return dirty; -} - - - -/* Called with iothread lock taken. */ -static void block_migration_cleanup_bmds(void) -{ - BlkMigDevState *bmds; - BlockDriverState *bs; - AioContext *ctx; - - unset_dirty_tracking(); - - while ((bmds = QSIMPLEQ_FIRST(&block_mig_state.bmds_list)) != NULL) { - QSIMPLEQ_REMOVE_HEAD(&block_mig_state.bmds_list, entry); - - bs = blk_bs(bmds->blk); - if (bs) { - bdrv_op_unblock_all(bs, bmds->blocker); - } - error_free(bmds->blocker); - - /* Save ctx, because bmds->blk can disappear during blk_unref. */ - ctx = blk_get_aio_context(bmds->blk); - aio_context_acquire(ctx); - blk_unref(bmds->blk); - aio_context_release(ctx); - - g_free(bmds->blk_name); - g_free(bmds->aio_bitmap); - g_free(bmds); - } -} - -/* Called with iothread lock taken. */ -static void block_migration_cleanup(void *opaque) -{ - BlkMigBlock *blk; - - bdrv_drain_all(); - - block_migration_cleanup_bmds(); - - blk_mig_lock(); - while ((blk = QSIMPLEQ_FIRST(&block_mig_state.blk_list)) != NULL) { - QSIMPLEQ_REMOVE_HEAD(&block_mig_state.blk_list, entry); - g_free(blk->buf); - g_free(blk); - } - blk_mig_unlock(); -} - -static int block_save_setup(QEMUFile *f, void *opaque) -{ - int ret; - - trace_migration_block_save("setup", block_mig_state.submitted, - block_mig_state.transferred); - - warn_report("block migration is deprecated. Use blockdev-mirror with" - "NBD instead."); - - qemu_mutex_lock_iothread(); - ret = init_blk_migration(f); - if (ret < 0) { - qemu_mutex_unlock_iothread(); - return ret; - } - - /* start track dirty blocks */ - ret = set_dirty_tracking(); - - qemu_mutex_unlock_iothread(); - - if (ret) { - return ret; - } - - ret = flush_blks(f); - blk_mig_reset_dirty_cursor(); - qemu_put_be64(f, BLK_MIG_FLAG_EOS); - - return ret; -} - -static int block_save_iterate(QEMUFile *f, void *opaque) -{ - int ret; - uint64_t last_bytes = qemu_file_transferred_noflush(f); - - trace_migration_block_save("iterate", block_mig_state.submitted, - block_mig_state.transferred); - - ret = flush_blks(f); - if (ret) { - return ret; - } - - blk_mig_reset_dirty_cursor(); - - /* control the rate of transfer */ - blk_mig_lock(); - while (block_mig_state.read_done * BLK_MIG_BLOCK_SIZE < - migration_rate_get() && - block_mig_state.submitted < MAX_PARALLEL_IO && - (block_mig_state.submitted + block_mig_state.read_done) < - MAX_IO_BUFFERS) { - blk_mig_unlock(); - if (block_mig_state.bulk_completed == 0) { - /* first finish the bulk phase */ - if (blk_mig_save_bulked_block(f) == 0) { - /* finished saving bulk on all devices */ - block_mig_state.bulk_completed = 1; - } - ret = 0; - } else { - /* Always called with iothread lock taken for - * simplicity, block_save_complete also calls it. - */ - qemu_mutex_lock_iothread(); - ret = blk_mig_save_dirty_block(f, 1); - qemu_mutex_unlock_iothread(); - } - if (ret < 0) { - return ret; - } - blk_mig_lock(); - if (ret != 0) { - /* no more dirty blocks */ - break; - } - } - blk_mig_unlock(); - - ret = flush_blks(f); - if (ret) { - return ret; - } - - qemu_put_be64(f, BLK_MIG_FLAG_EOS); - uint64_t delta_bytes = qemu_file_transferred_noflush(f) - last_bytes; - return (delta_bytes > 0); -} - -/* Called with iothread lock taken. */ - -static int block_save_complete(QEMUFile *f, void *opaque) -{ - int ret; - - trace_migration_block_save("complete", block_mig_state.submitted, - block_mig_state.transferred); - - ret = flush_blks(f); - if (ret) { - return ret; - } - - blk_mig_reset_dirty_cursor(); - - /* we know for sure that save bulk is completed and - all async read completed */ - blk_mig_lock(); - assert(block_mig_state.submitted == 0); - blk_mig_unlock(); - - do { - ret = blk_mig_save_dirty_block(f, 0); - if (ret < 0) { - return ret; - } - } while (ret == 0); - - /* report completion */ - qemu_put_be64(f, (100 << BDRV_SECTOR_BITS) | BLK_MIG_FLAG_PROGRESS); - - trace_migration_block_save_complete(); - - qemu_put_be64(f, BLK_MIG_FLAG_EOS); - - /* Make sure that our BlockBackends are gone, so that the block driver - * nodes can be inactivated. */ - block_migration_cleanup_bmds(); - - return 0; -} - -static void block_state_pending(void *opaque, uint64_t *must_precopy, - uint64_t *can_postcopy) -{ - /* Estimate pending number of bytes to send */ - uint64_t pending; - - qemu_mutex_lock_iothread(); - pending = get_remaining_dirty(); - qemu_mutex_unlock_iothread(); - - blk_mig_lock(); - pending += block_mig_state.submitted * BLK_MIG_BLOCK_SIZE + - block_mig_state.read_done * BLK_MIG_BLOCK_SIZE; - blk_mig_unlock(); - - /* Report at least one block pending during bulk phase */ - if (!pending && !block_mig_state.bulk_completed) { - pending = BLK_MIG_BLOCK_SIZE; - } - - trace_migration_block_state_pending(pending); - /* We don't do postcopy */ - *must_precopy += pending; -} - -static int block_load(QEMUFile *f, void *opaque, int version_id) -{ - static int banner_printed; - int len, flags; - char device_name[256]; - int64_t addr; - BlockBackend *blk, *blk_prev = NULL; - Error *local_err = NULL; - uint8_t *buf; - int64_t total_sectors = 0; - int nr_sectors; - int ret; - BlockDriverInfo bdi; - int cluster_size = BLK_MIG_BLOCK_SIZE; - - do { - addr = qemu_get_be64(f); - - flags = addr & (BDRV_SECTOR_SIZE - 1); - addr >>= BDRV_SECTOR_BITS; - - if (flags & BLK_MIG_FLAG_DEVICE_BLOCK) { - /* get device name */ - len = qemu_get_byte(f); - qemu_get_buffer(f, (uint8_t *)device_name, len); - device_name[len] = '\0'; - - blk = blk_by_name(device_name); - if (!blk) { - fprintf(stderr, "Error unknown block device %s\n", - device_name); - return -EINVAL; - } - - if (blk != blk_prev) { - blk_prev = blk; - total_sectors = blk_nb_sectors(blk); - if (total_sectors <= 0) { - error_report("Error getting length of block device %s", - device_name); - return -EINVAL; - } - - blk_activate(blk, &local_err); - if (local_err) { - error_report_err(local_err); - return -EINVAL; - } - - ret = bdrv_get_info(blk_bs(blk), &bdi); - if (ret == 0 && bdi.cluster_size > 0 && - bdi.cluster_size <= BLK_MIG_BLOCK_SIZE && - BLK_MIG_BLOCK_SIZE % bdi.cluster_size == 0) { - cluster_size = bdi.cluster_size; - } else { - cluster_size = BLK_MIG_BLOCK_SIZE; - } - } - - if (total_sectors - addr < BDRV_SECTORS_PER_DIRTY_CHUNK) { - nr_sectors = total_sectors - addr; - } else { - nr_sectors = BDRV_SECTORS_PER_DIRTY_CHUNK; - } - - if (flags & BLK_MIG_FLAG_ZERO_BLOCK) { - ret = blk_pwrite_zeroes(blk, addr * BDRV_SECTOR_SIZE, - nr_sectors * BDRV_SECTOR_SIZE, - BDRV_REQ_MAY_UNMAP); - } else { - int i; - int64_t cur_addr; - uint8_t *cur_buf; - - buf = g_malloc(BLK_MIG_BLOCK_SIZE); - qemu_get_buffer(f, buf, BLK_MIG_BLOCK_SIZE); - for (i = 0; i < BLK_MIG_BLOCK_SIZE / cluster_size; i++) { - cur_addr = addr * BDRV_SECTOR_SIZE + i * cluster_size; - cur_buf = buf + i * cluster_size; - - if ((!block_mig_state.zero_blocks || - cluster_size < BLK_MIG_BLOCK_SIZE) && - buffer_is_zero(cur_buf, cluster_size)) { - ret = blk_pwrite_zeroes(blk, cur_addr, - cluster_size, - BDRV_REQ_MAY_UNMAP); - } else { - ret = blk_pwrite(blk, cur_addr, cluster_size, cur_buf, - 0); - } - if (ret < 0) { - break; - } - } - g_free(buf); - } - - if (ret < 0) { - return ret; - } - } else if (flags & BLK_MIG_FLAG_PROGRESS) { - if (!banner_printed) { - printf("Receiving block device images\n"); - banner_printed = 1; - } - printf("Completed %d %%%c", (int)addr, - (addr == 100) ? '\n' : '\r'); - fflush(stdout); - } else if (!(flags & BLK_MIG_FLAG_EOS)) { - fprintf(stderr, "Unknown block migration flags: 0x%x\n", flags); - return -EINVAL; - } - ret = qemu_file_get_error(f); - if (ret != 0) { - return ret; - } - } while (!(flags & BLK_MIG_FLAG_EOS)); - - return 0; -} - -static bool block_is_active(void *opaque) -{ - return migrate_block(); -} - -static SaveVMHandlers savevm_block_handlers = { - .save_setup = block_save_setup, - .save_live_iterate = block_save_iterate, - .save_live_complete_precopy = block_save_complete, - .state_pending_exact = block_state_pending, - .state_pending_estimate = block_state_pending, - .load_state = block_load, - .save_cleanup = block_migration_cleanup, - .is_active = block_is_active, -}; - -void blk_mig_init(void) -{ - QSIMPLEQ_INIT(&block_mig_state.bmds_list); - QSIMPLEQ_INIT(&block_mig_state.blk_list); - qemu_mutex_init(&block_mig_state.lock); - - register_savevm_live("block", 0, 1, &savevm_block_handlers, - &block_mig_state); -} diff --git a/migration/colo.c b/migration/colo.c index 72f4f7b37e..9f9cd9d587 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -18,7 +18,6 @@ #include "qemu-file.h" #include "savevm.h" #include "migration/colo.h" -#include "block.h" #include "io/channel-buffer.h" #include "trace.h" #include "qemu/error-report.h" diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index e6f6785ad3..df5dd79169 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -147,15 +147,6 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict) } } - if (info->disk) { - monitor_printf(mon, "transferred disk: %" PRIu64 " kbytes\n", - info->disk->transferred >> 10); - monitor_printf(mon, "remaining disk: %" PRIu64 " kbytes\n", - info->disk->remaining >> 10); - monitor_printf(mon, "total disk: %" PRIu64 " kbytes\n", - info->disk->total >> 10); - } - if (info->xbzrle_cache) { monitor_printf(mon, "cache size: %" PRIu64 " bytes\n", info->xbzrle_cache->cache_size); @@ -685,24 +676,9 @@ static void hmp_migrate_status_cb(void *opaque) info = qmp_query_migrate(NULL); if (!info->has_status || info->status == MIGRATION_STATUS_ACTIVE || info->status == MIGRATION_STATUS_SETUP) { - if (info->disk) { - int progress; - - if (info->disk->remaining) { - progress = info->disk->transferred * 100 / info->disk->total; - } else { - progress = 100; - } - - monitor_printf(status->mon, "Completed %d %%\r", progress); - monitor_flush(status->mon); - } timer_mod(status->timer, qemu_clock_get_ms(QEMU_CLOCK_REALTIME) + 1000); } else { - if (migrate_block()) { - monitor_printf(status->mon, "\n"); - } if (info->error_desc) { error_report("%s", info->error_desc); } @@ -717,19 +693,11 @@ static void hmp_migrate_status_cb(void *opaque) void hmp_migrate(Monitor *mon, const QDict *qdict) { bool detach = qdict_get_try_bool(qdict, "detach", false); - bool blk = qdict_get_try_bool(qdict, "blk", false); bool resume = qdict_get_try_bool(qdict, "resume", false); const char *uri = qdict_get_str(qdict, "uri"); Error *err = NULL; - if (blk) { - monitor_printf(mon, "-b migrate option is deprecated, set the " - "'block' capability to 'true' instead.\n"); - return; - } - - qmp_migrate(uri, false, false, - false, false, true, resume, &err); + qmp_migrate(uri, false, false, true, resume, &err); if (hmp_handle_error(mon, err)) { return; } diff --git a/migration/migration.c b/migration/migration.c index d587c8f5b8..a8e9049391 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -46,7 +46,6 @@ #include "qapi/qmp/qerror.h" #include "qapi/qmp/qnull.h" #include "qemu/rcu.h" -#include "block.h" #include "postcopy-ram.h" #include "qemu/thread.h" #include "trace.h" @@ -160,7 +159,6 @@ void migration_object_init(void) migration_object_check(current_migration, &error_fatal); - blk_mig_init(); ram_mig_init(); dirty_bitmap_mig_init(); } @@ -994,16 +992,6 @@ static void populate_ram_info(MigrationInfo *info, MigrationState *s) } } -static void populate_disk_info(MigrationInfo *info) -{ - if (blk_mig_active()) { - info->disk = g_malloc0(sizeof(*info->disk)); - info->disk->transferred = blk_mig_bytes_transferred(); - info->disk->remaining = blk_mig_bytes_remaining(); - info->disk->total = blk_mig_bytes_total(); - } -} - static void fill_source_migration_info(MigrationInfo *info) { MigrationState *s = migrate_get_current(); @@ -1046,7 +1034,6 @@ static void fill_source_migration_info(MigrationInfo *info) /* TODO add some postcopy stats */ populate_time_info(info, s); populate_ram_info(info, s); - populate_disk_info(info); migration_populate_vfio_info(info); break; case MIGRATION_STATUS_COLO: @@ -1595,15 +1582,8 @@ bool migration_is_blocked(Error **errp) } /* Returns true if continue to migrate, or false if error detected */ -static bool migrate_prepare(MigrationState *s, bool blk, - bool resume, Error **errp) +static bool migrate_prepare(MigrationState *s, bool resume, Error **errp) { - if (blk) { - error_setg(errp, "@blk/-i migrate option is deprecated, set the " - "'block' capability to 'true' instead."); - return false; - } - if (resume) { if (s->state != MIGRATION_STATUS_POSTCOPY_PAUSED) { error_setg(errp, "Cannot resume if there is no " @@ -1660,8 +1640,7 @@ static bool migrate_prepare(MigrationState *s, bool blk, return true; } -void qmp_migrate(const char *uri, bool has_blk, bool blk, - bool has_detach, bool detach, +void qmp_migrate(const char *uri, bool has_detach, bool detach, bool has_resume, bool resume, Error **errp) { bool resume_requested; @@ -1675,8 +1654,7 @@ void qmp_migrate(const char *uri, bool has_blk, bool blk, } resume_requested = has_resume && resume; - if (!migrate_prepare(s, has_blk && blk, - resume_requested, errp)) { + if (!migrate_prepare(s, resume_requested, errp)) { /* Error detected, put into errp */ return; } diff --git a/migration/options.c b/migration/options.c index a4defb4e63..31f180679c 100644 --- a/migration/options.c +++ b/migration/options.c @@ -187,7 +187,6 @@ Property migration_properties[] = { MIGRATION_CAPABILITY_POSTCOPY_PREEMPT), DEFINE_PROP_MIG_CAP("x-colo", MIGRATION_CAPABILITY_X_COLO), DEFINE_PROP_MIG_CAP("x-release-ram", MIGRATION_CAPABILITY_RELEASE_RAM), - DEFINE_PROP_MIG_CAP("x-block", MIGRATION_CAPABILITY_BLOCK), DEFINE_PROP_MIG_CAP("x-return-path", MIGRATION_CAPABILITY_RETURN_PATH), DEFINE_PROP_MIG_CAP("x-multifd", MIGRATION_CAPABILITY_MULTIFD), DEFINE_PROP_MIG_CAP("x-background-snapshot", @@ -216,13 +215,6 @@ bool migrate_background_snapshot(void) return s->capabilities[MIGRATION_CAPABILITY_BACKGROUND_SNAPSHOT]; } -bool migrate_block(void) -{ - MigrationState *s = migrate_get_current(); - - return s->capabilities[MIGRATION_CAPABILITY_BLOCK]; -} - bool migrate_colo(void) { MigrationState *s = migrate_get_current(); @@ -461,19 +453,6 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) MigrationIncomingState *mis = migration_incoming_get_current(); ERRP_GUARD(); -#ifndef CONFIG_LIVE_BLOCK_MIGRATION - if (new_caps[MIGRATION_CAPABILITY_BLOCK]) { - error_setg(errp, "QEMU compiled without old-style (blk/-b, inc/-i) " - "block migration"); - error_append_hint(errp, "Use blockdev-mirror with NBD instead.\n"); - return false; - } -#endif - if (new_caps[MIGRATION_CAPABILITY_BLOCK]) { - warn_report("Block migration is deprecated. " - "Use blockdev-mirror with NBD instead."); - } - if (new_caps[MIGRATION_CAPABILITY_COMPRESS]) { warn_report("Old compression method is deprecated. " "Use multifd compression methods instead."); @@ -630,11 +609,6 @@ MigrationCapabilityStatusList *qmp_query_migrate_capabilities(Error **errp) int i; for (i = 0; i < MIGRATION_CAPABILITY__MAX; i++) { -#ifndef CONFIG_LIVE_BLOCK_MIGRATION - if (i == MIGRATION_CAPABILITY_BLOCK) { - continue; - } -#endif caps = g_malloc0(sizeof(*caps)); caps->capability = i; caps->state = s->capabilities[i]; diff --git a/migration/ram.c b/migration/ram.c index 2f5ce4d60b..d797243a68 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -53,7 +53,6 @@ #include "exec/target_page.h" #include "qemu/rcu_queue.h" #include "migration/colo.h" -#include "block.h" #include "sysemu/cpu-throttle.h" #include "savevm.h" #include "qemu/iov.h" @@ -1034,13 +1033,6 @@ static void migration_trigger_throttle(RAMState *rs) uint64_t bytes_dirty_period = rs->num_dirty_pages_period * TARGET_PAGE_SIZE; uint64_t bytes_dirty_threshold = bytes_xfer_period * threshold / 100; - /* During block migration the auto-converge logic incorrectly detects - * that ram migration makes no progress. Avoid this by disabling the - * throttling logic during the bulk phase of block migration. */ - if (blk_mig_bulk_active()) { - return; - } - /* * The following detection logic can be refined later. For now: * Check to see if the ratio between dirtied bytes and the approx. @@ -3099,13 +3091,6 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) int64_t t0; int done = 0; - if (blk_mig_bulk_active()) { - /* Avoid transferring ram during bulk phase of block migration as - * the bulk phase will usually take a long time and transferring - * ram updates during that time is pointless. */ - goto out; - } - /* * We'll take this lock a little bit long, but it's okay for two reasons. * Firstly, the only possible other thread to take it is who calls @@ -3181,7 +3166,6 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) */ ram_control_after_iterate(f, RAM_CONTROL_ROUND); -out: if (ret >= 0 && migration_is_setup_or_active(migrate_get_current()->state)) { if (migrate_multifd_flush_after_each_section()) { diff --git a/migration/savevm.c b/migration/savevm.c index 497ce02bd7..11d69afee2 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1649,11 +1649,6 @@ static int qemu_savevm_state(QEMUFile *f, Error **errp) return -EINVAL; } - if (migrate_block()) { - error_setg(errp, "Block migration and snapshots are incompatible"); - return -EINVAL; - } - ret = migrate_init(ms, errp); if (ret) { return ret; diff --git a/hmp-commands.hx b/hmp-commands.hx index ce26057e85..c56086266e 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -920,21 +920,17 @@ ERST { .name = "migrate", - .args_type = "detach:-d,blk:-b,resume:-r,uri:s", - .params = "[-d] [-b] [-r] uri", + .args_type = "detach:-d,resume:-r,uri:s", + .params = "[-d] [-r] uri", .help = "migrate to URI (using -d to not wait for completion)" - "\n\t\t\t -b for migration without shared storage with" - " full copy of disk\n\t\t\t -r to resume a paused migration", + "\n\t\t\t -r to resume a paused migration", .cmd = hmp_migrate, }, SRST -``migrate [-d] [-b]`` *uri* +``migrate [-d]`` *uri* Migrate to *uri* (using -d to not wait for completion). - - ``-b`` - for migration with full copy of disk ERST { diff --git a/migration/meson.build b/migration/meson.build index 92b1cc4297..af7350a21b 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -36,9 +36,6 @@ if get_option('replication').allowed() endif system_ss.add(when: rdma, if_true: files('rdma.c')) -if get_option('live_block_migration').allowed() - system_ss.add(files('block.c')) -endif system_ss.add(when: zstd, if_true: files('multifd-zstd.c')) specific_ss.add(when: 'CONFIG_SYSTEM_ONLY', -- 2.41.0
participants (2)
-
Juan Quintela -
Markus Armbruster