[libvirt] Question about save and restore OVS per-port data in live migration
Hi When we try to migrate a guest with 12 NICs, we find libvirt will try to get per-port data in finish phase, it makes the guest paused for several hundred milliseconds. qemuMigrationCookieAddNetwork -> virNetDevOpenvswitchGetMigrateData for per-nic -> run 'ovs-vsctl --if-exists get Interface vnetx external_ids:Port' (about 25ms) In our case, there's no portData available and it seems there is no need to run the cmd. Questions: 1. Is it necessary to get/set per-port data in migration finish phase? It will increase the downtime. 2. Is there any way (i.e. a switch) to not run the OVS cmd if there's no portData available?
On 5/29/19 6:23 AM, wangxin (U) wrote:
Hi
When we try to migrate a guest with 12 NICs, we find libvirt will try to get per-port data in finish phase, it makes the guest paused for several hundred milliseconds.
qemuMigrationCookieAddNetwork -> virNetDevOpenvswitchGetMigrateData for per-nic -> run 'ovs-vsctl --if-exists get Interface vnetx external_ids:Port' (about 25ms)
In our case, there's no portData available and it seems there is no need to run the cmd.
Questions: 1. Is it necessary to get/set per-port data in migration finish phase? It will increase the downtime. 2. Is there any way (i.e. a switch) to not run the OVS cmd if there's no portData available?
IIUC migrating port data is required for openvswitch functionality on the destination. Although looking at the commit that introduced this does not say that explicitly (v1.0.0-rc1~8). But I vaguely recall that being the case. We should revisit that decission and ask ovd guys if that's still the case. Unfortunately, to libvirt 'port data' are opaque blob. And I'm not sure we can know in advance what ports have data to transfer and which don't. Michal
-----Original Message----- From: Michal Privoznik [mailto:mprivozn@redhat.com] Sent: Wednesday, May 29, 2019 3:44 PM To: wangxin (U) <wangxinxin.wang@huawei.com>; libvir-list@redhat.com Cc: Lichunhe (Cloud Networking) <lichunhe@huawei.com> Subject: Re: Question about save and restore OVS per-port data in live migration
Hi
When we try to migrate a guest with 12 NICs, we find libvirt will try to get per-port data in finish phase, it makes the guest paused for several hundred milliseconds.
qemuMigrationCookieAddNetwork -> virNetDevOpenvswitchGetMigrateData for per-nic -> run 'ovs-vsctl --if-exists get Interface vnetx external_ids:Port' (about 25ms)
In our case, there's no portData available and it seems there is no need to run the cmd.
Questions: 1. Is it necessary to get/set per-port data in migration finish phase? It will increase the downtime. 2. Is there any way (i.e. a switch) to not run the OVS cmd if there's no
On 5/29/19 6:23 AM, wangxin (U) wrote: portData available?
IIUC migrating port data is required for openvswitch functionality on the destination. Although looking at the commit that introduced this does not say that explicitly (v1.0.0-rc1~8). But I vaguely recall that being the case. We should revisit that decission and ask ovd guys if that's still the case.
It will be helpful to do this. And, is there a special reason to migrating port data in finish phase? The finish phase should be as simple as possible. Tanks, Wangxin
Unfortunately, to libvirt 'port data' are opaque blob. And I'm not sure we can know in advance what ports have data to transfer and which don't.
Michal
On 5/30/19 3:50 AM, wangxin (U) wrote:
-----Original Message----- From: Michal Privoznik [mailto:mprivozn@redhat.com] Sent: Wednesday, May 29, 2019 3:44 PM To: wangxin (U) <wangxinxin.wang@huawei.com>; libvir-list@redhat.com Cc: Lichunhe (Cloud Networking) <lichunhe@huawei.com> Subject: Re: Question about save and restore OVS per-port data in live migration
Hi
When we try to migrate a guest with 12 NICs, we find libvirt will try to get per-port data in finish phase, it makes the guest paused for several hundred milliseconds.
qemuMigrationCookieAddNetwork -> virNetDevOpenvswitchGetMigrateData for per-nic -> run 'ovs-vsctl --if-exists get Interface vnetx external_ids:Port' (about 25ms)
In our case, there's no portData available and it seems there is no need to run the cmd.
Questions: 1. Is it necessary to get/set per-port data in migration finish phase? It will increase the downtime. 2. Is there any way (i.e. a switch) to not run the OVS cmd if there's no
On 5/29/19 6:23 AM, wangxin (U) wrote: portData available?
IIUC migrating port data is required for openvswitch functionality on the destination. Although looking at the commit that introduced this does not say that explicitly (v1.0.0-rc1~8). But I vaguely recall that being the case. We should revisit that decission and ask ovd guys if that's still the case.
It will be helpful to do this.
And, is there a special reason to migrating port data in finish phase? The finish phase should be as simple as possible.
Well, finish phase is the phase where we want to do this. Because until finish phase the domain is running on the source and if we'd transfer OVS private state before then there would be two different states - one on the source the other at the destination and we can't tell which one is valid. BTW this is exactly what's done with spice migration too. Well, qemu does it for us. But libvirt still has to wait for qemu to transfer spice internal state. Anyway, let me ask OVS guys. Done here: https://mail.openvswitch.org/pipermail/ovs-discuss/2019-May/048739.html Michal
-----Original Message----- From: Michal Privoznik [mailto:mprivozn@redhat.com] Sent: Friday, May 31, 2019 3:25 PM To: wangxin (U) <wangxinxin.wang@huawei.com>; libvir-list@redhat.com Cc: Lichunhe (Cloud Networking) <lichunhe@huawei.com> Subject: Re: Question about save and restore OVS per-port data in live migration
On 5/30/19 3:50 AM, wangxin (U) wrote:
-----Original Message----- From: Michal Privoznik [mailto:mprivozn@redhat.com] Sent: Wednesday, May 29, 2019 3:44 PM To: wangxin (U) <wangxinxin.wang@huawei.com>; libvir-list@redhat.com Cc: Lichunhe (Cloud Networking) <lichunhe@huawei.com> Subject: Re: Question about save and restore OVS per-port data in live migration
Hi
When we try to migrate a guest with 12 NICs, we find libvirt will try to get per-port data in finish phase, it makes the guest paused for several hundred milliseconds.
qemuMigrationCookieAddNetwork -> virNetDevOpenvswitchGetMigrateData for per-nic -> run 'ovs-vsctl --if-exists get Interface vnetx external_ids:Port' (about 25ms)
In our case, there's no portData available and it seems there is no need to run the cmd.
Questions: 1. Is it necessary to get/set per-port data in migration finish phase? It will increase the downtime. 2. Is there any way (i.e. a switch) to not run the OVS cmd if there's no
On 5/29/19 6:23 AM, wangxin (U) wrote: portData available?
IIUC migrating port data is required for openvswitch functionality on the destination. Although looking at the commit that introduced this does not say that explicitly (v1.0.0-rc1~8). But I vaguely recall that being the case. We should revisit that decission and ask ovd guys if that's still the case.
It will be helpful to do this.
And, is there a special reason to migrating port data in finish phase? The finish phase should be as simple as possible.
Well, finish phase is the phase where we want to do this. Because until finish phase the domain is running on the source and if we'd transfer OVS private state before then there would be two different states - one on the source the other at the destination and we can't tell which one is valid. BTW this is exactly what's done with spice migration too. Well, qemu does it for us. But libvirt still has to wait for qemu to transfer spice internal state.
I see, it's always safe to migrate device data after VM stopped.
Anyway, let me ask OVS guys.
Done here:
https://mail.openvswitch.org/pipermail/ovs-discuss/2019-May/048739.html
Michal
Thanks, Wangxin
participants (2)
-
Michal Privoznik -
wangxin (U)