[libvirt] [PATCH 1/1] qemu: libvirt live migration over RDMA of ipv6 addr failed
Using libvirt to do live migration over RDMA via ip v6 address failed.
For example:
# virsh migrate --live --migrateuri rdma://[deba::2222]:49152 \
rhel73_host1_guest1 qemu+ssh://[deba::2222]/system --verbose
root@deba::2222's password:
error: internal error: unable to execute QEMU command 'migrate': RDMA ERROR:
could not rdma_getaddrinfo address deba
As we can see, the ip v6 address used by rdma_getaddrinfo() has only "deba"
part. It should be "deba::2222".
1) According to rfc 3986, a literal ip v6 address should be enclosed
in '[' and ']'.
When using virsh command to do live migration via ip v6 addresss, user
will input the ip v6 address with brackets (i.e. rdma://[deba::2222]:49152).
libvirt will parse command line option by calling virURIParse().
Inside it calls virStringStripIPv6Brackets() to strip off the brackets.
The uri passed in to virURIParse() is:
"uri = rdma://[deba::2222]:49152"
Inside virURIParse() routine, it will strip off the bracket '[' and ']'
if
it's ip v6 address. Then save the ip v6 address in this format "deba::2222"
in the virURI->server field, and to be passed to qemu.
2) At the beginning of migration, in qemu's qemu_rdma_data_init(host_port)
routine, it calls inet_parse(host_port) routine to parse the ip v6 address and
port string obtained from libvirt.
The input string host_port passed to qemu_rdma_data_init() can be:
"hostname:port", or
"ipv4address:port", or
"[ipv6address]:port" (i.e "[deba::2222]:49152"), or
"ipv6address:port" (i.e "deba::2222:49152").
Existing qemu api inet_parse() can handle the above first 3 cases properly,
but didn't handle the last case ("ipv6address:port") correctly.
In this live migration over rdma via ip v6 address case, the server ip v6
address obtained from libvirt doesn't contain the brackets '[' and ']'
(i.e. "deba::2222:49152"). It caused inet_parse() to parse only "deba"
part,
and stopped at the 1st colon ':'. As the result, the subsequent
rdma_getaddrinfo() with ip address "deba" will fail.
NOTE:
If using libvirt to do live migration over TCP via ip v6 address:
# virsh migrate --live --migrateuri tcp://[deba::2222]:49152 \
rhel73_host1_guest1 qemu+ssh://[deba::2222]/system --verbose
It works fine.
In migrateuri of tcp case, libvirt will call virNetSocketNewConnectTCP()
directly to connect to remote "deba::2222" after it strips off
the brackets '[' and ']' for an ip v6 address.
On qemu side, fd_start_outgoing_migration() will be called to do migration.
It doesn't call inet_parse(). So we don't see issue in tcp case.
Solution:
Originally the patch was proposed to Qemu-devel commuity to changed code
in inet_parse() to handle ipv6 address w/o brackets ("ipv6:port" format).
Daniel's review comment is it's mandatory to use [] when providing a
numeric IPv6 address. So there's nothing broken in QEMU. libvirt needs
fixing to pass correct data to QEMU.
Now the new proposed patch is in libvirt's qemu driver, in
qemuMonitorMigrateToHost() routine, it will assemble uri based on host
and port values passed in. If the host passed in is an ipv6 address, we
will enclose brackets [] for the ipv6 address here before calling
qemuMonitorJSONMigrate().
The -incoming CLI part of enclosed brackets ipv6 address is already
taken care of in qemuMigrationPrepareIncoming() routine.
Signed-off-by: David Dai <zdai(a)linux.vnet.ibm.com>
---
src/qemu/qemu_monitor.c | 8 ++++++--
1 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c
index b7be5e7..258665c 100644
--- a/src/qemu/qemu_monitor.c
+++ b/src/qemu/qemu_monitor.c
@@ -2577,8 +2577,12 @@ qemuMonitorMigrateToHost(qemuMonitorPtr mon,
QEMU_CHECK_MONITOR(mon);
- if (virAsprintf(&uri, "%s:%s:%d", protocol, hostname, port) < 0)
- return -1;
+ if (strchr(hostname, ':')) {
+ if (virAsprintf(&uri, "%s:[%s]:%d", protocol, hostname, port) <
0)
+ return -1;
+ } else if (virAsprintf(&uri, "%s:%s:%d", protocol, hostname, port) <
0) {
+ return -1;
+ }
if (mon->json)
ret = qemuMonitorJSONMigrate(mon, flags, uri);
--
1.7.1