unable to migrate: virPortAllocatorSetUsed:299 : internal error: Failed to reserve port 49153
On libvirt 6.8.0 and qemu 5.1.0, when trying to live migrate "error:
internal error: Failed to reserve port" error is received and
migration does not succeed:
virsh # migrate cartridge qemu+tls://ratchet.lan/system --live
--persistent --undefinesource --copy-storage-all --verbose
error: internal error: Failed to reserve port 49153
virsh #
On target host with debug logs, nothing interesting but the error
itself is found in the logs
...
2020-10-12 02:11:33.852+0000: 6871: debug :
qemuMonitorJSONIOProcessLine:220 : Line [{"return": {}, "id":
"libvirt-373"}]
2020-10-12 02:11:33.852+0000: 6871: info :
qemuMonitorJSONIOProcessLine:239 : QEMU_MONITOR_RECV_REPLY:
mon=0x7fe784255020 reply={"return": {}, "id":
"libvirt-373"}
2020-10-12 02:11:33.852+0000: 6825: debug :
qemuDomainObjExitMonitorInternal:5615 : Exited monitor
(mon=0x7fe784255020 vm=0x55f086c81ea0 name=cartridge)
2020-10-12 02:11:33.852+0000: 6825: debug : qemuDomainObjEndJob:1140 :
Stopping job: async nested (async=migration in vm=0x55f086c81ea0
name=cartridge)
2020-10-12 02:11:33.852+0000: 6825: error :
virPortAllocatorSetUsed:299 : internal error: Failed to reserve port
49153
2020-10-12 02:11:33.852+0000: 6825: debug :
qemuMigrationParamsReset:1206 : Resetting migration parameters
0x7fe784257c30, flags 0x59
2020-10-12 02:11:33.852+0000: 6825: debug :
qemuDomainObjBeginJobInternal:835 : Starting job: job=async nested
agentJob=none asyncJob=none (vm=0x55f086c81ea0 name=cartridge, current
job=none agentJob=none async=migration in)
2020-10-12 02:11:33.852+0000: 6825: debug :
qemuDomainObjBeginJobInternal:887 : Started job: async nested
(async=migration in vm=0x55f086c81ea0 name=cartridge)
2020-10-12 02:11:33.852+0000: 6825: debug :
qemuDomainObjEnterMonitorInternal:5590 : Entering monitor
(mon=0x7fe784255020 vm=0x55f086c81ea0 name=cartridge)
2020-10-12 02:11:33.852+0000: 6825: debug :
qemuMonitorSetMigrationCapabilities:3853 : mon:0x7fe784255020
vm:0x55f086c81ea0 fd:30
2020-10-12 02:11:33.852+0000: 6825: info : qemuMonitorSend:944 :
QEMU_MONITOR_SEND_MSG: mon=0x7fe784255020
msg={"execute":"migrate-set-capabilities","arguments":{"capabilities":[{"capability":"xbzrle","state":false},{"capability":"auto-converge","state":false},{"capability":"rdma-pin-all","state":false},{"capability>
fd=-1
...
Full logs:
destination:
https://drive.google.com/file/d/1g986SbSVijvwZd8d7xDrJwo_AmKH-JnV/view?us...
source:
https://drive.google.com/file/d/1lsV2EOBxF7xH5-lgz2Psh9YSkOePXvAd/view?us...
On target host, nothing is listening on the target port:
ratchet /var/log/libvirt/qemu # netstat -lnp | grep 49153 | wc -l
0
and nc -l 49153 succeeds without issues.
Do you have any suggestions on how to proceed here?