4:59 a.m.
Hi,
I'm facing a strange issue while doing a migration from an hypervisor to another one.
The migration takes for ever to start moving the memory.
The VM had no workload what so ever, just a basic ubuntu image. The versions on the
hypervisors are: libvirt 1.2.21, qemu 1.2.3
Command to launche the migration:
virsh migrate --verbose --live --abort-on-error --tunnelled --p2p --auto-converge
--copy-storage-inc --xml vm-6160.xml 6160
qemu+tls://<destination_hypervisor>/system
Here is the log output, look at the time elapsed:
root@virt-hv009:~# virsh domjobinfo 6160
Job type: Unbounded
Time elapsed: 27518 ms
Data processed: 21.506 GiB
Data remaining: 29.003 GiB
Data total: 50.509 GiB
Memory processed: 0.000 B
Memory remaining: 520.820 MiB
Memory total: 520.820 MiB
File processed: 21.506 GiB
File remaining: 28.494 GiB
File total: 50.000 GiB
Constant pages: 0
Normal pages: 0
Normal data: 0.000 B
Expected downtime: 300 ms
Setup time: 6 ms
root@virt-hv009:~# virsh domjobinfo 6160
Job type: Unbounded
Time elapsed: 32331 ms
Data processed: 25.475 GiB
Data remaining: 25.034 GiB
Data total: 50.509 GiB
Memory processed: 0.000 B
Memory remaining: 520.820 MiB
Memory total: 520.820 MiB
File processed: 25.475 GiB
File remaining: 24.525 GiB
File total: 50.000 GiB
Constant pages: 0
Normal pages: 0
Normal data: 0.000 B
Expected downtime: 300 ms
Setup time: 6 ms
root@virt-hv009:~# virsh domjobinfo 6160
Job type: Unbounded
Time elapsed: 49543 ms
Data processed: 50.000 GiB
Data remaining: 520.820 MiB
Data total: 50.509 GiB
Memory processed: 0.000 B
Memory remaining: 520.820 MiB
Memory total: 520.820 MiB
File processed: 50.000 GiB
File remaining: 0.000 B
File total: 50.000 GiB
Constant pages: 0
Normal pages: 0
Normal data: 0.000 B
Expected downtime: 300 ms
Setup time: 6 ms
^^^^ Here after 49 sec, the disk has been process, but the memory hasn't started
I skip the logs, but below is the output 5minutes & 56 seconds later, still nothing:
root@virt-hv009:~# virsh domjobinfo 6160
Job type: Unbounded
Time elapsed: 356919 ms
Data processed: 50.000 GiB
Data remaining: 520.820 MiB
Data total: 50.509 GiB
Memory processed: 0.000 B
Memory remaining: 520.820 MiB
Memory total: 520.820 MiB
File processed: 50.000 GiB
File remaining: 0.000 B
File total: 50.000 GiB
Constant pages: 0
Normal pages: 0
Normal data: 0.000 B
Expected downtime: 300 ms
Setup time: 6 ms
Just after it started to move the memory, but 6 minutes later!
root@virt-hv009:~# virsh domjobinfo 6160
Job type: Unbounded
Time elapsed: 360092 ms
Data processed: 50.052 GiB
Data remaining: 453.895 MiB
Data total: 50.509 GiB
Memory processed: 53.224 MiB
Memory remaining: 453.895 MiB
Memory total: 520.820 MiB
Memory bandwidth: 98.172 MiB/s
File processed: 50.000 GiB
File remaining: 0.000 B
File total: 50.000 GiB
Constant pages: 3541
Normal pages: 13591
Normal data: 53.090 MiB
Expected downtime: 300 ms
Setup time: 6 ms
root@virt-hv009:~# virsh domjobinfo 6160
error: failed to get domain '6160'
error: Domain not found: no domain with matching name '6160'
Here the migration is done. But what made it wait more than 5 minutes to start moving the
memory? Does anyone knows? Or have any idea what could produce this latency.
- Marco