Re: [libvirt] virsh domjobinfo during storage migration
On Thu, Nov 24, 2016 at 22:54:10 +0100, Ruben Kerkhof wrote:
Hi all,
Running virsh domjobinfo on my CentOS 6 systems during a migration
with --copy-storage-all, the output used to look like this:
Job type: Unbounded
Time elapsed: 1830632 ms
Data processed: 37.212 GiB
Data remaining: 1.025 GiB
Data total: 16.016 GiB
Memory processed: 37.212 GiB
Memory remaining: 1.025 GiB
Memory total: 16.016 GiB
Memory bandwidth: 100.018 MiB/s
Constant pages: 618279
Normal pages: 9734623
Normal data: 37.135 GiB
Expected downtime: 1118 ms
Setup time: 61 ms
But when I run the same command on CentOS 7, libvirt-1.2.17-13 and
qemu-kvm-1.5.3-105, I just get this:
$ sudo virsh domjobinfo 58358fec-c35c-4a7a-a4dd-18ec2e1327bf
Job type: Unbounded
Time elapsed: 209616 ms
I tried to debug this myself, but I'm a bit stuck. I'd expected
libvirt to send query-migrate commands to qemu every 50 ms or so. To
verify I attached gdb and put a breakpoint at
qemuMonitorJSONGetMigrationStatus, but the breakpoint only hits after
the complete disk is mirrored.
That's expected since QEMU finally implemented a migration event, which
allowed us to stop polling every 50 ms. Libvirt will only call
query-migrate at the end of a migration or when virsh domjobinfo is
called.
The reason why you don't see anything interesting in the output is NBD
storage migration. With new enough QEMU libvirt will first migrate
storage using QEMU's integrated NBD server and then "migrate" QMP
command will called. With old QEMU storage migration was done by the
"migrate" command itself. Thus calling query-migrate while storage is
migrated (i.e., before migrate was called) does not provide anything. I
guess it should be possible to also check the progress of all running
block jobs so that we can report statistics about ongoing storage
migration even when NBD is used.
Jirka