[libvirt] speeding up qemu core dumps
Hi All, I've had user reports of "libvirt limiting KVM core dumps". This isn't quite true since libvirt just uses the default migration speed defined in qemu's migration.c /* Migration speed throttling */ static int64_t max_throttle = (32 << 20); AFAIK, there's not much interest in changing the qemu default http://lists.nongnu.org/archive/html/qemu-devel/2011-02/msg01156.html I thought there were more discussions on the topic but can only find the above thread ATM. Do folks here have objections to increasing the migration speed prior to core dump? If not, any suggestions for a value? I set the migration speed to 1G in doCoreDump() prior to calling qemuMigrationToFile() with following improvement dumping a 4G guest 32MiB/s migration speed (qemu default): # time virsh dump --live test /tmp/dump Domain test dumped to /tmp/dump real 0m10.121s user 0m0.004s sys 0m0.004s 1GiB/s migration speed: # time virsh dump --live test /tmp/dump Domain test dumped to /tmp/dump real 0m3.745s user 0m0.004s sys 0m0.004s I suppose the same applies for save. Regards, Jim
On 08/03/2011 08:37 AM, Jim Fehlig wrote:
http://lists.nongnu.org/archive/html/qemu-devel/2011-02/msg01156.html
I thought there were more discussions on the topic but can only find the above thread ATM.
Do folks here have objections to increasing the migration speed prior to core dump? If not, any suggestions for a value? I set the migration speed to 1G in doCoreDump() prior to calling qemuMigrationToFile() with following improvement dumping a 4G guest
Why any limit at all? When dumping to file rather than network, I can't think of any justification for a default limit when targetting a local drive (a particular installation may have a reasonable limit, especially if the file involved happens to use a network - think NFS). It seems like it would make sense to override the qemu defaults and go with unlimited speed for faster dump and save operations. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
Eric Blake wrote:
On 08/03/2011 08:37 AM, Jim Fehlig wrote:
http://lists.nongnu.org/archive/html/qemu-devel/2011-02/msg01156.html
I thought there were more discussions on the topic but can only find the above thread ATM.
Do folks here have objections to increasing the migration speed prior to core dump? If not, any suggestions for a value? I set the migration speed to 1G in doCoreDump() prior to calling qemuMigrationToFile() with following improvement dumping a 4G guest
Why any limit at all? When dumping to file rather than network, I can't think of any justification for a default limit when targetting a local drive (a particular installation may have a reasonable limit, especially if the file involved happens to use a network - think NFS). It seems like it would make sense to override the qemu defaults and go with unlimited speed for faster dump and save operations.
Agreed, 1G was just a test value, but I'm haven't yet figured out how to define unlimited. From libvirt json monitor connection I tried migrate_set_speed with INT64_MAX, but that was invalid. Using qemu monitor directly, I can go as high as INT64_MAX-512 (qemu) migrate_set_speed 9223372036854775295B (qemu) migrate_set_speed 9223372036854775296B invalid size Any hints on proper way to set unlimited migration speed? Regards, Jim
On Wed, Aug 03, 2011 at 09:42:23PM -0600, Jim Fehlig wrote:
Eric Blake wrote:
On 08/03/2011 08:37 AM, Jim Fehlig wrote:
http://lists.nongnu.org/archive/html/qemu-devel/2011-02/msg01156.html
I thought there were more discussions on the topic but can only find the above thread ATM.
Do folks here have objections to increasing the migration speed prior to core dump? If not, any suggestions for a value? I set the migration speed to 1G in doCoreDump() prior to calling qemuMigrationToFile() with following improvement dumping a 4G guest
Why any limit at all? When dumping to file rather than network, I can't think of any justification for a default limit when targetting a local drive (a particular installation may have a reasonable limit, especially if the file involved happens to use a network - think NFS). It seems like it would make sense to override the qemu defaults and go with unlimited speed for faster dump and save operations.
Agreed, 1G was just a test value, but I'm haven't yet figured out how to define unlimited. From libvirt json monitor connection I tried migrate_set_speed with INT64_MAX, but that was invalid. Using qemu monitor directly, I can go as high as INT64_MAX-512
(qemu) migrate_set_speed 9223372036854775295B (qemu) migrate_set_speed 9223372036854775296B invalid size
Any hints on proper way to set unlimited migration speed?
The code doesn't seem to have a way to set a truely unlimited rate, but even INT64_MAX / 2 would be more than sufficient for us to max out the host OS storage throughput Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
participants (3)
-
Daniel P. Berrange -
Eric Blake -
Jim Fehlig