[libvirt] qemu-kvm networking speed
How do I change the networking speed provided to my qemu-kvm guest? I have four gigabit ports bonded; however, my guest is showing just 100Mb/s. I have used iperf to confirm. I am not sure if the settings would be on the guest or server side. Any help would be greatly appreciated! Thank you, Bryan
On Thu, Mar 12, 2009 at 09:28:56AM -0400, Bryan_Coleman@dart.biz wrote:
How do I change the networking speed provided to my qemu-kvm guest? I have four gigabit ports bonded; however, my guest is showing just 100Mb/s. I have used iperf to confirm.
The reported 'speed' setting for emulated NICs is basically irrelevant since this is emulated hardware. The original real hardware may have been specced as 100mb/s, but that has no bearing on its speed in a virtual environment. The QEMU emulated devices will send/recv data until the hit a bottleneck, typically the CPU usage of the QEMU process is hit first. So you want a NIC that entails a low CPU overhead in QEMU. In order of prefernce you'll want to try virtio, e1000, rtl8139, ne2k_pci This can be set in libvirt xml with <model type='virtio'/> in the guest XML <interface> section. Obviously you need to adjust your guest OS to load correct drivers to match. Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
Thank you for the suggestion. Switching to virtio and/or e1000 helped but still was not able to exceed 450Mb/s. It took me awhile to figure out you had to either restart or reload libvirtd for the changes to take effect. Is there anything else I could check/change to approach 1000Mb/s or higher? Thank you, Bryan Coleman "Daniel P. Berrange" <berrange@redhat.com> Sent by: libvir-list-bounces@redhat.com 03/12/2009 09:42 AM Please respond to "Daniel P. Berrange" <berrange@redhat.com> To Bryan_Coleman@dart.biz cc libvir-list@redhat.com Subject Re: [libvirt] qemu-kvm networking speed On Thu, Mar 12, 2009 at 09:28:56AM -0400, Bryan_Coleman@dart.biz wrote:
How do I change the networking speed provided to my qemu-kvm guest? I have four gigabit ports bonded; however, my guest is showing just 100Mb/s. I have used iperf to confirm.
The reported 'speed' setting for emulated NICs is basically irrelevant since this is emulated hardware. The original real hardware may have been specced as 100mb/s, but that has no bearing on its speed in a virtual environment. The QEMU emulated devices will send/recv data until the hit a bottleneck, typically the CPU usage of the QEMU process is hit first. So you want a NIC that entails a low CPU overhead in QEMU. In order of prefernce you'll want to try virtio, e1000, rtl8139, ne2k_pci This can be set in libvirt xml with <model type='virtio'/> in the guest XML <interface> section. Obviously you need to adjust your guest OS to load correct drivers to match. Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- Libvir-list mailing list Libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Thu, Mar 12, 2009 at 04:18:27PM -0400, Bryan_Coleman@dart.biz wrote:
Thank you for the suggestion. Switching to virtio and/or e1000 helped but still was not able to exceed 450Mb/s. It took me awhile to figure out you had to either restart or reload libvirtd for the changes to take effect. Is there anything else I could check/change to approach 1000Mb/s or higher?
I don't think any of the hardware emulation layer can be faster than virtio, that would be very surprizing. Seems you're approaching the limit of the capacity of the machine, I would not be surprized if you were CPU bound at that point. There is probably tuning possibilities but that also probably depend on other factors than just libvirt or QEmu itself. Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
When I run iperf according to 'top' my CPU % does not even go above 2%. 'virt-manager' CPU usage did not go above 25%. Thoughts? Bryan Daniel Veillard <veillard@redhat.com> 03/12/2009 04:28 PM Please respond to veillard@redhat.com To Bryan_Coleman@dart.biz cc libvir-list@redhat.com Subject Re: [libvirt] qemu-kvm networking speed On Thu, Mar 12, 2009 at 04:18:27PM -0400, Bryan_Coleman@dart.biz wrote:
Thank you for the suggestion. Switching to virtio and/or e1000 helped but still was not able to exceed 450Mb/s. It took me awhile to figure out you had to either restart or reload libvirtd for the changes to take effect.
Is there anything else I could check/change to approach 1000Mb/s or higher?
I don't think any of the hardware emulation layer can be faster than virtio, that would be very surprizing. Seems you're approaching the limit of the capacity of the machine, I would not be surprized if you were CPU bound at that point. There is probably tuning possibilities but that also probably depend on other factors than just libvirt or QEmu itself. Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
On Thu, Mar 12, 2009 at 04:35:22PM -0400, Bryan_Coleman@dart.biz wrote:
When I run iperf according to 'top' my CPU % does not even go above 2%. 'virt-manager' CPU usage did not go above 25%.
Thoughts?
Then it sounds like a bug, if the system is idle while process/domain are trying to push data on that very system and blocked on processing this sounds like an I/O scheduling error or something is blocking in your test. An interesting point would be to test with virtIO between 2 domains hosted on the same machine, to make sure it's not the actual networking access which is the limiting factor, and would provide an interesting data point. Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
On Thu, Mar 12, 2009 at 04:18:27PM -0400, Bryan_Coleman@dart.biz wrote:
Thank you for the suggestion. Switching to virtio and/or e1000 helped but still was not able to exceed 450Mb/s. It took me awhile to figure out you had to either restart or reload libvirtd for the changes to take effect. Is there anything else I could check/change to approach 1000Mb/s or higher?
I think it really comes down to tuning of the host & guest OS at that point, and making sure you have the absolute latest KVM host side, and absolute latest guest kernel. Early VirtIO guest drivers were not hugely optimized. There has been alot of optimization work in .27, .28, .29 kernels getting better all the time. Also make sure things like checksum/segmentation offload are enabled (ethtool settings). Also things like iptables can have impact - so worth checking how that's setup in host - eg, turn off net.bridge.bridge-nf-call-{iptables,arptables,ip6tables} sysctl settings to avoid having bridged traffic in host pushed up through iptables. This isn't really my area of expertize, so you'll probably get more help asking on the kvm development mailing list Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
Thanks for all of the suggestions. I am using Fedora 10 on both the host and guests. One thing of note: I was previously using a realtek setup. When I would do something network intensive ( nessus scan, iperf ), the network on the guest would just stop working. Switching to the virtio setup everything works just fine. Thanks again, Bryan "Daniel P. Berrange" <berrange@redhat.com> Sent by: libvir-list-bounces@redhat.com 03/13/2009 06:13 AM Please respond to "Daniel P. Berrange" <berrange@redhat.com> To Bryan_Coleman@dart.biz cc libvir-list@redhat.com Subject Re: [libvirt] qemu-kvm networking speed On Thu, Mar 12, 2009 at 04:18:27PM -0400, Bryan_Coleman@dart.biz wrote:
Thank you for the suggestion. Switching to virtio and/or e1000 helped but still was not able to exceed 450Mb/s. It took me awhile to figure out you had to either restart or reload libvirtd for the changes to take effect.
Is there anything else I could check/change to approach 1000Mb/s or higher?
I think it really comes down to tuning of the host & guest OS at that point, and making sure you have the absolute latest KVM host side, and absolute latest guest kernel. Early VirtIO guest drivers were not hugely optimized. There has been alot of optimization work in .27, .28, .29 kernels getting better all the time. Also make sure things like checksum/segmentation offload are enabled (ethtool settings). Also things like iptables can have impact - so worth checking how that's setup in host - eg, turn off net.bridge.bridge-nf-call-{iptables,arptables,ip6tables} sysctl settings to avoid having bridged traffic in host pushed up through iptables. This isn't really my area of expertize, so you'll probably get more help asking on the kvm development mailing list Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- Libvir-list mailing list Libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
participants (3)
-
Bryan_Coleman＠dart.biz -
Daniel P. Berrange -
Daniel Veillard