Re: [libvirt-users] Need to increase the rx and tx buffer size of my interface

Hi Ashish, I have tested with your xml in the first mail, and it works for rx_queue_size(see below). multiqueue need to work with vhost backend driver. And when you set "queues=1" it will ignored. Please check your qemu-kvm-rhev package, should be newer than qemu-kvm-rhev-2.9.0-16.el7_4.2 And the logs? tx_queue_size='512' will not work in the guest with direct type interface, in fact, no matter what you set, it will not work and guest will get the default '256'. We only support vhost-user backend to have more than 256. refer to https://libvirt.org/formatdomain.html#elementsNICSEthernet tx_queue_size The optional tx_queue_size attribute controls the size of virtio ring for each queue as described above. The default value is hypervisor dependent and may change across its releases. Moreover, some hypervisors may pose some restrictions on actual value. For instance, QEMU v2.9 requires value to be a power of two from [256, 1024] range. In addition to that, this may work only for a subset of interface types, e.g. aforementioned QEMU enables this option only for vhostuser type. Since 3.7.0 (QEMU and KVM only) multiqueue only supports vhost as backend driver. # rpm -q libvirt qemu-kvm-rhev libvirt-3.2.0-14.el7_4.3.x86_64 qemu-kvm-rhev-2.9.0-16.el7_4.9.x86_64 1. the xml as below <interface type='direct'> <mac address='52:54:00:00:b5:99'/> <source dev='eno1' mode='vepa'/> <model type='virtio'/> <driver name='vhost' queues='5' rx_queue_size='512' tx_queue_size='512'> <host csum='off' gso='off' tso4='off' tso6='off' ecn='off' ufo='off' mrg_rxbuf='off'/> <guest csum='off' tso4='off' tso6='off' ecn='off' ufo='off'/> </driver> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> 2. after start the vm, check the qemu command line: *-netdev tap,fds=26:28:29:30:31,id=hostnet0,vhost=on,vhostfds=32:33:34:35:36* -device virtio-net-pci,csum=off,gso=off,host_tso4=off,host_tso6=off,host_ecn=off,host_ufo=off,mrg_rxbuf=off,guest_csum=off,guest_tso4=off,guest_tso6=off,guest_ecn=off,guest_ufo=off, *mq=on,vectors=12,rx_queue_size=512,tx_queue_size=512* ,netdev=hostnet0,id=net0,mac=52:54:00:00:b5:99,bus=pci.0,addr=0x3 3. check on guest # ethtool -g eth0 Ring parameters for eth0: Pre-set maximums: RX: *512 ==> rx_queue_size works* RX Mini: 0 RX Jumbo: 0 TX: *256 ===> no change* Current hardware settings: RX: *512 **==> rx_queue_size works* RX Mini: 0 RX Jumbo: 0 TX: *256 ===> no change* # ethtool -l eth0 Channel parameters for eth0: Pre-set maximums: RX: 0 TX: 0 Other: 0 Combined: *5 ==> queues what we set* Current hardware settings: RX: 0 TX: 0 Other: 0 Combined: 1 If change to qemu as driver, # virsh edit rhel7 .. <interface type='direct'> <mac address='52:54:00:00:b5:99'/> <source dev='eno1' mode='vepa'/> <model type='virtio'/> <driver name='qemu' queues='5' rx_queue_size='512' tx_queue_size='512'> <host csum='off' gso='off' tso4='off' tso6='off' ecn='off' ufo='off' mrg_rxbuf='off'/> <guest csum='off' tso4='off' tso6='off' ecn='off' ufo='off'/> </driver> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> .. Domain rhel7 XML configuration edited. ==> the xml can validate and save # virsh start rhel7 Domain rhel7 started # virsh dumpxml rhel7 | grep /interface -B9 <source dev='eno1' mode='vepa'/> <target dev='macvtap0'/> <model type='virtio'/> *<driver name='qemu' queues='5' rx_queue_size='512' tx_queue_size='512'>* <host csum='off' gso='off' tso4='off' tso6='off' ecn='off' ufo='off' mrg_rxbuf='off'/> <guest csum='off' tso4='off' tso6='off' ecn='off' ufo='off'/> </driver> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> * -netdev tap,fds=26:28:29:30:31*,id=hostnet0 -device virtio-net-pci,csum=off,gso=off,host_tso4=off,host_tso6=off,host_ecn=off,host_ufo=off,mrg_rxbuf=off,guest_csum=off,guest_tso4=off,guest_tso6=off,guest_ecn=off,guest_ufo=off, *rx_queue_size=512,tx_queue_size=512* ,netdev=hostnet0,id=net0,mac=52:54:00:00:b5:99,bus=pci.0,addr=0x3 *"mq=on,vectors=12" is missing*, indicates there is no multiqueue and check on guest # ethtool -l eth0 Channel parameters for eth0: Pre-set maximums: RX: 0 TX: 0 Other: 0 Combined: 1 ==> no multiqueue Current hardware settings: RX: 0 TX: 0 Other: 0 Combined: 1 # ethtool -g eth0 Ring parameters for eth0: Pre-set maximums: RX: *512* RX Mini: 0 RX Jumbo: 0 TX: 256 Current hardware settings: RX: *512* RX Mini: 0 RX Jumbo: 0 TX: 256 ------- Best Regards, Yalan Zhang IRC: yalzhang Internal phone: 8389413 On Thu, Oct 26, 2017 at 2:33 AM, Ashish Kurian <ashishbnv(a)gmail.com&gt; wrote: