8:53 a.m.
Hi all,
I want to use tc qdisc settings in a network coposed of several qemu VMs,
connected through bridges and tap interfaces.
I generate traffic with a spirent. Everything is fine when the scheduling
discipline is not installed but when I run the command to set taprio queues
traffic on the VM's NIC the traffic is dropped, i can send max 1mbps.
I think that there is something missing in the virtual NIC configuration or
setup. With ethtool i can see that queues are configured. I've also noticed
the BQL equals to 0, that is different than the physical machine
(BQL=18600) where everything works correctly.
I've read that it could be because NIC drivers do not support that setting.
Do you have any suggestions?
Thanks you all
Silvia
4:13 a.m.
On 3/27/21 2:53 PM, Silvia Fichera wrote:
Hi all,
I want to use tc qdisc settings in a network coposed of several qemu
VMs, connected through bridges and tap interfaces.
I generate traffic with a spirent. Everything is fine when the
scheduling discipline is not installed but when I run the command to set
taprio queues traffic on the VM's NIC the traffic is dropped, i can send
max 1mbps.
I think that there is something missing in the virtual NIC configuration
or setup. With ethtool i can see that queues are configured. I've also
noticed the BQL equals to 0, that is different than the physical machine
(BQL=18600) where everything works correctly.
I've read that it could be because NIC drivers do not support that setting.
Do you have any suggestions?
Hey,
I'm not familiar with taprio, but what's implemented in libvirt is htb
and sfq and that works well. Are you setting qdisc-s yourself or
modifying libvirt created structure?
Are you setting these qdiscs from the host, right?
I know that when changing QoS settings (when libvirt changes qdics/class
layout) for a brief moment packets are not transmitted from/to guest. I
suspect that kernel is freeing up queues or something. But this does not
look like your case, does it?
Michal
4:38 a.m.
Hi Michal,
these are the steps:
- start the vm with qemu
sudo qemu-system-x86_64-spice -m 2048 -enable-kvm -smp 3 -cdrom
/home/machine_A/ubuntu.iso -netdev
tap,ifname=tap0,vhost=on,id=n1,vhostforce=on,queues=3,script=/etc/tap0_ifup.sh,downscript=/etc/tap0_ifdown.sh
-device virtio-net-pci,netdev=n1,mac=9E:58:00:d2:53:03,mq=on,vectors=8
-netdev
tap,ifname=tap1,vhost=on,id=n2,vhostforce=on,queues=3,script=/etc/tap1_ifup.sh,downscript=/etc/tap1_ifdown.sh
-device virtio-net-pci,netdev=n2,mac=7A:53:00:d1:59:04,mq=on,vectors=8
-device virtio-net,netdev=network2 -netdev user,id=network2 -hda
switch_A.img
- attach the tap to the bridges in the host machine (to have the traffic
coming from the outsider traffic generator injected in the vm)
- in the VM:
- enable ip_forward
- enable promiscuous mode for the interface
- send the "tc qdisc" command to configure the output NIC with 2
traffic classes (TC) assigned to 2 different queues
- add the iptables mangle rules to the POSTROUTING chain to assign TC1
to the traffic with dport 7777 and TC0 to the traffic with dport 8888
I'm sending 2 UDP flows, I have no loss under 3Mbps each.
If I capture traffic with tcpdump on the ingress and on the egress nic (of
the VM), I see a difference of 50% of packets
When I did a test on the host machine, to check if it has the same problem,
I've reached an aggregated traffic of 90Mbps with no loss.
That's why I think that there is some misconfiguration on the virtual NIC.
Thanks
Silvia
On Mon, Mar 29, 2021 at 11:13 AM Michal Privoznik <mprivozn(a)redhat.com>
wrote:
On 3/27/21 2:53 PM, Silvia Fichera wrote:
> Hi all,
> I want to use tc qdisc settings in a network coposed of several qemu
> VMs, connected through bridges and tap interfaces.
> I generate traffic with a spirent. Everything is fine when the
> scheduling discipline is not installed but when I run the command to set
> taprio queues traffic on the VM's NIC the traffic is dropped, i can send
> max 1mbps.
> I think that there is something missing in the virtual NIC configuration
> or setup. With ethtool i can see that queues are configured. I've also
> noticed the BQL equals to 0, that is different than the physical machine
> (BQL=18600) where everything works correctly.
> I've read that it could be because NIC drivers do not support that
setting.
>
> Do you have any suggestions?
Hey,
I'm not familiar with taprio, but what's implemented in libvirt is htb
and sfq and that works well. Are you setting qdisc-s yourself or
modifying libvirt created structure?
Are you setting these qdiscs from the host, right?
I know that when changing QoS settings (when libvirt changes qdics/class
layout) for a brief moment packets are not transmitted from/to guest. I
suspect that kernel is freeing up queues or something. But this does not
look like your case, does it?
Michal
--
Silvia Fichera
5:04 a.m.
On 3/29/21 11:38 AM, Silvia Fichera wrote:
Hi Michal,
these are the steps:
- start the vm with qemu
sudo qemu-system-x86_64-spice -m 2048 -enable-kvm -smp 3 -cdrom
/home/machine_A/ubuntu.iso -netdev
tap,ifname=tap0,vhost=on,id=n1,vhostforce=on,queues=3,script=/etc/tap0_ifup.sh,downscript=/etc/tap0_ifdown.sh
-device virtio-net-pci,netdev=n1,mac=9E:58:00:d2:53:03,mq=on,vectors=8
-netdev
tap,ifname=tap1,vhost=on,id=n2,vhostforce=on,queues=3,script=/etc/tap1_ifup.sh,downscript=/etc/tap1_ifdown.sh
-device virtio-net-pci,netdev=n2,mac=7A:53:00:d1:59:04,mq=on,vectors=8
-device virtio-net,netdev=network2 -netdev user,id=network2 -hda
switch_A.img
- attach the tap to the bridges in the host machine (to have the traffic
coming from the outsider traffic generator injected in the vm)
- in the VM:
- enable ip_forward
- enable promiscuous mode for the interface
This sounds fishy. Why is this needed?
- send the "tc qdisc" command to configure the output
NIC with 2
traffic classes (TC) assigned to 2 different queues
- add the iptables mangle rules to the POSTROUTING chain to assign
TC1 to the traffic with dport 7777 and TC0 to the traffic with dport 8888
I'm sending 2 UDP flows, I have no loss under 3Mbps each.
If I capture traffic with tcpdump on the ingress and on the egress nic
(of the VM), I see a difference of 50% of packets
When I did a test on the host machine, to check if it has the same
problem, I've reached an aggregated traffic of 90Mbps with no loss.
That's why I think that there is some misconfiguration on the virtual NIC.
Well, that's fairly easy to test - create those two TAPs, do the setup
you're doing and instad of spawning qemu, run iperf or spirent and tell
it to use those TAPs. I think you'll find the problem elsewhere.
Anyway, this problem is not libvirt related really and I guess you can
find more elaborate replies on a list where TC developers hang out.
Looks like they're using latrc: https://lartc.org/#mailinglist
Michal
1215
days inactive
1217
days old
3 comments
2 participants
participants (2)
-
Michal Privoznik -
Silvia Fichera