[libvirt] problem with outbound limiting
Hi I am using 0.9.4 version of libvirt and want to limit bandwidth. I installed libvirt on ubuntu 11.04 using libvirt source. For this i am using the QOS configuration <devices> <interface type='network'> <source network='default'/> <target dev='vnet0'/> *<bandwidth> <inbound average='1000' peak='5000' burst='1024'/> <outbound average='128' peak='256' burst='256'/> </bandwidth>* </interface> Here the inbound limit is working properly but the outbound is limiting but not with values which is give Here are the scenarios i tested and the results when i gave average as 2048 --- it is limiting with 600 -- 700 kb average result ===================== 2048 (2mpbs) 600-700 kb 4096 (4mbps) 1 MB/s 10240 (10mbps) 2-3 MB/s 12288 (12 mbps) 3-4 MB/s 20480 (20 mbps) 16-18 MB/s Is this the expected behavior or some thing is missing If this is the expected behavior ,then in what ratio it is limiting. Please help me Thanks in advance. -- Thanks and Regards, Upendra.M
small correction it is not mbps ,it is MBps On Wed, Aug 10, 2011 at 12:06 PM, Upendra Moturi <upendra.m@hexagrid.com>wrote:
Hi
I am using 0.9.4 version of libvirt and want to limit bandwidth. I installed libvirt on ubuntu 11.04 using libvirt source.
For this i am using the QOS configuration
<devices> <interface type='network'>
<source network='default'/> <target dev='vnet0'/> *<bandwidth> <inbound average='1000' peak='5000' burst='1024'/> <outbound average='128' peak='256' burst='256'/>
</bandwidth>* </interface>
Here the inbound limit is working properly but the outbound is limiting but not with values which is give
Here are the scenarios i tested and the results
when i gave average as 2048 --- it is limiting with 600 -- 700 kb
average result ===================== 2048 (2mpbs) 600-700 kb 4096 (4mbps) 1 MB/s 10240 (10mbps) 2-3 MB/s 12288 (12 mbps) 3-4 MB/s 20480 (20 mbps) 16-18 MB/s
Is this the expected behavior or some thing is missing If this is the expected behavior ,then in what ratio it is limiting.
Please help me Thanks in advance.
-- Thanks and Regards, Upendra.M
-- Thanks and Regards, Upendra.M
On 08/10/2011 02:55 AM, Upendra Moturi wrote: Hi, does this patch solve your problem? I am setting the MTU to fixed 2kb. Doing tests with scp seems to indicate that this improved the situation -- at least for me. Stefan Signed-off-by: Stefan Berger <stefanb@linux.vnet.ibm.com> --- src/util/network.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) Index: libvirt-acl/src/util/network.c =================================================================== --- libvirt-acl.orig/src/util/network.c +++ libvirt-acl/src/util/network.c @@ -1156,8 +1156,8 @@ virBandwidthEnable(virBandwidthPtr bandw virCommandFree(cmd); cmd = virCommandNew(TC); - virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent", - "1:", "classid", "1:1", "htb", NULL); + virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent", + "1:", "classid", "1:1", "htb", NULL); virCommandAddArgList(cmd, "rate", average, NULL); if (peak) @@ -1170,9 +1170,9 @@ virBandwidthEnable(virBandwidthPtr bandw virCommandFree(cmd); cmd = virCommandNew(TC); - virCommandAddArgList(cmd,"filter", "add", "dev", iface, "parent", - "1:0", "protocol", "ip", "handle", "1", "fw", - "flowid", "1", NULL); + virCommandAddArgList(cmd,"filter", "add", "dev", iface, "parent", + "1:0", "protocol", "ip", "handle", "1", "fw", + "flowid", "1", NULL); if (virCommandRun(cmd, NULL) < 0) goto cleanup; @@ -1191,8 +1191,8 @@ virBandwidthEnable(virBandwidthPtr bandw virCommandFree(cmd); cmd = virCommandNew(TC); - virCommandAddArgList(cmd, "qdisc", "add", "dev", iface, - "ingress", NULL); + virCommandAddArgList(cmd, "qdisc", "add", "dev", iface, + "ingress", NULL); if (virCommandRun(cmd, NULL) < 0) goto cleanup; @@ -1202,7 +1202,7 @@ virBandwidthEnable(virBandwidthPtr bandw virCommandAddArgList(cmd, "filter", "add", "dev", iface, "parent", "ffff:", "protocol", "ip", "u32", "match", "ip", "src", "0.0.0.0/0", "police", "rate", average, - "burst", burst, "mtu", burst, "drop", "flowid", + "burst", burst, "mtu", "2kb", "drop", "flowid", ":1", NULL); if (virCommandRun(cmd, NULL) < 0)
small correction it is not mbps ,it is MBps
On Wed, Aug 10, 2011 at 12:06 PM, Upendra Moturi <upendra.m@hexagrid.com <mailto:upendra.m@hexagrid.com>> wrote:
Hi
I am using 0.9.4 version of libvirt and want to limit bandwidth. I installed libvirt on ubuntu 11.04 using libvirt source.
For this i am using the QOS configuration
<devices> <interface type='network'>
<source network='default'/> <target dev='vnet0'/> *<bandwidth> <inbound average='1000' peak='5000' burst='1024'/> <outbound average='128' peak='256' burst='256'/>
</bandwidth>* </interface>
Here the inbound limit is working properly but the outbound is limiting but not with values which is give
Here are the scenarios i tested and the results
when i gave average as 2048 --- it is limiting with 600 -- 700 kb
average result ===================== 2048 (2mpbs) 600-700 kb 4096 (4mbps) 1 MB/s 10240 (10mbps) 2-3 MB/s 12288 (12 mbps) 3-4 MB/s 20480 (20 mbps) 16-18 MB/s
Is this the expected behavior or some thing is missing If this is the expected behavior ,then in what ratio it is limiting.
Please help me Thanks in advance.
-- Thanks and Regards, Upendra.M
-- Thanks and Regards, Upendra.M
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On 08/10/2011 09:04 AM, Stefan Berger wrote:
On 08/10/2011 02:55 AM, Upendra Moturi wrote:
Hi,
does this patch solve your problem? I am setting the MTU to fixed 2kb. Doing tests with scp seems to indicate that this improved the situation -- at least for me.
Stefan
Signed-off-by: Stefan Berger <stefanb@linux.vnet.ibm.com>
--- src/util/network.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
Index: libvirt-acl/src/util/network.c =================================================================== --- libvirt-acl.orig/src/util/network.c +++ libvirt-acl/src/util/network.c @@ -1156,8 +1156,8 @@ virBandwidthEnable(virBandwidthPtr bandw
virCommandFree(cmd); cmd = virCommandNew(TC); - virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent", - "1:", "classid", "1:1", "htb", NULL); + virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent", + "1:", "classid", "1:1", "htb", NULL); virCommandAddArgList(cmd, "rate", average, NULL);
This hunk was just indentation; you should push the whitespace changes as a separate commit (and can push that now, under the trivial rule, without needing further review). [It doesn't help matters that thunderbird botched the whitespace in my reply]
@@ -1202,7 +1202,7 @@ virBandwidthEnable(virBandwidthPtr bandw virCommandAddArgList(cmd, "filter", "add", "dev", iface, "parent", "ffff:", "protocol", "ip", "u32", "match", "ip", "src", "0.0.0.0/0", "police", "rate", average, - "burst", burst, "mtu", burst, "drop", "flowid", + "burst", burst, "mtu", "2kb", "drop", "flowid", ":1", NULL);
Here's the meat of the proposed patch. Is 2kb always valid, or will we run into problems on NICs that insist on traditional limits near the 1518 mark? Should it be user-configurable? I'm reluctant to ACK this without a bit more understanding of why 2kb works, as well as feedback on test results with the patch in place. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
On 08/10/2011 11:20 AM, Eric Blake wrote:
On 08/10/2011 09:04 AM, Stefan Berger wrote:
On 08/10/2011 02:55 AM, Upendra Moturi wrote:
Hi,
does this patch solve your problem? I am setting the MTU to fixed 2kb. Doing tests with scp seems to indicate that this improved the situation -- at least for me.
Stefan
Signed-off-by: Stefan Berger <stefanb@linux.vnet.ibm.com>
--- src/util/network.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
Index: libvirt-acl/src/util/network.c =================================================================== --- libvirt-acl.orig/src/util/network.c +++ libvirt-acl/src/util/network.c @@ -1156,8 +1156,8 @@ virBandwidthEnable(virBandwidthPtr bandw
virCommandFree(cmd); cmd = virCommandNew(TC); - virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent", - "1:", "classid", "1:1", "htb", NULL); + virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent", + "1:", "classid", "1:1", "htb", NULL); virCommandAddArgList(cmd, "rate", average, NULL);
This hunk was just indentation; you should push the whitespace changes as a separate commit (and can push that now, under the trivial rule, without needing further review). [It doesn't help matters that thunderbird botched the whitespace in my reply]
@@ -1202,7 +1202,7 @@ virBandwidthEnable(virBandwidthPtr bandw virCommandAddArgList(cmd, "filter", "add", "dev", iface, "parent", "ffff:", "protocol", "ip", "u32", "match", "ip", "src", "0.0.0.0/0", "police", "rate", average, - "burst", burst, "mtu", burst, "drop", "flowid", + "burst", burst, "mtu", "2kb", "drop", "flowid", ":1", NULL);
Here's the meat of the proposed patch. Is 2kb always valid, or will we run into problems on NICs that insist on traditional limits near the 1518 mark? Should it be user-configurable? I'm reluctant to ACK this without a bit more understanding of why 2kb works, as well as feedback on test results with the patch in place.
Yes, I am not an exper on tc. I have seen examples with the mtu being set to '1' for dropping icmp traffic for example. So if a VM produces smaller than 2kb packets, which on traditional networks it probably should, then the packets should go through. We could also leave out the mtu parameter above and it would go to 2kb by default. Check via tc filter show dev <ifname> root The 2kb parameter doesn't explain why 2MB doesn't work, though. Even if this wasn't the final fix, it at least 'improves the situation' (for me). Stefan
Thanks for your reply. The patch it appears still has problem. I have changed only mtu value as 2kb instead of burst value in the network.c file With average as 2048 , burst as 2048 and peak 2048 scp copied at 30-40KB/s With average as 4096 , burst as 4096 and peak 4096 scp copied at same 30-40KB/s where as earlier it was 1 MB/s.(without patch) On Wed, Aug 10, 2011 at 9:16 PM, Stefan Berger <stefanb@linux.vnet.ibm.com> wrote:
On 08/10/2011 11:20 AM, Eric Blake wrote:
On 08/10/2011 09:04 AM, Stefan Berger wrote:
On 08/10/2011 02:55 AM, Upendra Moturi wrote:
Hi,
does this patch solve your problem? I am setting the MTU to fixed 2kb. Doing tests with scp seems to indicate that this improved the situation -- at least for me.
Stefan
Signed-off-by: Stefan Berger <stefanb@linux.vnet.ibm.com>
--- src/util/network.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
Index: libvirt-acl/src/util/network.c =================================================================== --- libvirt-acl.orig/src/util/network.c +++ libvirt-acl/src/util/network.c @@ -1156,8 +1156,8 @@ virBandwidthEnable(virBandwidthPtr bandw
virCommandFree(cmd); cmd = virCommandNew(TC); - virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent", - "1:", "classid", "1:1", "htb", NULL); + virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent", + "1:", "classid", "1:1", "htb", NULL); virCommandAddArgList(cmd, "rate", average, NULL);
This hunk was just indentation; you should push the whitespace changes as a separate commit (and can push that now, under the trivial rule, without needing further review). [It doesn't help matters that thunderbird botched the whitespace in my reply]
@@ -1202,7 +1202,7 @@ virBandwidthEnable(virBandwidthPtr bandw virCommandAddArgList(cmd, "filter", "add", "dev", iface, "parent", "ffff:", "protocol", "ip", "u32", "match", "ip", "src", "0.0.0.0/0", "police", "rate", average, - "burst", burst, "mtu", burst, "drop", "flowid", + "burst", burst, "mtu", "2kb", "drop", "flowid", ":1", NULL);
Here's the meat of the proposed patch. Is 2kb always valid, or will we run into problems on NICs that insist on traditional limits near the 1518 mark? Should it be user-configurable? I'm reluctant to ACK this without a bit more understanding of why 2kb works, as well as feedback on test results with the patch in place.
Yes, I am not an exper on tc. I have seen examples with the mtu being set to '1' for dropping icmp traffic for example. So if a VM produces smaller than 2kb packets, which on traditional networks it probably should, then the packets should go through. We could also leave out the mtu parameter above and it would go to 2kb by default. Check via
tc filter show dev <ifname> root
The 2kb parameter doesn't explain why 2MB doesn't work, though. Even if this wasn't the final fix, it at least 'improves the situation' (for me).
Stefan
-- Thanks and Regards, Upendra.M
On 08/11/2011 08:07 AM, Upendra Moturi wrote:
Thanks for your reply.
The patch it appears still has problem. I have changed only mtu value as 2kb instead of burst value in the network.c file
With average as 2048 , burst as 2048 and peak 2048 scp copied at 30-40KB/s All test I did were with the patch applied. I saw the throughput
collapsing yesterday at 2Mb mtu.
I generated a file using dd if=/dev/urandom of=testfile bs=1024 count=409600 I did the same test you did with 2048 without specifying a specific model of ethernet card and reach 1.9MB/s using scp'ing towards remote machine. When specifying <model type='virtio'/> I also get 1.9MB/s as result.
With average as 4096 , burst as 4096 and peak 4096 scp copied at same 30-40KB/s
With these numbers and using the virtio model I ended up getting a throughput of 3.8Mb/s. The host is a 3.0.0 Linux kernel. Here's the command line output of the tc command inspecting the tap interface of the VM (vnet0). # tc filter show dev vnet0 root filter parent ffff: protocol ip pref 49152 u32 filter parent ffff: protocol ip pref 49152 u32 fh 800: ht divisor 1 filter parent ffff: protocol ip pref 49152 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 match 00000000/00000000 at 12 police 0x5b rate 32768Kbit burst 4Mb mtu 2Kb action drop overhead 0b ref 1 bind 1 Maybe someone else can verify as well. Stefan
where as earlier it was 1 MB/s.(without patch)
On Wed, Aug 10, 2011 at 9:16 PM, Stefan Berger <stefanb@linux.vnet.ibm.com> wrote:
On 08/10/2011 11:20 AM, Eric Blake wrote:
On 08/10/2011 02:55 AM, Upendra Moturi wrote:
Hi,
does this patch solve your problem? I am setting the MTU to fixed 2kb. Doing tests with scp seems to indicate that this improved the situation -- at least for me.
Stefan
Signed-off-by: Stefan Berger<stefanb@linux.vnet.ibm.com>
--- src/util/network.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
Index: libvirt-acl/src/util/network.c =================================================================== --- libvirt-acl.orig/src/util/network.c +++ libvirt-acl/src/util/network.c @@ -1156,8 +1156,8 @@ virBandwidthEnable(virBandwidthPtr bandw
virCommandFree(cmd); cmd = virCommandNew(TC); - virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent", - "1:", "classid", "1:1", "htb", NULL); + virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent", + "1:", "classid", "1:1", "htb", NULL); virCommandAddArgList(cmd, "rate", average, NULL); This hunk was just indentation; you should push the whitespace changes as a separate commit (and can push that now, under the trivial rule, without needing further review). [It doesn't help matters that thunderbird botched
On 08/10/2011 09:04 AM, Stefan Berger wrote: the whitespace in my reply]
@@ -1202,7 +1202,7 @@ virBandwidthEnable(virBandwidthPtr bandw virCommandAddArgList(cmd, "filter", "add", "dev", iface, "parent", "ffff:", "protocol", "ip", "u32", "match", "ip", "src", "0.0.0.0/0", "police", "rate", average, - "burst", burst, "mtu", burst, "drop", "flowid", + "burst", burst, "mtu", "2kb", "drop", "flowid", ":1", NULL); Here's the meat of the proposed patch. Is 2kb always valid, or will we run into problems on NICs that insist on traditional limits near the 1518 mark? Should it be user-configurable? I'm reluctant to ACK this without a bit more understanding of why 2kb works, as well as feedback on test results with the patch in place.
Yes, I am not an exper on tc. I have seen examples with the mtu being set to '1' for dropping icmp traffic for example. So if a VM produces smaller than 2kb packets, which on traditional networks it probably should, then the packets should go through. We could also leave out the mtu parameter above and it would go to 2kb by default. Check via
tc filter show dev<ifname> root
The 2kb parameter doesn't explain why 2MB doesn't work, though. Even if this wasn't the final fix, it at least 'improves the situation' (for me).
Stefan
On 08/11/2011 08:07 AM, Upendra Moturi wrote:
Thanks for your reply.
The patch it appears still has problem. I have changed only mtu value as 2kb instead of burst value in the network.c file
With average as 2048 , burst as 2048 and peak 2048 scp copied at 30-40KB/s All test I did were with the patch applied. I saw the throughput
collapsing yesterday at 2Mb mtu. Let me clarify : Yesterday I saw the throughput collapse without the
On 08/11/2011 09:39 AM, Stefan Berger wrote: patch applied. Stefan
I generated a file using
dd if=/dev/urandom of=testfile bs=1024 count=409600
I did the same test you did with 2048 without specifying a specific model of ethernet card and reach 1.9MB/s using scp'ing towards remote machine. When specifying <model type='virtio'/> I also get 1.9MB/s as result.
With average as 4096 , burst as 4096 and peak 4096 scp copied at same 30-40KB/s
With these numbers and using the virtio model I ended up getting a throughput of 3.8Mb/s.
The host is a 3.0.0 Linux kernel. Here's the command line output of the tc command inspecting the tap interface of the VM (vnet0).
# tc filter show dev vnet0 root filter parent ffff: protocol ip pref 49152 u32 filter parent ffff: protocol ip pref 49152 u32 fh 800: ht divisor 1 filter parent ffff: protocol ip pref 49152 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 match 00000000/00000000 at 12 police 0x5b rate 32768Kbit burst 4Mb mtu 2Kb action drop overhead 0b ref 1 bind 1
Maybe someone else can verify as well.
Stefan
where as earlier it was 1 MB/s.(without patch)
On Wed, Aug 10, 2011 at 9:16 PM, Stefan Berger <stefanb@linux.vnet.ibm.com> wrote:
On 08/10/2011 11:20 AM, Eric Blake wrote:
On 08/10/2011 02:55 AM, Upendra Moturi wrote:
Hi,
does this patch solve your problem? I am setting the MTU to fixed 2kb. Doing tests with scp seems to indicate that this improved the situation -- at least for me.
Stefan
Signed-off-by: Stefan Berger<stefanb@linux.vnet.ibm.com>
--- src/util/network.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
Index: libvirt-acl/src/util/network.c =================================================================== --- libvirt-acl.orig/src/util/network.c +++ libvirt-acl/src/util/network.c @@ -1156,8 +1156,8 @@ virBandwidthEnable(virBandwidthPtr bandw
virCommandFree(cmd); cmd = virCommandNew(TC); - virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent", - "1:", "classid", "1:1", "htb", NULL); + virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent", + "1:", "classid", "1:1", "htb", NULL); virCommandAddArgList(cmd, "rate", average, NULL); This hunk was just indentation; you should push the whitespace changes as a separate commit (and can push that now, under the trivial rule, without needing further review). [It doesn't help matters that thunderbird botched
On 08/10/2011 09:04 AM, Stefan Berger wrote: the whitespace in my reply]
@@ -1202,7 +1202,7 @@ virBandwidthEnable(virBandwidthPtr bandw virCommandAddArgList(cmd, "filter", "add", "dev", iface, "parent", "ffff:", "protocol", "ip", "u32", "match", "ip", "src", "0.0.0.0/0", "police", "rate", average, - "burst", burst, "mtu", burst, "drop", "flowid", + "burst", burst, "mtu", "2kb", "drop", "flowid", ":1", NULL); Here's the meat of the proposed patch. Is 2kb always valid, or will we run into problems on NICs that insist on traditional limits near the 1518 mark? Should it be user-configurable? I'm reluctant to ACK this without a bit more understanding of why 2kb works, as well as feedback on test results with the patch in place.
Yes, I am not an exper on tc. I have seen examples with the mtu being set to '1' for dropping icmp traffic for example. So if a VM produces smaller than 2kb packets, which on traditional networks it probably should, then the packets should go through. We could also leave out the mtu parameter above and it would go to 2kb by default. Check via
tc filter show dev<ifname> root
The 2kb parameter doesn't explain why 2MB doesn't work, though. Even if this wasn't the final fix, it at least 'improves the situation' (for me).
Stefan
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Thanks for your clarification. I am using 2.6.38-8-server kernel. Will test with 3.0.0 kernel On Thu, Aug 11, 2011 at 7:18 PM, Stefan Berger <stefanb@linux.vnet.ibm.com> wrote:
On 08/11/2011 09:39 AM, Stefan Berger wrote:
On 08/11/2011 08:07 AM, Upendra Moturi wrote:
Thanks for your reply.
The patch it appears still has problem. I have changed only mtu value as 2kb instead of burst value in the network.c file
With average as 2048 , burst as 2048 and peak 2048 scp copied at 30-40KB/s
All test I did were with the patch applied. I saw the throughput collapsing yesterday at 2Mb mtu.
Let me clarify : Yesterday I saw the throughput collapse without the patch applied.
Stefan
I generated a file using
dd if=/dev/urandom of=testfile bs=1024 count=409600
I did the same test you did with 2048 without specifying a specific model of ethernet card and reach 1.9MB/s using scp'ing towards remote machine. When specifying <model type='virtio'/> I also get 1.9MB/s as result.
With average as 4096 , burst as 4096 and peak 4096 scp copied at same 30-40KB/s
With these numbers and using the virtio model I ended up getting a throughput of 3.8Mb/s.
The host is a 3.0.0 Linux kernel. Here's the command line output of the tc command inspecting the tap interface of the VM (vnet0).
# tc filter show dev vnet0 root filter parent ffff: protocol ip pref 49152 u32 filter parent ffff: protocol ip pref 49152 u32 fh 800: ht divisor 1 filter parent ffff: protocol ip pref 49152 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 match 00000000/00000000 at 12 police 0x5b rate 32768Kbit burst 4Mb mtu 2Kb action drop overhead 0b ref 1 bind 1
Maybe someone else can verify as well.
Stefan
where as earlier it was 1 MB/s.(without patch)
On Wed, Aug 10, 2011 at 9:16 PM, Stefan Berger <stefanb@linux.vnet.ibm.com> wrote:
On 08/10/2011 11:20 AM, Eric Blake wrote:
On 08/10/2011 09:04 AM, Stefan Berger wrote:
On 08/10/2011 02:55 AM, Upendra Moturi wrote:
Hi,
does this patch solve your problem? I am setting the MTU to fixed 2kb. Doing tests with scp seems to indicate that this improved the situation -- at least for me.
Stefan
Signed-off-by: Stefan Berger<stefanb@linux.vnet.ibm.com>
--- src/util/network.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
Index: libvirt-acl/src/util/network.c =================================================================== --- libvirt-acl.orig/src/util/network.c +++ libvirt-acl/src/util/network.c @@ -1156,8 +1156,8 @@ virBandwidthEnable(virBandwidthPtr bandw
virCommandFree(cmd); cmd = virCommandNew(TC); - virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent", - "1:", "classid", "1:1", "htb", NULL); + virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent", + "1:", "classid", "1:1", "htb", NULL); virCommandAddArgList(cmd, "rate", average, NULL);
This hunk was just indentation; you should push the whitespace changes as a separate commit (and can push that now, under the trivial rule, without needing further review). [It doesn't help matters that thunderbird botched the whitespace in my reply]
@@ -1202,7 +1202,7 @@ virBandwidthEnable(virBandwidthPtr bandw virCommandAddArgList(cmd, "filter", "add", "dev", iface, "parent", "ffff:", "protocol", "ip", "u32", "match", "ip", "src", "0.0.0.0/0", "police", "rate", average, - "burst", burst, "mtu", burst, "drop", "flowid", + "burst", burst, "mtu", "2kb", "drop", "flowid", ":1", NULL);
Here's the meat of the proposed patch. Is 2kb always valid, or will we run into problems on NICs that insist on traditional limits near the 1518 mark? Should it be user-configurable? I'm reluctant to ACK this without a bit more understanding of why 2kb works, as well as feedback on test results with the patch in place.
Yes, I am not an exper on tc. I have seen examples with the mtu being set to '1' for dropping icmp traffic for example. So if a VM produces smaller than 2kb packets, which on traditional networks it probably should, then the packets should go through. We could also leave out the mtu parameter above and it would go to 2kb by default. Check via
tc filter show dev<ifname> root
The 2kb parameter doesn't explain why 2MB doesn't work, though. Even if this wasn't the final fix, it at least 'improves the situation' (for me).
Stefan
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
-- Thanks and Regards, Upendra.M
On 08/11/2011 09:54 AM, Upendra Moturi wrote:
Thanks for your clarification. I am using 2.6.38-8-server kernel. Will test with 3.0.0 kernel Also works on 2.6.38.8-35.fc15.x86_64 kernel.
Stefan
On Thu, Aug 11, 2011 at 7:18 PM, Stefan Berger <stefanb@linux.vnet.ibm.com> wrote:
On 08/11/2011 09:39 AM, Stefan Berger wrote:
On 08/11/2011 08:07 AM, Upendra Moturi wrote:
Thanks for your reply.
The patch it appears still has problem. I have changed only mtu value as 2kb instead of burst value in the network.c file
With average as 2048 , burst as 2048 and peak 2048 scp copied at 30-40KB/s All test I did were with the patch applied. I saw the throughput
collapsing yesterday at 2Mb mtu. Let me clarify : Yesterday I saw the throughput collapse without the patch applied.
Stefan
I generated a file using
dd if=/dev/urandom of=testfile bs=1024 count=409600
I did the same test you did with 2048 without specifying a specific model of ethernet card and reach 1.9MB/s using scp'ing towards remote machine. When specifying<model type='virtio'/> I also get 1.9MB/s as result.
With average as 4096 , burst as 4096 and peak 4096 scp copied at same 30-40KB/s
With these numbers and using the virtio model I ended up getting a throughput of 3.8Mb/s.
The host is a 3.0.0 Linux kernel. Here's the command line output of the tc command inspecting the tap interface of the VM (vnet0).
# tc filter show dev vnet0 root filter parent ffff: protocol ip pref 49152 u32 filter parent ffff: protocol ip pref 49152 u32 fh 800: ht divisor 1 filter parent ffff: protocol ip pref 49152 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 match 00000000/00000000 at 12 police 0x5b rate 32768Kbit burst 4Mb mtu 2Kb action drop overhead 0b ref 1 bind 1
Maybe someone else can verify as well.
Stefan
where as earlier it was 1 MB/s.(without patch)
On Wed, Aug 10, 2011 at 9:16 PM, Stefan Berger <stefanb@linux.vnet.ibm.com> wrote:
On 08/10/2011 11:20 AM, Eric Blake wrote:
On 08/10/2011 09:04 AM, Stefan Berger wrote: > On 08/10/2011 02:55 AM, Upendra Moturi wrote: > > Hi, > > does this patch solve your problem? I am setting the MTU to fixed 2kb. > Doing tests with scp seems to indicate that this improved the > situation > -- at least for me. > > Stefan > > Signed-off-by: Stefan Berger<stefanb@linux.vnet.ibm.com> > > --- > src/util/network.c | 16 ++++++++-------- > 1 file changed, 8 insertions(+), 8 deletions(-) > > Index: libvirt-acl/src/util/network.c > =================================================================== > --- libvirt-acl.orig/src/util/network.c > +++ libvirt-acl/src/util/network.c > @@ -1156,8 +1156,8 @@ virBandwidthEnable(virBandwidthPtr bandw > > virCommandFree(cmd); > cmd = virCommandNew(TC); > - virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent", > - "1:", "classid", "1:1", "htb", NULL); > + virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent", > + "1:", "classid", "1:1", "htb", NULL); > virCommandAddArgList(cmd, "rate", average, NULL); This hunk was just indentation; you should push the whitespace changes as a separate commit (and can push that now, under the trivial rule, without needing further review). [It doesn't help matters that thunderbird botched the whitespace in my reply]
> @@ -1202,7 +1202,7 @@ virBandwidthEnable(virBandwidthPtr bandw > virCommandAddArgList(cmd, "filter", "add", "dev", iface, "parent", > "ffff:", "protocol", "ip", "u32", "match", "ip", > "src", "0.0.0.0/0", "police", "rate", average, > - "burst", burst, "mtu", burst, "drop", "flowid", > + "burst", burst, "mtu", "2kb", "drop", "flowid", > ":1", NULL); Here's the meat of the proposed patch. Is 2kb always valid, or will we run into problems on NICs that insist on traditional limits near the 1518 mark? Should it be user-configurable? I'm reluctant to ACK this without a bit more understanding of why 2kb works, as well as feedback on test results with the patch in place.
Yes, I am not an exper on tc. I have seen examples with the mtu being set to '1' for dropping icmp traffic for example. So if a VM produces smaller than 2kb packets, which on traditional networks it probably should, then the packets should go through. We could also leave out the mtu parameter above and it would go to 2kb by default. Check via
tc filter show dev<ifname> root
The 2kb parameter doesn't explain why 2MB doesn't work, though. Even if this wasn't the final fix, it at least 'improves the situation' (for me).
Stefan
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
participants (3)
-
Eric Blake -
Stefan Berger -
Upendra Moturi