12:33 p.m.
Recent autotest/virt-test testing on f20 discovered an anomaly in how
the bandwidth options are documented and used. This was discovered due
to a bug fix in the /sbin/tc utility found in iproute-3.11.0.1 (on f20)
in which overflow was actually caught and returned as an error. The fix
was first introduced in iproute-3.10 (search on iproute2 commit 'a303853e').
The autotest/virt-test test for virsh domiftune was attempting to send
the largest unsigned integer value (4294967295) for maximum value
testing. The libvirt xml implementation was designed to manage values
in kilobytes thus when this value was passed to /sbin/tc, it (now)
properly rejected the 4294967295kbps value.
Investigation of the problem discovered that formatdomain.html.in and
formatnetwork.html.in described the elements and property types slightly
differently, although they use the same code - virNetDevBandwidthParseRate()
(shared by portgroups, domains, and networks xml parsers). Rather than
have the descriptions in two places, this patch will combine and reword
the description under formatnetwork.html.in and have formatdomain.html.in
link to that description.
This documentation faux pas was continued into the virsh man page where
the bandwidth description for both 'attach-interface' and 'domiftune'
did not indicate the format of each value, thus leading to the test using
largest unsigned integer value assuming "bps" rather than "kbps",
which
ultimately was wrong.
Signed-off-by: John Ferlan <jferlan(a)redhat.com>
---
Changes since v1:
-> Reflection and some private IRC validation results in changing
to use "kilobyte" and not "kibibyte"
-> Although the smallest unit is 1 kb/kbps, it's felt that things
aren't headed in the going smaller direction, rather things are
desired to be larger. So the inability to "test" the maximum of
4294967295 bps in favor of 4294967 kbps shouldn't really matter.
docs/formatdomain.html.in | 35 +++-----------------
docs/formatnetwork.html.in | 81 ++++++++++++++++++++++++++++++++++------------
tools/virsh.pod | 10 ++++--
3 files changed, 73 insertions(+), 53 deletions(-)
diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in
index 4660983..ea1a97b 100644
--- a/docs/formatdomain.html.in
+++ b/docs/formatdomain.html.in
@@ -3750,37 +3750,10 @@ qemu-kvm -net nic,model=? /dev/null
<p>
This part of interface XML provides setting quality of service. Incoming
- and outgoing traffic can be shaped independently. The
- <code>bandwidth</code> element can have at most one
<code>inbound</code>
- and at most one <code>outbound</code> child elements. Leaving any of
these
- children element out result in no QoS applied on that traffic direction.
- So, when you want to shape only domain's incoming traffic, use
- <code>inbound</code> only, and vice versa. Each of these elements have
one
- mandatory attribute <code>average</code> (or
<code>floor</code> as
- described below). <code>average</code> specifies average bit rate on
- the interface being shaped. Then there are two optional attributes:
- <code>peak</code>, which specifies maximum rate at which interface can
send
- data, and <code>burst</code>, amount of bytes that can be burst at
- <code>peak</code> speed. Accepted values for attributes are integer
- numbers. The units for <code>average</code> and
<code>peak</code> attributes
- are kilobytes per second, and for the <code>burst</code> just
kilobytes.
- Note the limitation of implementation: the <code>peak</code> attribute
in
- <code>outbound</code> element is ignored (as linux ingress filters
don't
- know it yet). <span class="since">Since 0.9.4</span> The
<code>inbound</code> can
- optionally have <code>floor</code> attribute. This is there for
- guaranteeing minimal throughput for shaped interfaces. This, however,
- requires that all traffic goes through one point where QoS decisions can
- take place. That's why this attribute works only for virtual networks for
- now (that is <code><interface
type='network'/></code> with a
- forward type of route, nat, or no forward at all). Moreover, the
- virtual network the interface is connected to is required to have at least
- inbound QoS set (<code>average</code> at least). Moreover, with
- <code>floor</code> attribute users don't need to specify
- <code>average</code>. However, <code>peak</code> and
<code>burst</code>
- attributes still require <code>average</code>. Currently, linux kernel
- doesn't allow ingress qdiscs to have any classes therefore
- <code>floor</code> can be applied only on
<code>inbound</code> and not
- <code>outbound</code>. <span class="since">Since
1.0.1</span>
+ and outgoing traffic can be shaped independently.
+ The <code>bandwidth</code> element and its child elements are
described
+ in the <a href="formatnetwork.html#elementQoS">QoS</a>
section of
+ the Network XML.
</p>
<h5><a name="elementVlanTag">Setting VLAN tag (on supported
network types only)</a></h5>
diff --git a/docs/formatnetwork.html.in b/docs/formatnetwork.html.in
index 1ca1bec..d4c390a 100644
--- a/docs/formatnetwork.html.in
+++ b/docs/formatnetwork.html.in
@@ -412,40 +412,81 @@
<p>
The <code><bandwidth></code> element allows setting
- quality of service for a particular network.
- <span class="since">Since 0.9.4</span> The limits
specified
+ quality of service for a particular network
+ (<span class="since">since 0.9.4</span>). For a
<code>domain</code>
+ object, the limits specified are applied to the domain traffic.
+ For a <code>network</code> object, the limits specified
are applied to the aggregate of all traffic to/from all guest
interfaces attached to that network, <b>not</b> to each guest
- interface individually. Setting <code>bandwidth</code> for a
+ interface individually. Setting <code>bandwidth</code> for a
network is supported only for networks with
a <code><forward></code> mode
of <code>route</code>, <code>nat</code>, or no mode at
all
- (i.e. an "isolated" network). <code>bandwidth</code>
setting
+ (i.e. an "isolated" network). Setting
<code>bandwidth</code>
is <b>not</b> supported for forward modes
of <code>bridge</code>, <code>passthrough</code>,
<code>private</code>,
or <code>hostdev</code>, and attempts to do this will lead to
- a failure to define the network (or to create a transient
- network).
+ a failure to define the network (or to create a transient network).
</p>
<p>
Incoming and outgoing traffic can be shaped independently. The
- <code>bandwidth</code> element can have at most
- one <code>inbound</code> and at most one
<code>outbound</code>
+ <code>bandwidth</code> element can have at most one
+ <code>inbound</code> and at most one
<code>outbound</code>
child element. Leaving either of these children elements out
results in no QoS applied for that traffic direction. So,
- when you want to shape only a network's incoming traffic, use
+ when you want to shape only incoming traffic, use
<code>inbound</code> only, and vice versa. Each of these
- elements have one mandatory attribute - <code>average</code>,
- which specifies the desired average bit rate for the interface
- being shaped (in kibibytes/second). There are also two
- optional attributes: <code>peak</code>, which specifies the
- maximum rate at which the bridge can send data (again in
- kibibytes/second), and <code>burst</code> - the amount of
- bytes that can be transmitted in a single burst at
- <code>peak</code> speed (in kibibytes). Accepted values for
- attributes are integer numbers. The allotted bandwidth is
- shared equally between domains connected to the network.
+ elements have one mandatory attribute - <code>average</code> (or
+ <code>floor</code> as described below). The attributes are as
follows,
+ where accepted values for each attribute is an integer number.
</p>
+ <dl>
+ <dt><code>average</code></dt>
+ <dd>
+ Specifies the desired average bit rate for the interface
+ being shaped (in kilobytes/second).
+ </dd>
+ <dt><code>peak</code></dt>
+ <dd>
+ Optional attribute which specifies the maximum rate at
+ which the bridge can send data (in kilobytes/second).
+ Note the limitation of implementation: this attribute in the
+ <code>outbound</code> element is ignored (as Linux ingress
+ filters don't know it yet).
+ </dd>
+ <dt><code>burst</code></dt>
+ <dd>
+ Optional attribute which specifies the amount of kilobytes that
+ can be transmitted in a single burst at <code>peak</code> speed.
+ </dd>
+ <dt><code>floor</code></dt>
+ <dd>
+ Optional attribute available only for the <code>inbound</code>
+ element. This attribute guarantees minimal throughput for
+ shaped interfaces. This, however, requires that all traffic
+ goes through one point where QoS decisions can take place, hence
+ why this attribute works only for virtual networks for now
+ (that is <code><interface
type='network'/></code> with a
+ forward type of route, nat, or no forward at all). Moreover, the
+ virtual network the interface is connected to is required to have
+ at least inbound QoS set (<code>average</code> at least). If
+ using the <code>floor</code> attribute users don't need to
specify
+ <code>average</code>. However, <code>peak</code> and
+ <code>burst</code> attributes still require
<code>average</code>.
+ Currently, the Linux kernel doesn't allow ingress qdiscs to have
+ any classes therefore <code>floor</code> can be applied only
+ on <code>inbound</code> and not <code>outbound</code>.
+ </dd>
+ </dl>
+
+ <p>
+ Attributes <code>average</code>, <code>peak</code>, and
+ <code>burst</code> are available
+ <span class="since">since 0.9.4</span>, while the
+ <code>floor</code> attribute is available
+ <span class="since">since 1.0.1</span>.
+ </p>
+
<p>
A <code><portgroup></code> element can also include
a <code><bandwidth></code> element. In that case, the
@@ -561,7 +602,7 @@
network), and each portgroup has a name, as well as various
subelements associated with it. The currently supported
subelements are <code><bandwidth></code>
- (documented <a href="formatdomain.html#elementQoS">here</a>)
+ (described <a href="formatnetwork.html#elementQoS">here</a>)
and <code><virtualport></code>
(documented <a
href="formatdomain.html#elementsNICSDirect">here</a>).
If a domain interface definition specifies a portgroup (by
diff --git a/tools/virsh.pod b/tools/virsh.pod
index f221475..30dcbb8 100644
--- a/tools/virsh.pod
+++ b/tools/virsh.pod
@@ -680,7 +680,10 @@ or the MAC address.
If no I<--inbound> or I<--outbound> is specified, this command will
query and show the bandwidth settings. Otherwise, it will set the
inbound or outbound bandwidth. I<average,peak,burst> is the same as
-in command I<attach-interface>.
+in command I<attach-interface>. Values for I<average> and I<peak> are
+expressed in kilobytes per second, while I<burst> is expressed in kilobytes
+in a single burst at -I<peak> speed as described in the Network XML
+documentation at L<http://libvirt.org/formatnetwork.html#elementQoS>;.
If I<--live> is specified, affect a running guest.
If I<--config> is specified, affect the next boot of a persistent guest.
@@ -2019,7 +2022,10 @@ of the network interface. I<script> allows to specify a path
to a script
handling a bridge instead of the default one. I<model> allows to specify the
model type. I<inbound> and I<outbound> control the bandwidth of the
interface.
I<peak> and I<burst> are optional, so "average,peak",
"average,,burst" and
-"average" are also legal.
+"average" are also legal. Values for I<average> and I<peak> are
+expressed in kilobytes per second, while I<burst> is expressed in kilobytes
+in a single burst at -I<peak> speed as described in the Network XML
+documentation at L<http://libvirt.org/formatnetwork.html#elementQoS>;.
If I<--live> is specified, affect a running domain.
If I<--config> is specified, affect the next startup of a persistent domain.
--
1.8.4.2
8:53 a.m.
On 13.02.2014 19:33, John Ferlan wrote:
Recent autotest/virt-test testing on f20 discovered an anomaly in
how
the bandwidth options are documented and used. This was discovered due
to a bug fix in the /sbin/tc utility found in iproute-3.11.0.1 (on f20)
in which overflow was actually caught and returned as an error. The fix
was first introduced in iproute-3.10 (search on iproute2 commit 'a303853e').
The autotest/virt-test test for virsh domiftune was attempting to send
the largest unsigned integer value (4294967295) for maximum value
testing. The libvirt xml implementation was designed to manage values
in kilobytes thus when this value was passed to /sbin/tc, it (now)
properly rejected the 4294967295kbps value.
Investigation of the problem discovered that formatdomain.html.in and
formatnetwork.html.in described the elements and property types slightly
differently, although they use the same code - virNetDevBandwidthParseRate()
(shared by portgroups, domains, and networks xml parsers). Rather than
have the descriptions in two places, this patch will combine and reword
the description under formatnetwork.html.in and have formatdomain.html.in
link to that description.
This documentation faux pas was continued into the virsh man page where
the bandwidth description for both 'attach-interface' and 'domiftune'
did not indicate the format of each value, thus leading to the test using
largest unsigned integer value assuming "bps" rather than "kbps",
which
ultimately was wrong.
Signed-off-by: John Ferlan <jferlan(a)redhat.com>
---
Changes since v1:
-> Reflection and some private IRC validation results in changing
to use "kilobyte" and not "kibibyte"
-> Although the smallest unit is 1 kb/kbps, it's felt that things
aren't headed in the going smaller direction, rather things are
desired to be larger. So the inability to "test" the maximum of
4294967295 bps in favor of 4294967 kbps shouldn't really matter.
docs/formatdomain.html.in | 35 +++-----------------
docs/formatnetwork.html.in | 81 ++++++++++++++++++++++++++++++++++------------
tools/virsh.pod | 10 ++++--
3 files changed, 73 insertions(+), 53 deletions(-)
diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in
index 4660983..ea1a97b 100644
--- a/docs/formatdomain.html.in
+++ b/docs/formatdomain.html.in
@@ -3750,37 +3750,10 @@ qemu-kvm -net nic,model=? /dev/null
<p>
This part of interface XML provides setting quality of service. Incoming
- and outgoing traffic can be shaped independently. The
- <code>bandwidth</code> element can have at most one
<code>inbound</code>
- and at most one <code>outbound</code> child elements. Leaving any of
these
- children element out result in no QoS applied on that traffic direction.
- So, when you want to shape only domain's incoming traffic, use
- <code>inbound</code> only, and vice versa. Each of these elements have
one
- mandatory attribute <code>average</code> (or
<code>floor</code> as
- described below). <code>average</code> specifies average bit rate on
- the interface being shaped. Then there are two optional attributes:
- <code>peak</code>, which specifies maximum rate at which interface can
send
- data, and <code>burst</code>, amount of bytes that can be burst at
- <code>peak</code> speed. Accepted values for attributes are integer
- numbers. The units for <code>average</code> and
<code>peak</code> attributes
- are kilobytes per second, and for the <code>burst</code> just
kilobytes.
- Note the limitation of implementation: the <code>peak</code> attribute
in
- <code>outbound</code> element is ignored (as linux ingress filters
don't
- know it yet). <span class="since">Since 0.9.4</span> The
<code>inbound</code> can
- optionally have <code>floor</code> attribute. This is there for
- guaranteeing minimal throughput for shaped interfaces. This, however,
- requires that all traffic goes through one point where QoS decisions can
- take place. That's why this attribute works only for virtual networks for
- now (that is <code><interface
type='network'/></code> with a
- forward type of route, nat, or no forward at all). Moreover, the
- virtual network the interface is connected to is required to have at least
- inbound QoS set (<code>average</code> at least). Moreover, with
- <code>floor</code> attribute users don't need to specify
- <code>average</code>. However, <code>peak</code> and
<code>burst</code>
- attributes still require <code>average</code>. Currently, linux
kernel
- doesn't allow ingress qdiscs to have any classes therefore
- <code>floor</code> can be applied only on
<code>inbound</code> and not
- <code>outbound</code>. <span class="since">Since
1.0.1</span>
+ and outgoing traffic can be shaped independently.
+ The <code>bandwidth</code> element and its child elements are
described
+ in the <a href="formatnetwork.html#elementQoS">QoS</a>
section of
+ the Network XML.
</p>
<h5><a name="elementVlanTag">Setting VLAN tag (on supported
network types only)</a></h5>
diff --git a/docs/formatnetwork.html.in b/docs/formatnetwork.html.in
index 1ca1bec..d4c390a 100644
--- a/docs/formatnetwork.html.in
+++ b/docs/formatnetwork.html.in
@@ -412,40 +412,81 @@
<p>
The <code><bandwidth></code> element allows setting
- quality of service for a particular network.
- <span class="since">Since 0.9.4</span> The limits
specified
+ quality of service for a particular network
+ (<span class="since">since 0.9.4</span>). For a
<code>domain</code>
+ object, the limits specified are applied to the domain traffic.
I'm quite sure about the 'domain traffic'. The <bandwidth/> under domain
limits the particular <interface/> that has <bandwidth/>. Having 'domain
traffic' written here may sound like if the domain traffic was
aggregated and then shaped (which is done in network not in domain).
Maybe 'domain interface traffic'?
Other than that, the patch looks good.
Michal
9:45 a.m.
>
> <h5><a name="elementVlanTag">Setting VLAN tag (on
supported network types only)</a></h5>
> diff --git a/docs/formatnetwork.html.in b/docs/formatnetwork.html.in
> index 1ca1bec..d4c390a 100644
> --- a/docs/formatnetwork.html.in
> +++ b/docs/formatnetwork.html.in
> @@ -412,40 +412,81 @@
>
> <p>
> The <code><bandwidth></code> element allows
setting
> - quality of service for a particular network.
> - <span class="since">Since 0.9.4</span> The limits
specified
> + quality of service for a particular network
> + (<span class="since">since 0.9.4</span>). For a
<code>domain</code>
> + object, the limits specified are applied to the domain traffic.
I'm quite sure about the 'domain traffic'. The <bandwidth/> under
domain
limits the particular <interface/> that has <bandwidth/>. Having 'domain
traffic' written here may sound like if the domain traffic was
aggregated and then shaped (which is done in network not in domain).
Maybe 'domain interface traffic'?
Right - I struggled with this one when trying to merge things. Using
"domain interface traffic" does target a bit better. Of course the
context is discussing network bandwidth, so I guess I just assumed - hah
- that the reader would consider the domain traffic as the traffic for
that interface.
In the end, they both affect the same target interface; however, it
wasn't quite clear which would take precedence. If the interface had
one set of values on it, then the domain was defined with a different
set, then what happens? Or vice versa - a domain with a set of values
applied to an interface and then someone edits the network interface. It
seems it would be the "last" set to be applied that would "win".
Since this is an active topic for other recent patches, hopefully we can
come up with a wording that will stick going forward. I've copied Laine
on this response just to bring it to his attention and get his take as
well...
Tks,
John
4:58 a.m.
On 02/17/2014 05:45 PM, John Ferlan wrote:
>> <h5><a name="elementVlanTag">Setting VLAN tag (on
supported network types only)</a></h5>
>> diff --git a/docs/formatnetwork.html.in b/docs/formatnetwork.html.in
>> index 1ca1bec..d4c390a 100644
>> --- a/docs/formatnetwork.html.in
>> +++ b/docs/formatnetwork.html.in
>> @@ -412,40 +412,81 @@
>>
>> <p>
>> The <code><bandwidth></code> element allows
setting
>> - quality of service for a particular network.
>> - <span class="since">Since 0.9.4</span> The limits
specified
>> + quality of service for a particular network
>> + (<span class="since">since 0.9.4</span>). For a
<code>domain</code>
>> + object, the limits specified are applied to the domain traffic.
> I'm quite sure about the 'domain traffic'. The <bandwidth/> under
domain
> limits the particular <interface/> that has <bandwidth/>. Having
'domain
> traffic' written here may sound like if the domain traffic was
> aggregated and then shaped (which is done in network not in domain).
> Maybe 'domain interface traffic'?
>
Right - I struggled with this one when trying to merge things. Using
"domain interface traffic" does target a bit better. Of course the
context is discussing network bandwidth, so I guess I just assumed - hah
- that the reader would consider the domain traffic as the traffic for
that interface.
In the end, they both affect the same target interface; however, it
wasn't quite clear which would take precedence. If the interface had
one set of values on it, then the domain was defined with a different
set, then what happens?
That is a non-question, because there is no <bandwidth> subelement of a
<domain>! The <bandwidth> element can appear in 3 places:
1) as a subelement of a domain's <interface>. In this case, the
bandwidth applies to that one interface of that domain.
2) as a subelement of a <portgroup> in a <network>. In this case, if a
domain's <interface> has a "portgroup='xxx'" attribute in its
<source>
element:
<interface type='network'>
<source network='testnet' portgroup='admin'/>
...
*and* if the <interface itself has no <bandwidth> element, then the
<bandwidth> element of the portgroup is applied to that interface.
3) as a subelement of a <network>. In this case, the bandwidth specified
is intended to be to total aggregate bandwidth of all guest interfaces
attached to that network.
So, as you can see, there is no concept of "the bandwidth of a domain".
The places where there may be a question of precendence:
1) if an <interface> has a <bandwidth> *and* it belongs to a portgroup
that has a <bandwidth>. In this case, we decided that the bandwidth
under the individual <interface> would take precedence, since at the
time anyone who had permission to edit the domain (and its <interface>)
also had permission to edit the network (and its <portgroup>); so
allowing the specific config for one interface to override the general
config for many interfaces was more useful.
2) if a <network> has a bandwidth defined for the aggregate of all
interfaces attached to that network, and one interface has bandwidth
defined that is higher than the aggregate for the entire network. In
this case, the aggregate bandwidth for the network takes precedence, not
because of anything we do, but just because the two choke points are
independent of each other (the <interface> bandwidth control happens on
the interface's tap device, and the <network> bandwidth control happens
on the interface part of the bridge device created for that network).
Did you maybe mean to say
If the interface had one set of values on it, then the *NETWORK*
was defined with a different set, then what happens?
? If so, then I believe (2) above explains in (rather obtusely, of
course, but I hope it's possible to decipher it)
Or vice versa - a domain with a set of values
applied to an interface and then someone edits the network interface. It
seems it would be the "last" set to be applied that would "win".
What is "interface" vs. "network interface"? I can't tell if
you're
talking about the same thing with two slightly different names, or
incorrectly naming something in the <network> configuration.
But if you're talking about interface bandwidth vs. network aggregate
bandwidth, the way that it works is that both sets of rules are
installed, and the traffic for that guest's interface is shaped twice -
once as it goes in/out of the guest interface's tap device, then the
same traffic is shaped again (in aggregate with all other traffic for
the network) as it goes in/out of the interface that is a part of the
network's bridge device.
Since this is an active topic for other recent patches, hopefully we
can
come up with a wording that will stick going forward. I've copied Laine
on this response just to bring it to his attention and get his take as
well...
Although I understand the desire to eliminate nearly duplicated wording,
I'm a bit concerned about trying to combine two things that are almost
but not entirely the same, due to the potential of the combined
information being wrong in one of both cases, or at least becoming
unnecessarily confusing due to extra "exceptions" that are needed to
explain the differences. In spite of that, I'm fine with it being
combined - if people complain, we can always split it up again.
I can never really form a good opinion about documentation without
seeing it live on the web page and hearing how others interpret it, and
my burn-out level on documentation has lately led to my opinion either
being "yeah, that's okay", or "no, I don't like that, but I
don't know
how to / don't have time to fix it" :-P
(The recent change to the information for <bandwidth> in the network
documentation was a side effect of me making a small change to
validation of network xml to disallow specifying <bandwidth> on network
types where we can't support setting aggregate bandwidth for the entire
network. While I was doing that, I noticed that the documentation about
<bandwidth> in formatnetwork.html.in was obtuse/incorrect, so I made
some changes to fix those things; I didn't think to look at the
documentation for <bandwidth> in the domain (<interface>) XML docs,
because I hadn't changed anything there.)
11:16 a.m.
On 02/18/2014 05:58 AM, Laine Stump wrote:
On 02/17/2014 05:45 PM, John Ferlan wrote:
>
>
>>> <h5><a name="elementVlanTag">Setting VLAN tag (on
supported network types only)</a></h5>
>>> diff --git a/docs/formatnetwork.html.in b/docs/formatnetwork.html.in
>>> index 1ca1bec..d4c390a 100644
>>> --- a/docs/formatnetwork.html.in
>>> +++ b/docs/formatnetwork.html.in
>>> @@ -412,40 +412,81 @@
>>>
>>> <p>
>>> The <code><bandwidth></code> element
allows setting
>>> - quality of service for a particular network.
>>> - <span class="since">Since 0.9.4</span> The
limits specified
>>> + quality of service for a particular network
>>> + (<span class="since">since 0.9.4</span>). For
a <code>domain</code>
>>> + object, the limits specified are applied to the domain traffic.
>> I'm quite sure about the 'domain traffic'. The <bandwidth/>
under domain
>> limits the particular <interface/> that has <bandwidth/>. Having
'domain
>> traffic' written here may sound like if the domain traffic was
>> aggregated and then shaped (which is done in network not in domain).
>> Maybe 'domain interface traffic'?
>>
> Right - I struggled with this one when trying to merge things. Using
> "domain interface traffic" does target a bit better. Of course the
> context is discussing network bandwidth, so I guess I just assumed - hah
> - that the reader would consider the domain traffic as the traffic for
> that interface.
>
> In the end, they both affect the same target interface; however, it
> wasn't quite clear which would take precedence. If the interface had
> one set of values on it, then the domain was defined with a different
> set, then what happens?
That is a non-question, because there is no <bandwidth> subelement of a
<domain>! The <bandwidth> element can appear in 3 places:
1) as a subelement of a domain's <interface>. In this case, the
bandwidth applies to that one interface of that domain.
2) as a subelement of a <portgroup> in a <network>. In this case, if a
domain's <interface> has a "portgroup='xxx'" attribute in its
<source>
element:
<interface type='network'>
<source network='testnet' portgroup='admin'/>
...
*and* if the <interface itself has no <bandwidth> element, then the
<bandwidth> element of the portgroup is applied to that interface.
3) as a subelement of a <network>. In this case, the bandwidth specified
is intended to be to total aggregate bandwidth of all guest interfaces
attached to that network.
So, as you can see, there is no concept of "the bandwidth of a domain".
Semantics - I suppose. The "whole" network vs. the "slice" for the
domain. Trying to understand while flipping between two pages and using
"improper" or "inexact" words. I guess what I meant was the bandwidth
assigned to the interface of a domain vs. the bandwidth assigned to the
whole network. In any case, you've answered my questions below...
The places where there may be a question of precendence:
1) if an <interface> has a <bandwidth> *and* it belongs to a portgroup
that has a <bandwidth>. In this case, we decided that the bandwidth
under the individual <interface> would take precedence, since at the
time anyone who had permission to edit the domain (and its <interface>)
also had permission to edit the network (and its <portgroup>); so
allowing the specific config for one interface to override the general
config for many interfaces was more useful.
2) if a <network> has a bandwidth defined for the aggregate of all
interfaces attached to that network, and one interface has bandwidth
defined that is higher than the aggregate for the entire network. In
this case, the aggregate bandwidth for the network takes precedence, not
because of anything we do, but just because the two choke points are
independent of each other (the <interface> bandwidth control happens on
the interface's tap device, and the <network> bandwidth control happens
on the interface part of the bridge device created for that network).
Did you maybe mean to say
If the interface had one set of values on it, then the *NETWORK*
was defined with a different set, then what happens?
? If so, then I believe (2) above explains in (rather obtusely, of
course, but I hope it's possible to decipher it)
Your (2) is closer to what I was going for... and yes it makes sense.
The whole portgroup discussion (1) adds complexity to things, but it is
described in the formatnetwork page, although it was in a "different"
paragraph which is why I ignored it at first. I've since moved it (see
below)
> Or vice versa - a domain with a set of values
> applied to an interface and then someone edits the network interface. It
> seems it would be the "last" set to be applied that would "win".
What is "interface" vs. "network interface"? I can't tell if
you're
talking about the same thing with two slightly different names, or
incorrectly naming something in the <network> configuration.
Again some semantics, but it's becoming more clear now - thanks. The
reference to "network interface" was the "<network>" object as
opposed
to "an interface" which was the "<domain><interface>"
object (again I
may not be as semantically correct as you expect, but hopefully you see
what I was intending).
But if you're talking about interface bandwidth vs. network aggregate
bandwidth, the way that it works is that both sets of rules are
installed, and the traffic for that guest's interface is shaped twice -
once as it goes in/out of the guest interface's tap device, then the
same traffic is shaped again (in aggregate with all other traffic for
the network) as it goes in/out of the interface that is a part of the
network's bridge device.
A discussion which I think would be good to add to formatnetwork...
> Since this is an active topic for other recent patches, hopefully
we can
> come up with a wording that will stick going forward. I've copied Laine
> on this response just to bring it to his attention and get his take as
> well...
>
Although I understand the desire to eliminate nearly duplicated wording,
I'm a bit concerned about trying to combine two things that are almost
but not entirely the same, due to the potential of the combined
information being wrong in one of both cases, or at least becoming
unnecessarily confusing due to extra "exceptions" that are needed to
explain the differences. In spite of that, I'm fine with it being
combined - if people complain, we can always split it up again.
Understood; however, considering your last paragraph regarding missing
or not thinking about formatdomain is just cause enough to try and put
the discussion in one place. Keeping things in two places usually
results in one being forgotten during editing. I used cscope to find
relevant bandwidth discussions...
I can never really form a good opinion about documentation without
seeing it live on the web page and hearing how others interpret it, and
my burn-out level on documentation has lately led to my opinion either
being "yeah, that's okay", or "no, I don't like that, but I
don't know
how to / don't have time to fix it" :-P
(The recent change to the information for <bandwidth> in the network
documentation was a side effect of me making a small change to
validation of network xml to disallow specifying <bandwidth> on network
types where we can't support setting aggregate bandwidth for the entire
network. While I was doing that, I noticed that the documentation about
<bandwidth> in formatnetwork.html.in was obtuse/incorrect, so I made
some changes to fix those things; I didn't think to look at the
documentation for <bandwidth> in the domain (<interface>) XML docs,
because I hadn't changed anything there.)
Because I know it's easier to read without all the git +/-, here's what
I have now in the QoS section on the formatnetwork (hopefully I've
captured everything correctly):
The <bandwidth> element allows setting quality of service for a
particular network (since 0.9.4). Setting bandwidth for a network is
supported only for networks with a <forward> mode of route, nat, or no
mode at all (i.e. an "isolated" network). Setting bandwidth is not
supported for forward modes of bridge, passthrough, private, or hostdev.
Attempts to do this will lead to a failure to define the network or to
create a transient network.
The <bandwidth> element can only be a subelement of a domain's
<interface>, a subelement of a <network>, or a subelement of a
<portgroup> in a <network>.
As a subelement of a domain's <interface>, the bandwidth only applies to
that one interface of the domain. As a subelement of a <network>, the
bandwidth is a total aggregate bandwidth to/from all guest interfaces
attached to that network, not to each guest interface individually. If a
domain's <interface> has <bandwidth> element values higher than the
aggregate for the entire network, then the aggregate bandwidth for the
<network> takes precedence. This is because the two choke points are
independent of each other where the domain's <interface> bandwidth
control is applied on the interface's tap device, while the <network>
bandwidth control is applied on the interface part of the bridge device
created for that network.
As a subelement of a <portgroup> in a <network>, if a domain's
<interface> has a portgroup attribute in its <source> element and if the
<interface> itself has no <bandwidth> element, then the <bandwidth>
element of the portgroup will be applied individually to each guest
interface defined to be a member of that portgroup. Any <bandwidth>
element in the domain's <interface> definition will override the setting
in the portgroup (since 1.0.1).
Incoming and outgoing traffic can be shaped independently. The bandwidth
element can have at most one inbound and at most one outbound child
element. Leaving either of these children elements out results in no QoS
applied for that traffic direction. So, when you want to shape only
incoming traffic, use inbound only, and vice versa. Each of these
elements have one mandatory attribute - average (or floor as described
below). The attributes are as follows, where accepted values for each
attribute is an integer number.
average
Specifies the desired average bit rate for the interface being
shaped (in kilobytes/second).
peak
Optional attribute which specifies the maximum rate at which the
bridge can send data (in kilobytes/second). Note the limitation of
implementation: this attribute in the outbound element is ignored (as
Linux ingress filters don't know it yet).
burst
Optional attribute which specifies the amount of kilobytes that can
be transmitted in a single burst at peak speed.
floor
Optional attribute available only for the inbound element. This
attribute guarantees minimal throughput for shaped interfaces. This,
however, requires that all traffic goes through one point where QoS
decisions can take place, hence why this attribute works only for
virtual networks for now (that is <interface type='network'/> with a
forward type of route, nat, or no forward at all). Moreover, the virtual
network the interface is connected to is required to have at least
inbound QoS set (average at least). If using the floor attribute users
don't need to specify average. However, peak and burst attributes still
require average. Currently, the Linux kernel doesn't allow ingress
qdiscs to have any classes therefore floor can be applied only on
inbound and not outbound.
Attributes average, peak, and burst are available since 0.9.4, while the
floor attribute is available since 1.0.1.
8:47 a.m.
On 02/18/2014 07:16 PM, John Ferlan wrote:
Because I know it's easier to read without all the git +/-,
here's what
I have now in the QoS section on the formatnetwork (hopefully I've
captured everything correctly):
This all looks fine. ACK.
The <bandwidth> element allows setting quality of service for a
particular network (since 0.9.4). Setting bandwidth for a network is
supported only for networks with a <forward> mode of route, nat, or no
mode at all (i.e. an "isolated" network). Setting bandwidth is not
supported for forward modes of bridge, passthrough, private, or hostdev.
Attempts to do this will lead to a failure to define the network or to
create a transient network.
The <bandwidth> element can only be a subelement of a domain's
<interface>, a subelement of a <network>, or a subelement of a
<portgroup> in a <network>.
As a subelement of a domain's <interface>, the bandwidth only applies to
that one interface of the domain. As a subelement of a <network>, the
bandwidth is a total aggregate bandwidth to/from all guest interfaces
attached to that network, not to each guest interface individually. If a
domain's <interface> has <bandwidth> element values higher than the
aggregate for the entire network, then the aggregate bandwidth for the
<network> takes precedence. This is because the two choke points are
independent of each other where the domain's <interface> bandwidth
control is applied on the interface's tap device, while the <network>
bandwidth control is applied on the interface part of the bridge device
created for that network.
As a subelement of a <portgroup> in a <network>, if a domain's
<interface> has a portgroup attribute in its <source> element and if the
<interface> itself has no <bandwidth> element, then the <bandwidth>
element of the portgroup will be applied individually to each guest
interface defined to be a member of that portgroup. Any <bandwidth>
element in the domain's <interface> definition will override the setting
in the portgroup (since 1.0.1).
Incoming and outgoing traffic can be shaped independently. The bandwidth
element can have at most one inbound and at most one outbound child
element. Leaving either of these children elements out results in no QoS
applied for that traffic direction. So, when you want to shape only
incoming traffic, use inbound only, and vice versa. Each of these
elements have one mandatory attribute - average (or floor as described
below). The attributes are as follows, where accepted values for each
attribute is an integer number.
average
Specifies the desired average bit rate for the interface being
shaped (in kilobytes/second).
peak
Optional attribute which specifies the maximum rate at which the
bridge can send data (in kilobytes/second). Note the limitation of
implementation: this attribute in the outbound element is ignored (as
Linux ingress filters don't know it yet).
burst
Optional attribute which specifies the amount of kilobytes that can
be transmitted in a single burst at peak speed.
floor
Optional attribute available only for the inbound element. This
attribute guarantees minimal throughput for shaped interfaces. This,
however, requires that all traffic goes through one point where QoS
decisions can take place, hence why this attribute works only for
virtual networks for now (that is <interface type='network'/> with a
forward type of route, nat, or no forward at all). Moreover, the virtual
network the interface is connected to is required to have at least
inbound QoS set (average at least). If using the floor attribute users
don't need to specify average. However, peak and burst attributes still
require average. Currently, the Linux kernel doesn't allow ingress
qdiscs to have any classes therefore floor can be applied only on
inbound and not outbound.
Attributes average, peak, and burst are available since 0.9.4, while the
floor attribute is available since 1.0.1.
3928
days inactive
3935
days old
5 comments
3 participants
participants (3)
-
John Ferlan -
Laine Stump -
Michal Privoznik