1:50 p.m.
Are there any plans to support configuring 802.1q VLAN tagging through
libvirt?
My understanding is that libvirt would use vconfig to create tagged
interfaces, while using a physical interface as trunk (e.g., eth0 is the
trunk, eth0.20 the interface with the '20' vlan tag).
Then it would add the tagged interface (eth0.20) to a bridge with the
guest virtual interface.
I don't know how libvirt would need to be extended to be able to
properly 'manage' those 802.1q-enabled virtual switches. Another
question would be how libvirt-CIM would support this.
Is anyone interested/working on this?
Thanks,
-Klaus
--
Klaus Heinrich Kiwi <klausk(a)linux.vnet.ibm.com>
Linux Security Development, IBM Linux Technology Center
1:52 p.m.
On Thu, Apr 02, 2009 at 03:50:32PM -0300, Klaus Heinrich Kiwi wrote:
Are there any plans to support configuring 802.1q VLAN tagging
through
libvirt?
My understanding is that libvirt would use vconfig to create tagged
interfaces, while using a physical interface as trunk (e.g., eth0 is the
trunk, eth0.20 the interface with the '20' vlan tag).
Yes, this type of configuration is within scope of the network
interface management APis, that Laine & David Lutterkort are
currently working on. THe underlying config will be done by netcf
library.
I don't know how libvirt would need to be extended to be able to
properly 'manage' those 802.1q-enabled virtual switches. Another
question would be how libvirt-CIM would support this.
Once libvirt provides the network interface functionality, then the
libvirt-CIM developers can map it into the appropriate DMTF schema
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
6:38 p.m.
> I don't know how libvirt would need to be extended to be able to
> properly 'manage' those 802.1q-enabled virtual switches. Another
> question would be how libvirt-CIM would support this.
Once libvirt provides the network interface functionality, then the
libvirt-CIM developers can map it into the appropriate DMTF schema
Right - the support for the VLANs would be similar to how libvirt-cim
represents the virtual network pools.
--
Kaitlin Rupert
IBM Linux Technology Center
kaitlin(a)linux.vnet.ibm.com
6:34 p.m.
Hi Klaus,
On Thu, 2009-04-02 at 15:50 -0300, Klaus Heinrich Kiwi wrote:
My understanding is that libvirt would use vconfig to create tagged
interfaces, while using a physical interface as trunk (e.g., eth0 is the
trunk, eth0.20 the interface with the '20' vlan tag).
Then it would add the tagged interface (eth0.20) to a bridge with the
guest virtual interface.
as Dan said, the actual functionality will be provided by netcf[1]
VLAN is very high on the list of things that need to be done next - and
any help with that would be much appreciated ;)
David
[1] https://fedorahosted.org/netcf/
4:39 p.m.
On Thu, 2009-04-02 at 16:34 -0700, David Lutterkort wrote:
Hi Klaus,
On Thu, 2009-04-02 at 15:50 -0300, Klaus Heinrich Kiwi wrote:
> My understanding is that libvirt would use vconfig to create tagged
> interfaces, while using a physical interface as trunk (e.g., eth0 is the
> trunk, eth0.20 the interface with the '20' vlan tag).
>
> Then it would add the tagged interface (eth0.20) to a bridge with the
> guest virtual interface.
I was thinking about the semantics I described above. It ultimately
means that we'll have a bridge for each VLAN tag that crosses the trunk
interface. So for example if guests A, B and C are all associated with
VLAN ID 20, then:
eth0 -> eth0.20 -> br0 -> [tap0, tap1, tap2]
(where tap[0-3] are associated with guests A, B, C respectively)
Adding a new guest to a VLAN would mean searching the host system and
check if there is already a bridge running on that VLAN ID. Create a new
one if needed, or use the existing one.
The things that concerns me the most are:
1) How scalable this really is
2) The semantics are really different from how physical, 802.1q-enabled
switches would work.
Because (2) really creates new switches for each new VLAN tag, I wonder
how management would be different from what we have today with physical
switches (i.e., defining a port with a VLAN ID, assigning that port to a
physical machine) - unless we hide it behind libvirt somehow.
Still, things like SNMP-management can have issues with such approach
(snmp-based network managers would go crazy identifying several
switches/bridges per box - not really useful from a management PoV).
Are there other options? Since a tagged interface like eth0.20 is kind
of a virtual interface itself, would it be appropriate to use those
directly?
Or maybe extending the existing bridging code to be 802.1q-aware?
Thanks,
-Klaus
as Dan said, the actual functionality will be provided by netcf[1]
VLAN is very high on the list of things that need to be done next - and
any help with that would be much appreciated ;)
David
[1] https://fedorahosted.org/netcf/
--
Klaus Heinrich Kiwi <klausk(a)linux.vnet.ibm.com>
Linux Security Development, IBM Linux Technology Center
4:44 p.m.
On Tue, Apr 07, 2009 at 06:39:17PM -0300, Klaus Heinrich Kiwi wrote:
I was thinking about the semantics I described above. It ultimately
I just caught this discussion. We have implemented basic vlan support
(for xend, bridged only). Is there an existing patch from you guys for
the user interface? Below is our current libvirt patch
regards
john
Support setting of vlan ID for network interfaces
Signed-off-by: John Levon <john.levon(a)sun.com>
Signed-off-by: Max Zhen <max.zhen(a)sun.com>
diff --git a/docs/schemas/domain.rng b/docs/schemas/domain.rng
--- a/docs/schemas/domain.rng
+++ b/docs/schemas/domain.rng
@@ -692,6 +692,13 @@
</optional>
</element>
</optional>
+ <optional>
+ <element name='vlan'>
+ <attribute name='id'>
+ <ref name='unsignedInt'/>
+ </attribute>
+ </element>
+ </optional>
</interleave>
</define>
diff --git a/src/domain_conf.c b/src/domain_conf.c
--- a/src/domain_conf.c
+++ b/src/domain_conf.c
@@ -918,6 +918,7 @@ virDomainNetDefParseXML(virConnectPtr co
char *address = NULL;
char *port = NULL;
char *model = NULL;
+ char *vlanid = NULL;
if (VIR_ALLOC(def) < 0) {
virReportOOMError(conn);
@@ -983,6 +984,8 @@ virDomainNetDefParseXML(virConnectPtr co
model = virXMLPropString(cur, "type");
} else if (xmlStrEqual (cur->name, BAD_CAST "networkresource"))
{
virDomainNetDefParseXMLRate(def, cur);
+ } else if (xmlStrEqual (cur->name, BAD_CAST "vlan")) {
+ vlanid = virXMLPropString(cur, "id");
}
}
cur = cur->next;
@@ -1093,6 +1096,16 @@ virDomainNetDefParseXML(virConnectPtr co
model = NULL;
}
+ def->vlanid = 0;
+
+ if (vlanid != NULL) {
+ if (virStrToLong_i(vlanid, NULL, 10, &def->vlanid) < 0) {
+ virDomainReportError(conn, VIR_ERR_INTERNAL_ERROR, "%s",
+ _("Cannot parse vlan ID"));
+ goto error;
+ }
+ }
+
cleanup:
VIR_FREE(macaddr);
VIR_FREE(network);
@@ -1104,6 +1117,7 @@ cleanup:
VIR_FREE(bridge);
VIR_FREE(model);
VIR_FREE(type);
+ VIR_FREE(vlanid);
return def;
@@ -3035,6 +3049,9 @@ virDomainNetDefFormat(virConnectPtr conn
unit, period, def->rate.value);
}
+ if (def->vlanid)
+ virBufferVSprintf(buf, " <vlan id='%d' />\n",
def->vlanid);
+
virBufferAddLit(buf, " </interface>\n");
return 0;
diff --git a/src/domain_conf.h b/src/domain_conf.h
--- a/src/domain_conf.h
+++ b/src/domain_conf.h
@@ -186,6 +186,7 @@ struct _virDomainNetDef {
int period;
long value;
} rate;
+ int vlanid;
};
enum virDomainChrSrcType {
diff --git a/src/virsh.c b/src/virsh.c
--- a/src/virsh.c
+++ b/src/virsh.c
@@ -4762,6 +4762,7 @@ static const vshCmdOptDef opts_attach_in
{"mac", VSH_OT_DATA, 0, gettext_noop("MAC address")},
{"script", VSH_OT_DATA, 0, gettext_noop("script used to bridge network
interface")},
{"capped-bandwidth", VSH_OT_STRING, 0, gettext_noop("bandwidth limit
for this interface")},
+ {"vlanid", VSH_OT_INT, 0, gettext_noop("VLAN ID attached to this
interface")},
{NULL, 0, 0, NULL}
};
@@ -4769,7 +4770,7 @@ cmdAttachInterface(vshControl *ctl, cons
cmdAttachInterface(vshControl *ctl, const vshCmd *cmd)
{
virDomainPtr dom = NULL;
- char *mac, *target, *script, *type, *source, *bw;
+ char *mac, *target, *script, *type, *source, *bw, *vlanid;
int typ, ret = FALSE;
char *buf = NULL, *tmp = NULL;
@@ -4787,6 +4788,7 @@ cmdAttachInterface(vshControl *ctl, cons
mac = vshCommandOptString(cmd, "mac", NULL);
script = vshCommandOptString(cmd, "script", NULL);
bw = vshCommandOptString(cmd, "capped-bandwidth", NULL);
+ vlanid = vshCommandOptString(cmd, "vlanid", NULL);
/* check interface type */
if (STREQ(type, "network")) {
@@ -4866,6 +4868,21 @@ cmdAttachInterface(vshControl *ctl, cons
tmp = vshRealloc(ctl, tmp, strlen(unit) + strlen(bw) + strlen(format));
if (!tmp) goto cleanup;
sprintf(tmp, format, unit, r);
+ buf = vshRealloc(ctl, buf, strlen(buf) + strlen(tmp) + 1);
+ if (!buf) goto cleanup;
+ strcat(buf, tmp);
+ }
+
+ if (vlanid != NULL) {
+ char *left;
+ long r = strtol(vlanid, &left, 10);
+ if ((r <= 0) || (r >= 4095) || (*left != '\0')) {
+ vshError(ctl, FALSE, _("Bad VLAN ID: %s in command
'attach-interface'"), vlanid);
+ goto cleanup;
+ }
+ tmp = vshRealloc(ctl, tmp, strlen(vlanid) + 20);
+ if (!tmp) goto cleanup;
+ sprintf(tmp, " <vlan id='%s'/>\n", vlanid);
buf = vshRealloc(ctl, buf, strlen(buf) + strlen(tmp) + 1);
if (!buf) goto cleanup;
strcat(buf, tmp);
diff --git a/src/xend_internal.c b/src/xend_internal.c
--- a/src/xend_internal.c
+++ b/src/xend_internal.c
@@ -1896,7 +1896,10 @@ xend_parse_rate(const char *ratestr, int
if (sscanf(ratestr, "%lu,%lu", &amount, &interval) == 2) {
*unit = VIR_DOMAIN_NET_RATE_KB;
*period = VIR_DOMAIN_NET_PERIOD_S;
- *val = (amount * 8) / (interval / 1000000);
+ /* bytes to kilobits */
+ *val = (amount / 125);
+ /* factor in period interval */
+ *val /= (interval / 1000000);
return 0;
}
@@ -2046,6 +2049,15 @@ xenDaemonParseSxprNets(virConnectPtr con
_("ignoring malformed rate limit
'%s'"), tmp);
} else {
net->rate.enabled = 1;
+ }
+ }
+
+ tmp = sexpr_node(node, "device/vif/vlanid");
+ if (tmp) {
+ if (virStrToLong_i(tmp, NULL, 0, &net->vlanid) != 0) {
+ virXendError(conn, VIR_ERR_INTERNAL_ERROR,
+ _("malformed vlanid '%s'"), tmp);
+ goto cleanup;
}
}
@@ -5553,6 +5565,9 @@ xenDaemonFormatSxprNet(virConnectPtr con
unit, period);
}
+ if (def->vlanid)
+ virBufferVSprintf(buf, "(vlanid '%d')", def->vlanid);
+
/*
* apparently (type ioemu) breaks paravirt drivers on HVM so skip this
* from Xen 3.1.0
8:32 p.m.
On Tue, 2009-04-07 at 18:39 -0300, Klaus Heinrich Kiwi wrote:
I was thinking about the semantics I described above. It ultimately
means that we'll have a bridge for each VLAN tag that crosses the trunk
interface. So for example if guests A, B and C are all associated with
VLAN ID 20, then:
eth0 -> eth0.20 -> br0 -> [tap0, tap1, tap2]
(where tap[0-3] are associated with guests A, B, C respectively)
Yes, I think that's how it should work; it would also mean that you'd
first set up eth0 as a separate interface, and new bridge/vlan interface
combos afterwards. AFAIK, for the bridge, only bootproto=none would make
sense.
The things that concerns me the most are:
1) How scalable this really is
I don't know either ... we'll find out ;)
2) The semantics are really different from how physical,
802.1q-enabled
switches would work.
Because (2) really creates new switches for each new VLAN tag, I wonder
how management would be different from what we have today with physical
switches (i.e., defining a port with a VLAN ID, assigning that port to a
physical machine) - unless we hide it behind libvirt somehow.
The reason we are creating all those bridges isn't the VLAN's - it's
that we want to share the same physical interface amongst several
guests. And I don't know of another way to do that.
Are there other options? Since a tagged interface like eth0.20 is
kind
of a virtual interface itself, would it be appropriate to use those
directly?
You can use it directly, I just don't know how else you would share it
amongst VM's without a bridge.
David
4:31 a.m.
On Tue, Apr 07, 2009 at 06:32:43PM -0700, David Lutterkort wrote:
On Tue, 2009-04-07 at 18:39 -0300, Klaus Heinrich Kiwi wrote:
> I was thinking about the semantics I described above. It ultimately
> means that we'll have a bridge for each VLAN tag that crosses the trunk
> interface. So for example if guests A, B and C are all associated with
> VLAN ID 20, then:
>
> eth0 -> eth0.20 -> br0 -> [tap0, tap1, tap2]
>
> (where tap[0-3] are associated with guests A, B, C respectively)
Yes, I think that's how it should work; it would also mean that you'd
first set up eth0 as a separate interface, and new bridge/vlan interface
combos afterwards. AFAIK, for the bridge, only bootproto=none would make
sense.
> The things that concerns me the most are:
> 1) How scalable this really is
I don't know either ... we'll find out ;)
I don't think that's really a scalability problem from libvirt's POV. I
know people use this setup quite widely already even with plain ifcfg-XXX
scripts. Any scalability problems nmost likely fall into the kernel /
networking code and whether it is good at avoiding unneccessary data
copies when you have stacked NIC -> VLAN -> BRIDGE -> TAP
> 2) The semantics are really different from how physical, 802.1q-enabled
> switches would work.
>
> Because (2) really creates new switches for each new VLAN tag, I wonder
> how management would be different from what we have today with physical
> switches (i.e., defining a port with a VLAN ID, assigning that port to a
> physical machine) - unless we hide it behind libvirt somehow.
I think one thing to consider is the difference between the physical and
logical models. The libvirt API / representation here is fairly low
level, dealing in individuals NICs. I think management apps would likely
want to present this in a slightly alternate way dealing more in logical
entities than physical NICs. eg oVirt's network model is closer to the
one you describe where the user defines a new switch for each VLAN tag.
It then maps this into the low level physical model of individual NICs
as needed. I think it si important that libvirt use the physical model
here to give apps flexibility in how they expose it to users.
The reason we are creating all those bridges isn't the VLAN's - it's
that we want to share the same physical interface amongst several
guests. And I don't know of another way to do that.
> Are there other options? Since a tagged interface like eth0.20 is kind
> of a virtual interface itself, would it be appropriate to use those
> directly?
You can use it directly, I just don't know how else you would share it
amongst VM's without a bridge.
In the (nearish) future NICs will start appearing with SR-IOV capabilities.
This gives you one physical PCI device, whcih exposes multiple functions.
So a single physical NIC appears as 8 NICs to the OS. You can thus directly
assign each of these virtual NICs to a different VM directly,a voiding the
need to bridge them.
I don't think its worth spending too much time trying to come up with other
non-bridged NIC sharing setups when hardware is about todo it all for us :-)
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
6:06 a.m.
Anno domini 2009 Daniel P. Berrange scripsit:
[...]
In the (nearish) future NICs will start appearing with SR-IOV
capabilities.
This gives you one physical PCI device, whcih exposes multiple functions.
So a single physical NIC appears as 8 NICs to the OS. You can thus directly
assign each of these virtual NICs to a different VM directly,a voiding the
need to bridge them.
I don't think its worth spending too much time trying to come up
with other
non-bridged NIC sharing setups when hardware is about todo it all for us :-)
Is this hope or do you know about any timelime for this?
Ciao
Max
--
Follow the white penguin.
6:09 a.m.
On Wed, Apr 08, 2009 at 01:06:48PM +0200, Maximilian Wilhelm wrote:
Anno domini 2009 Daniel P. Berrange scripsit:
[...]
> In the (nearish) future NICs will start appearing with SR-IOV capabilities.
> This gives you one physical PCI device, whcih exposes multiple functions.
> So a single physical NIC appears as 8 NICs to the OS. You can thus directly
> assign each of these virtual NICs to a different VM directly,a voiding the
> need to bridge them.
> I don't think its worth spending too much time trying to come up with other
> non-bridged NIC sharing setups when hardware is about todo it all for us :-)
Is this hope or do you know about any timelime for this?
Hardware exists today & patches are avialable for upstream Linux.
http://lwn.net/Articles/308238/
The question mark is more over how long it takes before the hardware is
widely available & shipped to consumers by vendors.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
6:13 a.m.
Anno domini 2009 Daniel P. Berrange scripsit:
On Wed, Apr 08, 2009 at 01:06:48PM +0200, Maximilian Wilhelm wrote:
> Anno domini 2009 Daniel P. Berrange scripsit:
That was a *really* _fast_ reply :)
> [...]
> > In the (nearish) future NICs will start appearing with SR-IOV capabilities.
> > This gives you one physical PCI device, whcih exposes multiple functions.
> > So a single physical NIC appears as 8 NICs to the OS. You can thus directly
> > assign each of these virtual NICs to a different VM directly,a voiding the
> > need to bridge them.
> > I don't think its worth spending too much time trying
to come up with other
> > non-bridged NIC sharing setups when hardware is about todo it all for us :-)
> Is this hope or do you know about any timelime for this?
Hardware exists today & patches are avialable for upstream Linux.
Oh, nice. That went below my radar.
The question mark is more over how long it takes before the hardware
is
widely available & shipped to consumers by vendors.
*nod*
Let's hope and see :)
Thanks for the info!
Ciao
Max
--
Follow the white penguin.
3:33 a.m.
On Wed, Apr 08, 2009 at 12:09:06PM +0100, Daniel P. Berrange wrote:
On Wed, Apr 08, 2009 at 01:06:48PM +0200, Maximilian Wilhelm wrote:
> Anno domini 2009 Daniel P. Berrange scripsit:
>
> [...]
> > In the (nearish) future NICs will start appearing with SR-IOV capabilities.
> > This gives you one physical PCI device, whcih exposes multiple functions.
> > So a single physical NIC appears as 8 NICs to the OS. You can thus directly
> > assign each of these virtual NICs to a different VM directly,a voiding the
> > need to bridge them.
>
> > I don't think its worth spending too much time trying to come up with
other
> > non-bridged NIC sharing setups when hardware is about todo it all for us :-)
>
> Is this hope or do you know about any timelime for this?
Hardware exists today & patches are avialable for upstream Linux.
http://lwn.net/Articles/308238/
The question mark is more over how long it takes before the hardware is
widely available & shipped to consumers by vendors.
The hardware is not appropriate for all environments. For example,
using the hardware removes the ability of the host to manage the
packet flow to the guest (filtering, bandwidth limits, etc.).
Domain migration is also complicated, though not impossible.
SR-IOV isn't the answer to all problems.
5688
days inactive
5713
days old
11 comments
7 participants
participants (7)
-
Daniel P. Berrange -
David Edmondson -
David Lutterkort -
John Levon -
Kaitlin Rupert -
Klaus Heinrich Kiwi -
Maximilian Wilhelm