9:03 p.m.
In Xen it is possible to configure a VIF to enslave with an arbitrary
bridge device using a fragment of XML like:
<interface type='bridge'>
<source bridge='xenbr0'/>
<mac address='00:16:3E:66:94:9C'/>
</interface>
In this example I've enslaved to xenbr0 which is created by the Xen networking
script, but this will work with any bridge device really.
In QEMU the only bridging is available via the 'virtual networks' support
in libvirt, by referencing a named bridge. For example
<interface type='network'>
<source network='default'/>
<mac address='00:16:3E:66:94:9C'/>
</interface>
Now 'default' is merely defined to be backed by a bridge device 'virbr0'
(although it does not neccessary get backed by a bridge) so the basic
impl is basically just enslaving a TAP device.
Anyway, we now have two different syntaxes in the XML for defining what
is more or less the same thing which is a real PITA for the tools. It
is useful to have the distinct representations - one is for 'managed'
bridges, and the other is for 'unmanaged' bridges - we can do some much
more interesting things with managed bridges.
To improve life for the tools though I'd like to do two things as a high
priority
- Support managed networks in Xen - we can simply lookup the bridge
device associated with a network and tell Xen to use vif-bridge with
this device. For the reverse Xen will tell us what bridge device a
guest is using, and we can reverse lookup the corresponding network
- Support non-managed networks in QEMU - we can simply enslave the
QEMU tap device to an arbitrary user specific bridge device.
Until one of these two options is done, its going to be a PITA to support
networking across QEMU & Xen in virt-manager/virt-install.
Regards,
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
8:27 a.m.
On Fri, 2007-03-09 at 03:03 +0000, Daniel P. Berrange wrote:
Anyway, we now have two different syntaxes in the XML for defining
what
is more or less the same thing which is a real PITA for the tools. It
is useful to have the distinct representations - one is for 'managed'
bridges, and the other is for 'unmanaged' bridges - we can do some much
more interesting things with managed bridges.
As I said in another mail, the way I'd imagined virt-install working
would be that by default it would connect guest NICs to the default
network (i.e. not the bridge associated with the default network), but
you could still use --bridge to explicitly connect the guest to a
bridge. And we'd have a --network option.
To improve life for the tools though I'd like to do two things as
a high
priority
- Support managed networks in Xen - we can simply lookup the bridge
device associated with a network and tell Xen to use vif-bridge with
this device.
We already have this in libvirt. See virDomainParseXMLIfDesc()
For the reverse Xen will tell us what bridge device a
guest is using, and we can reverse lookup the corresponding network
I'm not sure why you want this?
- Support non-managed networks in QEMU - we can simply enslave the
QEMU tap device to an arbitrary user specific bridge device.
Yes, we should definitely have this.
Cheers,
Mark.
8:32 a.m.
On Fri, Mar 09, 2007 at 02:27:02PM +0000, Mark McLoughlin wrote:
On Fri, 2007-03-09 at 03:03 +0000, Daniel P. Berrange wrote:
> To improve life for the tools though I'd like to do two things as a high
> priority
>
> - Support managed networks in Xen - we can simply lookup the bridge
> device associated with a network and tell Xen to use vif-bridge with
> this device.
We already have this in libvirt. See virDomainParseXMLIfDesc()
Ahh I missed that. I will have to write some tests for that I think,
although it'll require extending the test:///default driver to support
the networking APIs :-)
> For the reverse Xen will tell us what bridge device a
> guest is using, and we can reverse lookup the corresponding network
I'm not sure why you want this?
Without this you be sending in
<interface type='network'>
<source network='foo'>
</interface>
And be getting back
<interface type='bridge'>
<source bridge='virbr0'>
</interface>
As far as is possible the XML used to create a VM should be identical
to that you get back from libvirt once it is running.
> - Support non-managed networks in QEMU - we can simply enslave
the
> QEMU tap device to an arbitrary user specific bridge device.
Yes, we should definitely have this.
Ok, I'll add that - it should be trivial since the tricky enslaving
code is already there.
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
10:20 a.m.
On Fri, 2007-03-09 at 14:32 +0000, Daniel P. Berrange wrote:
On Fri, Mar 09, 2007 at 02:27:02PM +0000, Mark McLoughlin wrote:
> On Fri, 2007-03-09 at 03:03 +0000, Daniel P. Berrange wrote:
> > For the reverse Xen will tell us what bridge device a
> > guest is using, and we can reverse lookup the corresponding network
>
> I'm not sure why you want this?
Without this you be sending in
<interface type='network'>
<source network='foo'>
</interface>
And be getting back
<interface type='bridge'>
<source bridge='virbr0'>
</interface>
As far as is possible the XML used to create a VM should be identical
to that you get back from libvirt once it is running.
Absolutely, very good point.
There's a few places where we return the current running state rather
than the initial configuration we provided, but this would be a
particularly bad case.
It's a bit fugly, though - should libvirt do ListDomains(),
GetBridgName() and match the bridge name, or should we have
virNetworkLookupByBridge() and make qemud do the lookup itself?
Cheers,
Mark.
10:22 a.m.
On Fri, Mar 09, 2007 at 04:20:58PM +0000, Mark McLoughlin wrote:
On Fri, 2007-03-09 at 14:32 +0000, Daniel P. Berrange wrote:
> On Fri, Mar 09, 2007 at 02:27:02PM +0000, Mark McLoughlin wrote:
> > On Fri, 2007-03-09 at 03:03 +0000, Daniel P. Berrange wrote:
> > > For the reverse Xen will tell us what bridge device a
> > > guest is using, and we can reverse lookup the corresponding network
> >
> > I'm not sure why you want this?
>
> Without this you be sending in
>
> <interface type='network'>
> <source network='foo'>
> </interface>
>
> And be getting back
>
> <interface type='bridge'>
> <source bridge='virbr0'>
> </interface>
>
> As far as is possible the XML used to create a VM should be identical
> to that you get back from libvirt once it is running.
Absolutely, very good point.
There's a few places where we return the current running state rather
than the initial configuration we provided, but this would be a
particularly bad case.
It's a bit fugly, though - should libvirt do ListDomains(),
GetBridgName() and match the bridge name, or should we have
virNetworkLookupByBridge() and make qemud do the lookup itself?
I was actually thinking virNetworkLookupByBridge could be a very handy
API to have
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
8:40 a.m.
On Fri, Mar 09, 2007 at 02:27:02PM +0000, Mark McLoughlin wrote:
On Fri, 2007-03-09 at 03:03 +0000, Daniel P. Berrange wrote:
> Anyway, we now have two different syntaxes in the XML for defining what
> is more or less the same thing which is a real PITA for the tools. It
> is useful to have the distinct representations - one is for 'managed'
> bridges, and the other is for 'unmanaged' bridges - we can do some much
> more interesting things with managed bridges.
As I said in another mail, the way I'd imagined virt-install working
would be that by default it would connect guest NICs to the default
network (i.e. not the bridge associated with the default network), but
you could still use --bridge to explicitly connect the guest to a
bridge. And we'd have a --network option.
I think we need to explore this a little further - the APIs for virtual
networks are very nice froma tools POV because we can trivially enumerate
networks & connect VMs to them. Its not pleasent to have to enumerate the
regular non-virtual network devices using a different set of APIs - indeed
its impossible if we're talking to a remote libvirt. So I think we need to
look at also exposing the 'shared physical device' style networks via the
libvirt APIs too.
This could be expressed with something like:
<network>
<name>default</name>
<bridge name="virbr0" />
<device name="eth0" />
</network>
Which would setup a virbr0 bridge,and enslave eth0 to it. It would assume
that the network eth0 is connected to has DHCP already, so there's no need
to define an IP for the bridge itself, or run dnsmasq in this case.
Is it actually possible to take an already configured eth0 device and
enslave it to virbr0 without eth0 loosing its address config/routes ?
If we have this kind of support could we auto-generate network configs at
runtime based on any bridge/network device the adminsitrator has already
manually configured via /etc/sysconfig/networking-scripts ?
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
10:44 a.m.
On Fri, 2007-03-09 at 14:40 +0000, Daniel P. Berrange wrote:
On Fri, Mar 09, 2007 at 02:27:02PM +0000, Mark McLoughlin wrote:
> On Fri, 2007-03-09 at 03:03 +0000, Daniel P. Berrange wrote:
>
> > Anyway, we now have two different syntaxes in the XML for defining what
> > is more or less the same thing which is a real PITA for the tools. It
> > is useful to have the distinct representations - one is for 'managed'
> > bridges, and the other is for 'unmanaged' bridges - we can do some
much
> > more interesting things with managed bridges.
>
> As I said in another mail, the way I'd imagined virt-install working
> would be that by default it would connect guest NICs to the default
> network (i.e. not the bridge associated with the default network), but
> you could still use --bridge to explicitly connect the guest to a
> bridge. And we'd have a --network option.
I think we need to explore this a little further - the APIs for virtual
networks are very nice froma tools POV because we can trivially enumerate
networks & connect VMs to them. Its not pleasent to have to enumerate the
regular non-virtual network devices using a different set of APIs - indeed
its impossible if we're talking to a remote libvirt. So I think we need to
look at also exposing the 'shared physical device' style networks via the
libvirt APIs too.
Took me a second to understand you, but what I'm saying implies a
dialog like:
---
Add Virtual Network Interface:
Mac Address : [ ]
Connect to:
* Network : [ Default ]
* Physical Interface : [ eth0 ]
---
And your point is that in order to populate the latter dropdown list,
you need to use a sucky API which doesn't work in the remote case.
Rather you suggest that if people want to use bridging, then they
should modify the default network XML config by hand and not have the
latter option in the UI?
I dunno, the concept of "connect the guest to a physical network"
should be in the UI IMHO, and if you do then you need a list of physical
interfaces ...
This could be expressed with something like:
<network>
<name>default</name>
<bridge name="virbr0" />
<device name="eth0" />
</network>
Which would setup a virbr0 bridge,and enslave eth0 to it. It would assume
that the network eth0 is connected to has DHCP already, so there's no need
to define an IP for the bridge itself, or run dnsmasq in this case.
Yeah, it's like taking your virtual netgear router, turning off IP
forwarding and DHCP and enabling bridging.
That introduces this user visible notion of a bridge vs. a router,
which is just horrible. But, I guess you're saying you wouldn't have it
in the UI.
Is it actually possible to take an already configured eth0 device
and
enslave it to virbr0 without eth0 loosing its address config/routes ?
Well, where would the address from eth0 go if you had an ip address
configured for the network already? It would have to be an either or
thing then ...
Also, what if you try and bridge multiple networks out over the
physical interface?
That's why I'm suggesting that if you want to bridge guests over a
physical interface, then you need to change the interfaces configuration
so that it's "shared" ... which in effect, makes it a bridge which you
can add guest's backend interfaces too.
...
I guess it all comes down to two options:
a) Expose, in the UI, the concept of "either connect the guest to a
virtual network or to a shared physical interface" or
b) Make it an admin decision, not available in the UI, whether the
default network is a private network or bridged to a physical
interface
I prefer a), but I can see the argument for b).
If we have this kind of support could we auto-generate network
configs at
runtime based on any bridge/network device the adminsitrator has already
manually configured via /etc/sysconfig/networking-scripts ?
Not really following ...
Cheers,
Mark.
12:52 p.m.
On Fri, Mar 09, 2007 at 04:44:17PM +0000, Mark McLoughlin wrote:
On Fri, 2007-03-09 at 14:40 +0000, Daniel P. Berrange wrote:
> On Fri, Mar 09, 2007 at 02:27:02PM +0000, Mark McLoughlin wrote:
> >
> > As I said in another mail, the way I'd imagined virt-install working
> > would be that by default it would connect guest NICs to the default
> > network (i.e. not the bridge associated with the default network), but
> > you could still use --bridge to explicitly connect the guest to a
> > bridge. And we'd have a --network option.
>
> I think we need to explore this a little further - the APIs for virtual
> networks are very nice froma tools POV because we can trivially enumerate
> networks & connect VMs to them. Its not pleasent to have to enumerate the
> regular non-virtual network devices using a different set of APIs - indeed
> its impossible if we're talking to a remote libvirt. So I think we need to
> look at also exposing the 'shared physical device' style networks via the
> libvirt APIs too.
Took me a second to understand you, but what I'm saying implies a
dialog like:
---
Add Virtual Network Interface:
Mac Address : [ ]
Connect to:
* Network : [ Default ]
* Physical Interface : [ eth0 ]
---
And your point is that in order to populate the latter dropdown list,
you need to use a sucky API which doesn't work in the remote case.
Pretty much, yes.
Rather you suggest that if people want to use bridging, then they
should modify the default network XML config by hand and not have the
latter option in the UI?
How they configure the network XML is a completely separate issue - we could
easily have UI in virt-manager for creating/deleteing/editing networks in
the same way we have UI for creating/deleting/editing domains. So while today
they'd have to edit the XML to bridge to the physical network, that need only
be a short term issue.
I dunno, the concept of "connect the guest to a physical
network"
should be in the UI IMHO, and if you do then you need a list of physical
interfaces ...
So yes, we need list of physical interfaces / networks, in addition to the
existing list of virtual networks. Perhaps the same libvirt API can be
used to provide both listings, or maybe we need separate APIs. Even if
we used a single API at the libvirt level for listing virtual & physical
networks, we can have the option of displaying a dialog with the struture
you outline above - just key off some attribute in the XML to determine
whether to put a particular virNetworkPtr in the 'Network' drop down
vs the 'physical interface' drop down (or whatever they end up being called).
> This could be expressed with something like:
>
> <network>
> <name>default</name>
> <bridge name="virbr0" />
> <device name="eth0" />
> </network>
>
> Which would setup a virbr0 bridge,and enslave eth0 to it. It would assume
> that the network eth0 is connected to has DHCP already, so there's no need
> to define an IP for the bridge itself, or run dnsmasq in this case.
Yeah, it's like taking your virtual netgear router, turning off IP
forwarding and DHCP and enabling bridging.
That introduces this user visible notion of a bridge vs. a router,
which is just horrible. But, I guess you're saying you wouldn't have it
in the UI.
Just because the different bridge vs routed confoigs are represented in the
libvirt XML one way, doesn't force our hand in our we present it to the user.
Any network with a '<device name='eth0'> tag could be displayed in the
'physical interfaces' drop down, while any without that tag would be
in the 'Network' drop down. My concerns at this stage are really about the
being able to use a consistent API in libvirt for everything we ever want to
do with networking, and not ever have to call out to non-libvirt APIs ourselves.
> Is it actually possible to take an already configured eth0
device and
> enslave it to virbr0 without eth0 loosing its address config/routes ?
Well, where would the address from eth0 go if you had an ip address
configured for the network already? It would have to be an either or
thing then ...
In my example I'm not configuring any address for the network - after initial
enslaving, the only device which would have an address would be eth0 which
preserved its address config from before it was enslaved. If that's possible -
i've not actually tested this theory ? I know Xen has horrible manual address
copying, but I figure that is only a result of the need to add in netloop.
Also, what if you try and bridge multiple networks out over the
physical interface?
That'd be impossible because eth0 can only be in one bridge at once - it
only really makes sense to bridge one network into any physical device in
any case. Semantically bridging multiple networks is just the same has
having one larger network, so there'd be no point in trying to bridge
multiple networks into one physical device.
That's why I'm suggesting that if you want to bridge guests
over a
physical interface, then you need to change the interfaces configuration
so that it's "shared" ... which in effect, makes it a bridge which you
can add guest's backend interfaces too.
...
I guess it all comes down to two options:
a) Expose, in the UI, the concept of "either connect the guest to a
virtual network or to a shared physical interface" or
b) Make it an admin decision, not available in the UI, whether the
default network is a private network or bridged to a physical
interface
I prefer a), but I can see the argument for b).
> If we have this kind of support could we auto-generate network configs at
> runtime based on any bridge/network device the adminsitrator has already
> manually configured via /etc/sysconfig/networking-scripts ?
Not really following ...
ie, if the user has create /etc/sysconfig/networking-scripts/ifcrg-br5 for
whatever reason, then we can automatically instantiate a network at the
libvirt level to represent this externally created network.
Semantically an admin creating arbitrary bridge devices with their regular
distro specific network init scripts is no different to libvirt createing
bridge devices. In both cases you end up with a bridge device, to which
you'd like to connect guests. In both cases you may or may not have a
physical interface enslaved in the bridge. So if the admin has created
some bridge device I'd like it to appear when doing 'virsh net-list'.
Regards,
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
1:17 p.m.
On Fri, 2007-03-09 at 18:52 +0000, Daniel P. Berrange wrote:
On Fri, Mar 09, 2007 at 04:44:17PM +0000, Mark McLoughlin wrote:
> Rather you suggest that if people want to use bridging, then
they
> should modify the default network XML config by hand and not have the
> latter option in the UI?
How they configure the network XML is a completely separate issue - we could
easily have UI in virt-manager for creating/deleteing/editing networks in
the same way we have UI for creating/deleting/editing domains.
... except you'd again have need an API for iterating physical network
devices ...
> That introduces this user visible notion of a bridge vs. a
router,
> which is just horrible. But, I guess you're saying you wouldn't have it
> in the UI.
Just because the different bridge vs routed confoigs are represented in the
libvirt XML one way, doesn't force our hand in our we present it to the user.
Any network with a '<device name='eth0'> tag could be displayed in the
'physical interfaces' drop down, while any without that tag would be
in the 'Network' drop down.
Well, except that you're suggesting "connect to physical interface
eth0" should be a property of a virtual network, but I think it makes
more sense for it to be the property of a virtual nic.
e.g. connect your qemu guests to the default network, connect your Xen
guests to the eth0 bridge.
Cheers,
Mark.
1:38 p.m.
On Fri, Mar 09, 2007 at 07:17:04PM +0000, Mark McLoughlin wrote:
On Fri, 2007-03-09 at 18:52 +0000, Daniel P. Berrange wrote:
> On Fri, Mar 09, 2007 at 04:44:17PM +0000, Mark McLoughlin wrote:
> > Rather you suggest that if people want to use bridging, then they
> > should modify the default network XML config by hand and not have the
> > latter option in the UI?
>
> How they configure the network XML is a completely separate issue - we could
> easily have UI in virt-manager for creating/deleteing/editing networks in
> the same way we have UI for creating/deleting/editing domains.
... except you'd again have need an API for iterating physical network
devices ...
You say that like its a bad thing ? Virt-manager or other apps will need to
be able to enumerate physical devices somehow. So if libvirt is already
providing APIs for virtual networks a VM can connect to, why not also provide
APIs for physical devices (or figure out a way to let the existing virtual
networks API also deal with phys devices).
> > That introduces this user visible notion of a bridge vs. a router,
> > which is just horrible. But, I guess you're saying you wouldn't have
it
> > in the UI.
>
> Just because the different bridge vs routed confoigs are represented in the
> libvirt XML one way, doesn't force our hand in our we present it to the user.
> Any network with a '<device name='eth0'> tag could be displayed in
the
> 'physical interfaces' drop down, while any without that tag would be
> in the 'Network' drop down.
Well, except that you're suggesting "connect to physical interface
eth0" should be a property of a virtual network, but I think it makes
more sense for it to be the property of a virtual nic.
Well actually I'm suggesting 'connect to the physical network that eth0
is a part of' which doesn't feel much different to 'connect to the
virtual network'
NB, in all this discussion I'm talking about a scenario where the guest
can be connect to eth0 via a bridge. I've not even thought about a case
where you'd NAT a VIF directly into eth0 - that's probably irrelevant
use case now we have proper virtual networks - i only bring it up because
that is what xen's vif-route or vif-nat scripts do.
e.g. connect your qemu guests to the default network, connect your
Xen
guests to the eth0 bridge.
I'm just wondering whether this is making a distinction, where no real
distinction exists? If you run 'ifconfig' or 'brctl show' in either
of these two cases its going to look basically identical to the admin.
ie, there a bridge device, with one of more NICs in it, some virtual
NICs or TAPs, some physical NICs. If you run 'virsh net-list' you're
only going to see one of those cases
Regards,
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
2:04 p.m.
On Fri, 2007-03-09 at 19:38 +0000, Daniel P. Berrange wrote:
On Fri, Mar 09, 2007 at 07:17:04PM +0000, Mark McLoughlin wrote:
> On Fri, 2007-03-09 at 18:52 +0000, Daniel P. Berrange wrote:
> > On Fri, Mar 09, 2007 at 04:44:17PM +0000, Mark McLoughlin wrote:
>
> > > Rather you suggest that if people want to use bridging, then they
> > > should modify the default network XML config by hand and not have the
> > > latter option in the UI?
> >
> > How they configure the network XML is a completely separate issue - we could
> > easily have UI in virt-manager for creating/deleteing/editing networks in
> > the same way we have UI for creating/deleting/editing domains.
>
> ... except you'd again have need an API for iterating physical network
> devices ...
You say that like its a bad thing ?
Not at all, I'm just saying that making the physical interface bridging
configuration a part of the virtual network description does not make
the problem of listing physical interfaces go away.
> e.g. connect your qemu guests to the default network, connect
your Xen
> guests to the eth0 bridge.
I'm just wondering whether this is making a distinction, where no real
distinction exists? If you run 'ifconfig' or 'brctl show' in either
of these two cases its going to look basically identical to the admin.
ie, there a bridge device, with one of more NICs in it, some virtual
NICs or TAPs, some physical NICs. If you run 'virsh net-list' you're
only going to see one of those cases
I think there's a big distinction between the concepts which we want
users to understand, but not such a big distinction in how they are
implemented.
Cheers,
Mark.
6498
days inactive
6498
days old
10 comments
2 participants
participants (2)
-
Daniel P. Berrange -
Mark McLoughlin