3 a.m.
Hi David,
On Fri, 2009-01-16 at 01:13 +0000, David Lutterkort wrote:
For certain applications, we want libvirt to be able to configure
host
network interfaces in a variety of ways; currently, we are most
interested in teaching libvirt how to set up ordinary ethernet
interfaces, bridges, bonding and vlan's.
Below is a high-level proposal of how that could be done. Please comment
copiously ;)
1. XML Format
=============
The first question is how the user would describe the host interfaces they
want. Below are some sketches of what an XML representation of the various
kinds of interfaces would look like. This covers the minimal amount of
settings for these to be useful, though I am sure we'll need to add more
over time.
<interface device="eth0" onboot="yes">
<hwaddr>00:19:d2:9f:88:96</hwaddr>
<dhcp peerdns="yes"/>
</interface>
<interface device="eth1" onboot="yes">
<hwaddr>00:19:d2:9f:88:97</hwaddr>
<static ipaddr="192.168.0.5" gateway="192.168.0.1"
netmask="255.255.255.0"/>
</interface>
<interface device="eth2" onboot="yes">
<hwaddr>00:19:d2:9f:88:98</hwaddr>
</interface>
You mention MTU below - yes, you'll need that.
<bond name="bond00" onboot="yes"
mode="active-backup">
<slave device="eth0" primary="yes"/>
<slave device="eth1"/>
</bond>
<bridge name="br0" stp="off" onboot="yes">
<member device="eth2"/>
<dhcp peerdns="yes"/>
</bridge>
I don't think we want to define a bridge here, but more that an
interface is shared - i.e. this is a property of eth2.
The main concern is that this is the way I'd expect NetworkManager to
support it - i.e. that you could configure NetworkManager to share eth0,
rather than ask it to create br0 and add eth0 to it.
If you just want to create a bridge, you can creati a virtual network.
<vlan device="eth0" tag="42"
reorder_hdr="yes"/>
I think these last three are also interfaces and should be described
with an <interface> tag e.g.
<interface device="bond0" onboot="yes">
<bond mode="active-backup"/>
<slave device="eth0" primary="yes"/>
<slave device="eth1"/>
<dhcp/>
</interface>
<interface device="eth2" onboot="yes">
<hwaddr>00:19:d2:9f:88:98</hwaddr>
<shared bridge="br0"/>
<dhcp/>
</interface>
<interface device="eth0">
<vlan tag="42" reorder_hdr="yes"/>
</interface>
All of these should also allow a <uuid> element for specifying
a uuid; I
omitted that for brevity.
2. API Changes
==============
There are two options for dealing with network interfaces: (1) use the
existing virNetwork* calls or (2) add completely new API calls.
Repurposing existing virNetwork* calls
--------------------------------------
The existing calls map well to the operations we need for managing
interfaces, with a few exceptions:
- virNetworkGetAutostart/SetAutostart: depending on how we implement all
this (see below), 'autostart' might actually mean 'on boot', not
'when
libvirtd starts'
- virNetworkGetBridgeName doesn't make sense for interfaces, and should
return NULL for interfaces
We'll probably also end up adding some functions to query details about an
interface, in particular, a call to see what kind of network/interface a
virNetworkPtr represents
I don't think I like this much - these APIs manage a "virtual network"
which I see as a distinct concept from configuring host hardware.
Really, I think keeping the two things separate will actually reduce
confusion in the long run.
Add completely new virInterface* calls
--------------------------------------
This would add roughly the same API calls as the virNetwork* calls,
i.e. we'd have something like
typedef struct virInterface *virInterfacePtr;
int virInterfaceCreate(virInterfacePtr);
virInterfacePtr virInterfaceCreateXML(..);
...
plus some calls to extract information from a virInterfacePtr
This sounds good, but "interface" is pretty damn generic :-)
virNetInterface
virNetDevice
...
What I don't like about any of these is that I've always imagined we
might add further APIs to libvirt for changing a domain's configuration
without munging XML e.g.
int virDomainGetInterfaces(virDomainPtr domain,
virInterfacePtr interfaces,
int *ninterfaces);
int virInterfaceSetMacAddress(virInterfacePtr interface,
const uint8_t mac[6]);
So, you can see the potential namespace conflicts ...
(But that's pretty irrelevant since I think I'm alone in thinking APIs
like that would make sense :-)
The second option seems cleaner to me and easier to implement, and
avoids
any subtle changes in the behavior of existing API, though I don't like
that we'll be adding another 20 or so calls to libvirt's public API, with
attendant churn both in the drivers and in virsh.
I don't think the amount of boilerplate you need to add for new APIs
should stop you - the more of this crud we add, the more incentive we'll
have to figure out ways to reduce it :-)
3. Implementation
=================
Configuring network interfaces is highly OS and OS-variant/distro
dependant. There are at least two different ways how libvirt can go about
modifying the host to create interfaces:
1. Modify the system's network setup scripts (ifcfg-XXX on RH)
2. Directly use the system's network utilities like ifconfig
3. Rely on NetworkManager (not an option right now, as NM doesn't know
about bridges and the like)
It has to be an option - even if it only supports a subset of what the
other options support.
I really wouldn't like to see us push ahead with this until we have a
plan for how this will work with NetworkManager (and a rough agreement
for how future support for bonding etc. in NetworkManager might be
configured)
Option (1) saves us from replicating every bit of network setup
functionality that those scripts already have - besides configuring the
interface, we also might have to setup routes, run dhclient etc.
Option (2) would require far fewer backend implementations than (1) - we
should be able to get away with one implementation for Linux, rather than
one for Fedora/RHEL, one for Debian, one for SuSe, three for gentoo
etc.
The thing is, this has to integrate with existing configuration -
there's no point in futzing about with "ip link set eth0 ..." if the
user has configured eth0 with NetworkManager or the distro's networking
scripts.
If we want 'autostart' for an interface to mean 'bring up
the interface
as soon as the system boots', we are pretty much forced to go with
option (1).
Why?
All in all, option (1) seems more attractive, since it should save us
from
dealing with a lot of low-level details of network setup, and the distro
scripts should be much better integrated with the rest of the system than
what we come up with for libvirt.
IMHO, (1) and (3) are requirements and we'll probably end up doing (2)
too in the long run.
Cheers,
Mark.