7:13 p.m.
For certain applications, we want libvirt to be able to configure host
network interfaces in a variety of ways; currently, we are most
interested in teaching libvirt how to set up ordinary ethernet
interfaces, bridges, bonding and vlan's.
Below is a high-level proposal of how that could be done. Please comment
copiously ;)
1. XML Format
=============
The first question is how the user would describe the host interfaces they
want. Below are some sketches of what an XML representation of the various
kinds of interfaces would look like. This covers the minimal amount of
settings for these to be useful, though I am sure we'll need to add more
over time.
<interface device="eth0" onboot="yes">
<hwaddr>00:19:d2:9f:88:96</hwaddr>
<dhcp peerdns="yes"/>
</interface>
<interface device="eth1" onboot="yes">
<hwaddr>00:19:d2:9f:88:97</hwaddr>
<static ipaddr="192.168.0.5" gateway="192.168.0.1"
netmask="255.255.255.0"/>
</interface>
<interface device="eth2" onboot="yes">
<hwaddr>00:19:d2:9f:88:98</hwaddr>
</interface>
<bond name="bond00" onboot="yes"
mode="active-backup">
<slave device="eth0" primary="yes"/>
<slave device="eth1"/>
</bond>
<bridge name="br0" stp="off" onboot="yes">
<member device="eth2"/>
<dhcp peerdns="yes"/>
</bridge>
<vlan device="eth0" tag="42" reorder_hdr="yes"/>
All of these should also allow a <uuid> element for specifying a uuid; I
omitted that for brevity.
2. API Changes
==============
There are two options for dealing with network interfaces: (1) use the
existing virNetwork* calls or (2) add completely new API calls.
Repurposing existing virNetwork* calls
--------------------------------------
The existing calls map well to the operations we need for managing
interfaces, with a few exceptions:
- virNetworkGetAutostart/SetAutostart: depending on how we implement all
this (see below), 'autostart' might actually mean 'on boot', not
'when
libvirtd starts'
- virNetworkGetBridgeName doesn't make sense for interfaces, and should
return NULL for interfaces
We'll probably also end up adding some functions to query details about an
interface, in particular, a call to see what kind of network/interface a
virNetworkPtr represents
Add completely new virInterface* calls
--------------------------------------
This would add roughly the same API calls as the virNetwork* calls,
i.e. we'd have something like
typedef struct virInterface *virInterfacePtr;
int virInterfaceCreate(virInterfacePtr);
virInterfacePtr virInterfaceCreateXML(..);
...
plus some calls to extract information from a virInterfacePtr
The second option seems cleaner to me and easier to implement, and avoids
any subtle changes in the behavior of existing API, though I don't like
that we'll be adding another 20 or so calls to libvirt's public API, with
attendant churn both in the drivers and in virsh.
3. Implementation
=================
Configuring network interfaces is highly OS and OS-variant/distro
dependant. There are at least two different ways how libvirt can go about
modifying the host to create interfaces:
1. Modify the system's network setup scripts (ifcfg-XXX on RH)
2. Directly use the system's network utilities like ifconfig
3. Rely on NetworkManager (not an option right now, as NM doesn't know
about bridges and the like)
Option (1) saves us from replicating every bit of network setup
functionality that those scripts already have - besides configuring the
interface, we also might have to setup routes, run dhclient etc.
Option (2) would require far fewer backend implementations than (1) - we
should be able to get away with one implementation for Linux, rather than
one for Fedora/RHEL, one for Debian, one for SuSe, three for gentoo
etc.
If we want 'autostart' for an interface to mean 'bring up the interface
as soon as the system boots', we are pretty much forced to go with
option (1).
All in all, option (1) seems more attractive, since it should save us from
dealing with a lot of low-level details of network setup, and the distro
scripts should be much better integrated with the rest of the system than
what we come up with for libvirt.
4. Misc issues
==============
* Should interfaces have labels/roles ('data-interface') to help admins
make sense of the current config ?
* Do we expect interfaces to be in a specific state before we create them
or do we just tear them down and reconfigure them no matter what ?
* Are there crucial config options that are not covered by the sketches
above (e.g., setting an explicit MTU) ? Are there things in the XML
sketches above that will be impossible to implement on some OS ?
* Should this even be done as part of libvirt ? It seems like a very
generic network config tool, and libvirt merely the conduit to exposing
this through an API, most importantly, a remotable API.
David
7:26 p.m.
On Fri, Jan 16, 2009 at 01:13:18AM +0000, David Lutterkort wrote:
* Should this even be done as part of libvirt ? It seems like a
very
generic network config tool, and libvirt merely the conduit to exposing
this through an API, most importantly, a remotable API.
My humble opinion would be "no". This argument equally applies to
everything you could configure on a host, and I don't think anyone wants
to turn libvirt into libmanagement. Would it be feasible for libvirtd to
have a 'passthrough' mode that feeds unknown stuff off to some other
daemon?
Especially in this case - interface management is very complex as you
note.
regards
john
8:59 p.m.
On Fri, 2009-01-16 at 01:26 +0000, John Levon wrote:
On Fri, Jan 16, 2009 at 01:13:18AM +0000, David Lutterkort wrote:
> * Should this even be done as part of libvirt ? It seems like a very
> generic network config tool, and libvirt merely the conduit to exposing
> this through an API, most importantly, a remotable API.
My humble opinion would be "no". This argument equally applies to
everything you could configure on a host, and I don't think anyone wants
to turn libvirt into libmanagement. Would it be feasible for libvirtd to
have a 'passthrough' mode that feeds unknown stuff off to some other
daemon?
Especially in this case - interface management is very complex as you
note.
I am not disagreeing with you, but either way, libvirt needs _some_ way
to control host interfaces. And seeing how there are no tools that do
something like the above in a portable way, it seems worthwhile, too.
The question is probably more whether some of this functionality should
be split out in a way that can be used independenty of libvirt.
David
3:06 a.m.
On Fri, 2009-01-16 at 03:29 +0000, John Levon wrote:
On Fri, Jan 16, 2009 at 02:59:17AM +0000, David Lutterkort wrote:
> I am not disagreeing with you, but either way, libvirt needs _some_ way
> to control host interfaces.
This is far from obvious to me. Could you explain more?
You're right to ask, IMHO.
The rationale will be better explained by others, I'm sure, but here
goes:
Any serious virtualization management tool needs to be able to
configure the networking interfaces on a remote host.
libvirt aims to be an all-encompassing API for such management
tools.
The obvious worry is how will we decide where to draw the line? If
someone is writing a virt management tool that e.g. needs to install
packages on hosts, will we add support for that?
Cheers,
Mark.
4:11 a.m.
On Fri, Jan 16, 2009 at 09:06:05AM +0000, Mark McLoughlin wrote:
On Fri, 2009-01-16 at 03:29 +0000, John Levon wrote:
> On Fri, Jan 16, 2009 at 02:59:17AM +0000, David Lutterkort wrote:
>
> > I am not disagreeing with you, but either way, libvirt needs _some_ way
> > to control host interfaces.
>
> This is far from obvious to me. Could you explain more?
You're right to ask, IMHO.
The rationale will be better explained by others, I'm sure, but here
goes:
Any serious virtualization management tool needs to be able to
configure the networking interfaces on a remote host.
libvirt aims to be an all-encompassing API for such management
tools.
The obvious worry is how will we decide where to draw the line? If
someone is writing a virt management tool that e.g. needs to install
packages on hosts, will we add support for that?
I think the difference that, is that installing RPMS is not anything todo
with virtualization in the general case - it would just be a particular
apps approach to working with its OS - as such it would be out of scope
for libvirt.
Integrating with host networking meanwhile is a fundamental requirement
for virtualization for all apps using libvirt, since guests need network
connectivity, and thus managing NICs should be within scope.
Also note there is existing precedent as the CIM APi, Xen-API, VMWare
and VirtualIron APIs all provide for management of host NICs.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
7:24 a.m.
On Fri, Jan 16, 2009 at 10:11:02AM +0000, Daniel P. Berrange wrote:
Integrating with host networking meanwhile is a fundamental
requirement
for virtualization for all apps using libvirt, since guests need network
connectivity, and thus managing NICs should be within scope.
I don't think that's much of an argument. Plenty of things can be
considered fundamental. My kernel version certainly is, so why isn't
libvirt letting me upgrade that? What about my firewall? Why isn't
libvirt configuring my iSCSI target for me?
We should be considering why libvirt is /well-placed/ to configure the
host. I think it should be pretty clear that it's actually not: the
problems around distro differences alone is a good indication. The
proposed API is anaemic enough to not be of much use.
This is way beyond carving out the physical system into virtual chunks
and it's a big step towards lib*virt* becoming libmanagement.
It seems David's main justification for this was:
- you want to configure host NICs
- libvirt already has scaffolding for remote management
So I'm sympathetic to something like spinning out the RPC into a generic
layer, and modularising what it can talk, with a libvirt module. But
that's a very different beast.
regards
john
7:41 a.m.
On Fri, Jan 16, 2009 at 01:24:53PM +0000, John Levon wrote:
On Fri, Jan 16, 2009 at 10:11:02AM +0000, Daniel P. Berrange wrote:
> Integrating with host networking meanwhile is a fundamental requirement
> for virtualization for all apps using libvirt, since guests need network
> connectivity, and thus managing NICs should be within scope.
I don't think that's much of an argument. Plenty of things can be
considered fundamental. My kernel version certainly is, so why isn't
libvirt letting me upgrade that? What about my firewall? Why isn't
libvirt configuring my iSCSI target for me?
The kernel version isn't fundamental to the task of provisioning and
configuring a guest VM. When deploying a VM there is no general
requirement to upgrade the host kernel. When deploying a VM there
very much is a requirement to configure physical resources in the
host such as storage, and networking.
We should be considering why libvirt is /well-placed/ to configure
the
host. I think it should be pretty clear that it's actually not: the
problems around distro differences alone is a good indication. The
proposed API is anaemic enough to not be of much use.
The existance of many different impls is exactly the reason for libvirt
to have this capability. Libvirt is providing a consistent mgmt API
for management of guests and host networking interfaces is as much a
part of this as the storage management. Libvirt is providing this
capability across virtualization technology. If we did not include
the NIC mgmt then apps using libvirt would not only need to write
different code for each OS, but also for Xen, VMWare, etc each of
have their own APIs for network management which is just not helpful
for apps using libvirt. We already have this problem in apps using
libvirt needing to cope with different OS networking setups and thus
duplicating all this horrible code in every user of libvirt, which
is why networking mgmt is a important goal within scope.
This is way beyond carving out the physical system into virtual
chunks
and it's a big step towards lib*virt* becoming libmanagement.
Not really - it is precisely about carving up / managing physical
resources that are related to the guest configuration. It is not
about arbitrary OS management - we don't want an API to deploy
RPMs, or configure Apache, or any other things related to general
OS management.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
7:55 a.m.
On Fri, Jan 16, 2009 at 01:41:19PM +0000, Daniel P. Berrange wrote:
> I don't think that's much of an argument. Plenty of
things can be
> considered fundamental. My kernel version certainly is, so why isn't
> libvirt letting me upgrade that? What about my firewall? Why isn't
> libvirt configuring my iSCSI target for me?
The kernel version isn't fundamental to the task of provisioning and
configuring a guest VM. When deploying a VM there is no general
requirement to upgrade the host kernel. When deploying a VM there
very much is a requirement to configure physical resources in the
host such as storage, and networking.
So it sounds like you think libvirt /should/ be going out and
configuring shared storage. What about VLAN set up on my router
hardware? That too, right?
The existance of many different impls is exactly the reason for
libvirt
to have this capability. Libvirt is providing a consistent mgmt API
No - it's exactly the reason for SOME common API. No-one is arguing that
a common API for host networking is a bad idea.
"There isn't an API, and it's sometimes needed for management" is not
an
argument for it be part of libvirt's scope.
Again, what makes libvirt a good place for this management? I don't
accept "because it's there" as a reasonable justification...
regards,
john
8:20 a.m.
Just to be clear - I am very sympathetic to the need for this stuff and
the conclusion that it belongs in libvirt. I just think we need to be
fairly clear on where the line is.
On Fri, 2009-01-16 at 13:41 +0000, Daniel P. Berrange wrote:
On Fri, Jan 16, 2009 at 01:24:53PM +0000, John Levon wrote:
> On Fri, Jan 16, 2009 at 10:11:02AM +0000, Daniel P. Berrange wrote:
>
> > Integrating with host networking meanwhile is a fundamental requirement
> > for virtualization for all apps using libvirt, since guests need network
> > connectivity, and thus managing NICs should be within scope.
>
> I don't think that's much of an argument. Plenty of things can be
> considered fundamental. My kernel version certainly is, so why isn't
> libvirt letting me upgrade that? What about my firewall? Why isn't
> libvirt configuring my iSCSI target for me?
The kernel version isn't fundamental to the task of provisioning and
configuring a guest VM. When deploying a VM there is no general
requirement to upgrade the host kernel.
There may be a requirement to install certain packages before VMs can be
deployed.
When deploying a VM there
very much is a requirement to configure physical resources in the
host such as storage, and networking.
When "deploying a VM" or when "configuring a host to run VMs"?
i.e. is this isn't an API to list the available bridges which a user
could choose to connect a VM to, this is an API to configure bonded NICs
etc.
The Virtual Network and Storage Pool stuff is different as well - it's
about virtualizing those resources. Configuring a bonded NIC or setting
a static IP address on eth0 is not about virtualizing eth0.
With the exception, perhaps, of configuring a NIC to be a shared
physical interface, this stuff seems to me to be about host
configuration rather than helping with VM configuration or virtualizing
resources.
...
What remote management tools can benefit from is piggy backing on top of
libvirt's authenticated connection to a host. What would be so wrong
with adding a TCP stream tunnel (c.f. SSH tunnel) over the libvirt
connection and allow the management tool talk to a separate host
configuration agent?
Cheers,
Mark.
8:45 a.m.
On Fri, Jan 16, 2009 at 02:20:27PM +0000, Mark McLoughlin wrote:
Just to be clear - I am very sympathetic to the need for this stuff
and
the conclusion that it belongs in libvirt. I just think we need to be
fairly clear on where the line is.
On Fri, 2009-01-16 at 13:41 +0000, Daniel P. Berrange wrote:
> On Fri, Jan 16, 2009 at 01:24:53PM +0000, John Levon wrote:
> > On Fri, Jan 16, 2009 at 10:11:02AM +0000, Daniel P. Berrange wrote:
> >
> > > Integrating with host networking meanwhile is a fundamental requirement
> > > for virtualization for all apps using libvirt, since guests need network
> > > connectivity, and thus managing NICs should be within scope.
> >
> > I don't think that's much of an argument. Plenty of things can be
> > considered fundamental. My kernel version certainly is, so why isn't
> > libvirt letting me upgrade that? What about my firewall? Why isn't
> > libvirt configuring my iSCSI target for me?
>
> The kernel version isn't fundamental to the task of provisioning and
> configuring a guest VM. When deploying a VM there is no general
> requirement to upgrade the host kernel.
There may be a requirement to install certain packages before VMs can be
deployed.
That is true, you also need to installl an OS before you can manage
a physical host, and you need to build a machine before you can
install an OS on it. We could go on for ever here, which is why
I'm trying to make define the boundary line as being related to
the management of hardware resources on the host. WRT the need to
install certain packages for, say KVM, libvirt's role is to provide
information about what capabilities are present. So the host XML
for capablities indicates whether KVM virtualization is available.
Libvirt is managing what is available on the host, not trying to
install more thing. Installing RPMs is not hardware resource
management, hence I class it out of scope.
> When deploying a VM there
> very much is a requirement to configure physical resources in the
> host such as storage, and networking.
When "deploying a VM" or when "configuring a host to run VMs"?
Configuring resourcs on the host to connect to a VM. So finding or
creating a LVM LUN, creating a VLAN devices on NIC and putting it
into a bridge to guest to a guest.
i.e. is this isn't an API to list the available bridges which a
user
could choose to connect a VM to, this is an API to configure bonded NICs
etc.
Mere listing of host devices is already provided by the node device
enumation APIs.
The Virtual Network and Storage Pool stuff is different as well - it's
about virtualizing those resources. Configuring a bonded NIC or setting
a static IP address on eth0 is not about virtualizing eth0.
I don't think there is such a distinction between the storage pool
stuff and the network stuff. In storage pools we are taking phys
devices and putting them into a LVM volume groups - this is directly
akin to taking two NICs and bonding them. Likewise managing FibreChanel
with NPIV is directly akin to managing VLANs on a NIC. So I class
configuration of bonding, bridges and VLANs within scope fo the
network mgmt API.
What remote management tools can benefit from is piggy backing on top
of
libvirt's authenticated connection to a host. What would be so wrong
with adding a TCP stream tunnel (c.f. SSH tunnel) over the libvirt
connection and allow the management tool talk to a separate host
configuration agent?
That isn't providing an API that is consistnet across virt manangement
platforms. It is not providing an API and XML model that is consistnet
with the rest of the model exposed through libvirt's hypervisor, node
capability, node device & virtual network APIs. Network interface APIs
are the core missing piece of libvirt API functionality IMHO.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
8:49 a.m.
On Fri, Jan 16, 2009 at 01:24:53PM +0000, John Levon wrote:
We should be considering why libvirt is /well-placed/ to configure
the
host. I think it should be pretty clear that it's actually not: the
problems around distro differences alone is a good indication. The
proposed API is anaemic enough to not be of much use.
This is precisely why it is merely a proposal - if you can elaborate on
why is insufficiently useful we can discuss how to address the shortcomings
or whether the extra needs should be declared out of scope.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
3:14 p.m.
On Fri, 2009-01-16 at 03:29 +0000, John Levon wrote:
On Fri, Jan 16, 2009 at 02:59:17AM +0000, David Lutterkort wrote:
> I am not disagreeing with you, but either way, libvirt needs _some_ way
> to control host interfaces.
This is far from obvious to me. Could you explain more?
The reasoning goes - similar to what Mark said - something like:
virtualization is concerned with splitting up the physical resources of
a host and presenting them as virtual resources to guests.
Physical resources are (at a minimum) CPU, memory, storage and
networking; in order to split up resources, we need very fine-grained
and low-level control over them - we have that in libvirt for CPU and
memory (via the hypervisor) and for storage through libvirt's storage
API. Adding API to control host interfaces rounds that picture out by
adding fine-grained control for the last major physical resource libvirt
isn't managing yet.
David
9:54 p.m.
Hi all,
I think I agree with David here
I think that some method of configuring host network interfaces is inline with configuring
CPU or RAM allocations for a guest.
You want to provision say, a new server, you might want to add a tagged VLAN bridge for it
to sit off as well. Or perhaps establish a private "bridge" or subnet for it.
At the moment, you have to set that up manually. While it can be easily standardized ( I
think ) - there are some complexities ( ie MTU settings - jumbo frames, or allowing for
tagged vlan overhead without appropriate hardware.. ).
Am I inline with your thinkings, David? ;)
cheers;
//Chris
----- Original Message -----
From: "John Levon" <levon(a)movementarian.org>
To: "David Lutterkort" <lutter(a)redhat.com>
Cc: libvir-list(a)redhat.com
Sent: Friday, 16 January, 2009 12:29:43 PM GMT +08:00 Beijing / Chongqing / Hong Kong /
Urumqi
Subject: Re: [libvirt] RFC: configuring host interfaces with libvirt
On Fri, Jan 16, 2009 at 02:59:17AM +0000, David Lutterkort wrote:
I am not disagreeing with you, but either way, libvirt needs _some_
way
to control host interfaces.
This is far from obvious to me. Could you explain more?
regards
john
--
Libvir-list mailing list
Libvir-list(a)redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
3 a.m.
Hi David,
On Fri, 2009-01-16 at 01:13 +0000, David Lutterkort wrote:
For certain applications, we want libvirt to be able to configure
host
network interfaces in a variety of ways; currently, we are most
interested in teaching libvirt how to set up ordinary ethernet
interfaces, bridges, bonding and vlan's.
Below is a high-level proposal of how that could be done. Please comment
copiously ;)
1. XML Format
=============
The first question is how the user would describe the host interfaces they
want. Below are some sketches of what an XML representation of the various
kinds of interfaces would look like. This covers the minimal amount of
settings for these to be useful, though I am sure we'll need to add more
over time.
<interface device="eth0" onboot="yes">
<hwaddr>00:19:d2:9f:88:96</hwaddr>
<dhcp peerdns="yes"/>
</interface>
<interface device="eth1" onboot="yes">
<hwaddr>00:19:d2:9f:88:97</hwaddr>
<static ipaddr="192.168.0.5" gateway="192.168.0.1"
netmask="255.255.255.0"/>
</interface>
<interface device="eth2" onboot="yes">
<hwaddr>00:19:d2:9f:88:98</hwaddr>
</interface>
You mention MTU below - yes, you'll need that.
<bond name="bond00" onboot="yes"
mode="active-backup">
<slave device="eth0" primary="yes"/>
<slave device="eth1"/>
</bond>
<bridge name="br0" stp="off" onboot="yes">
<member device="eth2"/>
<dhcp peerdns="yes"/>
</bridge>
I don't think we want to define a bridge here, but more that an
interface is shared - i.e. this is a property of eth2.
The main concern is that this is the way I'd expect NetworkManager to
support it - i.e. that you could configure NetworkManager to share eth0,
rather than ask it to create br0 and add eth0 to it.
If you just want to create a bridge, you can creati a virtual network.
<vlan device="eth0" tag="42"
reorder_hdr="yes"/>
I think these last three are also interfaces and should be described
with an <interface> tag e.g.
<interface device="bond0" onboot="yes">
<bond mode="active-backup"/>
<slave device="eth0" primary="yes"/>
<slave device="eth1"/>
<dhcp/>
</interface>
<interface device="eth2" onboot="yes">
<hwaddr>00:19:d2:9f:88:98</hwaddr>
<shared bridge="br0"/>
<dhcp/>
</interface>
<interface device="eth0">
<vlan tag="42" reorder_hdr="yes"/>
</interface>
All of these should also allow a <uuid> element for specifying
a uuid; I
omitted that for brevity.
2. API Changes
==============
There are two options for dealing with network interfaces: (1) use the
existing virNetwork* calls or (2) add completely new API calls.
Repurposing existing virNetwork* calls
--------------------------------------
The existing calls map well to the operations we need for managing
interfaces, with a few exceptions:
- virNetworkGetAutostart/SetAutostart: depending on how we implement all
this (see below), 'autostart' might actually mean 'on boot', not
'when
libvirtd starts'
- virNetworkGetBridgeName doesn't make sense for interfaces, and should
return NULL for interfaces
We'll probably also end up adding some functions to query details about an
interface, in particular, a call to see what kind of network/interface a
virNetworkPtr represents
I don't think I like this much - these APIs manage a "virtual network"
which I see as a distinct concept from configuring host hardware.
Really, I think keeping the two things separate will actually reduce
confusion in the long run.
Add completely new virInterface* calls
--------------------------------------
This would add roughly the same API calls as the virNetwork* calls,
i.e. we'd have something like
typedef struct virInterface *virInterfacePtr;
int virInterfaceCreate(virInterfacePtr);
virInterfacePtr virInterfaceCreateXML(..);
...
plus some calls to extract information from a virInterfacePtr
This sounds good, but "interface" is pretty damn generic :-)
virNetInterface
virNetDevice
...
What I don't like about any of these is that I've always imagined we
might add further APIs to libvirt for changing a domain's configuration
without munging XML e.g.
int virDomainGetInterfaces(virDomainPtr domain,
virInterfacePtr interfaces,
int *ninterfaces);
int virInterfaceSetMacAddress(virInterfacePtr interface,
const uint8_t mac[6]);
So, you can see the potential namespace conflicts ...
(But that's pretty irrelevant since I think I'm alone in thinking APIs
like that would make sense :-)
The second option seems cleaner to me and easier to implement, and
avoids
any subtle changes in the behavior of existing API, though I don't like
that we'll be adding another 20 or so calls to libvirt's public API, with
attendant churn both in the drivers and in virsh.
I don't think the amount of boilerplate you need to add for new APIs
should stop you - the more of this crud we add, the more incentive we'll
have to figure out ways to reduce it :-)
3. Implementation
=================
Configuring network interfaces is highly OS and OS-variant/distro
dependant. There are at least two different ways how libvirt can go about
modifying the host to create interfaces:
1. Modify the system's network setup scripts (ifcfg-XXX on RH)
2. Directly use the system's network utilities like ifconfig
3. Rely on NetworkManager (not an option right now, as NM doesn't know
about bridges and the like)
It has to be an option - even if it only supports a subset of what the
other options support.
I really wouldn't like to see us push ahead with this until we have a
plan for how this will work with NetworkManager (and a rough agreement
for how future support for bonding etc. in NetworkManager might be
configured)
Option (1) saves us from replicating every bit of network setup
functionality that those scripts already have - besides configuring the
interface, we also might have to setup routes, run dhclient etc.
Option (2) would require far fewer backend implementations than (1) - we
should be able to get away with one implementation for Linux, rather than
one for Fedora/RHEL, one for Debian, one for SuSe, three for gentoo
etc.
The thing is, this has to integrate with existing configuration -
there's no point in futzing about with "ip link set eth0 ..." if the
user has configured eth0 with NetworkManager or the distro's networking
scripts.
If we want 'autostart' for an interface to mean 'bring up
the interface
as soon as the system boots', we are pretty much forced to go with
option (1).
Why?
All in all, option (1) seems more attractive, since it should save us
from
dealing with a lot of low-level details of network setup, and the distro
scripts should be much better integrated with the rest of the system than
what we come up with for libvirt.
IMHO, (1) and (3) are requirements and we'll probably end up doing (2)
too in the long run.
Cheers,
Mark.
7:31 a.m.
On Fri, Jan 16, 2009 at 09:00:16AM +0000, Mark McLoughlin wrote:
What I don't like about any of these is that I've always
imagined we
might add further APIs to libvirt for changing a domain's configuration
without munging XML e.g.
I'm glad to hear this is being considered: you're most certainly not
alone. This is a significant wart in the current API IMHO.
regards
john
7:49 a.m.
On Fri, Jan 16, 2009 at 09:00:16AM +0000, Mark McLoughlin wrote:
Hi David,
On Fri, 2009-01-16 at 01:13 +0000, David Lutterkort wrote:
> 3. Implementation
> =================
>
> Configuring network interfaces is highly OS and OS-variant/distro
> dependant. There are at least two different ways how libvirt can go about
> modifying the host to create interfaces:
>
> 1. Modify the system's network setup scripts (ifcfg-XXX on RH)
>
> 2. Directly use the system's network utilities like ifconfig
>
> 3. Rely on NetworkManager (not an option right now, as NM doesn't know
> about bridges and the like)
It has to be an option - even if it only supports a subset of what the
other options support.
I really wouldn't like to see us push ahead with this until we have a
plan for how this will work with NetworkManager (and a rough agreement
for how future support for bonding etc. in NetworkManager might be
configured)
Absolutely - NetworkManager is a fundamental requirement for this to be
viable, since it is the default mgmt backend for Linux desktop deployments,
and potentially even for Server deployments in future.
Also need to take into account the network mgmt capabilities of other
virt APIs such as VMWare / XenAPI, since it is very desirable to be able
to target these as a backend impl when using those hypervisor drivers
> Option (1) saves us from replicating every bit of network setup
> functionality that those scripts already have - besides configuring the
> interface, we also might have to setup routes, run dhclient etc.
The important benefit of (1), is that libvirt plays nicely with
distro scripts, rather than fighting/racing with them.
> Option (2) would require far fewer backend implementations than
(1) - we
> should be able to get away with one implementation for Linux, rather than
> one for Fedora/RHEL, one for Debian, one for SuSe, three for gentoo
> etc.
The thing is, this has to integrate with existing configuration -
there's no point in futzing about with "ip link set eth0 ..." if the
user has configured eth0 with NetworkManager or the distro's networking
scripts.
Yes, I think option 2 is pretty much doomed to fail. If you don't
integrate with the native distro mgmt scripts, or with NetworkManager
then you are forever going to have a fight between who's managing
what. You can already see this with Xen's network-bridge script
which cna be considered option 2. If you restart your network using
the distro scripts, after Xen's network-bridge script has run, then
you end up wwith a completely trashed host network.
> All in all, option (1) seems more attractive, since it should
save us from
> dealing with a lot of low-level details of network setup, and the distro
> scripts should be much better integrated with the rest of the system than
> what we come up with for libvirt.
IMHO, (1) and (3) are requirements and we'll probably end up doing (2)
too in the long run.
I think (2) should really be avoided - integration with the OS
is of primary importance to avoid getting things into a total
mess.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
4:28 p.m.
On Fri, 2009-01-16 at 13:49 +0000, Daniel P. Berrange wrote:
On Fri, Jan 16, 2009 at 09:00:16AM +0000, Mark McLoughlin wrote:
>
> The thing is, this has to integrate with existing configuration -
> there's no point in futzing about with "ip link set eth0 ..." if the
> user has configured eth0 with NetworkManager or the distro's networking
> scripts.
Yes, I think option 2 is pretty much doomed to fail. If you don't
integrate with the native distro mgmt scripts, or with NetworkManager
then you are forever going to have a fight between who's managing
what. You can already see this with Xen's network-bridge script
which cna be considered option 2. If you restart your network using
the distro scripts, after Xen's network-bridge script has run, then
you end up wwith a completely trashed host network.
That goes for pretty much any config munging task - don't ever change
the canonical place where config data is stored; for network devices
that's dictated by the distro. Whatever we do for libvirt can only ever
be an (updatable) view of that data.
David
4:23 p.m.
Hi Mark,
On Fri, 2009-01-16 at 09:00 +0000, Mark McLoughlin wrote:
Hi David,
On Fri, 2009-01-16 at 01:13 +0000, David Lutterkort wrote:
>
> <bond name="bond00" onboot="yes"
mode="active-backup">
> <slave device="eth0" primary="yes"/>
> <slave device="eth1"/>
> </bond>
>
> <bridge name="br0" stp="off" onboot="yes">
> <member device="eth2"/>
> <dhcp peerdns="yes"/>
> </bridge>
I don't think we want to define a bridge here, but more that an
interface is shared - i.e. this is a property of eth2.
The main concern is that this is the way I'd expect NetworkManager to
support it - i.e. that you could configure NetworkManager to share eth0,
rather than ask it to create br0 and add eth0 to it.
If you just want to create a bridge, you can creati a virtual network.
Is it possible to use DHCP to configure the bridge device itself using
that, similar to what's described in [1] ? And have guest's DHCP
requests forwarded across the bridge ? The docs only talk about static
IP assignments for the bridge device.
> <vlan device="eth0" tag="42"
reorder_hdr="yes"/>
I think these last three are also interfaces and should be described
with an <interface> tag e.g.
Agreed.
I don't think I like this much - these APIs manage a
"virtual network"
which I see as a distinct concept from configuring host hardware.
Really, I think keeping the two things separate will actually reduce
confusion in the long run.
Agreed.
This sounds good, but "interface" is pretty damn generic
:-)
virNetInterface
virNetDevice
...
Sure, virInterface is pretty generic; maybe virNetDevice, though we'd
have to call it <net-device> in the XML for consistency. Naming is
NP-hard ;)
What I don't like about any of these is that I've always
imagined we
might add further APIs to libvirt for changing a domain's configuration
without munging XML e.g.
int virDomainGetInterfaces(virDomainPtr domain,
virInterfacePtr interfaces,
int *ninterfaces);
int virInterfaceSetMacAddress(virInterfacePtr interface,
const uint8_t mac[6]);
So, you can see the potential namespace conflicts ...
Not really, since this still follows the naming convention 'vir' +
class_name + method_name. There's a virDomainInterfaceStats, and it
doesn't seem all that confusing to me to distinguish that from a
virInterfaceStats (i.e. stats for all interfaces on the host)
> 3. Implementation
> =================
>
> Configuring network interfaces is highly OS and OS-variant/distro
> dependant. There are at least two different ways how libvirt can go about
> modifying the host to create interfaces:
>
> 1. Modify the system's network setup scripts (ifcfg-XXX on RH)
>
> 2. Directly use the system's network utilities like ifconfig
>
> 3. Rely on NetworkManager (not an option right now, as NM doesn't know
> about bridges and the like)
It has to be an option - even if it only supports a subset of what the
other options support.
I really wouldn't like to see us push ahead with this until we have a
plan for how this will work with NetworkManager (and a rough agreement
for how future support for bonding etc. in NetworkManager might be
configured)
Yeah, I started talking to Dan Williams about their plans etc. Of
course, their main use case (wireless networking) is unimportant for
libvirt. But we should be able to find a way to share some of the
backend bits and the general model for interfaces, umm, net-devices ;)
The thing is, this has to integrate with existing configuration -
there's no point in futzing about with "ip link set eth0 ..." if the
user has configured eth0 with NetworkManager or the distro's networking
scripts.
Agreed.
> If we want 'autostart' for an interface to mean
'bring up the interface
> as soon as the system boots', we are pretty much forced to go with
> option (1).
Why?
Because those are what brings up networking at boot - otherwise we'd
define 'autostart' as 'whenever libvirtd starts' - but it doesn't
seem
like nay of this is controversial, anyway.
David
[1] http://wiki.libvirt.org/page/Networking#Fedora.2FRHEL_Bridging
12:48 p.m.
Hi David,
On Fri, 2009-01-16 at 22:23 +0000, David Lutterkort wrote:
> > <bridge name="br0" stp="off"
onboot="yes">
> > <member device="eth2"/>
> > <dhcp peerdns="yes"/>
> > </bridge>
>
> I don't think we want to define a bridge here, but more that an
> interface is shared - i.e. this is a property of eth2.
Note this line.
> The main concern is that this is the way I'd expect
NetworkManager to
> support it - i.e. that you could configure NetworkManager to share eth0,
> rather than ask it to create br0 and add eth0 to it.
>
> If you just want to create a bridge, you can creati a virtual network.
Is it possible to use DHCP to configure the bridge device itself using
that, similar to what's described in [1] ? And have guest's DHCP
requests forwarded across the bridge ? The docs only talk about static
IP assignments for the bridge device.
That configuration is what I'm calling a "shared physical interface".
I wouldn't be surprised if NetworkManager could only be used for this
exact configuration rather than any possible variation of bridge
configuration.
Which is why I'm suggesting just using a "shared" property on a
interface - this property would imply the creation of a bridge.
Cheers,
Mark.
3:34 p.m.
Mark McLoughlin wrote:
Hi David,
On Fri, 2009-01-16 at 22:23 +0000, David Lutterkort wrote:
> > > <bridge name="br0" stp="off"
onboot="yes">
>>> <member device="eth2"/>
>>> <dhcp peerdns="yes"/>
>>> </bridge>
>> I don't think we want to define a bridge here, but more that an
>> interface is shared - i.e. this is a property of eth2.
Note this line.
>> The main concern is that this is the way I'd expect NetworkManager to
>> support it - i.e. that you could configure NetworkManager to share eth0,
>> rather than ask it to create br0 and add eth0 to it.
>>
>> If you just want to create a bridge, you can creati a virtual network.
Sorry to chime in so late... the virtual network support only allows
the user to define bridges with NAT/routed forwarding.
> Is it possible to use DHCP to configure the bridge device itself
using
> that, similar to what's described in [1] ? And have guest's DHCP
> requests forwarded across the bridge ? The docs only talk about static
> IP assignments for the bridge device.
That configuration is what I'm calling a "shared physical interface".
I wouldn't be surprised if NetworkManager could only be used for this
exact configuration rather than any possible variation of bridge
configuration.
Which is why I'm suggesting just using a "shared" property on a
interface - this property would imply the creation of a bridge.
Not sure if I'm missing the mark here..
Would the still behave as a virtual network pool in this case? If
multiple guests are tied to the same bridge, it would be useful to
represent this as some kind of pool or grouping.
--
Kaitlin Rupert
IBM Linux Technology Center
kaitlin(a)linux.vnet.ibm.com
12:22 p.m.
On Thu, 2009-01-29 at 13:34 -0800, Kaitlin Rupert wrote:
Mark McLoughlin wrote:
>
>>> I don't think we want to define a bridge here, but more that an
>>> interface is shared - i.e. this is a property of eth2.
>
> Note this line.
>
>>> The main concern is that this is the way I'd expect NetworkManager to
>>> support it - i.e. that you could configure NetworkManager to share eth0,
>>> rather than ask it to create br0 and add eth0 to it.
>>>
>>> If you just want to create a bridge, you can creati a virtual network.
Sorry to chime in so late... the virtual network support only allows
the user to define bridges with NAT/routed forwarding.
I tripped over this the first time, too; to put words in Mark's mouth,
he meant 'bridge' here in the sense of a 'vritual network'. He calls a
bridge with one enslaved physical NIC a 'shared network interface' (and
doesn't much care for bridges with more NIC's ;)
After talking to NM people, there doesn't seem to be a concern with NM
compat - they are fine with the more explicit representation of a
bridge.
Would the still behave as a virtual network pool in this case? If
multiple guests are tied to the same bridge, it would be useful to
represent this as some kind of pool or grouping.
Not sure what you mean here ... with all this, you'd still set your
guests up the way you do today, with appropriate sources for their
network interfaces, e.g. <source bridge='br0'/>
David
5:41 p.m.
David Lutterkort wrote:
On Thu, 2009-01-29 at 13:34 -0800, Kaitlin Rupert wrote:
> Mark McLoughlin wrote:
>>>> I don't think we want to define a bridge here, but more that an
>>>> interface is shared - i.e. this is a property of eth2.
>> Note this line.
>>
>>>> The main concern is that this is the way I'd expect NetworkManager
to
>>>> support it - i.e. that you could configure NetworkManager to share eth0,
>>>> rather than ask it to create br0 and add eth0 to it.
>>>>
>>>> If you just want to create a bridge, you can creati a virtual network.
> Sorry to chime in so late... the virtual network support only allows
> the user to define bridges with NAT/routed forwarding.
I tripped over this the first time, too; to put words in Mark's mouth,
he meant 'bridge' here in the sense of a 'vritual network'. He calls a
bridge with one enslaved physical NIC a 'shared network interface' (and
doesn't much care for bridges with more NIC's ;)
Ah, okay. This makes sense. Thanks for the clarification.
After talking to NM people, there doesn't seem to be a concern with NM
compat - they are fine with the more explicit representation of a
bridge.
> Would the still behave as a virtual network pool in this case? If
> multiple guests are tied to the same bridge, it would be useful to
> represent this as some kind of pool or grouping.
Not sure what you mean here ... with all this, you'd still set your
guests up the way you do today, with appropriate sources for their
network interfaces, e.g. <source bridge='br0'/>
Would there be the bridge equivalents of
virConnectListNetworks()/virConnectListDefinedNetworks(): a way to list
the available bridges?
After reading through all the mails again, I believe this is what's
being proposed, so I think I might have answered my own question. =)
--
Kaitlin Rupert
IBM Linux Technology Center
kaitlin(a)linux.vnet.ibm.com
6:35 a.m.
On Fri, Jan 16, 2009 at 01:13:18AM +0000, David Lutterkort wrote:
For certain applications, we want libvirt to be able to configure
host
network interfaces in a variety of ways; currently, we are most
interested in teaching libvirt how to set up ordinary ethernet
interfaces, bridges, bonding and vlan's.
[...]
3. Implementation
=================
Configuring network interfaces is highly OS and OS-variant/distro
dependant. There are at least two different ways how libvirt can go about
modifying the host to create interfaces:
1. Modify the system's network setup scripts (ifcfg-XXX on RH)
2. Directly use the system's network utilities like ifconfig
3. Rely on NetworkManager (not an option right now, as NM doesn't know
about bridges and the like)
I must be missing something here ... how can libvirt modify stuff
inside the guest?
Rich.
--
Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top
6:41 a.m.
On Fri, Jan 16, 2009 at 12:35:55PM +0000, Richard W.M. Jones wrote:
On Fri, Jan 16, 2009 at 01:13:18AM +0000, David Lutterkort wrote:
> For certain applications, we want libvirt to be able to configure host
> network interfaces in a variety of ways; currently, we are most
> interested in teaching libvirt how to set up ordinary ethernet
> interfaces, bridges, bonding and vlan's.
[...]
> 3. Implementation
> =================
>
> Configuring network interfaces is highly OS and OS-variant/distro
> dependant. There are at least two different ways how libvirt can go about
> modifying the host to create interfaces:
>
> 1. Modify the system's network setup scripts (ifcfg-XXX on RH)
>
> 2. Directly use the system's network utilities like ifconfig
>
> 3. Rely on NetworkManager (not an option right now, as NM doesn't know
> about bridges and the like)
I must be missing something here ... how can libvirt modify stuff
inside the guest?
It can't - this is all about configuring & managing networking on the
host OS.
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
6:44 a.m.
On Fri, Jan 16, 2009 at 12:41:42PM +0000, Daniel P. Berrange wrote:
On Fri, Jan 16, 2009 at 12:35:55PM +0000, Richard W.M. Jones wrote:
> On Fri, Jan 16, 2009 at 01:13:18AM +0000, David Lutterkort wrote:
> > For certain applications, we want libvirt to be able to configure host
> > network interfaces in a variety of ways; currently, we are most
> > interested in teaching libvirt how to set up ordinary ethernet
> > interfaces, bridges, bonding and vlan's.
> [...]
> > 3. Implementation
> > =================
> >
> > Configuring network interfaces is highly OS and OS-variant/distro
> > dependant. There are at least two different ways how libvirt can go about
> > modifying the host to create interfaces:
> >
> > 1. Modify the system's network setup scripts (ifcfg-XXX on RH)
> >
> > 2. Directly use the system's network utilities like ifconfig
> >
> > 3. Rely on NetworkManager (not an option right now, as NM doesn't know
> > about bridges and the like)
>
> I must be missing something here ... how can libvirt modify stuff
> inside the guest?
It can't - this is all about configuring & managing networking on the
host OS.
Well OK then, how does the information get passed into the guest?
Rich.
--
Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/
6:53 a.m.
On Fri, Jan 16, 2009 at 12:44:21PM +0000, Richard W.M. Jones wrote:
On Fri, Jan 16, 2009 at 12:41:42PM +0000, Daniel P. Berrange wrote:
> On Fri, Jan 16, 2009 at 12:35:55PM +0000, Richard W.M. Jones wrote:
> > On Fri, Jan 16, 2009 at 01:13:18AM +0000, David Lutterkort wrote:
> > > For certain applications, we want libvirt to be able to configure host
> > > network interfaces in a variety of ways; currently, we are most
> > > interested in teaching libvirt how to set up ordinary ethernet
> > > interfaces, bridges, bonding and vlan's.
> > [...]
> > > 3. Implementation
> > > =================
> > >
> > > Configuring network interfaces is highly OS and OS-variant/distro
> > > dependant. There are at least two different ways how libvirt can go about
> > > modifying the host to create interfaces:
> > >
> > > 1. Modify the system's network setup scripts (ifcfg-XXX on RH)
> > >
> > > 2. Directly use the system's network utilities like ifconfig
> > >
> > > 3. Rely on NetworkManager (not an option right now, as NM doesn't
know
> > > about bridges and the like)
> >
> > I must be missing something here ... how can libvirt modify stuff
> > inside the guest?
>
> It can't - this is all about configuring & managing networking on the
> host OS.
Well OK then, how does the information get passed into the guest?
There isn't anything to pass into the guest . This is about how to
configure host network interfaces for bridging, bonding, vlans, etc
eg, So you can take an host with a NIC, eth0, put that NIC into a
bridge, br0, and thus now have the ability to create guests using
a bridged networking configuration.
As it currently stands admins have to manually configure bridges
in their host by editing /etc/sysconfig/networking-scripts/ifcfg-eth0
This proposal is to provide an API for doing this in the host instead
of requiring that admin do it manually ahead of time.
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
7:07 a.m.
On Fri, Jan 16, 2009 at 12:53:28PM +0000, Daniel P. Berrange wrote:
There isn't anything to pass into the guest . This is about how
to
configure host network interfaces for bridging, bonding, vlans, etc
eg, So you can take an host with a NIC, eth0, put that NIC into a
bridge, br0, and thus now have the ability to create guests using
a bridged networking configuration.
OK, got you. I thought originally this was for guest configuration.
Rich.
--
Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones
Read my OCaml programming blog: http://camltastic.blogspot.com/
Fedora now supports 68 OCaml packages (the OPEN alternative to F#)
http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora
9:06 a.m.
On Fri, Jan 16, 2009 at 01:13:18AM +0000, David Lutterkort wrote:
1. XML Format
=============
The first question is how the user would describe the host interfaces they
want. Below are some sketches of what an XML representation of the various
kinds of interfaces would look like. This covers the minimal amount of
settings for these to be useful, though I am sure we'll need to add more
over time.
<interface device="eth0" onboot="yes">
<hwaddr>00:19:d2:9f:88:96</hwaddr>
<dhcp peerdns="yes"/>
</interface>
<interface device="eth1" onboot="yes">
<hwaddr>00:19:d2:9f:88:97</hwaddr>
<static ipaddr="192.168.0.5" gateway="192.168.0.1"
netmask="255.255.255.0"/>
</interface>
<interface device="eth2" onboot="yes">
<hwaddr>00:19:d2:9f:88:98</hwaddr>
</interface>
<bond name="bond00" onboot="yes"
mode="active-backup">
<slave device="eth0" primary="yes"/>
<slave device="eth1"/>
</bond>
<bridge name="br0" stp="off" onboot="yes">
<member device="eth2"/>
<dhcp peerdns="yes"/>
</bridge>
<vlan device="eth0" tag="42" reorder_hdr="yes"/>
From a style point of view I'd prefer these to all have the same
top level XML element name, and use type='phys|bond|vlan|bridge'
attribute for distinction, since this follows the pattern we use
in the XML for other objects we describe. Type speciic sub-elements
can be used where there are specific extra metadat pieces we need
for that type. Also prefer to see it follow the syntax
used elsewhere for mac addresses, and finally agreed with te idea
that 'onboot' should actually be handled by the get/set-autostart
API, again to match the model used elsehwere in the APIs.
<interface type='physical'>
<name>eth0</name>
<mac address='00:19:d2:9f:88:96'/>
<dhcp peerdns="yes"/>
</interface>
<interface type='physical'
<name>eth1</name>
<mac address='00:19:d2:9f:88:97'/>
<ip address="192.168.0.5" gateway="192.168.0.1"
netmask="255.255.255.0"/>
</interface>
<interface type='physical'
<name>eth2</name>
<mac address='00:19:d2:9f:88:98'/>
</interface>
<interface type='bond'>
<name>bond0</name>
<bond mode='active-backup'>
<interface name="eth0" primary="yes"/>
<interface name="eth1"/>
</bond>
</bond>
<interface type='bridge'>
<name>br0</name>
<bridge stp='off'>
<interface name="eth2"/>
</bridge>
<dhcp peerdns="yes"/>
</bridge>
<interface type='vlan'>
<name>vlan0</name>
<vlan tag="42" reorder_hdr="yes">
<interface name='eth0'>
</vlan>
</inteface>
All of these should also allow a <uuid> element for specifying
a uuid; I
omitted that for brevity.
2. API Changes
==============
There are two options for dealing with network interfaces: (1) use the
existing virNetwork* calls or (2) add completely new API calls.
Repurposing existing virNetwork* calls
--------------------------------------
The existing calls map well to the operations we need for managing
interfaces, with a few exceptions:
- virNetworkGetAutostart/SetAutostart: depending on how we implement all
this (see below), 'autostart' might actually mean 'on boot', not
'when
libvirtd starts'
- virNetworkGetBridgeName doesn't make sense for interfaces, and should
return NULL for interfaces
We'll probably also end up adding some functions to query details about an
interface, in particular, a call to see what kind of network/interface a
virNetworkPtr represents
I think the fact that the XML description of the devices is quite
different, and the underling impl will be quite different, suggests
that re-using existing APIs is not worthwhile.
Add completely new virInterface* calls
--------------------------------------
This would add roughly the same API calls as the virNetwork* calls,
i.e. we'd have something like
typedef struct virInterface *virInterfacePtr;
int virInterfaceCreate(virInterfacePtr);
virInterfacePtr virInterfaceCreateXML(..);
...
plus some calls to extract information from a virInterfacePtr
The second option seems cleaner to me and easier to implement, and avoids
any subtle changes in the behavior of existing API, though I don't like
that we'll be adding another 20 or so calls to libvirt's public API, with
attendant churn both in the drivers and in virsh.
I think the number of public API entry points is not worth
worrying about, since that's such a tiny part of the code
effort. Any saving from reusing the virNetwork API entry
points is dwarfed by the code to actually implemnent these
network capabilities internally.
4. Misc issues
==============
* Should interfaces have labels/roles ('data-interface') to help admins
make sense of the current config ?
Those are application defined semantics really, so I don't think
they need direct representation in libvirt. It would also entail
an extra look-aside config cache, because initscript / networkmanager
(or other backend impl) don't provide any means to store such
labels/roles.
* Do we expect interfaces to be in a specific state before we
create them
or do we just tear them down and reconfigure them no matter what ?
I think we should expect the mgmt app to have put the device
in suitable state. eg, if you're creating a bridge containing
eth0, we should expect that the eth0 has been taken offline
already.
* Are there crucial config options that are not covered by the
sketches
above (e.g., setting an explicit MTU) ? Are there things in the XML
sketches above that will be impossible to implement on some OS ?
There are almost certainly things we won't be able to implement for certain
backends. XenAPI's API does not allow for VLANs for example. The important
thing would be to provide a way to query what capabilities are curently
available. So akin to the host virt capabilities, we'd need network
API capabilities data.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
10:15 a.m.
On Fri, Jan 16, 2009 at 03:06:16PM +0000, Daniel P. Berrange wrote:
<interface type='physical'>
<name>eth0</name>
<mac address='00:19:d2:9f:88:96'/>
<dhcp peerdns="yes"/>
</interface>
<interface type='physical'
<name>eth1</name>
<mac address='00:19:d2:9f:88:97'/>
<ip address="192.168.0.5" gateway="192.168.0.1"
netmask="255.255.255.0"/>
</interface>
<interface type='physical'
<name>eth2</name>
<mac address='00:19:d2:9f:88:98'/>
</interface>
<interface type='bond'>
<name>bond0</name>
<bond mode='active-backup'>
<interface name="eth0" primary="yes"/>
<interface name="eth1"/>
</bond>
</bond>
<interface type='bridge'>
<name>br0</name>
<bridge stp='off'>
<interface name="eth2"/>
</bridge>
<dhcp peerdns="yes"/>
</bridge>
<interface type='vlan'>
<name>vlan0</name>
<vlan tag="42" reorder_hdr="yes">
<interface name='eth0'>
</vlan>
</inteface>
> * Are there crucial config options that are not covered by the
sketches
> above (e.g., setting an explicit MTU) ? Are there things in the XML
> sketches above that will be impossible to implement on some OS ?
There are almost certainly things we won't be able to implement for certain
backends. XenAPI's API does not allow for VLANs for example. The important
thing would be to provide a way to query what capabilities are curently
available. So akin to the host virt capabilities, we'd need network
API capabilities data.
Actually I tell a lie - XenAPI does allow for bonding + vlans, using a
generic 'interface' class and a master/slave relation to associate
bonds with their interfaces, and interfaces with vlans. So the XML format
and functionality should map onto that quite well.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
3:50 p.m.
On Fri, 2009-01-16 at 15:06 +0000, Daniel P. Berrange wrote:
On Fri, Jan 16, 2009 at 01:13:18AM +0000, David Lutterkort wrote:
From a style point of view I'd prefer these to all have the same
top level XML element name, and use type='phys|bond|vlan|bridge'
attribute for distinction, since this follows the pattern we use
in the XML for other objects we describe. Type speciic sub-elements
can be used where there are specific extra metadat pieces we need
for that type.
Agreed - I initially thought you couldn't describe that with Relax-NG
(element content being completely different depending on the value of an
attribute), but found out that you actually can do that.
I think the fact that the XML description of the devices is quite
different, and the underling impl will be quite different, suggests
that re-using existing APIs is not worthwhile.
Agreed - that seems to be the consensus.
> * Do we expect interfaces to be in a specific state before we
create them
> or do we just tear them down and reconfigure them no matter what ?
I think we should expect the mgmt app to have put the device
in suitable state. eg, if you're creating a bridge containing
eth0, we should expect that the eth0 has been taken offline
already.
What's the mgmt app in this context ? I am thinking that if you tell
libvirt to change the config of eth0, it should just nuke whatever
config it may already have.
> * Are there crucial config options that are not covered by the
sketches
> above (e.g., setting an explicit MTU) ? Are there things in the XML
> sketches above that will be impossible to implement on some OS ?
There are almost certainly things we won't be able to implement for certain
backends. XenAPI's API does not allow for VLANs for example.
This brings up an important wrinkle: selection of the 'interface' driver
will mostly be independent of the HV, since it depends more on the
OS/distro you are running as your host than on your HV. I've been hoping
we could just sidestep Xen's network handling altogether and just focus
on doing the right thing for the host platform, regardless of HV. Since
we need that for kvm anyway, dealing with the networking part of Xen
would just add redundant driver implementations.
David
5:05 p.m.
David Lutterkort wrote:
For certain applications, we want libvirt to be able to configure
host
network interfaces in a variety of ways; currently, we are most
interested in teaching libvirt how to set up ordinary ethernet
interfaces, bridges, bonding and vlan's.
I agree that carving up host physical resources, including networking,
is appropriate in libvirt. Others have such support in their
virutalization APIs an the DMTF is currently working on models in SVPC.
I had requests for this functionality in libvirt several months back but
no cycles to seriously investigate.
[...]
3. Implementation
=================
Configuring network interfaces is highly OS and OS-variant/distro
dependant. There are at least two different ways how libvirt can go about
modifying the host to create interfaces:
1. Modify the system's network setup scripts (ifcfg-XXX on RH)
2. Directly use the system's network utilities like ifconfig
3. Rely on NetworkManager (not an option right now, as NM doesn't know
about bridges and the like)
Option (1) saves us from replicating every bit of network setup
functionality that those scripts already have - besides configuring the
interface, we also might have to setup routes, run dhclient etc.
This needs to be supported to play well with all of the existing
scripts. I think Daniel P already pointed this out in another response.
Option (2) would require far fewer backend implementations than (1) -
we
should be able to get away with one implementation for Linux, rather than
one for Fedora/RHEL, one for Debian, one for SuSe, three for gentoo
etc.
Xen's network-bridge script uses this approach and it's disastrous IMO.
Try rcnetwork restart on a network-bridge configured host :-). In
SLE11, we dropped this script and manage host network resources via
YaST. But, users want a nice remote interface that will work across
heterogeneous hosts ...
Your Option 3 needs to be supported as well and is one reason I have not
tackled this subject. I don't know anything about NetworkManager and
have not had time to dig in. That's about to change with SLE11 reaching
late RC phase :-).
If we want 'autostart' for an interface to mean 'bring up
the interface
as soon as the system boots', we are pretty much forced to go with
option (1).
Right. As in STARTMODE='onboot'
All in all, option (1) seems more attractive, since it should save us
from
dealing with a lot of low-level details of network setup, and the distro
scripts should be much better integrated with the rest of the system than
what we come up with for libvirt.
Although I'm no expert in SuSE sysconfig scripts, I think this approach
works best for SuSE distros. I'll have to alert a sysconfig maintainer
of this thread.
Thanks for raising this topic :-).
Jim
12:35 p.m.
David Lutterkort wrote:
For certain applications, we want libvirt to be able to configure
host
network interfaces in a variety of ways; currently, we are most
interested in teaching libvirt how to set up ordinary ethernet
interfaces, bridges, bonding and vlan's.
Below is a high-level proposal of how that could be done. Please comment
copiously ;)
1. XML Format
=============
The first question is how the user would describe the host interfaces they
want. Below are some sketches of what an XML representation of the various
kinds of interfaces would look like. This covers the minimal amount of
settings for these to be useful, though I am sure we'll need to add more
over time.
<interface device="eth0" onboot="yes">
<hwaddr>00:19:d2:9f:88:96</hwaddr>
<dhcp peerdns="yes"/>
</interface>
For onboot="yes", something like startmode="onboot" would be better
IMO. A startmode attribute would allow also using "manual",
"hotplug",
"ifplug", "nfsroot", etc.
WRT dhcp element, would it make sense to consider hostname (hostname to
send to server) and if to change the local hostname to the hostname
delivered via dhcp? E.g.
hostname="foo" (default `hostname`)
sethostname
Also route:
setrouting (default "yes")
setdefaultroute (default "yes")
and NIS:
nis (default "yes")
setnisdomain (default "yes")
What about dhcpv6? A separate <dhcp6 /> element?
<interface device="eth1" onboot="yes">
<hwaddr>00:19:d2:9f:88:97</hwaddr>
<static ipaddr="192.168.0.5" gateway="192.168.0.1"
netmask="255.255.255.0"/>
</interface>
Similarly, support for IPv6 here. Either 2 formats:
ipaddr="192.168.0.5" netmask="255.255.255.0"
ip6addr="2001:DB8:C::5/64"
or allow /len for both:
ipaddr="192.168.0.5/24"
ipaddr="2001:DB8:ABCD::1/64"
As for routing info, how about a separate route element:
<route gateway="192.168.0.1" /> # destination=default
<route destination="default" gateway="192.168.0.1" />
<route destination="10.0.0.0/8" gateway="192.168.0.2" />
<route destination="2001:DB8:C::/64" gateway="2001:DB8:C::1" />
<route destination="2001:DB8::/32"> # unrecheable route (loopback)
It would perhaps make sense to use iproute2 names, that is prefix
instead of destination and nexthop instead of gateway.
Don't recall if this was already brought up, but need a way to specify
MTU and perhaps ethtool settings as well.
<interface device="eth2" onboot="yes">
<hwaddr>00:19:d2:9f:88:98</hwaddr>
</interface>
<bond name="bond00" onboot="yes"
mode="active-backup">
<slave device="eth0" primary="yes"/>
<slave device="eth1"/>
</bond>
In addition to mode attribute for bonds, support for miimon,
arp_interval, and arp_ip_target?
<bridge name="br0" stp="off"
onboot="yes">
<member device="eth2"/>
<dhcp peerdns="yes"/>
</bridge>
Attributes to support hellotime, forwarddelay, and maxage? This would allow
<bridge name="br0" stp="on" hellotime="1"
maxage="4" fowarddelay="4"
..>
Regards,
Jim
4:51 p.m.
On Tue, 2009-01-20 at 11:35 -0700, Jim Fehlig wrote:
David Lutterkort wrote:
> For certain applications, we want libvirt to be able to configure host
> network interfaces in a variety of ways; currently, we are most
> interested in teaching libvirt how to set up ordinary ethernet
> interfaces, bridges, bonding and vlan's.
>
> Below is a high-level proposal of how that could be done. Please comment
> copiously ;)
>
> 1. XML Format
> =============
>
> The first question is how the user would describe the host interfaces they
> want. Below are some sketches of what an XML representation of the various
> kinds of interfaces would look like. This covers the minimal amount of
> settings for these to be useful, though I am sure we'll need to add more
> over time.
>
> <interface device="eth0" onboot="yes">
> <hwaddr>00:19:d2:9f:88:96</hwaddr>
> <dhcp peerdns="yes"/>
> </interface>
>
For onboot="yes", something like startmode="onboot" would be better
IMO. A startmode attribute would allow also using "manual",
"hotplug",
"ifplug", "nfsroot", etc.
This makes only sense if these additional modes are completely
orthogonal; I am not sure what the start modes 'manual', 'ifplug', and
'nfsroot' should do. For onboot, I assume you mean that they translate
into assignments in ifcfg in the following way
no startmode attribute -> ONBOOT=no
startmode='onboot' -> ONBOOT=yes
startmode='hotplug' -> ONBOOT=yes and HOTPLUG=yes
WRT dhcp element, would it make sense to consider hostname (hostname
to
send to server) and if to change the local hostname to the hostname
delivered via dhcp? E.g.
hostname="foo" (default `hostname`)
sethostname
Also route:
setrouting (default "yes")
setdefaultroute (default "yes")
and NIS:
nis (default "yes")
setnisdomain (default "yes")
Yes, they all make sense, though I would probably not support them in
the very first cut.
What about dhcpv6? A separate <dhcp6 /> element?
Yeah, we probably need a separate element for that since it pulls in a
slew of other config options.
> <interface device="eth1"
onboot="yes">
> <hwaddr>00:19:d2:9f:88:97</hwaddr>
> <static ipaddr="192.168.0.5" gateway="192.168.0.1"
netmask="255.255.255.0"/>
> </interface>
>
Similarly, support for IPv6 here. Either 2 formats:
ipaddr="192.168.0.5" netmask="255.255.255.0"
ip6addr="2001:DB8:C::5/64"
or allow /len for both:
ipaddr="192.168.0.5/24"
ipaddr="2001:DB8:ABCD::1/64"
Looks a little strange, but it would simplify notation.
As for routing info, how about a separate route element:
<route gateway="192.168.0.1" /> # destination=default
<route destination="default" gateway="192.168.0.1" />
<route destination="10.0.0.0/8" gateway="192.168.0.2" />
<route destination="2001:DB8:C::/64" gateway="2001:DB8:C::1"
/>
<route destination="2001:DB8::/32"> # unrecheable route (loopback)
It would perhaps make sense to use iproute2 names, that is prefix
instead of destination and nexthop instead of gateway.
For now, I want to stay out of setting up static routes, but I think
that has to come sooner or later.
Don't recall if this was already brought up, but need a way to
specify
MTU and perhaps ethtool settings as well.
Yeah, that is definitely needed.
> <interface device="eth2"
onboot="yes">
> <hwaddr>00:19:d2:9f:88:98</hwaddr>
> </interface>
>
> <bond name="bond00" onboot="yes"
mode="active-backup">
> <slave device="eth0" primary="yes"/>
> <slave device="eth1"/>
> </bond>
>
In addition to mode attribute for bonds, support for miimon,
arp_interval, and arp_ip_target?
Sure .. if the initscript suports it, it's easy enough to do ;)
> <bridge name="br0" stp="off"
onboot="yes">
> <member device="eth2"/>
> <dhcp peerdns="yes"/>
> </bridge>
>
Attributes to support hellotime, forwarddelay, and maxage? This would allow
<bridge name="br0" stp="on" hellotime="1"
maxage="4" fowarddelay="4"
..>
My main concern with these is that only forwarddelay, stp, and gcint are
controllable by Fedora/RHEL networking scripts. For hellotime and
maxage, we'd need to hook into the various ifup scripts atthe right
place. Are these directly supported by the SuSe infrastructure ?
David
4:39 p.m.
Adding Marius (SuSE sysconfig maintainer) in case I'm misspeaking :-)
David Lutterkort wrote:
On Tue, 2009-01-20 at 11:35 -0700, Jim Fehlig wrote:
> David Lutterkort wrote:
>
>> For certain applications, we want libvirt to be able to configure host
>> network interfaces in a variety of ways; currently, we are most
>> interested in teaching libvirt how to set up ordinary ethernet
>> interfaces, bridges, bonding and vlan's.
>>
>> Below is a high-level proposal of how that could be done. Please comment
>> copiously ;)
>>
>> 1. XML Format
>> =============
>>
>> The first question is how the user would describe the host interfaces they
>> want. Below are some sketches of what an XML representation of the various
>> kinds of interfaces would look like. This covers the minimal amount of
>> settings for these to be useful, though I am sure we'll need to add more
>> over time.
>>
>> <interface device="eth0" onboot="yes">
>> <hwaddr>00:19:d2:9f:88:96</hwaddr>
>> <dhcp peerdns="yes"/>
>> </interface>
>>
>>
> For onboot="yes", something like startmode="onboot" would be
better
> IMO. A startmode attribute would allow also using "manual",
"hotplug",
> "ifplug", "nfsroot", etc.
>
This makes only sense if these additional modes are completely
orthogonal; I am not sure what the start modes 'manual', 'ifplug', and
'nfsroot' should do.
manual
Interface will be set up if ifup is called manually.
ifplugd
Interface will be controlled from ifplugd
nfsroot
Nearly like auto, but interfaces with this startmode will never be
shut down via rcnetwork stop
For onboot, I assume you mean that they translate
into assignments in ifcfg in the following way
no startmode attribute -> ONBOOT=no
startmode='onboot' -> ONBOOT=yes
startmode='hotplug' -> ONBOOT=yes and HOTPLUG=yes
On SuSE, the related ifcfg syntax is
STARTMODE={manual|auto|hotplug|ifplugd|nfsroot|off}
> WRT dhcp element, would it make sense to consider hostname
(hostname to
> send to server) and if to change the local hostname to the hostname
> delivered via dhcp? E.g.
> hostname="foo" (default `hostname`)
> sethostname
>
> Also route:
> setrouting (default "yes")
> setdefaultroute (default "yes")
>
> and NIS:
> nis (default "yes")
> setnisdomain (default "yes")
>
Yes, they all make sense, though I would probably not support them in
the very first cut.
> What about dhcpv6? A separate <dhcp6 /> element?
>
Yeah, we probably need a separate element for that since it pulls in a
slew of other config options.
>> <interface device="eth1" onboot="yes">
>> <hwaddr>00:19:d2:9f:88:97</hwaddr>
>> <static ipaddr="192.168.0.5" gateway="192.168.0.1"
netmask="255.255.255.0"/>
>> </interface>
>>
>>
> Similarly, support for IPv6 here. Either 2 formats:
>
> ipaddr="192.168.0.5" netmask="255.255.255.0"
> ip6addr="2001:DB8:C::5/64"
>
> or allow /len for both:
>
> ipaddr="192.168.0.5/24"
> ipaddr="2001:DB8:ABCD::1/64"
>
Looks a little strange, but it would simplify notation.
> As for routing info, how about a separate route element:
>
> <route gateway="192.168.0.1" /> # destination=default
> <route destination="default" gateway="192.168.0.1" />
> <route destination="10.0.0.0/8" gateway="192.168.0.2" />
> <route destination="2001:DB8:C::/64" gateway="2001:DB8:C::1"
/>
> <route destination="2001:DB8::/32"> # unrecheable route (loopback)
>
> It would perhaps make sense to use iproute2 names, that is prefix
> instead of destination and nexthop instead of gateway.
>
For now, I want to stay out of setting up static routes, but I think
that has to come sooner or later.
> Don't recall if this was already brought up, but need a way to specify
> MTU and perhaps ethtool settings as well.
>
Yeah, that is definitely needed.
>> <interface device="eth2" onboot="yes">
>> <hwaddr>00:19:d2:9f:88:98</hwaddr>
>> </interface>
>>
>> <bond name="bond00" onboot="yes"
mode="active-backup">
>> <slave device="eth0" primary="yes"/>
>> <slave device="eth1"/>
>> </bond>
>>
>>
> In addition to mode attribute for bonds, support for miimon,
> arp_interval, and arp_ip_target?
>
Sure .. if the initscript suports it, it's easy enough to do ;)
>> <bridge name="br0" stp="off" onboot="yes">
>> <member device="eth2"/>
>> <dhcp peerdns="yes"/>
>> </bridge>
>>
>>
> Attributes to support hellotime, forwarddelay, and maxage? This would allow
> <bridge name="br0" stp="on" hellotime="1"
maxage="4" fowarddelay="4"
> ..>
>
My main concern with these is that only forwarddelay, stp, and gcint are
controllable by Fedora/RHEL networking scripts. For hellotime and
maxage, we'd need to hook into the various ifup scripts atthe right
place. Are these directly supported by the SuSe infrastructure ?
This is probably best answered by Marius.
Jim
3:59 a.m.
On Thu, Jan 22, 2009 at 03:39:41PM -0700, Jim Fehlig wrote:
Adding Marius (SuSE sysconfig maintainer) in case I'm misspeaking
:-)
David Lutterkort wrote:
>On Tue, 2009-01-20 at 11:35 -0700, Jim Fehlig wrote:
>
>>David Lutterkort wrote:
>>
>>>For certain applications, we want libvirt to be able to configure host
>>>network interfaces in a variety of ways; currently, we are most
>>>interested in teaching libvirt how to set up ordinary ethernet
>>>interfaces, bridges, bonding and vlan's.
>>>
>>>Below is a high-level proposal of how that could be done. Please comment
>>>copiously ;)
>>>
>>>1. XML Format
>>>=============
>>>
>>>The first question is how the user would describe the host interfaces
>>>they
>>>want. Below are some sketches of what an XML representation of the
>>>various
>>>kinds of interfaces would look like. This covers the minimal amount of
>>>settings for these to be useful, though I am sure we'll need to add more
>>>over time.
>>>
>>><interface device="eth0" onboot="yes">
>>> <hwaddr>00:19:d2:9f:88:96</hwaddr>
>>> <dhcp peerdns="yes"/>
>>></interface>
>>>
>>>
>>For onboot="yes", something like startmode="onboot" would be
better
>>IMO. A startmode attribute would allow also using "manual",
"hotplug",
>>"ifplug", "nfsroot", etc.
>>
>
>This makes only sense if these additional modes are completely
>orthogonal; I am not sure what the start modes 'manual', 'ifplug',
and
>'nfsroot' should do.
manual
Interface will be set up if ifup is called manually.
ifplugd
Interface will be controlled from ifplugd
nfsroot
Nearly like auto, but interfaces with this startmode will never be
shut down via rcnetwork stop
On SUSE we have:
auto == on == onboot:
Interface will be set up as soon as it is available
and service network was started.
hotplug:
The difference between auto and hotplug is that the
latter does not make rcnetwork fail (or wait for it)
if the interface cannot be brought up.
> For onboot, I assume you mean that they translate
>into assignments in ifcfg in the following way
>
> no startmode attribute -> ONBOOT=no
> startmode='onboot' -> ONBOOT=yes
> startmode='hotplug' -> ONBOOT=yes and HOTPLUG=yes
On SUSE there is a STARTMODE variable instead of separate ONBOOT,
HOTPLUG, ...
On SuSE, the related ifcfg syntax is
STARTMODE={manual|auto|hotplug|ifplugd|nfsroot|off}
>>WRT dhcp element, would it make sense to consider hostname (hostname to
>>send to server) and if to change the local hostname to the hostname
>>delivered via dhcp? E.g.
>> hostname="foo" (default `hostname`)
>> sethostname
>>
>>Also route:
>> setrouting (default "yes")
>> setdefaultroute (default "yes")
>>
>>and NIS:
>> nis (default "yes")
>> setnisdomain (default "yes")
>>
>
>Yes, they all make sense, though I would probably not support them in
>the very first cut.
>
>
>>What about dhcpv6? A separate <dhcp6 /> element?
>>
>
>Yeah, we probably need a separate element for that since it pulls in a
>slew of other config options.
Well, what I mean is just, that it IMO makes sense to consider it as far
as possible in the design.
>>><interface device="eth1"
onboot="yes">
>>> <hwaddr>00:19:d2:9f:88:97</hwaddr>
>>> <static ipaddr="192.168.0.5" gateway="192.168.0.1"
>>> netmask="255.255.255.0"/>
>>></interface>
>>>
>>>
>>Similarly, support for IPv6 here. Either 2 formats:
>>
>>ipaddr="192.168.0.5" netmask="255.255.255.0"
>>ip6addr="2001:DB8:C::5/64"
>>
>>or allow /len for both:
>>
>>ipaddr="192.168.0.5/24"
>>ipaddr="2001:DB8:ABCD::1/64"
>>
>
>Looks a little strange, but it would simplify notation.
We're using IPADDR[suffix] = {IPv4Addr|IPv6Addr}[/prefixlength] on
SUSE; the notation with NETMASK, BROADCAST or separate PREFIXLEN
is still supported, but used as fallback only.
>>As for routing info, how about a separate route element:
>>
>><route gateway="192.168.0.1" /> # destination=default
>><route destination="default" gateway="192.168.0.1" />
>><route destination="10.0.0.0/8" gateway="192.168.0.2"
/>
>><route destination="2001:DB8:C::/64"
gateway="2001:DB8:C::1" />
>><route destination="2001:DB8::/32"> # unrecheable route
(loopback)
>>
>>It would perhaps make sense to use iproute2 names, that is prefix
>>instead of destination and nexthop instead of gateway.
>>
>
>For now, I want to stay out of setting up static routes, but I think
>that has to come sooner or later.
It is OK to limit routes to the default route for now, but IMO
it is better to use separate xml nodes, e.g.:
<static ipaddr="192.168.0.5" [netmask,broadcast,...] />
and something like:
<route gateway="192.168.0.1" /> # implicit destination=default
<route destination="default" gateway="192.168.0.1" />
rather than mixing the gateway into the IP address related attributes:
<static ipaddr="192.168.0.5" gateway="192.168.0.1"
netmask="255.255.255.0"/>
because as soon as you start to support static routes, there are two
nodes/places where the default gateway can be defined.
>>Don't recall if this was already brought up, but need a
way to specify
>>MTU and perhaps ethtool settings as well.
>>
>
>Yeah, that is definitely needed.
>
>
>>><interface device="eth2" onboot="yes">
>>> <hwaddr>00:19:d2:9f:88:98</hwaddr>
>>></interface>
>>>
>>><bond name="bond00" onboot="yes"
mode="active-backup">
>>> <slave device="eth0" primary="yes"/>
>>> <slave device="eth1"/>
>>></bond>
>>>
>>>
>>In addition to mode attribute for bonds, support for miimon,
>>arp_interval, and arp_ip_target?
>>
>
>Sure .. if the initscript suports it, it's easy enough to do ;)
On SUSE it does... ;-) This are critical parameters and every bond
interface has to use either miimon (default) or the arp_interval +
arp_ip_target combination or serious network degradation will occur
during link failures.
>>><bridge name="br0" stp="off"
onboot="yes">
>>> <member device="eth2"/>
>>> <dhcp peerdns="yes"/>
>>></bridge>
>>>
>>>
>>Attributes to support hellotime, forwarddelay, and maxage? This would
>>allow
>> <bridge name="br0" stp="on" hellotime="1"
maxage="4" fowarddelay="4"
>>..>
>>
>
>My main concern with these is that only forwarddelay, stp, and gcint are
>controllable by Fedora/RHEL networking scripts. For hellotime and
>maxage, we'd need to hook into the various ifup scripts atthe right
>place. Are these directly supported by the SuSe infrastructure ?
>
This is probably best answered by Marius.
Yes, we support all bridge parameters in SUSE ifcfg files.
The problem occurs as soon as STP is on, because a bridge using the
IEEE default settings, needs 50 seconds to become fully operational
and no traffic goes through in this time... so there is a need to
tune at least the hellotime, maxage, and fowarddelay or all sort of
another timeout settings (e.g. dhcp).
See http://tldp.org/HOWTO/BRIDGE-STP-HOWTO/advanced-bridge.html
Another way would be to say, there is either no STP parameter at all
(and use always stp="off" + fowarddelay=0) or only the STP parameter
and the backend implementation has to handle the another parameters
and write them "using real world defaults" into the ifcfg file.
Gruesse / Regards,
Marius Tomaschewski <mt(a)suse.de>
--
Server Technologies Team, SUSE LINUX Products GmbH,
Nuernberg; GF: Markus Rex; HRB 16746 (AG Nuernberg)
PGP public key on: http://www.suse.de/~mt/mt.pgp
DF17 271A AD15 006A 5BB9 6C96 CA2F F3F7 373A 1CC0
6:43 p.m.
On Mon, 2009-02-02 at 10:59 +0100, Marius Tomaschewski wrote:
> >For now, I want to stay out of setting up static routes, but
I think
> >that has to come sooner or later.
It is OK to limit routes to the default route for now, but IMO
it is better to use separate xml nodes, e.g.:
<static ipaddr="192.168.0.5" [netmask,broadcast,...] />
and something like:
<route gateway="192.168.0.1" /> # implicit destination=default
<route destination="default" gateway="192.168.0.1" />
rather than mixing the gateway into the IP address related attributes:
<static ipaddr="192.168.0.5" gateway="192.168.0.1"
netmask="255.255.255.0"/>
because as soon as you start to support static routes, there are two
nodes/places where the default gateway can be defined.
Yeah, that's a good catch; I'll change the schema accordingly.
Another way would be to say, there is either no STP parameter at all
(and use always stp="off" + fowarddelay=0) or only the STP parameter
and the backend implementation has to handle the another parameters
and write them "using real world defaults" into the ifcfg file.
A third option would be to allow specifying parameters that only _some_
backends support, and produce an error, e.g. if you try to set maxage on
Fedora. We don't necessarily have to support only the lcd.
David
5770
days inactive
5791
days old
36 comments
9 participants
participants (9)
-
Chris Hoy Poy -
Daniel P. Berrange -
David Lutterkort -
Jim Fehlig -
John Levon -
Kaitlin Rupert -
Marius Tomaschewski -
Mark McLoughlin -
Richard W.M. Jones