[libvirt] [RFC] Network interface XML for containers
We never really settled on the XML format for container network interfaces. I know a little more about what these look like now and have been working on the code so would like to get this sorted out. With network namespaces enabled, processes within the container will not be able to see any network devices outside of the container. A veth device pair will be used to transport traffic into and out off the container. One end of the veth pair will be attached to a bridge in the parent namespace. The other end of will be moved into the container namespace. We need to be able to represent the following in the XML: Network or bridge name Name for parent side veth device Name for container side veth device inet address for container side veth device So this should end up looking quite a bit like the formats for Virtual network and Bridge to LAN with a couple new items. The formats I've been kicking around are: Virtual network <devices> <interface type='network'> <source network='default' dev='veth0'/> <target dev='veth1' address='192.168.0.150'/> </interface> </devices> Bridge to LAN <devices> <interface type='bridge'> <source bridge='virbr0' dev='veth4'/> <target dev='veth5' address='192.168.0.155'/> </interface> </devices> All comments welcome. Thanks! -- Best Regards, Dave Leskovec IBM Linux Technology Center Open Virtualization
On Fri, 9 May 2008, Dave Leskovec wrote:
We never really settled on the XML format for container network interfaces. I know a little more about what these look like now and have been working on the code so would like to get this sorted out.
With network namespaces enabled, processes within the container will not be able to see any network devices outside of the container.
1) How does it solve 'mac address spoofing' between virtual machines? 2) How does it solve 'dupplicate mac addresses' in a cluster of processing units? 3) How does it solve 'migration arp issues'? Stefan
On Fri, May 09, 2008 at 07:26:07PM +0200, Stefan de Konink wrote:
On Fri, 9 May 2008, Dave Leskovec wrote:
We never really settled on the XML format for container network interfaces. I know a little more about what these look like now and have been working on the code so would like to get this sorted out.
With network namespaces enabled, processes within the container will not be able to see any network devices outside of the container.
1) How does it solve 'mac address spoofing' between virtual machines?
That's an irrelevant question for the libvirt XML format. We're simply defining what MAC address a guest has. Its upto the underlying virtualization technology to do the neccessary mac address spoofing protection based on the define MAC libvirt assignes to a guest
2) How does it solve 'dupplicate mac addresses' in a cluster of processing units?
It doesn't. Libvirt does not provide policy, only mechanism. Its upto tools using libvirt to ensure uniqueness.
3) How does it solve 'migration arp issues'?
Again, not libvirt's problem. Its the underlying virt technology to deal with. Dan -- |: Red Hat, Engineering, Boston -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Fri, May 09, 2008 at 10:16:38AM -0700, Dave Leskovec wrote:
We never really settled on the XML format for container network interfaces. I know a little more about what these look like now and have been working on the code so would like to get this sorted out.
With network namespaces enabled, processes within the container will not be able to see any network devices outside of the container. A veth device pair will be used to transport traffic into and out off the container. One end of the veth pair will be attached to a bridge in the parent namespace. The other end of will be moved into the container namespace. We need to be able to represent the following in the XML:
Network or bridge name Name for parent side veth device Name for container side veth device
Do you really need to be able to specify the guest side of the NIC name in the XML ? I'd rather we just left it out unless there was a clear need & it can't be automatically determined by the driver. With other virt drivers, the guest side of the NIC just gets name sequentially assigned eth0, eth1,... Linux lets you rename the NICs anyway so IMHO the only important thing is the MAC address since that's guarenteeed persistent & unique property for NICs, which a name is not.
inet address for container side veth device
Again, why are you specifying an IP address manually ? THe guest IP is typically determined by the guest admin, either statically or via DHCP. It shouldn't need to be configured in the host side if using the libvirt virtual networking or bridging.
So this should end up looking quite a bit like the formats for Virtual network and Bridge to LAN with a couple new items. The formats I've been kicking around are: Virtual network <devices> <interface type='network'> <source network='default' dev='veth0'/> <target dev='veth1' address='192.168.0.150'/> </interface> </devices>
Due to a historical accident the <target> element in the interface XML is actually refering to the host side of the NIC. We don't have any existing element in our XML format describing the guest name of the NIC - its just implied by ordering. THe <target> element is also treated as read-only by existing drivers, being automagically assigned unique name by the virt driver. The containers should follow the existing XML format 100% <interface type='network'> <source network='default'> <target dev='veth0'/> </interface>
Bridge to LAN <devices> <interface type='bridge'> <source bridge='virbr0' dev='veth4'/> <target dev='veth5' address='192.168.0.155'/> </interface> </devices>
Again I think including the guest NIC name is redundant, likewise for ip address & we can follow the existing XML format 100% <interface type='bridge'> <source network='br0'> <target dev='veth4'/> </interface> NB, We do have an IP address element in the interface XML format, however, this is not for configuring the guest IP address directly. Rather it is to setup a filter on the host side to reject any IP traffic other than than coming from the designate IP address. This basically sets a firewall rule on the host side of the NIC matching on IP. This is only implemented in Xen though and even then I don't think anyone really uses it in practice. Regards, Daniel. -- |: Red Hat, Engineering, Boston -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
Daniel P. Berrange wrote:
On Fri, May 09, 2008 at 10:16:38AM -0700, Dave Leskovec wrote:
We never really settled on the XML format for container network interfaces. I know a little more about what these look like now and have been working on the code so would like to get this sorted out.
...
Network or bridge name Name for parent side veth device Name for container side veth device
Do you really need to be able to specify the guest side of the NIC name in the XML ? I'd rather we just left it out unless there was a clear need & it can't be automatically determined by the driver. With other virt drivers, the guest side of the NIC just gets name sequentially assigned eth0, eth1,... Linux lets you rename the NICs anyway so IMHO the only important thing is the MAC address since that's guarenteeed persistent & unique property for NICs, which a name is not.
No the guest side doesn't necessarily have to be specified. It will get auto-assigned. The interface for creating veth devices (ip link add type veth) does make it difficult to determine the names it assigned. It does not print the names of the new devices and running a second command to get them is problematic. Hmmmm... also doesn't seem to like specifying just one of the device names but is fine if you specify both. I'm all for not specifying the guest side if I can make it work.
inet address for container side veth device
Again, why are you specifying an IP address manually ? THe guest IP is typically determined by the guest admin, either statically or via DHCP. It shouldn't need to be configured in the host side if using the libvirt virtual networking or bridging.
So the users init would be responsible for configuring the container veth device then? At that point they can assign a static ip if desired? This would mean that for a container that wanted to just run an application, <init> would have to point to a script that configured the network and then ran the application. Not sure that's a problem, just stating the consequence.
So this should end up looking quite a bit like the formats for Virtual network and Bridge to LAN with a couple new items. The formats I've been kicking around are: Virtual network <devices> <interface type='network'> <source network='default' dev='veth0'/> <target dev='veth1' address='192.168.0.150'/> </interface> </devices>
Due to a historical accident the <target> element in the interface XML is actually refering to the host side of the NIC. We don't have any existing element in our XML format describing the guest name of the NIC - its just implied by ordering. THe <target> element is also treated as read-only by existing drivers, being automagically assigned unique name by the virt driver.
Ah, ok.
The containers should follow the existing XML format 100%
<interface type='network'> <source network='default'> <target dev='veth0'/> </interface>
Right. I agree, this is what I'd like to see.
Bridge to LAN <devices> <interface type='bridge'> <source bridge='virbr0' dev='veth4'/> <target dev='veth5' address='192.168.0.155'/> </interface> </devices>
Again I think including the guest NIC name is redundant, likewise for ip address & we can follow the existing XML format 100%
<interface type='bridge'> <source network='br0'> <target dev='veth4'/> </interface>
Yep. -- Best Regards, Dave Leskovec IBM Linux Technology Center Open Virtualization
DL> So the users init would be responsible for configuring the DL> container veth device then? At that point they can assign a DL> static ip if desired? This would mean that for a container that DL> wanted to just run an application, <init> would have to point to a DL> script that configured the network and then ran the application. DL> Not sure that's a problem, just stating the consequence. I think that specifying the IP in the XML is a nice shortcut, but I wonder about two things: First, if you're just starting a single application in a container, what are the chances you want that single application to have an interface and IP address of its own? Second, the IP address that shows up in the libvirt config would imply to viewers that they can access the guest in that way. However, the guest could certainly have changed the address of its interface, thus invalidating the IP information that libvirt has. This problem exists with MAC addresses as well, although I think it's less severe because people don't usually try to connect to a guest based on its MAC. A grand total of 30 seconds of pondering on the above makes me lean in the direction of not specifying the IP address in the XML. -- Dan Smith IBM Linux Technology Center Open Hypervisor Team email: danms@us.ibm.com
On Fri, May 09, 2008 at 12:29:39PM -0700, Dan Smith wrote:
DL> So the users init would be responsible for configuring the DL> container veth device then? At that point they can assign a DL> static ip if desired? This would mean that for a container that DL> wanted to just run an application, <init> would have to point to a DL> script that configured the network and then ran the application. DL> Not sure that's a problem, just stating the consequence.
I think that specifying the IP in the XML is a nice shortcut, but I wonder about two things:
First, if you're just starting a single application in a container, what are the chances you want that single application to have an interface and IP address of its own?
And you could trivially write a short shell script around the app that sets the IP that it requirs.
Second, the IP address that shows up in the libvirt config would imply to viewers that they can access the guest in that way. However, the guest could certainly have changed the address of its interface, thus invalidating the IP information that libvirt has.
Yes, this is a compelling reason not to expose it to me - its simply not data that can be reliably determined from the host OS side where libvirt is living. Dan. -- |: Red Hat, Engineering, Boston -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
participants (4)
-
Dan Smith -
Daniel P. Berrange -
Dave Leskovec -
Stefan de Konink