[Libvir] Patch for routed virtual networks
Greetings! The attached patch adds support for having routed virtual networks, in addition to the masquerading setup possible with the "<forward />" stanza. I have added a <route dev="ethX" /> stanza (dev is optional), completely equivalent to the <forward /> stanza. Summary of changes: * Added <route /> stanza to XML parsing/creation * Refactored qemudAddIptablesRules to allow for the routed network type * In iptables.c: * Renamed iptables(.*)ForwardAllowIn to iptables(.*)ForwardAllowRelatedIn, to better reflect their function * Added iptables(.*)ForwardAllowIn functions, that do not require traffic to be related Comments are very much appreciated :-) -- Mads Chr. Olesen <shiyee@shiyee.dk> shiyee.dk
lør, 08 03 2008 kl. 16:33 +0100, skrev Mads Chr. Olesen:
The attached patch adds support for having routed virtual networks, in addition to the masquerading setup possible with the "<forward />" stanza.
Just found a small error, now it actually compiles. -- Mads Chr. Olesen <shiyee@shiyee.dk> shiyee.dk
On Sat, Mar 08, 2008 at 04:33:32PM +0100, Mads Chr. Olesen wrote:
Greetings!
The attached patch adds support for having routed virtual networks, in addition to the masquerading setup possible with the "<forward />" stanza.
I have added a <route dev="ethX" /> stanza (dev is optional), completely equivalent to the <forward /> stanza.
This is still forwarding of traffic, so I think we should just use the existing <forward/> element and have an extra attribute to indiciate the type of forwarding, eg <forward/> (defaults to mode="nat" for compat) <forward mode="nat"/> <forward mode="route"/> <forward mode="nat" dev="ethX"/> <forward mode="route" dev="ethX"/>
Summary of changes: * Added <route /> stanza to XML parsing/creation * Refactored qemudAddIptablesRules to allow for the routed network type * In iptables.c: * Renamed iptables(.*)ForwardAllowIn to iptables(.*)ForwardAllowRelatedIn, to better reflect their function * Added iptables(.*)ForwardAllowIn functions, that do not require traffic to be related
Comments are very much appreciated :-)
I'm a little unclear on how this actually works. You add iptables rules to allow traffic in/out, but you're not adding any routing table entries, nor turning on proxy_arp, so I don't see how this will actually work in practice. Are you assuming the admin has already added suitable routing rules & turned on proxy arp ? Regards, Dan. -- |: Red Hat, Engineering, Boston -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
søn, 09 03 2008 kl. 21:09 +0000, skrev Daniel P. Berrange:
On Sat, Mar 08, 2008 at 04:33:32PM +0100, Mads Chr. Olesen wrote:
I have added a <route dev="ethX" /> stanza (dev is optional), completely equivalent to the <forward /> stanza.
This is still forwarding of traffic, so I think we should just use the existing <forward/> element and have an extra attribute to indiciate the type of forwarding, eg
<forward/> (defaults to mode="nat" for compat) <forward mode="nat"/> <forward mode="route"/> <forward mode="nat" dev="ethX"/> <forward mode="route" dev="ethX"/>
Sure, makes sense - an updated patch is attached.
I'm a little unclear on how this actually works. You add iptables rules to allow traffic in/out, but you're not adding any routing table entries, nor turning on proxy_arp, so I don't see how this will actually work in practice.
Are you assuming the admin has already added suitable routing rules & turned on proxy arp ?
Well, in my case (dedicated server, hetzner.de) this is all that is needed. My physical interface has IP 85.10.XXX.XXX, and then I have a secondary IP range which gets routed at that interface, IP range 78.47.YYY.YYY/30. I then setup my virtual interface with an IP in that range, by setting <ip address="78.47.YYY.YYY" netmask="255.255.255.248" /> Thus, to get packets routed at the virtual machines, it just needs to be allowed by iptables, and /proc/sys/net/ipv4/ip_forward needs to be set to 1. Other setups obviously might need more work. -- Mads Chr. Olesen <shiyee@shiyee.dk> shiyee.dk
Anything further I can do to help get this patch commited? I have been running with it, without problems across restarts, etc., for a couple of weeks now. man, 10 03 2008 kl. 22:09 +0100, skrev Mads Chr. Olesen:
On Sat, Mar 08, 2008 at 04:33:32PM +0100, Mads Chr. Olesen wrote:
I have added a <route dev="ethX" /> stanza (dev is optional), completely equivalent to the <forward /> stanza.
This is still forwarding of traffic, so I think we should just use
søn, 09 03 2008 kl. 21:09 +0000, skrev Daniel P. Berrange: the
existing <forward/> element and have an extra attribute to indiciate the type of forwarding, eg
<forward/> (defaults to mode="nat" for compat) <forward mode="nat"/> <forward mode="route"/> <forward mode="nat" dev="ethX"/> <forward mode="route" dev="ethX"/>
Sure, makes sense - an updated patch is attached.
I'm a little unclear on how this actually works. You add iptables rules to allow traffic in/out, but you're not adding any routing table entries, nor turning on proxy_arp, so I don't see how this will actually work in practice.
Are you assuming the admin has already added suitable routing rules & turned on proxy arp ?
Well, in my case (dedicated server, hetzner.de) this is all that is needed. My physical interface has IP 85.10.XXX.XXX, and then I have a secondary IP range which gets routed at that interface, IP range 78.47.YYY.YYY/30. I then setup my virtual interface with an IP in that range, by setting <ip address="78.47.YYY.YYY" netmask="255.255.255.248" />
Thus, to get packets routed at the virtual machines, it just needs to be allowed by iptables, and /proc/sys/net/ipv4/ip_forward needs to be set to 1.
Other setups obviously might need more work.
-- Mads Chr. Olesen <shiyee@shiyee.dk> shiyee.dk
On Mon, Mar 24, 2008 at 10:52:41AM +0100, Mads Chr. Olesen wrote:
Anything further I can do to help get this patch commited?
I have been running with it, without problems across restarts, etc., for a couple of weeks now.
I still don't see where the routing rules are defined / take place with this setup. There must be rules somewhere specifying the routing for the subnet 78.47.YYY.YYY/30, but its not being done in your patchset AFAICT Dan. -- |: Red Hat, Engineering, Boston -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
man, 24 03 2008 kl. 19:00 +0000, skrev Daniel P. Berrange:
On Mon, Mar 24, 2008 at 10:52:41AM +0100, Mads Chr. Olesen wrote:
Anything further I can do to help get this patch commited?
I have been running with it, without problems across restarts, etc., for a couple of weeks now.
I still don't see where the routing rules are defined / take place with this setup. There must be rules somewhere specifying the routing for the subnet 78.47.YYY.YYY/30, but its not being done in your patchset AFAICT
Do you mean entries in the routing table, or? This is my routing table: $ route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 78.47.YYY.YYY 0.0.0.0 255.255.255.248 U 0 0 0 virsubnetbr0 85.10.XXX.0 0.0.0.0 255.255.255.224 U 0 0 0 eth0 0.0.0.0 85.10.XXX.1 0.0.0.0 UG 100 0 0 eth0 The entry for virsubnetbr0 is setup automatically, by the definition in the XML-file: <ip address="78.47.YYY.YYY" netmask="255.255.255.248" /> -- Mads Chr. Olesen <shiyee@shiyee.dk> shiyee.dk
On Tue, Mar 25, 2008 at 12:56:31PM +0100, Mads Chr. Olesen wrote:
man, 24 03 2008 kl. 19:00 +0000, skrev Daniel P. Berrange:
On Mon, Mar 24, 2008 at 10:52:41AM +0100, Mads Chr. Olesen wrote:
Anything further I can do to help get this patch commited?
I have been running with it, without problems across restarts, etc., for a couple of weeks now.
I still don't see where the routing rules are defined / take place with this setup. There must be rules somewhere specifying the routing for the subnet 78.47.YYY.YYY/30, but its not being done in your patchset AFAICT
Do you mean entries in the routing table, or?
This is my routing table:
$ route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 78.47.YYY.YYY 0.0.0.0 255.255.255.248 U 0 0 0 virsubnetbr0 85.10.XXX.0 0.0.0.0 255.255.255.224 U 0 0 0 eth0 0.0.0.0 85.10.XXX.1 0.0.0.0 UG 100 0 0 eth0
Ok, so this takes care of letting the hsot correctly route guest traffic off the box. The local LAN router still also needs to have routing table setup to ensure other physical hosts can reach the guests. So, this patch is sufficient in this regard & I've committed it to CVS. In theory I think it might be possible to avoid the need to configure the local LAN router, by messing with proxy_arp, but I've not got it to work so far. Dan. -- |: Red Hat, Engineering, Boston -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Fri, Mar 28, 2008 at 08:40:19PM +0000, Daniel P. Berrange wrote:
In theory I think it might be possible to avoid the need to configure the local LAN router, by messing with proxy_arp, but I've not got it to work so far.
Unless I've misunderstood something, this won't work. If a machine on the LAN tries to reach one of the virtual machines (which is on a different subnet) it won't send out an ARP request for the IP in question, but just stick the MAC address of the the gateway onto the packet and let the gateway deal with it. AIUI proxy ARP is for when you have a subnet that's physically separated in which case the bridge will need to proxy the arp requests for hosts on the one side to hosts on the other side (and vice versa, obviously). -- Soren Hansen | Virtualisation specialist | Ubuntu Server Team Canonical Ltd. | http://www.ubuntu.com/
On Mon, Mar 24, 2008 at 07:00:48PM +0000, Daniel P. Berrange wrote:
Anything further I can do to help get this patch commited?
I have been running with it, without problems across restarts, etc., for a couple of weeks now. I still don't see where the routing rules are defined / take place with this setup. There must be rules somewhere specifying the routing for the subnet 78.47.YYY.YYY/30, but its not being done in your patchset AFAICT
I'm not sure what you mean? The host has an interface in the 78.46.xxx.yyy subnet, so there's automatically a route for the subnet set up on the host. Any further routing setup is out of scope, IMO. -- Soren Hansen | Virtualisation specialist | Ubuntu Server Team Canonical Ltd. | http://www.ubuntu.com/
participants (3)
-
Daniel P. Berrange -
Mads Chr. Olesen -
Soren Hansen