Hi Dan, On Thu, 2007-04-05 at 02:44 +0100, Daniel P. Berrange wrote:
Warning, this is a long & complicated email with lots of horrible details :-)
I've long been a little confused with the way iptables & bridging interacts, so set out to do some experiments. I added a -j LOG rule to every single chain in both the filter & nat tables, and then tried various traffic patterns, to see which chains were traversed & in which order.
Nice work ...
Scenario 2: Virtual network ===========================
net.bridge.bridge-nf-call-iptables = 1
Host: eth0 -> Internet virbr0 -> MASQUERADE to eth0
Guest: vif1.0 -> virbr0
Traffic: Guest -> Google ------------------------
Out:
NAT-PREROUTING IN=virbr0 OUT= PHYSIN=vif1.0 SRC=192.168.122.47 DST=64.233.167.99 FORWARD IN=virbr0 OUT=eth0 PHYSIN=vif1.0 SRC=192.168.122.47 DST=64.233.167.99 NAT-POSTROUTING IN= OUT=eth0 PHYSIN=vif1.0 SRC=192.168.122.47 DST=64.233.167.99
This really suprises me - I would have expected another one like: FORWARD IN=virbr0 OUT=virbr0 PHYSIN=vif1.0 PHYSOUT=virb0 SRC=192.168.122.47 DST=64.233.167.99 Is it because the packets are coming in on bridge interface we don't see any physdev matching? So, we would see it with Guest->Guest?
For virtual networks there are basically 3 types of networking config we need to represent in terms of iptables rules, and these need to work for scenrios 1 & 2 - ie regardless of the magic sysctl knob.
Well, IMHO, we should never be switching off the sysctl knob ourselves - i.e. we shouldn't have it in xen/scripts/network-bridge - but I take the point that a user might switch it off.
Problem: The INPUT rules are missing altogether for the isolated virtual network so potentially DHCP/DNS will be blocked Solution: Add them - simple bug.
I fixed this in CVS, didn't I?
Problem: The POSTROUTING rule is too generic so it matches pretty much any kind of traffic, from any virtual network, or even from VPN devices setup by VPNC. Solution: Only masquerade traffic whose source address is within the netmask associated with the virtual network in question
Problem: The FORWARD rule is too generic, forwarding traffic to/from the virtual network regardless of whether the dest/src IP address is within the netmask associated with the virtual network. Assuming the first problem is setup to only masquerade valid IP addresses from the virtual network, this rule would then allow guests to spoof their IP and have it forwarded off-host. Solution: Only forward packets whose IP address is within the netmask associated with the virtual network
Problem: The policy of the FORWARD rule is ACCEPT, and/or later user defined rules may inadvertently match on traffic from the virtual network, again allowing through spoof traffic, or traffic from what should be an isolated virtual network Solution: There needs to be a catch-all REJECT rule associated with every bridge device, in both directions
Problem: There is an extra physdev match per bridge device, and per guest device. This is basically unneccessary since the previous rule sets will already have allowed through the traffic. The physdev matches also only work if net.bridge.bridge-nf-call-iptables = 1 Solution: Simply remove the per-device matches
Problem: The POSTROUTING rule has a physdev match applied, which only works if net.bridge.bridge-nf-call-iptables = 1. Solution: Remove physdev match & masquerade based on network address associated with the virtual network
I guess the two main differences are 1) avoid physdev based rules because they don't work with net.bridge.bridge-nf-call-iptables = 1 and 2) use network address based rules which I avoided because of pure superstition and the feeling that IP based matching on a bridge was just ugly. I haven't spent long thinking about these changes, but on the face of it they all look well thought out and sensible. Definitely worth giving it a shot. Cheers, Mark.