Re: [libvirt-users] Networking issues with lxc containers in AWS EC2
On 04/11/2016 11:33 AM, Laine Stump wrote:
Interesting. That functionality was moved out of the kernel's
bridge
module into br_netfilter some time back, but that was done later than
the kernel 3.10 that is used by CentOS 7. Are you running some later
kernel version?
If your kernel doesn't have a message in dmesg that looks like this:
bridge: automatic filtering via arp/ip/ip6tables has been deprecated.
Update your scripts to load br_netfilter if you need this.
and the bridge driver is loaded, then that key should be available. Of
course if you don't have it, that's equivalent to having it set to 0,
so you should be okay regardless of why it's missing.
Ah, you were right. I'd forgot that the AMI I've using was one running
the 4.0.5 ml kernel. We discovered that bonded interfaces running with
mode 5 or 6 do not work with lxc containers (the host's ARP table does
not get updated). The issue was fixed in the 4.0.5 kernel so we ran for
a short time with this kernel, only to later abandon this kernel due to
a bug with software RAID.
I've reverted the kernel back to 3.10 on the AWS instances I'm using the
net.bridge.bridge-nf-call-iptables key is now present. It's already set
to 0 though so there is nothing that needs to be done here.
I wouldn't be too quick to judgement. First take a look at tcpdump on
the bridge interface that the containers are attached to, and on the
ethernet device that connects the bridge to the rest of Amazon's
infrastructure. If you see packets from the container's IP going out
but not coming back in, check the iptables rules (again - firewalld
uses iptables to setup its filtering) for a REJECT or DISCARD rule
that has an incrementing count. I use something like this to narrow
down the list I need to check:
while true; do iptables -v -S -Z | grep -v '^Zeroing' | grep -v "c 0
0" | grep -e '-c'; echo '**************'; sleep 1;
If you don't see any REJECT or DISCARD rules being triggered, then
maybe the problem is that AWS is providing an IP address to your
container's MAC, but isn't actually allowing traffic from that MAC out
onto the network.
I'll get this test setup. Unfortunately I'm not particularly
knowledgeable with iptables; we don't use it in our product so I've
never had to deal with it. I think you are right though about what's
happening--AWS doesn't recognize the MAC addresses for containers
running under another instance.