11:03 a.m.
I've run into a problem on my KVM host where a single guest will be
unreachable to other guests on the same host. This host has 2 bridged
devices and guests assigned to each have the same issue. I've noticed
that when I can't reach the problematic guest, the ARP entry for that
system is incorrect. This issue seems to only be a problem about 75%
of the time when making connections between the guests.
I've checked the logs and haven't seen anything indicating a problem.
Here's relavant information, if more is needed please let me know
CentOS 6.2 x86_64
libvirt-0.9.4-23.el6_2.4
kernel-2.6.32-220.4.1.el6
# brctl show
bridge name bridge id STP enabled interfaces
br0 8000.001b21a1cf76 no eth3
vnet0 # Problematic host
vnet1
vnet3
vnet4
vnet5
br1 8000.001b21a1ce4d no eth2
vnet6
virbr0 8000.525400a050ba yes virbr0-nic
vnet2
# ARP entry when it's the correct MAC
<omitted>.tamu.edu ether 52:54:00:e0:1e:6f C eth0
# ARP entry when it's wrong and ping or other connections time out
<omitted>.tamu.edu ether 00:1e:c9:bb:cd:d4 C eth0
The MAC ending in cd:d4 does not exist on any system on this network.
I've confirmed this with my campus IT who manages the switches and
routers.
# cat ifcfg-eth3
DEVICE="eth3"
HWADDR="00:1B:21:A1:CF:76"
NM_CONTROLLED="no"
ONBOOT="yes"
BRIDGE="br0"
# cat ifcfg-br0
DEVICE="br0"
ONBOOT="yes"
TYPE=Bridge
IPADDR="<omitted>"
NETMASK="255.255.255.192"
GATEWAY="<omitted>"
DOMAIN="tamu.edu"
# service iptables status
Table: filter
Chain INPUT (policy ACCEPT)
num target prot opt source destination
13 ACCEPT tcp -- <omitted> 0.0.0.0/0
multiport dports 10050 /* 001 zabbix agent port */
14 ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:53
15 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:53
16 ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:67
17 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:67
18 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:16509
19 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:16514
20 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0
state NEW tcp dpts:8002:8012
21 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:3493
22 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
state RELATED,ESTABLISHED
23 ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0
24 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
25 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0
state NEW tcp dpt:22
26 REJECT all -- 0.0.0.0/0 0.0.0.0/0
reject-with icmp-host-prohibited
Chain FORWARD (policy ACCEPT)
num target prot opt source destination
1 ACCEPT all -- 0.0.0.0/0 192.168.122.0/24
state RELATED,ESTABLISHED
2 ACCEPT all -- 192.168.122.0/24 0.0.0.0/0
3 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
4 REJECT all -- 0.0.0.0/0 0.0.0.0/0
reject-with icmp-port-unreachable
5 REJECT all -- 0.0.0.0/0 0.0.0.0/0
reject-with icmp-port-unreachable
6 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
PHYSDEV match --physdev-is-bridged
7 REJECT all -- 0.0.0.0/0 0.0.0.0/0
reject-with icmp-host-prohibited
Chain OUTPUT (policy ACCEPT)
num target prot opt source destination
Table: mangle
Chain PREROUTING (policy ACCEPT)
num target prot opt source destination
Chain INPUT (policy ACCEPT)
num target prot opt source destination
Chain FORWARD (policy ACCEPT)
num target prot opt source destination
Chain OUTPUT (policy ACCEPT)
num target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
num target prot opt source destination
1 CHECKSUM udp -- 0.0.0.0/0 0.0.0.0/0 udp
dpt:68 CHECKSUM fill
Table: nat
Chain PREROUTING (policy ACCEPT)
num target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
num target prot opt source destination
1 MASQUERADE tcp -- 192.168.122.0/24 !192.168.122.0/24
masq ports: 1024-65535
2 MASQUERADE udp -- 192.168.122.0/24 !192.168.122.0/24
masq ports: 1024-65535
3 MASQUERADE all -- 192.168.122.0/24 !192.168.122.0/24
Chain OUTPUT (policy ACCEPT)
num target prot opt source destination
Thanks
- Trey