---------- Forwarded message ---------- From: mallapadi niranjan <niranjan.ashok@gmail.com> Date: Mon, Nov 19, 2012 at 7:30 PM Subject: Re: [libvirt-users] unable to ping from guests in virbr0 to guests in virbr1 network To: Marwan Tanager <marwan.tngr@gmail.com> On Mon, Nov 19, 2012 at 7:51 PM, Marwan Tanager <marwan.tngr@gmail.com>wrote:

On Mon, Nov 19, 2012 at 12:09:53PM +0530, mallapadi niranjan wrote:

...

Hi all,

I have 3 guests (2-RHEL4 and 1 RHEL6) and have some issues regarding networking between them. The 2 RHEL4 system's use default bridge virbr0 and get ip's of range 192.168.122.0/24 (192.168.122.207, 192.168.122.167)

I created another bridge (virbr1) with NAT forwarding (no dhcp). The network i choose was 192.168.100.0/24. And the third system (RHEL6) was assigned static ip-addres 192.168.100.101,

...

...

From the RHEL6 system which uses virbr1 is able to ping systems in 192.168.122.0/24 series , but guest systems in 192.168.122.0/24 are not able to ping RHEL6 system (in virbr1) network.

...

...

From the RHEL4 guests i am able to ping the gateway ip's (192.168.122.1, 192.168.100.1) , but not the RHEL6 system

Versions: Fedora release 16 (Verne) libvirt-0.9.6.3-1.fc16.x86_64 qemu-kvm-0.15.1-8.fc16.x86_64

Any hints on what could be the problem

The problem is caused by the relative order of the iptables rules for those two networks.

When libvirt created virbr1 for the network 192.168.100.0/24 it inserted a couple of iptables rules on the FORWARDING chain for this interface, but it added them before the rules of virbr0 on the same chain. Those rules basically are ordered as follows (at least on my system which I suspect is different from yours since I also had the same problem):

1. Forward packets destined for the interface that are part of an established connection. 2. Forward packets coming from the interface. 3. Forward packets coming from and destined to the same interface (loopback). 4. Reject forwarding anything else to the interface. 5. Reject forwarding anything else from the interface.

okay that seems to be the behaviour: hain FORWARD (policy ACCEPT) target prot opt source destination ACCEPT all -- 0.0.0.0/0 192.168.100.0/24 state RELATED,ESTABLISHED ACCEPT all -- 192.168.100.0/24 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable ACCEPT all -- 0.0.0.0/0 192.168.200.0/24 state RELATED,ESTABLISHED ACCEPT all -- 192.168.200.0/24 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable ACCEPT all -- 0.0.0.0/0 192.168.122.0/24 state RELATED,ESTABLISHED ACCEPT all -- 192.168.122.0/24 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited I can ping from 192.168.100.0/24 -> 192.168.122.0/24 (but not the viceversa)

Since those rules are inserted for every interface libvirt adds in this order, you can only ping from one interface to another if the set of rules for the interface you are pinging from comes before that of the one you're pinging to. Otherwise, rule number 4 of the destination interface will get in the way since iptables matches the packets against the rules in a chain according to their order on it.

A possible workaround:

iptables-save >/path/to/iptables/dump/file

then, edit the file to move all relevant rules that REJECT things on the FORWARDING chain to the end of the chain.

then put this command in your rc.local script:

cat /path/to/iptables/dump/file | iptables-restore

This solution is not reliable though because you will need to update the file containing the rules every time you add a new virtual network or update the iptables rules for some other reason.

Right ,

Also, I hinted before at this problem on the list and someone pointed me that it's a bug, but nobody confirmed. Read my message on this topic with the subject: Inconsistent iptables forwarding rules for virtual networks, to get a clearer picture.

Okay, will look in to that mail

Marwan