---------- Forwarded message ----------
From: mallapadi niranjan <niranjan.ashok@gmail.com>
Date: Mon, Nov 19, 2012 at 7:30 PM
Subject: Re: [libvirt-users] unable to ping from guests in virbr0 to guests in virbr1 network
To: Marwan Tanager <marwan.tngr@gmail.com>




On Mon, Nov 19, 2012 at 7:51 PM, Marwan Tanager <marwan.tngr@gmail.com> wrote:
On Mon, Nov 19, 2012 at 12:09:53PM +0530, mallapadi niranjan wrote:
> Hi all,
>
> I have 3 guests (2-RHEL4 and 1 RHEL6) and have some issues regarding
> networking between them. The 2 RHEL4 system's use default bridge virbr0 and
> get ip's of range 192.168.122.0/24  (192.168.122.207, 192.168.122.167)
>
> I created another bridge (virbr1) with NAT forwarding (no dhcp). The
> network i choose was 192.168.100.0/24. And the third system (RHEL6) was
> assigned static ip-addres 192.168.100.101,
>
> >>From the RHEL6 system which uses virbr1 is able to ping systems in
> 192.168.122.0/24 series , but  guest systems in 192.168.122.0/24 are not
> able to ping RHEL6 system (in virbr1) network.
>
> >>From the RHEL4 guests i am able to ping the gateway ip's (192.168.122.1,
> 192.168.100.1) , but not the RHEL6 system
>
>
> Versions:
> Fedora release 16 (Verne)
> libvirt-0.9.6.3-1.fc16.x86_64
> qemu-kvm-0.15.1-8.fc16.x86_64
>
> Any hints on what could be the problem

The problem is caused by the relative order of the iptables rules for those two
networks.

When libvirt created virbr1 for the network 192.168.100.0/24 it inserted a
couple of iptables rules on the FORWARDING chain for this interface, but it
added them before the rules of virbr0 on the same chain. Those rules basically
are ordered as follows (at least on my system which I suspect is different from
yours since I also had the same problem):

1. Forward packets destined for the interface that are part of an established
connection.
2. Forward packets coming from the interface.
3. Forward packets coming from and destined to the same interface (loopback).
4. Reject forwarding anything else to the interface.
5. Reject forwarding anything else from the interface.

okay that seems to be the behaviour:
hain FORWARD (policy ACCEPT)
target     prot opt source               destination        
ACCEPT     all  --  0.0.0.0/0            192.168.100.0/24     state RELATED,ESTABLISHED
ACCEPT     all  --  192.168.100.0/24     0.0.0.0/0          
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0          
REJECT     all  --  0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable
REJECT     all  --  0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable
ACCEPT     all  --  0.0.0.0/0            192.168.200.0/24     state RELATED,ESTABLISHED
ACCEPT     all  --  192.168.200.0/24     0.0.0.0/0          
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0          
REJECT     all  --  0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable
REJECT     all  --  0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable
ACCEPT     all  --  0.0.0.0/0            192.168.122.0/24     state RELATED,ESTABLISHED
ACCEPT     all  --  192.168.122.0/24     0.0.0.0/0          
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0          
REJECT     all  --  0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable
REJECT     all  --  0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable
REJECT     all  --  0.0.0.0/0            0.0.0.0/0            reject-with icmp-host-prohibited

I can ping from 192.168.100.0/24 -> 192.168.122.0/24 (but not the viceversa)

 

Since those rules are inserted for every interface libvirt adds in this order,
you can only ping from one interface to another if the set of rules for the
interface you are pinging from comes before that of the one you're pinging to.
Otherwise, rule number 4 of the destination interface will get in the way since
iptables matches the packets against the rules in a chain according to their
order on it.

A possible workaround:

iptables-save >/path/to/iptables/dump/file

then, edit the file to move all relevant rules that REJECT things on the
FORWARDING chain to the end of the chain.

then put this command in your rc.local script:

cat /path/to/iptables/dump/file | iptables-restore

This solution is not reliable though because you will need to update the file
containing the rules every time you add a new virtual network or update the
iptables rules for some other reason.

Right ,
 

Also, I hinted before at this problem on the list and someone pointed me that
it's a bug, but nobody confirmed. Read my message on this topic with the
subject: Inconsistent iptables forwarding rules for virtual networks, to get a
clearer picture.

Okay, will look in to that mail

        Marwan