[libvirt-users] Modify Iptables Rules (virbr0 & virbr1)
Hi, I have some guests running in the "default" network (virbr0) and I've also created a similar (NAT) network (virbr1). Therefore, the FORWARD chain for the CentOS 6.4 host looks like this: http://fpaste.org/29229/75281379/ ...where line 3-7 are related to virbr0 and 8-12 to virbr1. My 2 questions: 1) I've noticed that I can ping from a guest within virbr0 to any guest on the virbr1 network. However, I can't do the same from virbr1 (ping guests on virbr0). This is because of "line 6" on the pastebin, where the traffic is REJECTED. I originally thought that If I create multiple NAT networks (just like the default virbr0) they would be able to talk to each other (the host doing the actual forwarding) but as you can see, based on the iptables rules that libvirt injects, this only happens for the first network. That is, the first network can reach all other networks (just because it happens to be the first one defined). Is this the intention (only default can talk to the others but not the other way around)? 2) I would like both networks to talk to each other. If I remove line #6 I can make virbr1 guests talk to virbr0 guests. What is the correct way to handle this? I obviously don't want to perform "iptables -D FORWARD line-number..." every time I start libvirt. (I really like to leave the networks as they are, NAT, dhcp running etc). Thanks! Jorge
On 07/31/2013 11:01 AM, Jorge Fábregas wrote:
That is, the first network can reach all other networks (just because it happens to be the first one defined). Is this the intention (only default can talk to the others but not the other way around)?
*Bump* I found this excellent post by Daniel Berrange: http://www.redhat.com/archives/libvir-list/2010-June/msg00762.html ...which explains all the firewall rules that libvirt creates based on the type of network you choose. Reading this I get the idea that, the intention for NAT virtual-networks, is to allow them to communicate with ANY other virtual-network on your system (since there's an allow rule for traffic coming out of it). In a nutshell, the problem is that there's a lack of consistency on how NAT virtual-networks communicate between each other. I think the traffic between these subnets should be either allowed or denied. Right now we have a mixed scenario where the decision to allow or deny the traffic is merely based on what position, of the firewall rules, your virtual-network happens to be. Here's what I mean: http://fpaste.org/30485/ Network 0 can reach any network due to line #3 Network 1 can only reach the networks defined below it (due to line #10) Network 1 can't reach Network 0 due to line #5 Network 2 can't reach any of the above networks due to #line 5 & 12 (reach = "initiate new connections") Summary: (Based on the order of firewall rules): virtual-networks can successfully initiate new connections to the networks defined below it but can't with networks defined above it. Comments are welcome. Thanks! Jorge
On 08/06/2013 06:38 PM, Jorge Fábregas wrote:
On 07/31/2013 11:01 AM, Jorge Fábregas wrote:
That is, the first network can reach all other networks (just because it happens to be the first one defined). Is this the intention (only default can talk to the others but not the other way around)? *Bump*
I found this excellent post by Daniel Berrange:
http://www.redhat.com/archives/libvir-list/2010-June/msg00762.html
...which explains all the firewall rules that libvirt creates based on the type of network you choose. Reading this I get the idea that, the intention for NAT virtual-networks, is to allow them to communicate with ANY other virtual-network on your system (since there's an allow rule for traffic coming out of it).
In a nutshell, the problem is that there's a lack of consistency on how NAT virtual-networks communicate between each other. I think the traffic between these subnets should be either allowed or denied. Right now we have a mixed scenario where the decision to allow or deny the traffic is merely based on what position, of the firewall rules, your virtual-network happens to be.
Here's what I mean:
Network 0 can reach any network due to line #3
Network 1 can only reach the networks defined below it (due to line #10) Network 1 can't reach Network 0 due to line #5
Network 2 can't reach any of the above networks due to #line 5 & 12
(reach = "initiate new connections")
Summary: (Based on the order of firewall rules): virtual-networks can successfully initiate new connections to the networks defined below it but can't with networks defined above it.
Correct. That is a known problem since 2008: https://bugzilla.redhat.com/show_bug.cgi?id=453580 Due to the large amount of work required to fix it relative to the apparent demand for a fix, it has remained unchanged. Note that if you want to have multiple virtual networks that can communicate with each other, you can define all the networks as <forward mode='route'/> (which gives them iptables rulesets that allow all access in both directions), then add in appropriate "blanket" NAT rules yourself in the host's iptables config.
Comments are welcome.
Thanks! Jorge
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
On 08/13/2013 06:31 AM, Laine Stump wrote:
Correct. That is a known problem since 2008:
Thanks Laine for confirming it is a known issue. I googled it a lot but couldn't find that bugzilla entry. Do you know if this is still the case with the upcoming Fedora 20 & firewalld? (these rules are still being created)?
Due to the large amount of work required to fix it relative to the apparent demand for a fix, it has remained unchanged.
I'm wondering if it really takes a lot of work. I think that by just changing the order of the rules everything gets fixed. If we group the rules *by functionality* instead of *by virtual-network* we can accomplish a particular goal (drop communication between virtual-networks or allow them): (Notice that I did not insert or delete any rule; just changed the order): - Allow communication between virtual-networks (regardless of direction): http://fpaste.org/31729/ - Block communication between virtual-networks (except for the LAN): http://fpaste.org/31731/
Note that if you want to have multiple virtual networks that can communicate with each other, you can define all the networks as <forward mode='route'/> (which gives them iptables rulesets that allow all access in both directions), then add in appropriate "blanket" NAT rules yourself in the host's iptables config.
Right, that's what I'm using now: just had to add a static route to my home router in order for them to be able to use the net. Again, thanks Laine for the feedback! -- Jorge
On 08/13/2013 07:07 AM, Jorge Fábregas wrote:
On 08/13/2013 06:31 AM, Laine Stump wrote:
Correct. That is a known problem since 2008:
https://bugzilla.redhat.com/show_bug.cgi?id=453580 Thanks Laine for confirming it is a known issue. I googled it a lot but couldn't find that bugzilla entry.
Do you know if this is still the case with the upcoming Fedora 20 & firewalld? (these rules are still being created)?
There hasn't been any substantial change in the iptables rules added by libvirt for virtual networks in a long time; libvirt's firewalld usage is in the form of sending firewall-cmd exactly the same rules that were previously sent directly to iptables.
Due to the large amount of work required to fix it relative to the apparent demand for a fix, it has remained unchanged. I'm wondering if it really takes a lot of work. I think that by just changing the order of the rules everything gets fixed. If we group the rules *by functionality* instead of *by virtual-network* we can accomplish a particular goal (drop communication between virtual-networks or allow them):
Sure, that's simple if you're going to start/stop all virtual networks together as a group. It's more complicated if you want each network to operate independently of the other (i.e. t obe able to start/stop each network without affecting the others). Possibly the way to do that would be to create separate chains for the allow and block. You're welcome to write a patch for it :-)
(Notice that I did not insert or delete any rule; just changed the order):
- Allow communication between virtual-networks (regardless of direction): http://fpaste.org/31729/
- Block communication between virtual-networks (except for the LAN): http://fpaste.org/31731/
Note that if you want to have multiple virtual networks that can communicate with each other, you can define all the networks as <forward mode='route'/> (which gives them iptables rulesets that allow all access in both directions), then add in appropriate "blanket" NAT rules yourself in the host's iptables config. Right, that's what I'm using now: just had to add a static route to my home router in order for them to be able to use the net.
Yes, that's another option, for those that have control over the routing tables of their network.
On 08/13/2013 07:23 AM, Laine Stump wrote:
There hasn't been any substantial change in the iptables rules added by libvirt for virtual networks in a long time;
I guess this is due to the fact that, in the enterprise (oVirt/RHEV), bridge networking is mainly used over "virtual-networks".
Sure, that's simple if you're going to start/stop all virtual networks together as a group. It's more complicated if you want each network to operate independently of the other (i.e. t obe able to start/stop each network without affecting the others). Possibly the way to do that would be to create separate chains for the allow and block.
You're right: that's the correct way to handle this (using chains).
You're welcome to write a patch for it :-)
Yeah I know it's easy to pinpoint a problem... I would have provided a patch If I were a coder, believe me :) I guess I can open an enhancement-request (perhaps for F21) with pseudo-code on how to handle the different events (something that would be easy for someone familiar with the code to implement). With the upcoming snapshot functionality in virt-manager I hope many end-users start using it more and subsequently the virtual-networks. Thanks! -- Jorge
participants (2)
-
Jorge Fábregas -
Laine Stump