[libvirt] ideas for custom iptables rules for libvirt networks.
We still periodically get requests to allow custom iptables rules for libvirt virtual networks (or, more commonly, a mode where libvirt simply leaves iptables alone, not adding or removing anything), and it's been a nagging item on my to-do list for a very long time. The problem is that, although the amount of code required to support *any* solution is very small, it's one of those things without a single obvious "only" way to do it. Anyway, I'm going to take one more stab at it. First, some background points: * For <forward mode='nat'> libvirt's iptables rules are essential to the operation of the forwarding, so we shouldn't mess with that. * For [no forward mode], libvirt's iptables rules are a part of what keeps the network isolated from the rest of the network, so we shouldn't mess with that either. * For <forward mode='route'> we currently allow all outgoing and incoming as long as it is to/from the IP address range defined for the network. So we really want something that can be used only for <forward mode='route'> I can see 3 different possibilities: 1) a new forward mode which is just like 'route', but doesn't add any iptables rules. (what to call it though? "filterless-route"? Too long and ugly :-/) 2) a new attribute to <forward> that takes effect only for mode='route'. Maybe call it "filter". We could have "filter='open'" (what it does currently, and will remain the default), "filter='outgoingOnly'", and "filter='none' (the most requested functionality - no iptables rules would be added for the network) 3) add a <filter> subelement to <forward> that allows specifying iptables rules for the network. Perhaps this could instead be a <filterref>, and use nwfilter to specify the rules? (that sounds really cool, and if it worked it would be a nice re-use of the nwfilter driver, but it may have undetermined pitfalls due to nwfilter being designed with guest traffic filtering in mind, would take a lot more work, and wouldn't address the most common request - "Don't mess with iptables! I want to do it myself!". Anyone have an opinion or alternate idea?
On 04/25/2016 01:48 PM, Laine Stump wrote:
We still periodically get requests to allow custom iptables rules for libvirt virtual networks (or, more commonly, a mode where libvirt simply leaves iptables alone, not adding or removing anything), and it's been a nagging item on my to-do list for a very long time. The problem is that, although the amount of code required to support *any* solution is very small, it's one of those things without a single obvious "only" way to do it. Anyway, I'm going to take one more stab at it.
First, some background points:
* For <forward mode='nat'> libvirt's iptables rules are essential to the operation of the forwarding, so we shouldn't mess with that.
* For [no forward mode], libvirt's iptables rules are a part of what keeps the network isolated from the rest of the network, so we shouldn't mess with that either.
* For <forward mode='route'> we currently allow all outgoing and incoming as long as it is to/from the IP address range defined for the network.
So we really want something that can be used only for <forward mode='route'>
I can see 3 different possibilities:
1) a new forward mode which is just like 'route', but doesn't add any iptables rules. (what to call it though? "filterless-route"? Too long and ugly :-/)
2) a new attribute to <forward> that takes effect only for mode='route'. Maybe call it "filter". We could have "filter='open'" (what it does currently, and will remain the default), "filter='outgoingOnly'", and "filter='none' (the most requested functionality - no iptables rules would be added for the network)
3) add a <filter> subelement to <forward> that allows specifying iptables rules for the network. Perhaps this could instead be a <filterref>, and use nwfilter to specify the rules? (that sounds really cool, and if it worked it would be a nice re-use of the nwfilter driver, but it may have undetermined pitfalls due to nwfilter being designed with guest traffic filtering in mind, would take a lot more work, and wouldn't address the most common request - "Don't mess with iptables! I want to do it myself!".
Anyone have an opinion or alternate idea?
I would suggest adding the bare minimum necessary :) Which I think is probably your second suggestion: <forward mode='route' filter='none'/> or maybe filter=automatic|manual , which takes the qualitative aspect out of the name. automatic=libvirt does whatever it wants with iptables, manual = you're on your own. I'd avoid adding anything extra like filter='outgoingOnly' until there's a real legitimate request for it. But also since routed networking generally requires external host config anyways it should be fine IMO to tell anyone that wants it to use filter=automatic and add there own rules on top to disable inbound traffic - Cole
On Mon, Apr 25, 2016 at 01:48:49PM -0400, Laine Stump wrote:
We still periodically get requests to allow custom iptables rules for libvirt virtual networks (or, more commonly, a mode where libvirt simply leaves iptables alone, not adding or removing anything), and it's been a nagging item on my to-do list for a very long time. The problem is that, although the amount of code required to support *any* solution is very small, it's one of those things without a single obvious "only" way to do it. Anyway, I'm going to take one more stab at it.
First, some background points:
* For <forward mode='nat'> libvirt's iptables rules are essential to the operation of the forwarding, so we shouldn't mess with that.
* For [no forward mode], libvirt's iptables rules are a part of what keeps the network isolated from the rest of the network, so we shouldn't mess with that either.
* For <forward mode='route'> we currently allow all outgoing and incoming as long as it is to/from the IP address range defined for the network.
So we really want something that can be used only for <forward mode='route'>
I can see 3 different possibilities:
1) a new forward mode which is just like 'route', but doesn't add any iptables rules. (what to call it though? "filterless-route"? Too long and ugly :-/)
I'd suggest this and just call it mode='bare' or mode='open', to avoid implying any specific semantics about the connectivity.
2) a new attribute to <forward> that takes effect only for mode='route'. Maybe call it "filter". We could have "filter='open'" (what it does currently, and will remain the default), "filter='outgoingOnly'", and "filter='none' (the most requested functionality - no iptables rules would be added for the network)
3) add a <filter> subelement to <forward> that allows specifying iptables rules for the network. Perhaps this could instead be a <filterref>, and use nwfilter to specify the rules? (that sounds really cool, and if it worked it would be a nice re-use of the nwfilter driver, but it may have undetermined pitfalls due to nwfilter being designed with guest traffic filtering in mind, would take a lot more work, and wouldn't address the most common request - "Don't mess with iptables! I want to do it myself!".
Anyone have an opinion or alternate idea?
Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 04/26/2016 04:15 AM, Daniel P. Berrange wrote:
On Mon, Apr 25, 2016 at 01:48:49PM -0400, Laine Stump wrote:
We still periodically get requests to allow custom iptables rules for libvirt virtual networks (or, more commonly, a mode where libvirt simply leaves iptables alone, not adding or removing anything), and it's been a nagging item on my to-do list for a very long time. The problem is that, although the amount of code required to support *any* solution is very small, it's one of those things without a single obvious "only" way to do it. Anyway, I'm going to take one more stab at it.
First, some background points:
* For <forward mode='nat'> libvirt's iptables rules are essential to the operation of the forwarding, so we shouldn't mess with that.
* For [no forward mode], libvirt's iptables rules are a part of what keeps the network isolated from the rest of the network, so we shouldn't mess with that either.
* For <forward mode='route'> we currently allow all outgoing and incoming as long as it is to/from the IP address range defined for the network.
So we really want something that can be used only for <forward mode='route'>
I can see 3 different possibilities:
1) a new forward mode which is just like 'route', but doesn't add any iptables rules. (what to call it though? "filterless-route"? Too long and ugly :-/) I'd suggest this and just call it mode='bare' or mode='open', to avoid implying any specific semantics about the connectivity.
Thinking about this more, I'm having disturbing memories of trying to combine multiple knobs into a single attribute in the past, only to have it backfire later (I can't recall anything specific though) - for example although I say right now that there's no reason to skip adding rules for isolated or nat networks, we may come up with a valid use case in the future. For that reason, I think it may be safer to add: <forward mode='route' filter='none'/> (or "manual" as Cole suggested). That way it's still obvious from mode that the traffic will be routed to other networks, and it will be possible to use the filter attribute for other modes (or add other filter settings even for mode='route') in the future. (Validation would be added to make sure that the filter attribute isn't set for any mode other than route). Does that make sense?
2) a new attribute to <forward> that takes effect only for mode='route'. Maybe call it "filter". We could have "filter='open'" (what it does currently, and will remain the default), "filter='outgoingOnly'", and "filter='none' (the most requested functionality - no iptables rules would be added for the network)
On Tue, Apr 26, 2016 at 11:06:23AM -0400, Laine Stump wrote:
On 04/26/2016 04:15 AM, Daniel P. Berrange wrote:
On Mon, Apr 25, 2016 at 01:48:49PM -0400, Laine Stump wrote:
We still periodically get requests to allow custom iptables rules for libvirt virtual networks (or, more commonly, a mode where libvirt simply leaves iptables alone, not adding or removing anything), and it's been a nagging item on my to-do list for a very long time. The problem is that, although the amount of code required to support *any* solution is very small, it's one of those things without a single obvious "only" way to do it. Anyway, I'm going to take one more stab at it.
First, some background points:
* For <forward mode='nat'> libvirt's iptables rules are essential to the operation of the forwarding, so we shouldn't mess with that.
* For [no forward mode], libvirt's iptables rules are a part of what keeps the network isolated from the rest of the network, so we shouldn't mess with that either.
* For <forward mode='route'> we currently allow all outgoing and incoming as long as it is to/from the IP address range defined for the network.
So we really want something that can be used only for <forward mode='route'>
I can see 3 different possibilities:
1) a new forward mode which is just like 'route', but doesn't add any iptables rules. (what to call it though? "filterless-route"? Too long and ugly :-/) I'd suggest this and just call it mode='bare' or mode='open', to avoid implying any specific semantics about the connectivity.
Thinking about this more, I'm having disturbing memories of trying to combine multiple knobs into a single attribute in the past, only to have it backfire later (I can't recall anything specific though) - for example although I say right now that there's no reason to skip adding rules for isolated or nat networks, we may come up with a valid use case in the future. For that reason, I think it may be safer to add:
<forward mode='route' filter='none'/>
(or "manual" as Cole suggested). That way it's still obvious from mode that the traffic will be routed to other networks, and it will be possible to use the filter attribute for other modes (or add other filter settings even for mode='route') in the future. (Validation would be added to make sure that the filter attribute isn't set for any mode other than route). Does that make sense?
I don't think this is actually encoding multiple things in one tunable. The only thing that the existing 'mode' attribute does is to influence what iptables rules get created by libvirt. ie the route vs nat distinction is only visible at the iptables level. So from that POV, your 'filter' attribute is really adding a new attribute to control the same functionality. Also, we're doing this so that users can create their own iptables rules without interference from libvirt. We've no idea what the user might do with their iptables rulkes - they may well create a setup that looks the same as mode=nat would for all we know. So IMHO it wouldbe misleading to label it 'mode=route' + 'filter=none' because we don't even know if the user's setup will allow routing outside of the network Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Tue, Apr 26, 2016 at 11:06:23AM -0400, Laine Stump wrote:
On 04/26/2016 04:15 AM, Daniel P. Berrange wrote:
On Mon, Apr 25, 2016 at 01:48:49PM -0400, Laine Stump wrote:
We still periodically get requests to allow custom iptables rules for libvirt virtual networks (or, more commonly, a mode where libvirt simply leaves iptables alone, not adding or removing anything), and it's been a nagging item on my to-do list for a very long time. The problem is that, although the amount of code required to support *any* solution is very small, it's one of those things without a single obvious "only" way to do it. Anyway, I'm going to take one more stab at it.
First, some background points:
* For <forward mode='nat'> libvirt's iptables rules are essential to the operation of the forwarding, so we shouldn't mess with that.
* For [no forward mode], libvirt's iptables rules are a part of what keeps the network isolated from the rest of the network, so we shouldn't mess with that either.
* For <forward mode='route'> we currently allow all outgoing and incoming as long as it is to/from the IP address range defined for the network.
So we really want something that can be used only for <forward mode='route'>
I can see 3 different possibilities:
1) a new forward mode which is just like 'route', but doesn't add any iptables rules. (what to call it though? "filterless-route"? Too long and ugly :-/) I'd suggest this and just call it mode='bare' or mode='open', to avoid implying any specific semantics about the connectivity.
Thinking about this more, I'm having disturbing memories of trying to combine multiple knobs into a single attribute in the past, only to have it backfire later (I can't recall anything specific though) - for example although I say right now that there's no reason to skip adding rules for isolated or nat networks, we may come up with a valid use case in the future. For that reason, I think it may be safer to add:
<forward mode='route' filter='none'/>
I would probably consider using sub-element <filter mode='none'/> or the mode='manual'. One example of that design backfiring right now is the <graphcis> device and it's listen attribute and listen sub-elements. I'm working on patches to move also ports into the listen elements and introducing new types of listen elements and there has to be extra code for backward compatibility. Pavel
(or "manual" as Cole suggested). That way it's still obvious from mode that the traffic will be routed to other networks, and it will be possible to use the filter attribute for other modes (or add other filter settings even for mode='route') in the future. (Validation would be added to make sure that the filter attribute isn't set for any mode other than route). Does that make sense?
participants (4)
-
Cole Robinson -
Daniel P. Berrange -
Laine Stump -
Pavel Hrdina