[libvirt] problem with nwfilter and ip6tables
Hi, I'm investigating using the nwfilter-functionality of libvirt to give my clients the possibility to block ports of their VPSes. The same mechanism allows me to restrict the outgoing traffic a VPS is generating. In the end, I want to restrict MAC, IPv4 and IPv6 traffic, while the client can also restrict traffic to UDP and TCP. All goes well, until I want to restrict the UDP/TCP traffic to certain IPv6 addresses. Where iptables shows the IPv4-restriction I've put up, ip6tables doesn't show anything. In the logs, I only see some ip6tables -D, -X and -F commands failing, which is expected when libvirt tries to delete/flush rules that were never there. I've built my nwfilter containing the following IPv6-rules, which I for instance reference once for all the TCP-ports which should be open. <!-- Allow established traffic --> <filter name='ipv6-allow-statefull' chain='ipv6'> <rule action='accept' direction='in' priority='500'> <all state='ESTABLISHED'/> </rule> <rule action='accept' direction='out' priority='500'> <all state='ESTABLISHED,RELATED'/> </rule> </filter> <!-- Allow TCP in $PORT --> <filter name='ipv6-allow-create-state-by-port' chain='ipv6'> <rule action='accept' direction='in' priority='500'> <tcp state='NEW' dstportstart='$PORT'/> </rule> <rule action='accept' direction='in' priority='500'> <udp state='NEW' dstportstart='$PORT'/> </rule> </filter> <!-- Allow IPv6 traffic from $RANGE --> <filter name='ipv6-allow-create-state-by-range' chain='ipv6'> <rule action='accept' direction='out' priority='500'> <ipv6 srcipaddr='$RANGE' srcipmask='64'/> </rule> </filter> <!-- Drop all other IPv6 traffic --> <filter name='ipv6-drop-stateless' chain='ipv6'> <rule action='drop' direction='inout' priority='999'> <all/> </rule> </filter> I use a similar approach for my IPv4 firewall, and it works perfectly. When I use these IPv6 rules, all IPv6 traffic is apparently dropped, but it's hard to debug when the result of this config is abscent in ip6tables. I'm using these version of software on debian 6.0 squeeze: virsh # version Compiled against library: libvir 0.9.2 Using library: libvir 0.9.2 Using API: QEMU 0.9.2 Running hypervisor: QEMU 0.15.0 Does anyone have any clues? Thanks in advance! Regards, Reinier Schoof -- TransIP BV | https://www.transip.nl/
On 11/21/2011 08:46 AM, Reinier Schoof wrote:
Hi,
I'm investigating using the nwfilter-functionality of libvirt to give my clients the possibility to block ports of their VPSes. The same mechanism allows me to restrict the outgoing traffic a VPS is generating. In the end, I want to restrict MAC, IPv4 and IPv6 traffic, while the client can also restrict traffic to UDP and TCP.
All goes well, until I want to restrict the UDP/TCP traffic to certain IPv6 addresses. Where iptables shows the IPv4-restriction I've put up, ip6tables doesn't show anything. In the logs, I only see some ip6tables -D, -X and -F commands failing, which is expected when libvirt tries to delete/flush rules that were never there.
I've built my nwfilter containing the following IPv6-rules, which I for instance reference once for all the TCP-ports which should be open.
<!-- Allow established traffic --> <filter name='ipv6-allow-statefull' chain='ipv6'> <rule action='accept' direction='in' priority='500'> <all state='ESTABLISHED'/> </rule> <rule action='accept' direction='out' priority='500'> <all state='ESTABLISHED,RELATED'/> </rule> </filter>
With the configuration as you have it, can you post the dump of ip6tables -L -n ? Can you put the two rules above on priority '100'.
<!-- Allow TCP in $PORT --> <filter name='ipv6-allow-create-state-by-port' chain='ipv6'> <rule action='accept' direction='in' priority='500'> <tcp state='NEW' dstportstart='$PORT'/> </rule> <rule action='accept' direction='in' priority='500'> <udp state='NEW' dstportstart='$PORT'/> </rule> </filter>
<!-- Allow IPv6 traffic from $RANGE --> <filter name='ipv6-allow-create-state-by-range' chain='ipv6'> <rule action='accept' direction='out' priority='500'> <ipv6 srcipaddr='$RANGE' srcipmask='64'/> </rule> </filter>
Can you put this one in priority '600'?
<!-- Drop all other IPv6 traffic --> <filter name='ipv6-drop-stateless' chain='ipv6'> <rule action='drop' direction='inout' priority='999'> <all/> </rule> </filter>
With the above comments 'implemented', does it 'work'? If not: - I'd be curious about the content of $RANGE that you're passing in. - how does the filter look like that ties all the above filters together. The order in which the above filters are referenced matters as long as you keep the rules at priority '500'. It should work with ipv6 as well as it does for ipv4, but I must say that I don't really test with IPv6 much... Stefan
I use a similar approach for my IPv4 firewall, and it works perfectly. When I use these IPv6 rules, all IPv6 traffic is apparently dropped, but it's hard to debug when the result of this config is abscent in ip6tables.
I'm using these version of software on debian 6.0 squeeze: virsh # version Compiled against library: libvir 0.9.2 Using library: libvir 0.9.2 Using API: QEMU 0.9.2 Running hypervisor: QEMU 0.15.0
Does anyone have any clues? Thanks in advance!
Regards,
Reinier Schoof
Hi Stefan, thanks for your quick response Op 21-11-2011 15:21, Stefan Berger schreef:
On 11/21/2011 08:46 AM, Reinier Schoof wrote:
Hi,
I'm investigating using the nwfilter-functionality of libvirt to give my clients the possibility to block ports of their VPSes. The same mechanism allows me to restrict the outgoing traffic a VPS is generating. In the end, I want to restrict MAC, IPv4 and IPv6 traffic, while the client can also restrict traffic to UDP and TCP.
All goes well, until I want to restrict the UDP/TCP traffic to certain IPv6 addresses. Where iptables shows the IPv4-restriction I've put up, ip6tables doesn't show anything. In the logs, I only see some ip6tables -D, -X and -F commands failing, which is expected when libvirt tries to delete/flush rules that were never there.
I've built my nwfilter containing the following IPv6-rules, which I for instance reference once for all the TCP-ports which should be open.
<!-- Allow established traffic --> <filter name='ipv6-allow-statefull' chain='ipv6'> <rule action='accept' direction='in' priority='500'> <all state='ESTABLISHED'/> </rule> <rule action='accept' direction='out' priority='500'> <all state='ESTABLISHED,RELATED'/> </rule> </filter>
With the configuration as you have it, can you post the dump of
ip6tables -L -n
?
Nothing to see there. All I see in my libvirt.log is some ip6tables -D, -X, -F and -E commands failing: /sbin/ip6tables -D libvirt-out -m physdev --physdev-out vpsif3 -g FO-vpsif3 /sbin/ip6tables -D libvirt-in -m physdev --physdev-in vpsif3 -g FI-vpsif3 /sbin/ip6tables -D libvirt-host-in -m physdev --physdev-in vpsif3 -g HI-vpsif3 [...] Command stderr: ip6tables v1.4.12: goto 'FO-vpsif3' is not a chain Try `ip6tables -h' or 'ip6tables --help' for more information. ip6tables v1.4.12: goto 'FI-vpsif3' is not a chain Try `ip6tables -h' or 'ip6tables --help' for more information. ip6tables v1.4.12: goto 'HI-vpsif3' is not a chain Try `ip6tables -h' or 'ip6tables --help' for more information. [...]
Can you put the two rules above on priority '100'.
<!-- Allow TCP in $PORT --> <filter name='ipv6-allow-create-state-by-port' chain='ipv6'> <rule action='accept' direction='in' priority='500'> <tcp state='NEW' dstportstart='$PORT'/> </rule> <rule action='accept' direction='in' priority='500'> <udp state='NEW' dstportstart='$PORT'/> </rule> </filter>
<!-- Allow IPv6 traffic from $RANGE --> <filter name='ipv6-allow-create-state-by-range' chain='ipv6'> <rule action='accept' direction='out' priority='500'> <ipv6 srcipaddr='$RANGE' srcipmask='64'/> </rule> </filter>
Can you put this one in priority '600'?
<!-- Drop all other IPv6 traffic --> <filter name='ipv6-drop-stateless' chain='ipv6'> <rule action='drop' direction='inout' priority='999'> <all/> </rule> </filter>
With the above comments 'implemented', does it 'work'?
Doesn't change a thing.
If not: - I'd be curious about the content of $RANGE that you're passing in.
<parameter name='RANGE' value='2a01:7c8:aaaa:5::'/>
- how does the filter look like that ties all the above filters together. The order in which the above filters are referenced matters as long as you keep the rules at priority '500'.
The filters are referenced in the exact order as I pasted them. This is the same as with the IPv4-version of the ruleset.
It should work with ipv6 as well as it does for ipv4, but I must say that I don't really test with IPv6 much...
Stefan
I use a similar approach for my IPv4 firewall, and it works perfectly. When I use these IPv6 rules, all IPv6 traffic is apparently dropped, but it's hard to debug when the result of this config is abscent in ip6tables.
I'm using these version of software on debian 6.0 squeeze: virsh # version Compiled against library: libvir 0.9.2 Using library: libvir 0.9.2 Using API: QEMU 0.9.2 Running hypervisor: QEMU 0.15.0
Does anyone have any clues? Thanks in advance!
Regards,
Reinier Schoof
-- TransIP BV | https://www.transip.nl/
On 11/21/2011 08:46 AM, Reinier Schoof wrote: Going back to the original email:
Hi,
I'm investigating using the nwfilter-functionality of libvirt to give my clients the possibility to block ports of their VPSes. The same mechanism allows me to restrict the outgoing traffic a VPS is generating. In the end, I want to restrict MAC, IPv4 and IPv6 traffic, while the client can also restrict traffic to UDP and TCP.
All goes well, until I want to restrict the UDP/TCP traffic to certain IPv6 addresses. Where iptables shows the IPv4-restriction I've put up, ip6tables doesn't show anything. In the logs, I only see some ip6tables -D, -X and -F commands failing, which is expected when libvirt tries to delete/flush rules that were never there.
I've built my nwfilter containing the following IPv6-rules, which I for instance reference once for all the TCP-ports which should be open.
<!-- Allow established traffic --> <filter name='ipv6-allow-statefull' chain='ipv6'> <rule action='accept' direction='in' priority='500'> <all state='ESTABLISHED'/> </rule> <rule action='accept' direction='out' priority='500'> <all state='ESTABLISHED,RELATED'/> </rule> </filter>
Replace with <filter name='ipv6-allow-statefull' chain='root'> <uuid>d7ca42fe-a2f5-6491-cdee-10d8a0956772</uuid> <rule action='accept' direction='in' priority='100'> <all-ipv6 state='ESTABLISHED'/> </rule> <rule action='accept' direction='out' priority='100'> <all-ipv6 state='ESTABLISHED,RELATED'/> </rule> </filter>
<!-- Allow TCP in $PORT --> <filter name='ipv6-allow-create-state-by-port' chain='ipv6'> <rule action='accept' direction='in' priority='500'> <tcp state='NEW' dstportstart='$PORT'/> </rule> <rule action='accept' direction='in' priority='500'> <udp state='NEW' dstportstart='$PORT'/> </rule> </filter>
Replace with <filter name='ipv6-allow-create-state-by-port' chain='root'> <uuid>ff97e825-712d-6b1a-c5d1-46fe635f9dd6</uuid> <rule action='accept' direction='in' priority='500'> <tcp-ipv6 state='NEW' dstportstart='$PORT'/> </rule> <rule action='accept' direction='in' priority='500'> <udp-ipv6 state='NEW' dstportstart='$PORT'/> </rule> </filter>
<!-- Allow IPv6 traffic from $RANGE --> <filter name='ipv6-allow-create-state-by-range' chain='ipv6'> <rule action='accept' direction='out' priority='500'> <ipv6 srcipaddr='$RANGE' srcipmask='64'/> </rule> </filter>
This probably should either be direction='in' or you may want to replace srcipaddr and srcipmask with dstipaddr and dstipmask. Replace with <filter name='ipv6-allow-create-state-by-range' chain='root'> <uuid>6e738070-9505-730d-14e6-ee01a6eb5885</uuid> <rule action='accept' direction='in' priority='500'> <all-ipv6 srcipaddr='$RANGE' srcipmask='62'/> </rule> </filter> You may want to add state='NEW' to the rule as well.
<!-- Drop all other IPv6 traffic --> <filter name='ipv6-drop-stateless' chain='ipv6'> <rule action='drop' direction='inout' priority='999'> <all/> </rule> </filter>
Replace with <filter name='ipv6-drop-stateless' chain='root'> <uuid>4377aca7-18fb-b373-4462-4ee2ba3db7cd</uuid> <rule action='drop' direction='inout' priority='999'> <all-ipv6/> </rule> </filter> You have to change the chain to 'root' and the protocol in the rules has to be tcp-ipv6, all-ipv6 etc. for ipv6 traffic. The reason is that most of these rules could be applied to either iptables or ip6tables and the network filtering system needs some more 'hints' whether it is indeed an ipv6 rule so it create ip6tables commands versus iptables commands. I hope this helps. 'ip6tables -L -n' here now shows: Chain FI-vnet0 (1 references) target prot opt source destination RETURN all ::/0 ::/0 state RELATED,ESTABLISHED RETURN all ::/0 ::/64 state ESTABLISHED ctdir ORIGINAL DROP all ::/0 ::/0 Chain FO-vnet0 (1 references) target prot opt source destination ACCEPT all ::/0 ::/0 state ESTABLISHED ACCEPT tcp ::/0 ::/0 tcp dpt:90 state NEW ACCEPT udp ::/0 ::/0 udp dpt:90 state NEW ACCEPT all ::/64 ::/0 state NEW,ESTABLISHED ctdir REPLY DROP all ::/0 ::/0 Chain HI-vnet0 (1 references) target prot opt source destination RETURN all ::/0 ::/0 state RELATED,ESTABLISHED RETURN all ::/0 ::/64 state ESTABLISHED ctdir ORIGINAL DROP all ::/0 ::/0 Stefan
I use a similar approach for my IPv4 firewall, and it works perfectly. When I use these IPv6 rules, all IPv6 traffic is apparently dropped, but it's hard to debug when the result of this config is abscent in ip6tables.
I'm using these version of software on debian 6.0 squeeze: virsh # version Compiled against library: libvir 0.9.2 Using library: libvir 0.9.2 Using API: QEMU 0.9.2 Running hypervisor: QEMU 0.15.0
Does anyone have any clues? Thanks in advance!
Regards,
Reinier Schoof
That actually works, thanks a lot! Op 21-11-2011 17:07, Stefan Berger schreef:
On 11/21/2011 08:46 AM, Reinier Schoof wrote:
Going back to the original email:
Hi,
I'm investigating using the nwfilter-functionality of libvirt to give my clients the possibility to block ports of their VPSes. The same mechanism allows me to restrict the outgoing traffic a VPS is generating. In the end, I want to restrict MAC, IPv4 and IPv6 traffic, while the client can also restrict traffic to UDP and TCP.
All goes well, until I want to restrict the UDP/TCP traffic to certain IPv6 addresses. Where iptables shows the IPv4-restriction I've put up, ip6tables doesn't show anything. In the logs, I only see some ip6tables -D, -X and -F commands failing, which is expected when libvirt tries to delete/flush rules that were never there.
I've built my nwfilter containing the following IPv6-rules, which I for instance reference once for all the TCP-ports which should be open.
<!-- Allow established traffic --> <filter name='ipv6-allow-statefull' chain='ipv6'> <rule action='accept' direction='in' priority='500'> <all state='ESTABLISHED'/> </rule> <rule action='accept' direction='out' priority='500'> <all state='ESTABLISHED,RELATED'/> </rule> </filter>
Replace with
<filter name='ipv6-allow-statefull' chain='root'> <uuid>d7ca42fe-a2f5-6491-cdee-10d8a0956772</uuid> <rule action='accept' direction='in' priority='100'> <all-ipv6 state='ESTABLISHED'/> </rule> <rule action='accept' direction='out' priority='100'> <all-ipv6 state='ESTABLISHED,RELATED'/> </rule> </filter>
<!-- Allow TCP in $PORT --> <filter name='ipv6-allow-create-state-by-port' chain='ipv6'> <rule action='accept' direction='in' priority='500'> <tcp state='NEW' dstportstart='$PORT'/> </rule> <rule action='accept' direction='in' priority='500'> <udp state='NEW' dstportstart='$PORT'/> </rule> </filter>
Replace with
<filter name='ipv6-allow-create-state-by-port' chain='root'> <uuid>ff97e825-712d-6b1a-c5d1-46fe635f9dd6</uuid> <rule action='accept' direction='in' priority='500'> <tcp-ipv6 state='NEW' dstportstart='$PORT'/> </rule> <rule action='accept' direction='in' priority='500'> <udp-ipv6 state='NEW' dstportstart='$PORT'/> </rule> </filter>
<!-- Allow IPv6 traffic from $RANGE --> <filter name='ipv6-allow-create-state-by-range' chain='ipv6'> <rule action='accept' direction='out' priority='500'> <ipv6 srcipaddr='$RANGE' srcipmask='64'/> </rule> </filter>
This probably should either be direction='in' or you may want to replace srcipaddr and srcipmask with dstipaddr and dstipmask.
Replace with
<filter name='ipv6-allow-create-state-by-range' chain='root'> <uuid>6e738070-9505-730d-14e6-ee01a6eb5885</uuid> <rule action='accept' direction='in' priority='500'> <all-ipv6 srcipaddr='$RANGE' srcipmask='62'/> </rule> </filter>
You may want to add state='NEW' to the rule as well.
<!-- Drop all other IPv6 traffic --> <filter name='ipv6-drop-stateless' chain='ipv6'> <rule action='drop' direction='inout' priority='999'> <all/> </rule> </filter>
Replace with
<filter name='ipv6-drop-stateless' chain='root'> <uuid>4377aca7-18fb-b373-4462-4ee2ba3db7cd</uuid> <rule action='drop' direction='inout' priority='999'> <all-ipv6/> </rule> </filter>
You have to change the chain to 'root' and the protocol in the rules has to be tcp-ipv6, all-ipv6 etc. for ipv6 traffic. The reason is that most of these rules could be applied to either iptables or ip6tables and the network filtering system needs some more 'hints' whether it is indeed an ipv6 rule so it create ip6tables commands versus iptables commands.
I hope this helps.
'ip6tables -L -n' here now shows:
Chain FI-vnet0 (1 references) target prot opt source destination RETURN all ::/0 ::/0 state RELATED,ESTABLISHED RETURN all ::/0 ::/64 state ESTABLISHED ctdir ORIGINAL DROP all ::/0 ::/0
Chain FO-vnet0 (1 references) target prot opt source destination ACCEPT all ::/0 ::/0 state ESTABLISHED ACCEPT tcp ::/0 ::/0 tcp dpt:90 state NEW ACCEPT udp ::/0 ::/0 udp dpt:90 state NEW ACCEPT all ::/64 ::/0 state NEW,ESTABLISHED ctdir REPLY DROP all ::/0 ::/0
Chain HI-vnet0 (1 references) target prot opt source destination RETURN all ::/0 ::/0 state RELATED,ESTABLISHED RETURN all ::/0 ::/64 state ESTABLISHED ctdir ORIGINAL DROP all ::/0 ::/0
Stefan
I use a similar approach for my IPv4 firewall, and it works perfectly. When I use these IPv6 rules, all IPv6 traffic is apparently dropped, but it's hard to debug when the result of this config is abscent in ip6tables.
I'm using these version of software on debian 6.0 squeeze: virsh # version Compiled against library: libvir 0.9.2 Using library: libvir 0.9.2 Using API: QEMU 0.9.2 Running hypervisor: QEMU 0.15.0
Does anyone have any clues? Thanks in advance!
Regards,
Reinier Schoof
-- TransIP BV | https://www.transip.nl/
participants (2)
-
Reinier Schoof -
Stefan Berger