[PATCH 0/2] network: support NAT networking for FreeBSD/pf
This series implements NAT networks support for FreeBSD using the Packet Filter (pf) firewall. The commit messages provide high-level details and limitations of the current implementation, and I'll use this cover letter to provide some more technical details and describe testing I have performed for this change. Libvirt FreeBSD/pf NAT testing For two networks: virsh # net-dumpxml default <network> <name>default</name> <uuid>68cd5419-9fda-4cf0-9ac6-2eb9c1ba41ed</uuid> <forward mode='nat'> <nat> <port start='1024' end='65535'/> </nat> </forward> <bridge name='virbr0' stp='on' delay='0'/> <mac address='52:54:00:db:0e:e5'/> <ip address='192.168.122.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.122.2' end='192.168.122.254'/> </dhcp> </ip> </network> virsh # net-dumpxml natnet <network> <name>natnet</name> <uuid>d3c59659-3ceb-4482-a625-1f839a54429c</uuid> <forward mode='nat'> <nat> <port start='1024' end='65535'/> </nat> </forward> <bridge name='virbr1' stp='on' delay='0'/> <mac address='52:54:00:0a:fc:1d'/> <ip address='10.0.100.1' netmask='255.255.255.0'> <dhcp> <range start='10.0.100.2' end='10.0.100.254'/> </dhcp> </ip> </network> virsh # The following rules are generated: $ sudo pfctl -a '*' -sn nat-anchor "libvirt/*" all { nat-anchor "default" all { nat pass on re0 inet from 192.168.122.0/24 to <natdst> -> (re0) port 1024:65535 round-robin } nat-anchor "natnet" all { nat pass on re0 inet from 10.0.100.0/24 to <natdst> -> (re0) port 1024:65535 round-robin } } $ $ sudo pfctl -a 'libvirt/default' -t natdst -T show 0.0.0.0/0 !192.168.122.0/24 !224.0.0.0/24 !255.255.255.255 $ sudo pfctl -a 'libvirt/natnet' -t natdst -T show 0.0.0.0/0 !10.0.100.0/24 !224.0.0.0/24 !255.255.255.255 $ $ sudo pfctl -a '*' -sr scrub all fragment reassemble anchor "libvirt/*" all { anchor "default" all { pass quick on virbr0 inet from 192.168.122.0/24 to 192.168.122.0/24 flags S/SA keep state pass quick on virbr0 inet from 192.168.122.0/24 to 224.0.0.0/24 flags S/SA keep state pass quick on virbr0 inet from 192.168.122.0/24 to 255.255.255.255 flags S/SA keep state block drop on virbr0 all } anchor "natnet" all { pass quick on virbr1 inet from 10.0.100.0/24 to 10.0.100.0/24 flags S/SA keep state pass quick on virbr1 inet from 10.0.100.0/24 to 224.0.0.0/24 flags S/SA keep state pass quick on virbr1 inet from 10.0.100.0/24 to 255.255.255.255 flags S/SA keep state block drop on virbr1 all } } pass all flags S/SA keep state $ Create two guests attached to the "default" network, vmA and vmB. vmA $ ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host noprefixroute valid_lft forever preferred_lft forever 2: enp0s4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:67:eb:de brd ff:ff:ff:ff:ff:ff inet 192.168.122.92/24 brd 192.168.122.255 scope global dynamic noprefixroute enp0s4 valid_lft 1082sec preferred_lft 1082sec inet6 fe80::5054:ff:fe67:ebde/64 scope link noprefixroute valid_lft forever preferred_lft forever vmA $ vmB $ ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host noprefixroute valid_lft forever preferred_lft forever 2: enp0s4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:d2:8b:41 brd ff:ff:ff:ff:ff:ff inet 192.168.122.154/24 metric 100 brd 192.168.122.255 scope global dynamic enp0s4 valid_lft 1040sec preferred_lft 1040sec inet6 fe80::5054:ff:fed2:8b41/64 scope link valid_lft forever preferred_lft forever vmB $ Test NAT rules: vmA $ ping -c 3 8.8.8.8 PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. 64 bytes from 8.8.8.8: icmp_seq=1 ttl=57 time=14.7 ms 64 bytes from 8.8.8.8: icmp_seq=2 ttl=57 time=10.7 ms 64 bytes from 8.8.8.8: icmp_seq=3 ttl=57 time=10.1 ms --- 8.8.8.8 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2006ms rtt min/avg/max/mdev = 10.099/11.835/14.710/2.047 ms vmA $ vmB $ ping -c 3 8.8.8.8 PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. 64 bytes from 8.8.8.8: icmp_seq=1 ttl=57 time=15.1 ms 64 bytes from 8.8.8.8: icmp_seq=2 ttl=57 time=11.0 ms 64 bytes from 8.8.8.8: icmp_seq=3 ttl=57 time=10.4 ms --- 8.8.8.8 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2006ms rtt min/avg/max/mdev = 10.434/12.198/15.113/2.075 ms vmB $ vmA $ curl wttr.in/?0Q Fog _ - _ - _ - +4(1) °C _ - _ - _ ↙ 11 km/h _ - _ - _ - 0 km 0.0 mm vmA $ vmB $ curl wttr.in/?0Q Fog _ - _ - _ - +4(1) °C _ - _ - _ ↙ 11 km/h _ - _ - _ - 0 km 0.0 mm vmB $ Inter-VM connectivity: vmA $ ping -c 3 192.168.122.154 PING 192.168.122.154 (192.168.122.154) 56(84) bytes of data. 64 bytes from 192.168.122.154: icmp_seq=1 ttl=64 time=0.253 ms 64 bytes from 192.168.122.154: icmp_seq=2 ttl=64 time=0.226 ms 64 bytes from 192.168.122.154: icmp_seq=3 ttl=64 time=0.269 ms --- 192.168.122.154 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2042ms rtt min/avg/max/mdev = 0.226/0.249/0.269/0.017 ms vmA $ vmA $ ssh 192.168.122.154 uname novel@192.168.122.154's password: Linux vmA $ Multicast test: vmA $ iperf -s -u -B 224.0.0.1 -i 1 ------------------------------------------------------------ Server listening on UDP port 5001 Joining multicast group 224.0.0.1 Server set to single client traffic mode (per multicast receive) UDP buffer size: 208 KByte (default) ------------------------------------------------------------ [ 1] local 224.0.0.1 port 5001 connected with 192.168.122.154 port 36963 [ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams [ 1] 0.00-1.00 sec 131 KBytes 1.07 Mbits/sec 0.030 ms 0/91 (0%) [ 1] 1.00-2.00 sec 128 KBytes 1.05 Mbits/sec 0.022 ms 0/89 (0%) [ 1] 2.00-3.00 sec 128 KBytes 1.05 Mbits/sec 0.021 ms 0/89 (0%) [ 1] 0.00-3.02 sec 389 KBytes 1.06 Mbits/sec 0.026 ms 0/271 (0%) vmB $ iperf -c 224.0.0.1 -u -T 32 -t 3 -i 1 ------------------------------------------------------------ Client connecting to 224.0.0.1, UDP port 5001 Sending 1470 byte datagrams, IPG target: 11215.21 us (kalman adjust) UDP buffer size: 208 KByte (default) ------------------------------------------------------------ [ 1] local 192.168.122.154 port 36963 connected with 224.0.0.1 port 5001 [ ID] Interval Transfer Bandwidth [ 1] 0.0000-1.0000 sec 131 KBytes 1.07 Mbits/sec [ 1] 1.0000-2.0000 sec 128 KBytes 1.05 Mbits/sec [ 1] 2.0000-3.0000 sec 128 KBytes 1.05 Mbits/sec [ 1] 0.0000-3.0173 sec 389 KBytes 1.06 Mbits/sec [ 1] Sent 272 datagrams vmB $ Broadcast test: vmA $ sudo sysctl -w net.ipv4.icmp_echo_ignore_broadcasts=0 net.ipv4.icmp_echo_ignore_broadcasts = 0 vmA $ vmB $ sudo sysctl -w net.ipv4.icmp_echo_ignore_broadcasts=0 net.ipv4.icmp_echo_ignore_broadcasts = 0 vmB $ host $ ping 192.168.122.255 PING 192.168.122.255 (192.168.122.255): 56 data bytes 64 bytes from 192.168.122.154: icmp_seq=0 ttl=64 time=0.199 ms 64 bytes from 192.168.122.92: icmp_seq=0 ttl=64 time=0.227 ms (DUP!) 64 bytes from 192.168.122.154: icmp_seq=1 ttl=64 time=0.209 ms 64 bytes from 192.168.122.92: icmp_seq=1 ttl=64 time=0.235 ms (DUP!) ^C --- 192.168.122.255 ping statistics --- 2 packets transmitted, 2 packets received, +2 duplicates, 0.0% packet loss round-trip min/avg/max/stddev = 0.199/0.218/0.235/0.014 ms This testing does not cover any negative scenarios which are probably not that important at this point. Roman Bogorodskiy (2): network: bridge_driver: add BSD implementation network: introduce Packet Filter firewall backend meson.build | 2 + po/POTFILES | 2 + src/network/bridge_driver_bsd.c | 107 +++++++++ src/network/bridge_driver_conf.c | 8 + src/network/bridge_driver_linux.c | 2 + src/network/bridge_driver_platform.c | 2 + src/network/meson.build | 1 + src/network/network_pf.c | 327 +++++++++++++++++++++++++++ src/network/network_pf.h | 26 +++ src/util/virfirewall.c | 4 +- src/util/virfirewall.h | 2 + 11 files changed, 482 insertions(+), 1 deletion(-) create mode 100644 src/network/bridge_driver_bsd.c create mode 100644 src/network/network_pf.c create mode 100644 src/network/network_pf.h -- 2.49.0
Add BSD-specific platform flavor of the bridge driver which will be used as a base for Packet Filter (pf) based NAT networking implementation. Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com> --- po/POTFILES | 1 + src/network/bridge_driver_bsd.c | 101 +++++++++++++++++++++++++++ src/network/bridge_driver_platform.c | 2 + 3 files changed, 104 insertions(+) create mode 100644 src/network/bridge_driver_bsd.c diff --git a/po/POTFILES b/po/POTFILES index 9747c38951..90664fe6e7 100644 --- a/po/POTFILES +++ b/po/POTFILES @@ -145,6 +145,7 @@ src/lxc/lxc_hostdev.c src/lxc/lxc_native.c src/lxc/lxc_process.c src/network/bridge_driver.c +src/network/bridge_driver_bsd.c src/network/bridge_driver_conf.c src/network/bridge_driver_linux.c src/network/bridge_driver_nop.c diff --git a/src/network/bridge_driver_bsd.c b/src/network/bridge_driver_bsd.c new file mode 100644 index 0000000000..93312fe6db --- /dev/null +++ b/src/network/bridge_driver_bsd.c @@ -0,0 +1,101 @@ +/* + * Copyright (C) 2025 FreeBSD Foundation + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + */ + +#include <config.h> + +#include "virlog.h" + +#define VIR_FROM_THIS VIR_FROM_NONE + +VIR_LOG_INIT("network.bridge_driver_bsd"); + +static virErrorPtr errInitV4; +static virErrorPtr errInitV6; + +void networkPreReloadFirewallRules(virNetworkDriverState *driver G_GNUC_UNUSED, + bool startup G_GNUC_UNUSED, + bool force G_GNUC_UNUSED) +{ +} + + +void networkPostReloadFirewallRules(bool startup G_GNUC_UNUSED) +{ +} + + +int networkCheckRouteCollision(virNetworkDef *def G_GNUC_UNUSED) +{ + return 0; +} + +int networkAddFirewallRules(virNetworkDef *def G_GNUC_UNUSED, + virFirewallBackend firewallBackend, + virFirewall **fwRemoval G_GNUC_UNUSED) +{ + if (def->forward.type == VIR_NETWORK_FORWARD_OPEN) { + VIR_DEBUG("No firewall rules to add for mode='open' network '%s'", def->name); + } else { + VIR_DEBUG("Adding firewall rules for mode='%s' network '%s' using %s", + virNetworkForwardTypeToString(def->forward.type), + def->name, + virFirewallBackendTypeToString(firewallBackend)); + + if (errInitV4 && + (virNetworkDefGetIPByIndex(def, AF_INET, 0) || + virNetworkDefGetRouteByIndex(def, AF_INET, 0))) { + virSetError(errInitV4); + return -1; + } + + if (errInitV6 && + (virNetworkDefGetIPByIndex(def, AF_INET6, 0) || + virNetworkDefGetRouteByIndex(def, AF_INET6, 0) || + def->ipv6nogw)) { + virSetError(errInitV6); + return -1; + } + + /* now actually add the rules */ + switch (firewallBackend) { + case VIR_FIREWALL_BACKEND_NONE: + virReportError(VIR_ERR_NO_SUPPORT, "%s", + _("No firewall backend is available")); + return -1; + + case VIR_FIREWALL_BACKEND_IPTABLES: + case VIR_FIREWALL_BACKEND_NFTABLES: + case VIR_FIREWALL_BACKEND_LAST: + virReportEnumRangeError(virFirewallBackend, firewallBackend); + return -1; + } + } + return 0; +} + +void +networkRemoveFirewallRules(virNetworkObj *obj, + bool unsetZone G_GNUC_UNUSED) +{ + virNetworkDef *def = virNetworkObjGetDef(obj); + if (def->forward.type == VIR_NETWORK_FORWARD_OPEN) { + VIR_DEBUG("No firewall rules to remove for mode='open' network '%s'", + def->name); + return; + } +} diff --git a/src/network/bridge_driver_platform.c b/src/network/bridge_driver_platform.c index 9ddcb71063..42fbcdbc0b 100644 --- a/src/network/bridge_driver_platform.c +++ b/src/network/bridge_driver_platform.c @@ -25,6 +25,8 @@ #if defined(__linux__) # include "bridge_driver_linux.c" +#elif defined(__FreeBSD__) +# include "bridge_driver_bsd.c" #else # include "bridge_driver_nop.c" #endif -- 2.49.0
On Sat, Apr 26, 2025 at 09:42:35AM +0200, Roman Bogorodskiy wrote:
Add BSD-specific platform flavor of the bridge driver which will be used as a base for Packet Filter (pf) based NAT networking implementation.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com> --- po/POTFILES | 1 + src/network/bridge_driver_bsd.c | 101 +++++++++++++++++++++++++++ src/network/bridge_driver_platform.c | 2 + 3 files changed, 104 insertions(+) create mode 100644 src/network/bridge_driver_bsd.c
diff --git a/po/POTFILES b/po/POTFILES index 9747c38951..90664fe6e7 100644 --- a/po/POTFILES +++ b/po/POTFILES @@ -145,6 +145,7 @@ src/lxc/lxc_hostdev.c src/lxc/lxc_native.c src/lxc/lxc_process.c src/network/bridge_driver.c +src/network/bridge_driver_bsd.c src/network/bridge_driver_conf.c src/network/bridge_driver_linux.c src/network/bridge_driver_nop.c diff --git a/src/network/bridge_driver_bsd.c b/src/network/bridge_driver_bsd.c new file mode 100644 index 0000000000..93312fe6db --- /dev/null +++ b/src/network/bridge_driver_bsd.c @@ -0,0 +1,101 @@ +/* + * Copyright (C) 2025 FreeBSD Foundation + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + */ + +#include <config.h> + +#include "virlog.h" + +#define VIR_FROM_THIS VIR_FROM_NONE + +VIR_LOG_INIT("network.bridge_driver_bsd"); + +static virErrorPtr errInitV4; +static virErrorPtr errInitV6; + +void networkPreReloadFirewallRules(virNetworkDriverState *driver G_GNUC_UNUSED, + bool startup G_GNUC_UNUSED, + bool force G_GNUC_UNUSED) +{ +} + + +void networkPostReloadFirewallRules(bool startup G_GNUC_UNUSED) +{ +} + + +int networkCheckRouteCollision(virNetworkDef *def G_GNUC_UNUSED) +{ + return 0; +} + +int networkAddFirewallRules(virNetworkDef *def G_GNUC_UNUSED, + virFirewallBackend firewallBackend, + virFirewall **fwRemoval G_GNUC_UNUSED) +{
You should report an error if "def->bridgeZone" is non-NULL, similar to the way Linux reports it when firewalld is not available.
+ if (def->forward.type == VIR_NETWORK_FORWARD_OPEN) { + VIR_DEBUG("No firewall rules to add for mode='open' network '%s'", def->name); + } else { + VIR_DEBUG("Adding firewall rules for mode='%s' network '%s' using %s", + virNetworkForwardTypeToString(def->forward.type), + def->name, + virFirewallBackendTypeToString(firewallBackend)); + + if (errInitV4 && + (virNetworkDefGetIPByIndex(def, AF_INET, 0) || + virNetworkDefGetRouteByIndex(def, AF_INET, 0))) { + virSetError(errInitV4); + return -1; + } + + if (errInitV6 && + (virNetworkDefGetIPByIndex(def, AF_INET6, 0) || + virNetworkDefGetRouteByIndex(def, AF_INET6, 0) || + def->ipv6nogw)) { + virSetError(errInitV6); + return -1; + } + + /* now actually add the rules */ + switch (firewallBackend) { + case VIR_FIREWALL_BACKEND_NONE: + virReportError(VIR_ERR_NO_SUPPORT, "%s", + _("No firewall backend is available")); + return -1; + + case VIR_FIREWALL_BACKEND_IPTABLES: + case VIR_FIREWALL_BACKEND_NFTABLES: + case VIR_FIREWALL_BACKEND_LAST: + virReportEnumRangeError(virFirewallBackend, firewallBackend); + return -1; + } + } + return 0; +} + +void +networkRemoveFirewallRules(virNetworkObj *obj, + bool unsetZone G_GNUC_UNUSED) +{ + virNetworkDef *def = virNetworkObjGetDef(obj); + if (def->forward.type == VIR_NETWORK_FORWARD_OPEN) { + VIR_DEBUG("No firewall rules to remove for mode='open' network '%s'", + def->name); + return; + } +} diff --git a/src/network/bridge_driver_platform.c b/src/network/bridge_driver_platform.c index 9ddcb71063..42fbcdbc0b 100644 --- a/src/network/bridge_driver_platform.c +++ b/src/network/bridge_driver_platform.c @@ -25,6 +25,8 @@
#if defined(__linux__) # include "bridge_driver_linux.c" +#elif defined(__FreeBSD__) +# include "bridge_driver_bsd.c" #else # include "bridge_driver_nop.c" #endif -- 2.49.0
With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Implement NAT networking support based on the Packet Filter (pf) firewall in FreeBSD. At this point, the implementation is very basic. It creates: - Essential NAT translation rules - Basic forwarding rules Implementation uses pf's anchor feature to group rules. All rules live in the "libvirt" anchor and every libvirt's network has its own sub-anchor. Currently there are some assumptions and limitations: - We assume that a user has created the "libvirt" (nat-)anchors. As they cannot be created on fly, it's better not to touch global pf configuration and let the user do the changes. If the user doesn't have these anchors configured, the rules will still be created in sub-anchors, but will not be effective until these anchors are activated. Should we check if these anchors are not active to give some runtime warning? - Currently, rule reloading is not smart: it always deletes rules, flushes rules and re-creates that. It would be better to do that more gracefully. - IPv6 configurations are currently not supported - For NAT, pf requires explicit IP address or an interface to NAT to. We try to obtain that from the network XML definition, and if it's not specified, we try to determine interface corresponding to the default route. Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com> --- meson.build | 2 + po/POTFILES | 1 + src/network/bridge_driver_bsd.c | 6 + src/network/bridge_driver_conf.c | 8 + src/network/bridge_driver_linux.c | 2 + src/network/meson.build | 1 + src/network/network_pf.c | 327 ++++++++++++++++++++++++++++++ src/network/network_pf.h | 26 +++ src/util/virfirewall.c | 4 +- src/util/virfirewall.h | 2 + 10 files changed, 378 insertions(+), 1 deletion(-) create mode 100644 src/network/network_pf.c create mode 100644 src/network/network_pf.h diff --git a/meson.build b/meson.build index 37b1caa566..b8a9be25bb 100644 --- a/meson.build +++ b/meson.build @@ -1642,6 +1642,8 @@ if not get_option('driver_network').disabled() and conf.has('WITH_LIBVIRTD') if firewall_backend_priority.length() == 0 if host_machine.system() == 'linux' firewall_backend_priority = ['nftables', 'iptables'] + elif host_machine.system() == 'freebsd' + firewall_backend_priority = ['pf'] else # No firewall impl on non-Linux so far, so force 'none' # as placeholder diff --git a/po/POTFILES b/po/POTFILES index 90664fe6e7..dc7293d0cd 100644 --- a/po/POTFILES +++ b/po/POTFILES @@ -152,6 +152,7 @@ src/network/bridge_driver_nop.c src/network/leaseshelper.c src/network/network_iptables.c src/network/network_nftables.c +src/network/network_pf.c src/node_device/node_device_driver.c src/node_device/node_device_udev.c src/nwfilter/nwfilter_dhcpsnoop.c diff --git a/src/network/bridge_driver_bsd.c b/src/network/bridge_driver_bsd.c index 93312fe6db..ad4fbd064b 100644 --- a/src/network/bridge_driver_bsd.c +++ b/src/network/bridge_driver_bsd.c @@ -19,6 +19,7 @@ #include <config.h> #include "virlog.h" +#include "network_pf.h" #define VIR_FROM_THIS VIR_FROM_NONE @@ -78,6 +79,9 @@ int networkAddFirewallRules(virNetworkDef *def G_GNUC_UNUSED, _("No firewall backend is available")); return -1; + case VIR_FIREWALL_BACKEND_PF: + return pfAddFirewallRules(def); + case VIR_FIREWALL_BACKEND_IPTABLES: case VIR_FIREWALL_BACKEND_NFTABLES: case VIR_FIREWALL_BACKEND_LAST: @@ -98,4 +102,6 @@ networkRemoveFirewallRules(virNetworkObj *obj, def->name); return; } + + pfRemoveFirewallRules(def); } diff --git a/src/network/bridge_driver_conf.c b/src/network/bridge_driver_conf.c index 738652390f..280c0f9c4f 100644 --- a/src/network/bridge_driver_conf.c +++ b/src/network/bridge_driver_conf.c @@ -129,6 +129,14 @@ virNetworkLoadDriverConfig(virNetworkDriverConfig *cfg G_GNUC_UNUSED, break; } + case VIR_FIREWALL_BACKEND_PF: { + g_autofree char *pfctlInPath = virFindFileInPath(PFCTL); + + if (pfctlInPath) + fwBackendSelected = true; + break; + } + case VIR_FIREWALL_BACKEND_LAST: virReportEnumRangeError(virFirewallBackend, fwBackends[i]); return -1; diff --git a/src/network/bridge_driver_linux.c b/src/network/bridge_driver_linux.c index 86f6a5915f..9077178c3e 100644 --- a/src/network/bridge_driver_linux.c +++ b/src/network/bridge_driver_linux.c @@ -58,6 +58,7 @@ networkFirewallSetupPrivateChains(virFirewallBackend backend, case VIR_FIREWALL_BACKEND_NFTABLES: return nftablesSetupPrivateChains(layer); + case VIR_FIREWALL_BACKEND_PF: case VIR_FIREWALL_BACKEND_LAST: virReportEnumRangeError(virFirewallBackend, backend); return -1; @@ -437,6 +438,7 @@ networkAddFirewallRules(virNetworkDef *def, case VIR_FIREWALL_BACKEND_NFTABLES: return nftablesAddFirewallRules(def, fwRemoval); + case VIR_FIREWALL_BACKEND_PF: case VIR_FIREWALL_BACKEND_LAST: virReportEnumRangeError(virFirewallBackend, firewallBackend); return -1; diff --git a/src/network/meson.build b/src/network/meson.build index 07cd5cda55..b527bab392 100644 --- a/src/network/meson.build +++ b/src/network/meson.build @@ -4,6 +4,7 @@ network_driver_sources = [ 'bridge_driver_platform.c', 'network_iptables.c', 'network_nftables.c', + 'network_pf.c', ] driver_source_files += files(network_driver_sources) diff --git a/src/network/network_pf.c b/src/network/network_pf.c new file mode 100644 index 0000000000..b12ef86089 --- /dev/null +++ b/src/network/network_pf.c @@ -0,0 +1,327 @@ +/* + * network_pf.c: pf-based firewall implementation for virtual networks + * + * Copyright (C) 2025 FreeBSD Foundation + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + */ + +/* + * pf(4) configuration principles/assumptions. + * + * All libvirt-managed firewall rule are configured within a pf anchor. + * Every libvirt network has a corresponding sub-anchor, like "libvirt/$network_name". + * Libvirt does not create the root anchors, so users are expected to specify them in + * their firewall configuration. Minimal configuration might look like: + * + * # cat /etc/pf.conf + * scrub all + * + * nat-anchor "libvirt\*" + * anchor "libvirt\*" + * + * pass all + * # + * + * Users are not expected to add/modify rules in the "libvirt\*" subanchors because + * the changes will be lost on restart. + * + * IPv6 NAT is currently not supported. + */ + +#include <config.h> + +#include <stdarg.h> +#include <unistd.h> +#include <fcntl.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <sys/socket.h> +#ifdef WITH_NET_IF_H +# include <net/if.h> +#endif +#if defined(__FreeBSD__) +# include <sys/sysctl.h> +# include <net/if_dl.h> +#endif +#include <net/route.h> +#include <netinet/in.h> + +#include "internal.h" +#include "virfirewalld.h" +#include "vircommand.h" +#include "virerror.h" +#include "virlog.h" +#include "virhash.h" +#include "virenum.h" +#include "virstring.h" +#include "network_pf.h" + +VIR_LOG_INIT("network.pf"); + +#define VIR_FROM_THIS VIR_FROM_NONE + + +static const char networkLocalMulticastIPv4[] = "224.0.0.0/24"; +static const char networkLocalBroadcast[] = "255.255.255.255/32"; + + +static char * +findDefaultRouteInterface(void) +{ +#if defined(__FreeBSD__) + int mib[6] = {CTL_NET, PF_ROUTE, 0, AF_INET, NET_RT_DUMP, 0}; + size_t needed; + g_autofree char *buf = NULL; + char *lim, *next; + struct rt_msghdr *rtm; + struct sockaddr *sa; + struct sockaddr_in *sin; + struct sockaddr_dl *sdl; + char *ifname; + size_t ifname_len; + size_t i; + + if (sysctl(mib, 6, NULL, &needed, NULL, 0) < 0) { + virReportSystemError(errno, + "%s", + _("Unable to get default interface name")); + return NULL; + } + + if (posix_memalign((void **)&buf, 8, needed) != 0) { + virReportSystemError(errno, + "%s", + _("Unable to get default interface name")); + return NULL; + } + + if (sysctl(mib, 6, buf, &needed, NULL, 0) < 0) { + virReportSystemError(errno, + "%s", + _("Unable to get default interface name")); + return NULL; + } + + lim = buf + needed; + next = buf; + + while (next < lim) { + rtm = (struct rt_msghdr *)(void *)next; + if (next + rtm->rtm_msglen > lim) + break; + + sin = (struct sockaddr_in *)(rtm + 1); + + if ((rtm->rtm_flags & RTF_GATEWAY) && sin->sin_addr.s_addr == INADDR_ANY) { + sdl = NULL; + sa = (struct sockaddr *)(sin + 1); + + for (i = 1; i < RTAX_MAX; i++) { + if (rtm->rtm_addrs & (1 << i)) { + if (i == RTAX_IFP && sa->sa_family == AF_LINK) { + sdl = (struct sockaddr_dl *)(void *)sa; + ifname_len = (sdl->sdl_nlen >= IFNAMSIZ) ? IFNAMSIZ - 1 : sdl->sdl_nlen; + ifname = g_new(char, ifname_len + 1); + virStrcpy(ifname, sdl->sdl_data, ifname_len + 1); + return ifname; + } + sa = (struct sockaddr *)((char *)sa + + ((sa->sa_len > 0) ? sa->sa_len : sizeof(struct sockaddr))); + } + } + } + + next += rtm->rtm_msglen; + } +#endif + + return NULL; +} + +static int +pfAddNatFirewallRules(virNetworkDef *def, + virNetworkIPDef *ipdef) +{ + /* + * # NAT rules + * table <natdst> persist + * { 0.0.0.0/0, ! 192.168.122.0/24, !224.0.0.0/24, !255.255.255.255 } + * nat pass log on $ext_if from 192.168.122.0/24 to <natdst> + * -> ($ext_if) port 1024:65535 + * + * # Filtering + * pass log quick on virbr0 from 192.168.122.0/24 to 192.168.122.0/24 + * pass out log quick on virbr0 from 192.168.122.0/24 to 224.0.0.0/24 + * pass out log quick on virbr0 from 192.168.122.0/24 to 255.255.255.255 + * block log on virbr0 + */ + int prefix = virNetworkIPDefPrefix(ipdef); + g_autofree const char *forwardIf = g_strdup(virNetworkDefForwardIf(def, 0)); + g_auto(virBuffer) pf_rules_buf = VIR_BUFFER_INITIALIZER; + g_autoptr(virCommand) cmd = virCommandNew(PFCTL); + virPortRange *portRange = &def->forward.port; + g_autofree char *portRangeStr = NULL; + + if (prefix < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Invalid prefix or netmask for '%1$s'"), + def->bridge); + return -1; + } + + if (portRange->start == 0 && portRange->end == 0) { + portRange->start = 1024; + portRange->end = 65535; + } + + if (portRange->start < portRange->end && portRange->end < 65536) { + portRangeStr = g_strdup_printf("%u:%u", + portRange->start, + portRange->end); + } else { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Invalid port range '%1$u-%2$u'."), + portRange->start, portRange->end); + return -1; + } + + if (!forwardIf) { + forwardIf = findDefaultRouteInterface(); + if (!forwardIf) { + virReportError(VIR_ERR_INTERNAL_ERROR, + "%s", + _("Cannot determine the default interface")); + return -1; + } + } + + virBufferAsprintf(&pf_rules_buf, + "table <natdst> persist { 0.0.0.0/0, ! %s/%d, ! %s, ! %s }\n", + virSocketAddrFormat(&ipdef->address), + prefix, + networkLocalMulticastIPv4, + networkLocalBroadcast); + virBufferAsprintf(&pf_rules_buf, + "nat pass on %s from %s/%d to <natdst> -> (%s) port %s\n", + forwardIf, + virSocketAddrFormat(&ipdef->address), + prefix, + forwardIf, + portRangeStr); + virBufferAsprintf(&pf_rules_buf, + "pass quick on %s from %s/%d to %s/%d\n", + def->bridge, + virSocketAddrFormat(&ipdef->address), + prefix, + virSocketAddrFormat(&ipdef->address), + prefix); + virBufferAsprintf(&pf_rules_buf, + "pass quick on %s from %s/%d to %s\n", + def->bridge, + virSocketAddrFormat(&ipdef->address), + prefix, + networkLocalMulticastIPv4); + virBufferAsprintf(&pf_rules_buf, + "pass quick on %s from %s/%d to %s\n", + def->bridge, + virSocketAddrFormat(&ipdef->address), + prefix, + networkLocalBroadcast); + virBufferAsprintf(&pf_rules_buf, + "block on %s\n", + def->bridge); + + /* pfctl -a libvirt/default -F all -f - */ + virCommandAddArg(cmd, "-a"); + virCommandAddArgFormat(cmd, "libvirt/%s", def->name); + virCommandAddArgList(cmd, "-F", "all", "-f", "-", NULL); + + virCommandSetInputBuffer(cmd, virBufferContentAndReset(&pf_rules_buf)); + + if (virCommandRun(cmd, NULL) < 0) { + VIR_WARN("Failed to create firewall rules for network %s", + def->name); + return -1; + } + return 0; +} + + +static int +pfAddRoutingFirewallRules(virNetworkDef *def, + virNetworkIPDef *ipdef G_GNUC_UNUSED) +{ + int prefix = virNetworkIPDefPrefix(ipdef); + + if (prefix < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Invalid prefix or netmask for '%1$s'"), + def->bridge); + return -1; + } + + /* TODO: routing rules */ + + return 0; +} + + +static int +pfAddIPSpecificFirewallRules(virNetworkDef *def, + virNetworkIPDef *ipdef) +{ + if (def->forward.type == VIR_NETWORK_FORWARD_NAT) { + if (VIR_SOCKET_ADDR_IS_FAMILY(&ipdef->address, AF_INET)) + return pfAddNatFirewallRules(def, ipdef); + else + return -1; + } else if (def->forward.type == VIR_NETWORK_FORWARD_ROUTE) { + return pfAddRoutingFirewallRules(def, ipdef); + } + return 0; +} + + +int +pfAddFirewallRules(virNetworkDef *def) +{ + size_t i; + virNetworkIPDef *ipdef; + + for (i = 0; + (ipdef = virNetworkDefGetIPByIndex(def, AF_UNSPEC, i)); + i++) { + if (pfAddIPSpecificFirewallRules(def, ipdef) < 0) + return -1; + } + + return 0; +} + + +void +pfRemoveFirewallRules(virNetworkDef *def) +{ + /* pfctl -a libvirt/default -F all */ + g_autoptr(virCommand) cmd = virCommandNew(PFCTL); + virCommandAddArg(cmd, "-a"); + virCommandAddArgFormat(cmd, "libvirt/%s", def->name); + virCommandAddArgList(cmd, "-F", "all", NULL); + + if (virCommandRun(cmd, NULL) < 0) + VIR_WARN("Failed to remove firewall rules for network %s", + def->name); +} diff --git a/src/network/network_pf.h b/src/network/network_pf.h new file mode 100644 index 0000000000..2cf5a1a6d9 --- /dev/null +++ b/src/network/network_pf.h @@ -0,0 +1,26 @@ +/* + * network_pf.h: helper APIs for managing pf in network driver + * + * Copyright (C) 2025 FreeBSD Foundation + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + */ + +#pragma once + +#include "network_conf.h" + +int pfAddFirewallRules(virNetworkDef *def); +void pfRemoveFirewallRules(virNetworkDef *def); diff --git a/src/util/virfirewall.c b/src/util/virfirewall.c index 9389bcf541..69521e2b46 100644 --- a/src/util/virfirewall.c +++ b/src/util/virfirewall.c @@ -39,7 +39,8 @@ VIR_ENUM_IMPL(virFirewallBackend, VIR_FIREWALL_BACKEND_LAST, "none", "iptables", - "nftables"); + "nftables", + "pf"); VIR_ENUM_DECL(virFirewallLayer); VIR_ENUM_IMPL(virFirewallLayer, @@ -847,6 +848,7 @@ virFirewallApplyCmd(virFirewall *firewall, return -1; break; + case VIR_FIREWALL_BACKEND_PF: case VIR_FIREWALL_BACKEND_LAST: default: virReportEnumRangeError(virFirewallBackend, diff --git a/src/util/virfirewall.h b/src/util/virfirewall.h index d42e60884b..030cb2c0a4 100644 --- a/src/util/virfirewall.h +++ b/src/util/virfirewall.h @@ -30,6 +30,7 @@ #define IPTABLES "iptables" #define IP6TABLES "ip6tables" #define NFT "nft" +#define PFCTL "pfctl" typedef struct _virFirewall virFirewall; @@ -48,6 +49,7 @@ typedef enum { VIR_FIREWALL_BACKEND_NONE, /* Always fails */ VIR_FIREWALL_BACKEND_IPTABLES, VIR_FIREWALL_BACKEND_NFTABLES, + VIR_FIREWALL_BACKEND_PF, VIR_FIREWALL_BACKEND_LAST, } virFirewallBackend; -- 2.49.0
On Sat, Apr 26, 2025 at 09:42:36AM +0200, Roman Bogorodskiy wrote:
Implement NAT networking support based on the Packet Filter (pf) firewall in FreeBSD. At this point, the implementation is very basic. It creates:
- Essential NAT translation rules - Basic forwarding rules
Implementation uses pf's anchor feature to group rules. All rules live in the "libvirt" anchor and every libvirt's network has its own sub-anchor.
Currently there are some assumptions and limitations:
- We assume that a user has created the "libvirt" (nat-)anchors. As they cannot be created on fly, it's better not to touch global pf configuration and let the user do the changes. If the user doesn't have these anchors configured, the rules will still be created in sub-anchors, but will not be effective until these anchors are activated. Should we check if these anchors are not active to give some runtime warning?
So IIUC a PF 'anchor' is essentially a top level namespace under which all libvirt rules will live, so they are isolated from other system rules ? This sounds similar to how we deal with the new 'nft' firewall on Linux, where we have to create top level tables to hold all our rules That has thrown up a tricky problem for us though around the prioritization. A packet has to be accepted by *every* top level table, for it to be allowed. So even if a libvirt rule allows it, potentiallly other system rules can still block it. This is basically unsolvable in the general case. We do a special hack in the firewalld case to tell it to put the bridge device into a separate libvirt "zone", which in turn means none of the other firewalld rules will ever get applied to those packets. Back to PF though. You're saying we can't create anchors on the fly - so that's saying the only way to create the anchors is to edit the /etc/ config file, and those get applied on system boot ? We should probably document this in docs/drvnetwork.rst... which I see does not actually exist :-( Probably good to start a skeleton of a page and put the FreeBSD specific notes there, and we can fll out Linux notes later
- Currently, rule reloading is not smart: it always deletes rules, flushes rules and re-creates that. It would be better to do that more gracefully.
That's not especially worse than what we do on Linux either
- IPv6 configurations are currently not supported
Probably worth reporting a fatal error if you see IPv6 config in the XML input, so attempts to use IPV6 are not silently ignored.
- For NAT, pf requires explicit IP address or an interface to NAT to. We try to obtain that from the network XML definition, and if it's not specified, we try to determine interface corresponding to the default route.
On Linux the defaults should work based on routing, so that's not too bad as a default behaviour. The difference is that FreeBSD would only work based on the single route, while the Linux behaviour could use all available routes. Not the end of the world, since most machines only have a single default route. The common exception being people using VPN clients.
diff --git a/src/network/network_pf.c b/src/network/network_pf.c new file mode 100644 index 0000000000..b12ef86089 --- /dev/null +++ b/src/network/network_pf.c @@ -0,0 +1,327 @@ +/* + * network_pf.c: pf-based firewall implementation for virtual networks + * + * Copyright (C) 2025 FreeBSD Foundation + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + */ + +/* + * pf(4) configuration principles/assumptions. + * + * All libvirt-managed firewall rule are configured within a pf anchor. + * Every libvirt network has a corresponding sub-anchor, like "libvirt/$network_name". + * Libvirt does not create the root anchors, so users are expected to specify them in + * their firewall configuration. Minimal configuration might look like: + * + * # cat /etc/pf.conf + * scrub all + * + * nat-anchor "libvirt\*" + * anchor "libvirt\*" + * + * pass all + * # + * + * Users are not expected to add/modify rules in the "libvirt\*" subanchors because + * the changes will be lost on restart. + * + * IPv6 NAT is currently not supported. + */ + +#include <config.h> + +#include <stdarg.h> +#include <unistd.h> +#include <fcntl.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <sys/socket.h> +#ifdef WITH_NET_IF_H +# include <net/if.h> +#endif +#if defined(__FreeBSD__)
Do you need these __FreeBSD__ conditions here ? Is 'pf' intended to be used on anything other than FreeBSD ? If not, we should ensure that it simply never built on any other platform at the meson level.
+# include <sys/sysctl.h> +# include <net/if_dl.h> +#endif +#include <net/route.h> +#include <netinet/in.h> + +#include "internal.h" +#include "virfirewalld.h" +#include "vircommand.h" +#include "virerror.h" +#include "virlog.h" +#include "virhash.h" +#include "virenum.h" +#include "virstring.h" +#include "network_pf.h" + +VIR_LOG_INIT("network.pf"); + +#define VIR_FROM_THIS VIR_FROM_NONE + + +static const char networkLocalMulticastIPv4[] = "224.0.0.0/24"; +static const char networkLocalBroadcast[] = "255.255.255.255/32"; + + +static char * +findDefaultRouteInterface(void) +{ +#if defined(__FreeBSD__) + int mib[6] = {CTL_NET, PF_ROUTE, 0, AF_INET, NET_RT_DUMP, 0}; + size_t needed; + g_autofree char *buf = NULL; + char *lim, *next; + struct rt_msghdr *rtm; + struct sockaddr *sa; + struct sockaddr_in *sin; + struct sockaddr_dl *sdl; + char *ifname; + size_t ifname_len; + size_t i; + + if (sysctl(mib, 6, NULL, &needed, NULL, 0) < 0) { + virReportSystemError(errno, + "%s", + _("Unable to get default interface name")); + return NULL; + } + + if (posix_memalign((void **)&buf, 8, needed) != 0) { + virReportSystemError(errno, + "%s", + _("Unable to get default interface name")); + return NULL; + } + + if (sysctl(mib, 6, buf, &needed, NULL, 0) < 0) { + virReportSystemError(errno, + "%s", + _("Unable to get default interface name")); + return NULL; + } + + lim = buf + needed; + next = buf; + + while (next < lim) { + rtm = (struct rt_msghdr *)(void *)next; + if (next + rtm->rtm_msglen > lim) + break; + + sin = (struct sockaddr_in *)(rtm + 1); + + if ((rtm->rtm_flags & RTF_GATEWAY) && sin->sin_addr.s_addr == INADDR_ANY) { + sdl = NULL; + sa = (struct sockaddr *)(sin + 1); + + for (i = 1; i < RTAX_MAX; i++) { + if (rtm->rtm_addrs & (1 << i)) { + if (i == RTAX_IFP && sa->sa_family == AF_LINK) { + sdl = (struct sockaddr_dl *)(void *)sa; + ifname_len = (sdl->sdl_nlen >= IFNAMSIZ) ? IFNAMSIZ - 1 : sdl->sdl_nlen; + ifname = g_new(char, ifname_len + 1); + virStrcpy(ifname, sdl->sdl_data, ifname_len + 1); + return ifname; + } + sa = (struct sockaddr *)((char *)sa + + ((sa->sa_len > 0) ? sa->sa_len : sizeof(struct sockaddr))); + } + } + } + + next += rtm->rtm_msglen; + } +#endif + + return NULL; +} + +static int +pfAddNatFirewallRules(virNetworkDef *def, + virNetworkIPDef *ipdef) +{ + /* + * # NAT rules + * table <natdst> persist + * { 0.0.0.0/0, ! 192.168.122.0/24, !224.0.0.0/24, !255.255.255.255 } + * nat pass log on $ext_if from 192.168.122.0/24 to <natdst> + * -> ($ext_if) port 1024:65535 + * + * # Filtering + * pass log quick on virbr0 from 192.168.122.0/24 to 192.168.122.0/24 + * pass out log quick on virbr0 from 192.168.122.0/24 to 224.0.0.0/24 + * pass out log quick on virbr0 from 192.168.122.0/24 to 255.255.255.255 + * block log on virbr0 + */ + int prefix = virNetworkIPDefPrefix(ipdef); + g_autofree const char *forwardIf = g_strdup(virNetworkDefForwardIf(def, 0)); + g_auto(virBuffer) pf_rules_buf = VIR_BUFFER_INITIALIZER; + g_autoptr(virCommand) cmd = virCommandNew(PFCTL); + virPortRange *portRange = &def->forward.port; + g_autofree char *portRangeStr = NULL; + + if (prefix < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Invalid prefix or netmask for '%1$s'"), + def->bridge); + return -1; + } + + if (portRange->start == 0 && portRange->end == 0) { + portRange->start = 1024; + portRange->end = 65535; + } + + if (portRange->start < portRange->end && portRange->end < 65536) { + portRangeStr = g_strdup_printf("%u:%u", + portRange->start, + portRange->end); + } else { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Invalid port range '%1$u-%2$u'."), + portRange->start, portRange->end); + return -1; + } + + if (!forwardIf) { + forwardIf = findDefaultRouteInterface(); + if (!forwardIf) { + virReportError(VIR_ERR_INTERNAL_ERROR, + "%s", + _("Cannot determine the default interface")); + return -1; + } + } + + virBufferAsprintf(&pf_rules_buf, + "table <natdst> persist { 0.0.0.0/0, ! %s/%d, ! %s, ! %s }\n", + virSocketAddrFormat(&ipdef->address), + prefix, + networkLocalMulticastIPv4, + networkLocalBroadcast); + virBufferAsprintf(&pf_rules_buf, + "nat pass on %s from %s/%d to <natdst> -> (%s) port %s\n", + forwardIf, + virSocketAddrFormat(&ipdef->address), + prefix, + forwardIf, + portRangeStr); + virBufferAsprintf(&pf_rules_buf, + "pass quick on %s from %s/%d to %s/%d\n", + def->bridge, + virSocketAddrFormat(&ipdef->address), + prefix, + virSocketAddrFormat(&ipdef->address), + prefix); + virBufferAsprintf(&pf_rules_buf, + "pass quick on %s from %s/%d to %s\n", + def->bridge, + virSocketAddrFormat(&ipdef->address), + prefix, + networkLocalMulticastIPv4); + virBufferAsprintf(&pf_rules_buf, + "pass quick on %s from %s/%d to %s\n", + def->bridge, + virSocketAddrFormat(&ipdef->address), + prefix, + networkLocalBroadcast); + virBufferAsprintf(&pf_rules_buf, + "block on %s\n", + def->bridge); + + /* pfctl -a libvirt/default -F all -f - */ + virCommandAddArg(cmd, "-a"); + virCommandAddArgFormat(cmd, "libvirt/%s", def->name); + virCommandAddArgList(cmd, "-F", "all", "-f", "-", NULL); + + virCommandSetInputBuffer(cmd, virBufferContentAndReset(&pf_rules_buf)); + + if (virCommandRun(cmd, NULL) < 0) { + VIR_WARN("Failed to create firewall rules for network %s", + def->name); + return -1; + } + return 0; +} + + +static int +pfAddRoutingFirewallRules(virNetworkDef *def, + virNetworkIPDef *ipdef G_GNUC_UNUSED) +{ + int prefix = virNetworkIPDefPrefix(ipdef); + + if (prefix < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Invalid prefix or netmask for '%1$s'"), + def->bridge); + return -1; + } + + /* TODO: routing rules */ + + return 0; +} + + +static int +pfAddIPSpecificFirewallRules(virNetworkDef *def, + virNetworkIPDef *ipdef) +{ + if (def->forward.type == VIR_NETWORK_FORWARD_NAT) { + if (VIR_SOCKET_ADDR_IS_FAMILY(&ipdef->address, AF_INET)) + return pfAddNatFirewallRules(def, ipdef); + else + return -1;
Should report an error message here about lack of IPV6
+ } else if (def->forward.type == VIR_NETWORK_FORWARD_ROUTE) { + return pfAddRoutingFirewallRules(def, ipdef);
Should this check for IPv6 too ? The linux impl doesn't seem to conditionalize based on IP family though.
+ } + return 0; +} + + +int +pfAddFirewallRules(virNetworkDef *def) +{ + size_t i; + virNetworkIPDef *ipdef; + + for (i = 0; + (ipdef = virNetworkDefGetIPByIndex(def, AF_UNSPEC, i)); + i++) { + if (pfAddIPSpecificFirewallRules(def, ipdef) < 0) + return -1; + } + + return 0; +} + + +void +pfRemoveFirewallRules(virNetworkDef *def) +{ + /* pfctl -a libvirt/default -F all */ + g_autoptr(virCommand) cmd = virCommandNew(PFCTL); + virCommandAddArg(cmd, "-a"); + virCommandAddArgFormat(cmd, "libvirt/%s", def->name); + virCommandAddArgList(cmd, "-F", "all", NULL); + + if (virCommandRun(cmd, NULL) < 0) + VIR_WARN("Failed to remove firewall rules for network %s", + def->name); +} diff --git a/src/util/virfirewall.h b/src/util/virfirewall.h index d42e60884b..030cb2c0a4 100644 --- a/src/util/virfirewall.h +++ b/src/util/virfirewall.h @@ -30,6 +30,7 @@ #define IPTABLES "iptables" #define IP6TABLES "ip6tables" #define NFT "nft" +#define PFCTL "pfctl"
typedef struct _virFirewall virFirewall;
@@ -48,6 +49,7 @@ typedef enum { VIR_FIREWALL_BACKEND_NONE, /* Always fails */ VIR_FIREWALL_BACKEND_IPTABLES, VIR_FIREWALL_BACKEND_NFTABLES, + VIR_FIREWALL_BACKEND_PF,
VIR_FIREWALL_BACKEND_LAST, } virFirewallBackend;
I'd suggest adding the new backend to virfirewall as a standalone patch behand, so this patch focuses exclusively on the freebsd pf impl in the network driver. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Daniel P. Berrangé wrote:
On Sat, Apr 26, 2025 at 09:42:36AM +0200, Roman Bogorodskiy wrote:
Implement NAT networking support based on the Packet Filter (pf) firewall in FreeBSD. At this point, the implementation is very basic. It creates:
- Essential NAT translation rules - Basic forwarding rules
Implementation uses pf's anchor feature to group rules. All rules live in the "libvirt" anchor and every libvirt's network has its own sub-anchor.
Currently there are some assumptions and limitations:
- We assume that a user has created the "libvirt" (nat-)anchors. As they cannot be created on fly, it's better not to touch global pf configuration and let the user do the changes. If the user doesn't have these anchors configured, the rules will still be created in sub-anchors, but will not be effective until these anchors are activated. Should we check if these anchors are not active to give some runtime warning?
So IIUC a PF 'anchor' is essentially a top level namespace under which all libvirt rules will live, so they are isolated from other system rules ?
Yes, that's correct.
This sounds similar to how we deal with the new 'nft' firewall on Linux, where we have to create top level tables to hold all our rules
That has thrown up a tricky problem for us though around the prioritization. A packet has to be accepted by *every* top level table, for it to be allowed. So even if a libvirt rule allows it, potentiallly other system rules can still block it. This is basically unsolvable in the general case. We do a special hack in the firewalld case to tell it to put the bridge device into a separate libvirt "zone", which in turn means none of the other firewalld rules will ever get applied to those packets.
It depends. In PF, rules could be created with the "quick" keyword. Quoting the manual page: Matching filter and translation rules marked with the quick option are final and abort the evaluation of the rules in other anchors and the main ruleset. If the anchor itself is marked with the quick option, ruleset evaluation will terminate when the anchor is exited if the packet is matched by any rule within the anchor. I'm using this keyword for some of the pass rules to avoid these packets occasionally blocked.
Back to PF though. You're saying we can't create anchors on the fly - so that's saying the only way to create the anchors is to edit the /etc/ config file, and those get applied on system boot ?
Yes, that's true. Well, in theory, we can edit the /etc/pf.conf, try to find an appropriate place for anchors definitions, and reload the rules. I saw some management software does that, but that seems to be too invasive for me, at least for now.
We should probably document this in docs/drvnetwork.rst... which I see does not actually exist :-( Probably good to start a skeleton of a page and put the FreeBSD specific notes there, and we can fll out Linux notes later
- Currently, rule reloading is not smart: it always deletes rules, flushes rules and re-creates that. It would be better to do that more gracefully.
That's not especially worse than what we do on Linux either
- IPv6 configurations are currently not supported
Probably worth reporting a fatal error if you see IPv6 config in the XML input, so attempts to use IPV6 are not silently ignored.
- For NAT, pf requires explicit IP address or an interface to NAT to. We try to obtain that from the network XML definition, and if it's not specified, we try to determine interface corresponding to the default route.
On Linux the defaults should work based on routing, so that's not too bad as a default behaviour. The difference is that FreeBSD would only work based on the single route, while the Linux behaviour could use all available routes. Not the end of the world, since most machines only have a single default route. The common exception being people using VPN clients.
Thanks for feedback, will address the comments in the follow up series. Roman
participants (2)
-
Daniel P. Berrangé -
Roman Bogorodskiy