9:27 a.m.
*sigh*
Found this is the code:
#ifdef VIR_NFTABLES_INCLUDE_HOST_RULES
/* These rules copied from the iptables backend, have been removed
* from the nftab because they are redundant since we are using our
own
* table that is default accept; there are no other users that
* could add a reject rule that we would need to / be able to
* override with these rules
*/
/* allow DHCP requests through to dnsmasq & back out */
nftablesAddTcpInput(fw, VIR_FIREWALL_LAYER_IPV4, def->bridge, 67);
nftablesAddUdpInput(fw, VIR_FIREWALL_LAYER_IPV4, def->bridge, 67);
nftablesAddTcpOutput(fw, VIR_FIREWALL_LAYER_IPV4, def->bridge, 68);
nftablesAddUdpOutput(fw, VIR_FIREWALL_LAYER_IPV4, def->bridge, 68);
Turns out this is correct, so it *was* an nftables ordering problem, I just
didn't know that libvirt didn't lay those rules down in nftables mode so it
looked broken.
Once I removed the relevant reject in my nftables config, it worked.
On Fri, Feb 21, 2025 at 12:06 AM Robin Powell <robinleepowell(a)gmail.com>
wrote:
Setting `firewall_backend = "iptables"` and allowing it to
use the
iptables-nft shim *did* get the right rules into place.
I then ran smack into the "my main input chain has a default deny rule",
but that's my problem. :)
On Thu, Feb 20, 2025 at 10:33 PM <robinleepowell(a)gmail.com> wrote:
> I do not think this is any of the similar issues that have been
> posted to this list; I've checked.
>
> In particular, this in *NOT* an issue with nft rules precedence; the
> rules are simply not being written by libvirt.
>
> I'm running vagrant which is running libvirt on a Fedora 41 host. I
> do not think this is a vagrant problem, but if people want me to run
> virsh commands directly I certainly can.
>
> Anyway, vagrant was failing until I noticed that the default for
> /etc/libvirt/network.conf is now nftables, which I did not have set
> up. When I set `firewall_backend = "iptables"`, everything worked
> fine.
>
> I want to emphasize that: the same vagrant / libvirt setup *was*
> working with iptables.
>
> But I took that as a sign that it was time to move to nftables, so I
> moved everything on this host and stuff is back to working.
>
> But libvirt just isn't writing out the right nft rules. Like, at
> all.
>
> Here's the network vagrant creates:
>
> $ sudo virsh net-dumpxml vagrant-libvirt
> <network connections='1' ipv6='yes'>
> <name>vagrant-libvirt</name>
> <uuid>b2d93ef4-b305-4382-a380-c1eca92d8ebd</uuid>
> <forward mode='nat'>
> <nat>
> <port start='1024' end='65535'/>
> </nat>
> </forward>
> <bridge name='virbr1' stp='on' delay='0'/>
> <mac address='52:54:00:63:d3:f6'/>
> <ip address='192.168.121.1' netmask='255.255.255.0'>
> <dhcp>
> <range start='192.168.121.1' end='192.168.121.254'/>
> </dhcp>
> </ip>
> </network>
>
> And here's the *entire* libvirt-related ruleset in nftables:
>
> table ip libvirt_network {
> chain forward {
> type filter hook forward priority filter; policy
> accept;
> counter packets 0 bytes 0 jump guest_cross
> counter packets 0 bytes 0 jump guest_input
> counter packets 0 bytes 0 jump guest_output
> }
>
> chain guest_output {
> ip saddr 192.168.121.0/24 iif "virbr1" counter
> packets 0 bytes 0 accept
> iif "virbr1" counter packets 0 bytes 0 reject
> }
>
> chain guest_input {
> oif "virbr1" ip daddr 192.168.121.0/24 ct state
> established,related counter packets 0 bytes 0 accept
> oif "virbr1" counter packets 0 bytes 0 reject
> }
>
> chain guest_cross {
> iif "virbr1" oif "virbr1" counter packets 0
bytes 0
> accept
> }
>
> chain guest_nat {
> type nat hook postrouting priority srcnat; policy
> accept;
> ip saddr 192.168.121.0/24 ip daddr 224.0.0.0/24
> counter packets 1 bytes 187 return
> ip saddr 192.168.121.0/24 ip daddr 255.255.255.255
> counter packets 0 bytes 0 return
> meta l4proto tcp ip saddr 192.168.121.0/24 ip daddr
> != 192.168.121.0/24 counter packets 0 bytes 0 masquerade to :1024-65535
> meta l4proto udp ip saddr 192.168.121.0/24 ip daddr
> != 192.168.121.0/24 counter packets 0 bytes 0 masquerade to :1024-65535
> ip saddr 192.168.121.0/24 ip daddr !=
> 192.168.121.0/24 counter packets 0 bytes 0 masquerade
> }
> }
> table ip6 libvirt_network {
> chain forward {
> type filter hook forward priority filter; policy
> accept;
> counter packets 0 bytes 0 jump guest_cross
> counter packets 0 bytes 0 jump guest_input
> counter packets 0 bytes 0 jump guest_output
> }
>
> chain guest_output {
> iif "virbr1" counter packets 0 bytes 0 reject
> }
>
> chain guest_input {
> oif "virbr1" counter packets 0 bytes 0 reject
> }
>
> chain guest_cross {
> iif "virbr1" oif "virbr1" counter packets 0
bytes 0
> accept
> }
>
> chain guest_nat {
> type nat hook postrouting priority srcnat; policy
> accept;
> }
> }
>
> Here's some log output (I have virtnetworkd running with -v):
>
> $ sudo journalctl -u virtnetworkd.service | grep -i nft
> [snip]
> Feb 20 21:36:42 stodi.digitalkingdom.org virtnetworkd[89119]: using
> firewall_backend: 'nftables'
> Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]:
> Applying 'nft list table ip libvirt_network'
> Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]:
> Applying 'nft list table ip6 libvirt_network'
> Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]:
> Applying 'nft -ae insert rule ip libvirt_network guest_output iif virbr1
> counter reject'
> Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]:
> Applying 'nft -ae insert rule ip libvirt_network guest_input oif virbr1
> counter reject'
> Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]:
> Applying 'nft -ae insert rule ip libvirt_network guest_cross iif virbr1 oif
> virbr1 counter accept'
> Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]:
> Applying 'nft -ae insert rule ip6 libvirt_network guest_output iif virbr1
> counter reject'
> Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]:
> Applying 'nft -ae insert rule ip6 libvirt_network guest_input oif virbr1
> counter reject'
> Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]:
> Applying 'nft -ae insert rule ip6 libvirt_network guest_cross iif virbr1
> oif virbr1 counter accept'
> Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]:
> Applying 'nft -ae insert rule ip libvirt_network guest_output ip saddr
> 192.168.121.0/24 iif virbr1 counter accept'
> Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]:
> Applying 'nft -ae insert rule ip libvirt_network guest_input oif virbr1 ip
> daddr 192.168.121.0/24 ct state related,established counter accept'
> Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]:
> Applying 'nft -ae insert rule ip libvirt_network guest_nat ip saddr
> 192.168.121.0/24 ip daddr '!=' 192.168.121.0/24 counter masquerade'
> Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]:
> Applying 'nft -ae insert rule ip libvirt_network guest_nat meta l4proto udp
> ip saddr 192.168.121.0/24 ip daddr '!=' 192.168.121.0/24 counter
> masquerade to :1024-65535'
> Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]:
> Applying 'nft -ae insert rule ip libvirt_network guest_nat meta l4proto tcp
> ip saddr 192.168.121.0/24 ip daddr '!=' 192.168.121.0/24 counter
> masquerade to :1024-65535'
> Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]:
> Applying 'nft -ae insert rule ip libvirt_network guest_nat ip saddr
> 192.168.121.0/24 ip daddr 255.255.255.255/32 counter return'
> Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]:
> Applying 'nft -ae insert rule ip libvirt_network guest_nat ip saddr
> 192.168.121.0/24 ip daddr 224.0.0.0/24 counter return'
>
> So it's not like trying to create the rules and failing, that I can
> see; it just isn't trying.
>
> To clarify this setup appears to be missing every rule that would be
> needed for DHCPv4 and DNS to work.
>
> Like https://gitlab.com/libvirt/libvirt/-/issues/88#note_694493261
> shows rules for udp 67 and 53 that are just 100% not being created
> at all.
>
> I have no idea what's going wrong here or even where to look; the
> code at
>
>
https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/GT...
> sure *looks like* it should be unconditionally adding those rules.
>
> Help?
>