Re: [libvirt PATCH 00/28] native support for nftables in virtual network driver
Laine Stump wrote:
On 6/10/24 2:54 PM, Roman Bogorodskiy wrote:
Laine Stump wrote:
This patch series enables libvirt to use nftables rules rather than iptables *when setting up virtual networks* (it does *not* add nftables support to the nwfilter driver). It accomplishes this by abstracting several iptables functions (from viriptables.[ch] called by the virtual network driver into a rudimentary "virNetfilter API" (in virnetfilter.[ch], having the virtual network driver call the virNetFilter API rather than calling the existing iptables functions directly, and then finally adding an equivalent virNftables backend that can be used instead of iptables (selected manually via a network.conf setting, or automatically if iptables isn't found on the host).
Hi,
Apparently, I'm late to the discussion.
I noticed that now I cannot use the bridge driver on FreeBSD as it's failing to initialize both iptables and nftables backends (which is expect).
Yeah, previously we wouldn't check if iptables was available until someone tried to start a network that would need to use it, and would then log an error (and just fail starting that network, but the network driver would remain running). But now we figure out which firewall backend to use immediately when the driver is loaded, and if we fail to fin a workable backend we fail the entire driver init.r
How did you use the network driver before? With a <forward mode='open'/> network? Truthfully I hadn't ever considered the case of someone using it with only network types that didn't need firewall rules. I wonder if there are other platforms we support that have a usable network driver for <forward mode='open'/> (MacOS?)
I'm using it with the following network configuration: virsh # net-dumpxml default <network> <name>default</name> <uuid>2a1415c9-325b-41e4-82c6-e805162d8934</uuid> <forward mode='nat'/> <bridge name='virbr0' stp='on' delay='0'/> <mac address='52:54:00:24:fa:43'/> <ip address='192.168.122.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.122.2' end='192.168.122.254'/> </dhcp> </ip> </network> So basically all the mechanics like creating tap devices, bridges, serving dhcp, etc, all these work for me. On top of that I had a few iterations of manual firewall configurations (with both ipfw and pf) to implement NAT on guests. Unfortunately, I don't have access to that setup anymore and I haven't re-created it yet. IIRC, it could probably show some warnings about missing iptables, but it didn't affect anything for me.
What would be a good way to address that? I see at least two options:
1. Add a Noop firewall driver 2. Implement a "real" FreeBSD driver based either on pf or ipfw (that's been on my TODO list forever, but I somehow got stuck on the very first step on choosing between pf and ipfw).
Why not both? :-)
This obviously will take much more time.
Maybe there are other options I'm missing.
Obviously (2) would be nicest, but I guess in the short term some variation of (1) would be quicker.
Another possibility could be to restore the old behavior of saving the error and only reporting it when a network requiring a firewall is loaded, but I think I remember a discussion about this during review of an earlier revision of the patches, and we agreed that it made the problem easier to find if it was reported immediately and the driver load failed.
I suppose in the long run the build-time option firewall_backend_priority should be used to control which backends are included in the build (rather than just which ones are checked at runtime), so that FreeBSD could completely skip all the iptables and nftables code (and firewalld when that's done), and Linux platforms could skip pf and ipfw.
What do you think?
I'm about to be offline for 3 weeks, but in the meantime if you'd like to try making a NULL backend that is only an option if it's listed in firewall_backend_priority (you'll need to remove the compile-time check that all possible backends are accounted for - I think that is the first of the two G_STATIC_ASSERTS at the top of virNetworkLoadDriverConfig()), always initializes successfully in bridge_driver_conf.c if it is listed in the options, and then in networkAddFirewallRules add a check to log an error and fail if backend == NULL (something about attempting to start a network type that would require firewall rules, but the system not having any of the supported types of firewallbackend or something - it's too late now and my brain is too fried and sleepy to think of good wording :-)). As long as it isn't a valid selection on Linux builds that are done with firewall_backend_priority=nftables,iptables, but *is* a valid selection if the setting is "firewall_backend_priority=null" that shouldn't be *too* controversial.
Ok, I think I can try making the NULL backend.
Later we can talk about pf and ipfw backends :-)
Yeah, that sounds good. My main problem with the choice is that ipfw is the most actively supported firewall, but it relies quite heavily on the rule numbering, which makes it a little hard to integrate with user-specific rules (i.e. defined outside of libvirt). The "pf" seem to be better in this regard (at least to my taste), but it's not "native" FreeBSD firewall and is not as active (at least, to my impression). Roman
On 6/12/24 2:32 PM, Roman Bogorodskiy wrote:
Laine Stump wrote:
On 6/10/24 2:54 PM, Roman Bogorodskiy wrote:
Laine Stump wrote:
This patch series enables libvirt to use nftables rules rather than iptables *when setting up virtual networks* (it does *not* add nftables support to the nwfilter driver). It accomplishes this by abstracting several iptables functions (from viriptables.[ch] called by the virtual network driver into a rudimentary "virNetfilter API" (in virnetfilter.[ch], having the virtual network driver call the virNetFilter API rather than calling the existing iptables functions directly, and then finally adding an equivalent virNftables backend that can be used instead of iptables (selected manually via a network.conf setting, or automatically if iptables isn't found on the host).
Hi,
Apparently, I'm late to the discussion.
I noticed that now I cannot use the bridge driver on FreeBSD as it's failing to initialize both iptables and nftables backends (which is expect).
Yeah, previously we wouldn't check if iptables was available until someone tried to start a network that would need to use it, and would then log an error (and just fail starting that network, but the network driver would remain running). But now we figure out which firewall backend to use immediately when the driver is loaded, and if we fail to fin a workable backend we fail the entire driver init.r
How did you use the network driver before? With a <forward mode='open'/> network? Truthfully I hadn't ever considered the case of someone using it with only network types that didn't need firewall rules. I wonder if there are other platforms we support that have a usable network driver for <forward mode='open'/> (MacOS?)
I'm using it with the following network configuration:
virsh # net-dumpxml default <network> <name>default</name> <uuid>2a1415c9-325b-41e4-82c6-e805162d8934</uuid> <forward mode='nat'/> <bridge name='virbr0' stp='on' delay='0'/> <mac address='52:54:00:24:fa:43'/> <ip address='192.168.122.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.122.2' end='192.168.122.254'/> </dhcp> </ip> </network>
So basically all the mechanics like creating tap devices, bridges, serving dhcp, etc, all these work for me. On top of that I had a few iterations of manual firewall configurations (with both ipfw and pf) to implement NAT on guests.
Okay, so essentially it was effectively <forward mode='open'/> (since the only difference is the lack of libvirt-added firewall rules).
Unfortunately, I don't have access to that setup anymore and I haven't re-created it yet. IIRC, it could probably show some warnings about missing iptables, but it didn't affect anything for me.
I'm surprised that there wasn't a fatal error while starting the network though.
I'm about to be offline for 3 weeks, but in the meantime if you'd like to try making a NULL backend that is only an option if it's listed in firewall_backend_priority (you'll need to remove the compile-time check that all possible backends are accounted for - I think that is the first of the two G_STATIC_ASSERTS at the top of virNetworkLoadDriverConfig()), always initializes successfully in bridge_driver_conf.c if it is listed in the options, and then in networkAddFirewallRules add a check to log an error and fail if backend == NULL (something about attempting to start a network type that would require firewall rules, but the system not having any of the supported types of firewallbackend or something - it's too late now and my brain is too fried and sleepy to think of good wording :-)). As long as it isn't a valid selection on Linux builds that are done with firewall_backend_priority=nftables,iptables, but *is* a valid selection if the setting is "firewall_backend_priority=null" that shouldn't be *too* controversial.
Ok, I think I can try making the NULL backend.
Yeah, Daniel's patch will actually disable the network driver completely, which will allow the rest of libvirt to still work, but sounds like it's not completely what you want. I think even with a NULL firewall backend it still should log an error and fail if someone tries to start a network that requires firewall rules though (i.e. your above config would still fail, unless it's changed to <forward mode='open'/>)
Later we can talk about pf and ipfw backends :-)
Yeah, that sounds good. My main problem with the choice is that ipfw is the most actively supported firewall, but it relies quite heavily on the rule numbering, which makes it a little hard to integrate with user-specific rules (i.e. defined outside of libvirt).
Well, there is a difficulty with any of the filtering systems wrt coordinating rules from multiple independent users, so nothing new there :-).
The "pf" seem to be better in this regard (at least to my taste), but it's not "native" FreeBSD firewall and is not as active (at least, to my impression).
I guess I should read up on both of those - I haven't done anything with packet filtering in a BSD since ipfilter back in the mid-90's :-)
On Wed, Jun 12, 2024 at 07:31:51PM GMT, Laine Stump wrote:
On 6/12/24 2:32 PM, Roman Bogorodskiy wrote:
I'm using it with the following network configuration:
virsh # net-dumpxml default <network> <name>default</name> <uuid>2a1415c9-325b-41e4-82c6-e805162d8934</uuid> <forward mode='nat'/> <bridge name='virbr0' stp='on' delay='0'/> <mac address='52:54:00:24:fa:43'/> <ip address='192.168.122.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.122.2' end='192.168.122.254'/> </dhcp> </ip> </network>
My configuration is the same (obtained from copying the file shipped as /usr/local/share/examples/libvirt/networks/default.xml, which is identical to src/network/default.xml.in in the libvirt tree) and I get an error when I try to start the network: # virsh net-start default error: Failed to start network default error: Unable to create bridge device: Invalid argument The debug log reveals the source of the error to be virNetDevBridgeCreate:474 I don't understand how that would work for you. My setup is completely vanilla, just a plain FreeBSD 14.1 install. The only thing that could possibly be making any difference is that the host's network interface is a wireless one.
So basically all the mechanics like creating tap devices, bridges, serving dhcp, etc, all these work for me. On top of that I had a few iterations of manual firewall configurations (with both ipfw and pf) to implement NAT on guests.
Okay, so essentially it was effectively <forward mode='open'/> (since the only difference is the lack of libvirt-added firewall rules).
[...]
I think even with a NULL firewall backend it still should log an error and fail if someone tries to start a network that requires firewall rules though (i.e. your above config would still fail, unless it's changed to <forward mode='open'/>)
I agree. The fact that this was allowed in the first place is arguably a bug
I'm about to be offline for 3 weeks, but in the meantime if you'd like to try making a NULL backend that is only an option if it's listed in firewall_backend_priority (you'll need to remove the compile-time check that all possible backends are accounted for - I think that is the first of the two G_STATIC_ASSERTS at the top of virNetworkLoadDriverConfig()), always initializes successfully in bridge_driver_conf.c if it is listed in the options, and then in networkAddFirewallRules add a check to log an error and fail if backend == NULL (something about attempting to start a network type that would require firewall rules, but the system not having any of the supported types of firewallbackend or something - it's too late now and my brain is too fried and sleepy to think of good wording :-)). As long as it isn't a valid selection on Linux builds that are done with firewall_backend_priority=nftables,iptables, but *is* a valid selection if the setting is "firewall_backend_priority=null" that shouldn't be *too* controversial.
I think we can treat the null backend just like the other ones. New default: -Dfirewall_backend_priority=nftables,iptables,null Works fine on Linux (even for qemu:///session) and FreeBSD. If you don't want firewall rules to ever be created (rules out mode=nat) for whatever reason: firewall_backend = "null" Same as above, but at compile time: -Dfirewall_backend_priority=null,nftables,iptables If you changed your mind later but don't want to recompile, or disagree with the package maintainer: firewall_backend = "iptables" Seems pretty reasonable and seamless to me. -- Andrea Bolognani / Red Hat / Virtualization
Andrea Bolognani wrote:
On Wed, Jun 12, 2024 at 07:31:51PM GMT, Laine Stump wrote:
On 6/12/24 2:32 PM, Roman Bogorodskiy wrote:
I'm using it with the following network configuration:
virsh # net-dumpxml default <network> <name>default</name> <uuid>2a1415c9-325b-41e4-82c6-e805162d8934</uuid> <forward mode='nat'/> <bridge name='virbr0' stp='on' delay='0'/> <mac address='52:54:00:24:fa:43'/> <ip address='192.168.122.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.122.2' end='192.168.122.254'/> </dhcp> </ip> </network>
My configuration is the same (obtained from copying the file shipped as /usr/local/share/examples/libvirt/networks/default.xml, which is identical to src/network/default.xml.in in the libvirt tree) and I get an error when I try to start the network:
# virsh net-start default error: Failed to start network default error: Unable to create bridge device: Invalid argument
The debug log reveals the source of the error to be
virNetDevBridgeCreate:474
I don't understand how that would work for you. My setup is completely vanilla, just a plain FreeBSD 14.1 install. The only thing that could possibly be making any difference is that the host's network interface is a wireless one.
Looks like you don't have the if_bridge kernel module loaded. If you run 'virt-host-validate', it should show if something's missing. Roman
On Thu, Jun 13, 2024 at 07:45:52PM GMT, Roman Bogorodskiy wrote:
My configuration is the same (obtained from copying the file shipped as /usr/local/share/examples/libvirt/networks/default.xml, which is identical to src/network/default.xml.in in the libvirt tree) and I get an error when I try to start the network:
# virsh net-start default error: Failed to start network default error: Unable to create bridge device: Invalid argument
The debug log reveals the source of the error to be
virNetDevBridgeCreate:474
I don't understand how that would work for you. My setup is completely vanilla, just a plain FreeBSD 14.1 install. The only thing that could possibly be making any difference is that the host's network interface is a wireless one.
Looks like you don't have the if_bridge kernel module loaded.
If you run 'virt-host-validate', it should show if something's missing.
Yeah, that was it. Probably worth adding a hint to the message displayed upon package installation? -- Andrea Bolognani / Red Hat / Virtualization
Andrea Bolognani wrote:
On Thu, Jun 13, 2024 at 07:45:52PM GMT, Roman Bogorodskiy wrote:
My configuration is the same (obtained from copying the file shipped as /usr/local/share/examples/libvirt/networks/default.xml, which is identical to src/network/default.xml.in in the libvirt tree) and I get an error when I try to start the network:
# virsh net-start default error: Failed to start network default error: Unable to create bridge device: Invalid argument
The debug log reveals the source of the error to be
virNetDevBridgeCreate:474
I don't understand how that would work for you. My setup is completely vanilla, just a plain FreeBSD 14.1 install. The only thing that could possibly be making any difference is that the host's network interface is a wireless one.
Looks like you don't have the if_bridge kernel module loaded.
If you run 'virt-host-validate', it should show if something's missing.
Yeah, that was it. Probably worth adding a hint to the message displayed upon package installation?
It's already referring to https://libvirt.org/drvbhyve.html which mentions that additional kernel modules should be loaded, so I think that should be good enough. Roman
On Fri, Jun 14, 2024 at 03:34:41PM GMT, Roman Bogorodskiy wrote:
Andrea Bolognani wrote:
On Thu, Jun 13, 2024 at 07:45:52PM GMT, Roman Bogorodskiy wrote:
Looks like you don't have the if_bridge kernel module loaded.
If you run 'virt-host-validate', it should show if something's missing.
Yeah, that was it. Probably worth adding a hint to the message displayed upon package installation?
It's already referring to https://libvirt.org/drvbhyve.html which mentions that additional kernel modules should be loaded, so I think that should be good enough.
I respectfully disagree, but then again I'm not going to write a patch myself either so you're absolutely entitled to disregard this suggestion :) -- Andrea Bolognani / Red Hat / Virtualization
Laine Stump wrote:
On 6/12/24 2:32 PM, Roman Bogorodskiy wrote:
Laine Stump wrote:
On 6/10/24 2:54 PM, Roman Bogorodskiy wrote:
Laine Stump wrote:
This patch series enables libvirt to use nftables rules rather than iptables *when setting up virtual networks* (it does *not* add nftables support to the nwfilter driver). It accomplishes this by abstracting several iptables functions (from viriptables.[ch] called by the virtual network driver into a rudimentary "virNetfilter API" (in virnetfilter.[ch], having the virtual network driver call the virNetFilter API rather than calling the existing iptables functions directly, and then finally adding an equivalent virNftables backend that can be used instead of iptables (selected manually via a network.conf setting, or automatically if iptables isn't found on the host).
Hi,
Apparently, I'm late to the discussion.
I noticed that now I cannot use the bridge driver on FreeBSD as it's failing to initialize both iptables and nftables backends (which is expect).
Yeah, previously we wouldn't check if iptables was available until someone tried to start a network that would need to use it, and would then log an error (and just fail starting that network, but the network driver would remain running). But now we figure out which firewall backend to use immediately when the driver is loaded, and if we fail to fin a workable backend we fail the entire driver init.r
How did you use the network driver before? With a <forward mode='open'/> network? Truthfully I hadn't ever considered the case of someone using it with only network types that didn't need firewall rules. I wonder if there are other platforms we support that have a usable network driver for <forward mode='open'/> (MacOS?)
I'm using it with the following network configuration:
virsh # net-dumpxml default <network> <name>default</name> <uuid>2a1415c9-325b-41e4-82c6-e805162d8934</uuid> <forward mode='nat'/> <bridge name='virbr0' stp='on' delay='0'/> <mac address='52:54:00:24:fa:43'/> <ip address='192.168.122.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.122.2' end='192.168.122.254'/> </dhcp> </ip> </network>
So basically all the mechanics like creating tap devices, bridges, serving dhcp, etc, all these work for me. On top of that I had a few iterations of manual firewall configurations (with both ipfw and pf) to implement NAT on guests.
Okay, so essentially it was effectively <forward mode='open'/> (since the only difference is the lack of libvirt-added firewall rules).
Yes, I've changed it to mode='open' and it still works for me, i.e. I have host<->guest connectivity. So it would make more sense to change the default to 'open', at least until nat is not supported. I think I'd better do that on the packaging level. Roman
Unfortunately, I don't have access to that setup anymore and I haven't re-created it yet. IIRC, it could probably show some warnings about missing iptables, but it didn't affect anything for me.
I'm surprised that there wasn't a fatal error while starting the network though.
I'm about to be offline for 3 weeks, but in the meantime if you'd like to try making a NULL backend that is only an option if it's listed in firewall_backend_priority (you'll need to remove the compile-time check that all possible backends are accounted for - I think that is the first of the two G_STATIC_ASSERTS at the top of virNetworkLoadDriverConfig()), always initializes successfully in bridge_driver_conf.c if it is listed in the options, and then in networkAddFirewallRules add a check to log an error and fail if backend == NULL (something about attempting to start a network type that would require firewall rules, but the system not having any of the supported types of firewallbackend or something - it's too late now and my brain is too fried and sleepy to think of good wording :-)). As long as it isn't a valid selection on Linux builds that are done with firewall_backend_priority=nftables,iptables, but *is* a valid selection if the setting is "firewall_backend_priority=null" that shouldn't be *too* controversial.
Ok, I think I can try making the NULL backend.
Yeah, Daniel's patch will actually disable the network driver completely, which will allow the rest of libvirt to still work, but sounds like it's not completely what you want. I think even with a NULL firewall backend it still should log an error and fail if someone tries to start a network that requires firewall rules though (i.e. your above config would still fail, unless it's changed to <forward mode='open'/>)
Later we can talk about pf and ipfw backends :-)
Yeah, that sounds good. My main problem with the choice is that ipfw is the most actively supported firewall, but it relies quite heavily on the rule numbering, which makes it a little hard to integrate with user-specific rules (i.e. defined outside of libvirt).
Well, there is a difficulty with any of the filtering systems wrt coordinating rules from multiple independent users, so nothing new there :-).
The "pf" seem to be better in this regard (at least to my taste), but it's not "native" FreeBSD firewall and is not as active (at least, to my impression).
I guess I should read up on both of those - I haven't done anything with packet filtering in a BSD since ipfilter back in the mid-90's :-)
On Thu, Jun 13, 2024 at 06:24:00PM GMT, Roman Bogorodskiy wrote:
Laine Stump wrote:
On 6/12/24 2:32 PM, Roman Bogorodskiy wrote:
So basically all the mechanics like creating tap devices, bridges, serving dhcp, etc, all these work for me. On top of that I had a few iterations of manual firewall configurations (with both ipfw and pf) to implement NAT on guests.
Okay, so essentially it was effectively <forward mode='open'/> (since the only difference is the lack of libvirt-added firewall rules).
Yes, I've changed it to mode='open' and it still works for me, i.e. I have host<->guest connectivity.
So it would make more sense to change the default to 'open', at least until nat is not supported. I think I'd better do that on the packaging level.
If you're touching the package, can you also look into the other question that I've raised about it? Namely that, since the QEMU driver is compiled out by default, the same should probably be the case for the network driver too. If someone goes and builds the port locally with QEMU support enabled, then and only then the network driver should be included. Honestly I'm not entirely sure it makes much sense to have the network driver and especially the default network if you need to bring your own firewall rules, but that can be a separate discussion. -- Andrea Bolognani / Red Hat / Virtualization
Andrea Bolognani wrote:
On Thu, Jun 13, 2024 at 06:24:00PM GMT, Roman Bogorodskiy wrote:
Laine Stump wrote:
On 6/12/24 2:32 PM, Roman Bogorodskiy wrote:
So basically all the mechanics like creating tap devices, bridges, serving dhcp, etc, all these work for me. On top of that I had a few iterations of manual firewall configurations (with both ipfw and pf) to implement NAT on guests.
Okay, so essentially it was effectively <forward mode='open'/> (since the only difference is the lack of libvirt-added firewall rules).
Yes, I've changed it to mode='open' and it still works for me, i.e. I have host<->guest connectivity.
So it would make more sense to change the default to 'open', at least until nat is not supported. I think I'd better do that on the packaging level.
If you're touching the package, can you also look into the other question that I've raised about it? Namely that, since the QEMU driver is compiled out by default, the same should probably be the case for the network driver too. If someone goes and builds the port locally with QEMU support enabled, then and only then the network driver should be included.
Honestly I'm not entirely sure it makes much sense to have the network driver and especially the default network if you need to bring your own firewall rules, but that can be a separate discussion.
Hm, I think the network driver is quite usable without QEMU, e.g. I use it with bhyve. I also find it quite useful even without firewall rules. Most of the time internal connectivity is enough for my guests. Configuring NAT on per-network basis is also fairly easy. For more advanced scenarios hooks could be used, though I haven't done that specifically. For now I'm inclined only to update 'nat' to 'open' as 'nat' is a misleading configuration and was working only by incidence. Roman
-- Andrea Bolognani / Red Hat / Virtualization
On Thu, Jun 13, 2024 at 08:00:32PM GMT, Roman Bogorodskiy wrote:
Andrea Bolognani wrote:
Honestly I'm not entirely sure it makes much sense to have the network driver and especially the default network if you need to bring your own firewall rules, but that can be a separate discussion.
Hm, I think the network driver is quite usable without QEMU, e.g. I use it with bhyve.
Okay, I didn't realize that was an option. Which leads me to open a different can of worms then: if libvirt networks can be used with drivers other than QEMU, wouldn't it make sense for their configuration to live in /etc/libvirt/network instead of /etc/libvirt/qemu/networks? How difficult would it be to adopt the new path without breaking existing setups?
I also find it quite useful even without firewall rules. Most of the time internal connectivity is enough for my guests. Configuring NAT on per-network basis is also fairly easy. For more advanced scenarios hooks could be used, though I haven't done that specifically.
VMs with no connectivity to the outside world are of very limited use IMO. At the very least, a warning about the fact that connectivity is limited could be displayed upon package installation. -- Andrea Bolognani / Red Hat / Virtualization
On Fri, Jun 14, 2024 at 04:41:25AM -0700, Andrea Bolognani wrote:
On Thu, Jun 13, 2024 at 08:00:32PM GMT, Roman Bogorodskiy wrote:
Andrea Bolognani wrote:
Honestly I'm not entirely sure it makes much sense to have the network driver and especially the default network if you need to bring your own firewall rules, but that can be a separate discussion.
Hm, I think the network driver is quite usable without QEMU, e.g. I use it with bhyve.
Okay, I didn't realize that was an option.
Which leads me to open a different can of worms then: if libvirt networks can be used with drivers other than QEMU, wouldn't it make sense for their configuration to live in /etc/libvirt/network instead of /etc/libvirt/qemu/networks? How difficult would it be to adopt the new path without breaking existing setups?
We can deal with the upgrade path easily enough. On startup, if the new location is empty, and the old location has files, then move the files to the new location. Downgrading libvirt will be broken, but so be it, we've never guaranteed that to work. I kinda wish we'd moved this a decade ago :-) The next best time is of course today. I think its especially beneficial now we have split modular daemons, as it would let us write SELinux policy for virtnetworkd which does not clash with virtqemud, or require privileges over the /etc/libvirt/qemu directory. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Wed, Jun 12, 2024 at 07:31:51PM -0400, Laine Stump wrote:
On 6/12/24 2:32 PM, Roman Bogorodskiy wrote:
Laine Stump wrote:
On 6/10/24 2:54 PM, Roman Bogorodskiy wrote:
Laine Stump wrote:
This patch series enables libvirt to use nftables rules rather than iptables *when setting up virtual networks* (it does *not* add nftables support to the nwfilter driver). It accomplishes this by abstracting several iptables functions (from viriptables.[ch] called by the virtual network driver into a rudimentary "virNetfilter API" (in virnetfilter.[ch], having the virtual network driver call the virNetFilter API rather than calling the existing iptables functions directly, and then finally adding an equivalent virNftables backend that can be used instead of iptables (selected manually via a network.conf setting, or automatically if iptables isn't found on the host).
Hi,
Apparently, I'm late to the discussion.
I noticed that now I cannot use the bridge driver on FreeBSD as it's failing to initialize both iptables and nftables backends (which is expect).
Yeah, previously we wouldn't check if iptables was available until someone tried to start a network that would need to use it, and would then log an error (and just fail starting that network, but the network driver would remain running). But now we figure out which firewall backend to use immediately when the driver is loaded, and if we fail to fin a workable backend we fail the entire driver init.r
How did you use the network driver before? With a <forward mode='open'/> network? Truthfully I hadn't ever considered the case of someone using it with only network types that didn't need firewall rules. I wonder if there are other platforms we support that have a usable network driver for <forward mode='open'/> (MacOS?)
I'm using it with the following network configuration:
virsh # net-dumpxml default <network> <name>default</name> <uuid>2a1415c9-325b-41e4-82c6-e805162d8934</uuid> <forward mode='nat'/> <bridge name='virbr0' stp='on' delay='0'/> <mac address='52:54:00:24:fa:43'/> <ip address='192.168.122.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.122.2' end='192.168.122.254'/> </dhcp> </ip> </network>
So basically all the mechanics like creating tap devices, bridges, serving dhcp, etc, all these work for me. On top of that I had a few iterations of manual firewall configurations (with both ipfw and pf) to implement NAT on guests.
Okay, so essentially it was effectively <forward mode='open'/> (since the only difference is the lack of libvirt-added firewall rules).
Unfortunately, I don't have access to that setup anymore and I haven't re-created it yet. IIRC, it could probably show some warnings about missing iptables, but it didn't affect anything for me.
I'm surprised that there wasn't a fatal error while starting the network though.
Having actually looked at the code again, the answer is amuzingly/depressingly obvious.... we already implemented a NULL firewall driver, which is used on non-Linux, many years ago: See bridge_driver_platform.c, which does: #if defined(__linux__) # include "bridge_driver_linux.c" #else # include "bridge_driver_nop.c" #endif The bridge_driver_nop.c simply does nothing and returns success for everything. So all the different virtual network modes "work" in so much as libvirt can start them, but they don't work in the sense that we're never creating the firewall rules to implement NAT, etc. So the root cause here is that our "firewall_backend" config logic completely forgot that the 'nop' driver already existed, and offers no way to configure it. This is quite simple to address. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Thu, Jun 13, 2024 at 05:33:43PM +0100, Daniel P. Berrangé wrote:
On Wed, Jun 12, 2024 at 07:31:51PM -0400, Laine Stump wrote:
On 6/12/24 2:32 PM, Roman Bogorodskiy wrote:
Unfortunately, I don't have access to that setup anymore and I haven't re-created it yet. IIRC, it could probably show some warnings about missing iptables, but it didn't affect anything for me.
I'm surprised that there wasn't a fatal error while starting the network though.
Having actually looked at the code again, the answer is amuzingly/depressingly obvious.... we already implemented a NULL firewall driver, which is used on non-Linux, many years ago:
See bridge_driver_platform.c, which does:
#if defined(__linux__) # include "bridge_driver_linux.c" #else # include "bridge_driver_nop.c" #endif
The bridge_driver_nop.c simply does nothing and returns success for everything. So all the different virtual network modes "work" in so much as libvirt can start them, but they don't work in the sense that we're never creating the firewall rules to implement NAT, etc.
So the root cause here is that our "firewall_backend" config logic completely forgot that the 'nop' driver already existed, and offers no way to configure it. This is quite simple to address.
I've sent a patch that ought to fix this problem, though I admit I have not actually tried it on FreeBSD. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
participants (4)
-
Andrea Bolognani -
Daniel P. Berrangé -
Laine Stump -
Roman Bogorodskiy