[PATCH v2] network: introduce a "none" firewall backend type
There are two scenarios identified after the recent firewall backend selection was introduced, which result in libvirtd failing to startup due to an inability to find either iptables/nftables - On Linux if running unprivileged with $PATH lacking the dir containing iptables/nftables - On non-Linux where iptables/nftables never existed In the former case, it is preferrable to restore the behaviour whereby the driver starts successfully. Users will get an error reported when attempting to start any virtual network, due to the lack of permissions needed to create bridge devices. This makes the missing firewall backend irrelevant. In the latter case, the network driver calls the 'nop' platform implementation which does not attempt to implement any firewall logic, just allowing the network to start without firewall rules. To solve this are number of changes are required * Introduce VIR_FIREWALL_BACKEND_NONE, which does nothing except report a fatal error from virFirewallApply(). This code path is unreachable, since we'll never create a virFirewall object with with VIR_FIREWALL_BACKEND_NONE, so the error reporting is just a sanity check. * Ignore the compile time backend defaults and assume use of the 'none' backend if running unprivileged. This fixes the first regression, avoiding the failure to start libvirtd on Linux in unprivileged context, instead allowing use of the driver and expecting a permission denied when creating a bridge. * Reject the use of compile time backend defaults no non-Linux and hardcode the 'none' backend. The non-Linux platforms have no firewall implementation at all currently, so there's no reason to permit the use of 'firewall_backend_priority' meson option. This fixes the second regression, avoiding the failure to start libvirtd on non-Linux hosts due to non-existant Linux binaries. * Change the Linux platform backend to raise an error if the firewall backend is 'none'. Again this code path is unreachable by default since we'll fail to create the bridge before getting here, but if someone modified network.conf to request the 'none' backend, this will stop further progress. * Change the nop platform backend to raise an error if the firewall backend is 'iptables' or 'nftables'. Again this code path is unreachable, since we should already have failed to find the iptables/nftables binaries on non-Linux hosts, so this is just a sanity check. * 'none' is not permited as a value in 'firewall_backend_priority' meson option, since it is conceptually meaningless to ask for that on Linux. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- Changed in v2: - Fix build problems on FreeBSD (changes proposed by Roman) meson.build | 26 +++++++++++++++++++------- meson_options.txt | 2 +- src/network/bridge_driver_conf.c | 19 ++++++++++++++----- src/network/bridge_driver_linux.c | 10 ++++++++++ src/network/bridge_driver_nop.c | 15 ++++++++++++++- src/util/virfirewall.c | 6 ++++++ src/util/virfirewall.h | 1 + 7 files changed, 65 insertions(+), 14 deletions(-) diff --git a/meson.build b/meson.build index 5c7cd7ec2e..2e8b87280d 100644 --- a/meson.build +++ b/meson.build @@ -1647,15 +1647,27 @@ if not get_option('driver_network').disabled() and conf.has('WITH_LIBVIRTD') conf.set('WITH_NETWORK', 1) firewall_backend_priority = get_option('firewall_backend_priority') - if (not firewall_backend_priority.contains('nftables') or - not firewall_backend_priority.contains('iptables') or - firewall_backend_priority.length() != 2) - error('invalid value for firewall_backend_priority option') + if firewall_backend_priority.length() == 0 + if host_machine.system() == 'linux' + firewall_backend_priority = ['nftables', 'iptables'] + else + # No firewall impl on non-Linux so far, so force 'none' + # as placeholder + firewall_backend_priority = ['none'] + endif + else + if host_machine.system() != 'linux' + error('firewall backend priority only supported on linux hosts') + endif endif - conf.set('FIREWALL_BACKEND_PRIORITY_0', 'VIR_FIREWALL_BACKEND_' + firewall_backend_priority[0].to_upper()) - conf.set('FIREWALL_BACKEND_PRIORITY_1', 'VIR_FIREWALL_BACKEND_' + firewall_backend_priority[1].to_upper()) - conf.set('FIREWALL_BACKEND_PRIORITY_NUM', firewall_backend_priority.length()) + backends = [] + foreach backend: firewall_backend_priority + backend = 'VIR_FIREWALL_BACKEND_' + backend.to_upper() + backends += backend + endforeach + + conf.set('FIREWALL_BACKENDS', ', '.join(backends)) elif get_option('driver_network').enabled() error('libvirtd must be enabled to build the network driver') endif diff --git a/meson_options.txt b/meson_options.txt index 50d71427cb..2d440c63d8 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -117,7 +117,7 @@ option('dtrace', type: 'feature', value: 'auto', description: 'use dtrace for st option('firewalld', type: 'feature', value: 'auto', description: 'firewalld support') # dep:firewalld option('firewalld_zone', type: 'feature', value: 'auto', description: 'whether to install firewalld libvirt zone') -option('firewall_backend_priority', type: 'array', choices: ['nftables', 'iptables'], description: 'order in which to try firewall backends') +option('firewall_backend_priority', type: 'array', choices: ['nftables', 'iptables'], value: [], description: 'order in which to try firewall backends') option('host_validate', type: 'feature', value: 'auto', description: 'build virt-host-validate') option('init_script', type: 'combo', choices: ['systemd', 'openrc', 'check', 'none'], value: 'check', description: 'Style of init script to install') option('loader_nvram', type: 'string', value: '', description: 'Pass list of pairs of <loader>:<nvram> paths. Both pairs and list items are separated by a colon.') diff --git a/src/network/bridge_driver_conf.c b/src/network/bridge_driver_conf.c index e2f3613a41..9da5e790b7 100644 --- a/src/network/bridge_driver_conf.c +++ b/src/network/bridge_driver_conf.c @@ -61,6 +61,7 @@ networkGetDnsmasqCaps(virNetworkDriverState *driver) static int virNetworkLoadDriverConfig(virNetworkDriverConfig *cfg G_GNUC_UNUSED, + bool privileged, const char *filename) { g_autoptr(virConf) conf = NULL; @@ -68,13 +69,17 @@ virNetworkLoadDriverConfig(virNetworkDriverConfig *cfg G_GNUC_UNUSED, bool fwBackendSelected = false; size_t i; int fwBackends[] = { - FIREWALL_BACKEND_PRIORITY_0, - FIREWALL_BACKEND_PRIORITY_1, + FIREWALL_BACKENDS }; - G_STATIC_ASSERT(G_N_ELEMENTS(fwBackends) == VIR_FIREWALL_BACKEND_LAST); - G_STATIC_ASSERT(G_N_ELEMENTS(fwBackends) == FIREWALL_BACKEND_PRIORITY_NUM); + G_STATIC_ASSERT(G_N_ELEMENTS(fwBackends) > 0 && + G_N_ELEMENTS(fwBackends) <= VIR_FIREWALL_BACKEND_LAST); int nFwBackends = G_N_ELEMENTS(fwBackends); + if (!privileged) { + fwBackends[0] = VIR_FIREWALL_BACKEND_NONE; + nFwBackends = 1; + } + if (access(filename, R_OK) == 0) { conf = virConfReadFile(filename, 0); @@ -104,6 +109,10 @@ virNetworkLoadDriverConfig(virNetworkDriverConfig *cfg G_GNUC_UNUSED, for (i = 0; i < nFwBackends && !fwBackendSelected; i++) { switch ((virFirewallBackend)fwBackends[i]) { + case VIR_FIREWALL_BACKEND_NONE: + fwBackendSelected = true; + break; + case VIR_FIREWALL_BACKEND_IPTABLES: { g_autofree char *iptablesInPath = virFindFileInPath(IPTABLES); @@ -187,7 +196,7 @@ virNetworkDriverConfigNew(bool privileged) configfile = g_strconcat(configdir, "/network.conf", NULL); - if (virNetworkLoadDriverConfig(cfg, configfile) < 0) + if (virNetworkLoadDriverConfig(cfg, privileged, configfile) < 0) return NULL; if (g_mkdir_with_parents(cfg->stateDir, 0777) < 0) { diff --git a/src/network/bridge_driver_linux.c b/src/network/bridge_driver_linux.c index 35e6bd1154..fe7c6e193c 100644 --- a/src/network/bridge_driver_linux.c +++ b/src/network/bridge_driver_linux.c @@ -47,6 +47,11 @@ networkFirewallSetupPrivateChains(virFirewallBackend backend, virFirewallLayer layer) { switch (backend) { + case VIR_FIREWALL_BACKEND_NONE: + virReportError(VIR_ERR_NO_SUPPORT, "%s", + _("No firewall backend is available")); + return -1; + case VIR_FIREWALL_BACKEND_IPTABLES: return iptablesSetupPrivateChains(layer); @@ -417,6 +422,11 @@ networkAddFirewallRules(virNetworkDef *def, } switch (firewallBackend) { + case VIR_FIREWALL_BACKEND_NONE: + virReportError(VIR_ERR_NO_SUPPORT, "%s", + _("No firewall backend is available")); + return -1; + case VIR_FIREWALL_BACKEND_IPTABLES: return iptablesAddFirewallRules(def, fwRemoval); diff --git a/src/network/bridge_driver_nop.c b/src/network/bridge_driver_nop.c index 537b9234f8..2114d521d1 100644 --- a/src/network/bridge_driver_nop.c +++ b/src/network/bridge_driver_nop.c @@ -19,6 +19,8 @@ #include <config.h> +#define VIR_FROM_THIS VIR_FROM_NETWORK + void networkPreReloadFirewallRules(virNetworkDriverState *driver G_GNUC_UNUSED, bool startup G_GNUC_UNUSED, bool force G_GNUC_UNUSED) @@ -37,9 +39,20 @@ int networkCheckRouteCollision(virNetworkDef *def G_GNUC_UNUSED) } int networkAddFirewallRules(virNetworkDef *def G_GNUC_UNUSED, - virFirewallBackend firewallBackend G_GNUC_UNUSED, + virFirewallBackend firewallBackend, virFirewall **fwRemoval G_GNUC_UNUSED) { + /* + * Shouldn't be possible, since virNetworkLoadDriverConfig + * ought to fail to find the required binaries when loading, + * so this is just a sanity check + */ + if (firewallBackend != VIR_FIREWALL_BACKEND_NONE) { + virReportError(VIR_ERR_NO_SUPPORT, + _("Firewall backend '%s' not available on this platform"), + virFirewallBackendTypeToString(firewallBackend)); + return -1; + } return 0; } diff --git a/src/util/virfirewall.c b/src/util/virfirewall.c index 2219506b18..d374f54b64 100644 --- a/src/util/virfirewall.c +++ b/src/util/virfirewall.c @@ -37,6 +37,7 @@ VIR_LOG_INIT("util.firewall"); VIR_ENUM_IMPL(virFirewallBackend, VIR_FIREWALL_BACKEND_LAST, + "none", "iptables", "nftables"); @@ -815,6 +816,11 @@ virFirewallApplyCmd(virFirewall *firewall, } switch (virFirewallGetBackend(firewall)) { + case VIR_FIREWALL_BACKEND_NONE: + virReportError(VIR_ERR_NO_SUPPORT, + _("Firewall backend is not implemented")); + return -1; + case VIR_FIREWALL_BACKEND_IPTABLES: if (virFirewallCmdIptablesApply(firewall, fwCmd, &output) < 0) return -1; diff --git a/src/util/virfirewall.h b/src/util/virfirewall.h index 302a6a4e5b..bce51259d2 100644 --- a/src/util/virfirewall.h +++ b/src/util/virfirewall.h @@ -44,6 +44,7 @@ typedef enum { } virFirewallLayer; typedef enum { + VIR_FIREWALL_BACKEND_NONE, /* Always fails */ VIR_FIREWALL_BACKEND_IPTABLES, VIR_FIREWALL_BACKEND_NFTABLES, -- 2.45.1
On 6/14/24 16:43, Daniel P. Berrangé wrote:
There are two scenarios identified after the recent firewall backend selection was introduced, which result in libvirtd failing to startup due to an inability to find either iptables/nftables
- On Linux if running unprivileged with $PATH lacking the dir containing iptables/nftables - On non-Linux where iptables/nftables never existed
In the former case, it is preferrable to restore the behaviour whereby the driver starts successfully. Users will get an error reported when attempting to start any virtual network, due to the lack of permissions needed to create bridge devices. This makes the missing firewall backend irrelevant.
In the latter case, the network driver calls the 'nop' platform implementation which does not attempt to implement any firewall logic, just allowing the network to start without firewall rules.
To solve this are number of changes are required
* Introduce VIR_FIREWALL_BACKEND_NONE, which does nothing except report a fatal error from virFirewallApply(). This code path is unreachable, since we'll never create a virFirewall object with with VIR_FIREWALL_BACKEND_NONE, so the error reporting is just a sanity check.
* Ignore the compile time backend defaults and assume use of the 'none' backend if running unprivileged.
This fixes the first regression, avoiding the failure to start libvirtd on Linux in unprivileged context, instead allowing use of the driver and expecting a permission denied when creating a bridge.
* Reject the use of compile time backend defaults no non-Linux and hardcode the 'none' backend. The non-Linux platforms have no firewall implementation at all currently, so there's no reason to permit the use of 'firewall_backend_priority' meson option.
This fixes the second regression, avoiding the failure to start libvirtd on non-Linux hosts due to non-existant Linux binaries.
* Change the Linux platform backend to raise an error if the firewall backend is 'none'. Again this code path is unreachable by default since we'll fail to create the bridge before getting here, but if someone modified network.conf to request the 'none' backend, this will stop further progress.
* Change the nop platform backend to raise an error if the firewall backend is 'iptables' or 'nftables'. Again this code path is unreachable, since we should already have failed to find the iptables/nftables binaries on non-Linux hosts, so this is just a sanity check.
* 'none' is not permited as a value in 'firewall_backend_priority' meson option, since it is conceptually meaningless to ask for that on Linux.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> ---
Changed in v2:
- Fix build problems on FreeBSD (changes proposed by Roman)
meson.build | 26 +++++++++++++++++++------- meson_options.txt | 2 +- src/network/bridge_driver_conf.c | 19 ++++++++++++++----- src/network/bridge_driver_linux.c | 10 ++++++++++ src/network/bridge_driver_nop.c | 15 ++++++++++++++- src/util/virfirewall.c | 6 ++++++ src/util/virfirewall.h | 1 + 7 files changed, 65 insertions(+), 14 deletions(-)
Reviewed-by: Michal Privoznik <mprivozn@redhat.com> Michal
On Fri, Jun 14, 2024 at 03:43:53PM GMT, Daniel P. Berrangé wrote:
meson.build | 26 +++++++++++++++++++------- meson_options.txt | 2 +- src/network/bridge_driver_conf.c | 19 ++++++++++++++----- src/network/bridge_driver_linux.c | 10 ++++++++++ src/network/bridge_driver_nop.c | 15 ++++++++++++++- src/util/virfirewall.c | 6 ++++++ src/util/virfirewall.h | 1 + 7 files changed, 65 insertions(+), 14 deletions(-)
The test suite no longer passes after applying this. At the very least, you need to squash in the diff at the bottom of this message.
firewall_backend_priority = get_option('firewall_backend_priority') - if (not firewall_backend_priority.contains('nftables') or - not firewall_backend_priority.contains('iptables') or - firewall_backend_priority.length() != 2) - error('invalid value for firewall_backend_priority option') + if firewall_backend_priority.length() == 0 + if host_machine.system() == 'linux' + firewall_backend_priority = ['nftables', 'iptables'] + else + # No firewall impl on non-Linux so far, so force 'none' + # as placeholder + firewall_backend_priority = ['none'] + endif + else + if host_machine.system() != 'linux' + error('firewall backend priority only supported on linux hosts') + endif endif
This implementation allows things such as -Dfirewall_backend_priority=nftables and -Dfirewall_backend_priority=iptables,iptables At least -Dfirewall_backend_priority=iptables,nftables,iptables will be blocked, but only because it results in a compilation error: meson will happily accept it. Are we okay with that? It's IMO inferior to the much stricter checking that's performed today.
@@ -815,6 +816,11 @@ virFirewallApplyCmd(virFirewall *firewall, switch (virFirewallGetBackend(firewall)) { + case VIR_FIREWALL_BACKEND_NONE: + virReportError(VIR_ERR_NO_SUPPORT, + _("Firewall backend is not implemented")); + return -1;
This check (and the other ones you've introduced) are running too late, which leads to this behavior: $ cat default-session.xml <network> <name>default-session</name> <forward mode='nat'/> <bridge name='virbr0' stp='on' delay='0'/> <ip address='192.168.123.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.123.2' end='192.168.123.254'/> </dhcp> </ip> </network> $ virsh -c qemu:///session net-define default-session.xml Network default-session defined from default-session.xml $ virsh -c qemu:///session net-start default-session error: Failed to start network default-session error: error creating bridge interface virbr0: Operation not permitted Since <forward mode='nat'/> requires firewall rules, we shouldn't allow a network that uses it to be defined in the first place. diff --git a/po/POTFILES b/po/POTFILES index 4bfbb91164..1ed4086d2c 100644 --- a/po/POTFILES +++ b/po/POTFILES @@ -143,6 +143,7 @@ src/lxc/lxc_process.c src/network/bridge_driver.c src/network/bridge_driver_conf.c src/network/bridge_driver_linux.c +src/network/bridge_driver_nop.c src/network/leaseshelper.c src/network/network_iptables.c src/network/network_nftables.c diff --git a/src/network/bridge_driver_nop.c b/src/network/bridge_driver_nop.c index 2114d521d1..323990cf17 100644 --- a/src/network/bridge_driver_nop.c +++ b/src/network/bridge_driver_nop.c @@ -49,8 +49,8 @@ int networkAddFirewallRules(virNetworkDef *def G_GNUC_UNUSED, */ if (firewallBackend != VIR_FIREWALL_BACKEND_NONE) { virReportError(VIR_ERR_NO_SUPPORT, - _("Firewall backend '%s' not available on this platform"), - virFirewallBackendTypeToString(firewallBackend)); + _("Firewall backend '%1$s' not available on this platform"), + virFirewallBackendTypeToSTring(firewallBackend)); return -1; } return 0; diff --git a/src/util/virfirewall.c b/src/util/virfirewall.c index d374f54b64..090dbcdbed 100644 --- a/src/util/virfirewall.c +++ b/src/util/virfirewall.c @@ -817,7 +817,7 @@ virFirewallApplyCmd(virFirewall *firewall, switch (virFirewallGetBackend(firewall)) { case VIR_FIREWALL_BACKEND_NONE: - virReportError(VIR_ERR_NO_SUPPORT, + virReportError(VIR_ERR_NO_SUPPORT, "%s", _("Firewall backend is not implemented")); return -1; -- Andrea Bolognani / Red Hat / Virtualization
On Fri, Jun 14, 2024 at 12:22:50PM -0400, Andrea Bolognani wrote:
On Fri, Jun 14, 2024 at 03:43:53PM GMT, Daniel P. Berrangé wrote:
meson.build | 26 +++++++++++++++++++------- meson_options.txt | 2 +- src/network/bridge_driver_conf.c | 19 ++++++++++++++----- src/network/bridge_driver_linux.c | 10 ++++++++++ src/network/bridge_driver_nop.c | 15 ++++++++++++++- src/util/virfirewall.c | 6 ++++++ src/util/virfirewall.h | 1 + 7 files changed, 65 insertions(+), 14 deletions(-)
The test suite no longer passes after applying this. At the very least, you need to squash in the diff at the bottom of this message.
Well that's very wierd as I ran a CI pipeline with this commit which worked ! https://gitlab.com/berrange/libvirt/-/pipelines/1332837446 but you are correct that it fails tests, so huh ?
firewall_backend_priority = get_option('firewall_backend_priority') - if (not firewall_backend_priority.contains('nftables') or - not firewall_backend_priority.contains('iptables') or - firewall_backend_priority.length() != 2) - error('invalid value for firewall_backend_priority option') + if firewall_backend_priority.length() == 0 + if host_machine.system() == 'linux' + firewall_backend_priority = ['nftables', 'iptables'] + else + # No firewall impl on non-Linux so far, so force 'none' + # as placeholder + firewall_backend_priority = ['none'] + endif + else + if host_machine.system() != 'linux' + error('firewall backend priority only supported on linux hosts') + endif endif
This implementation allows things such as
-Dfirewall_backend_priority=nftables
That's good, it allows preventing any fallback to iptables by default.
and
-Dfirewall_backend_priority=iptables,iptables
At least
-Dfirewall_backend_priority=iptables,nftables,iptables
will be blocked, but only because it results in a compilation error: meson will happily accept it.
Are we okay with that? It's IMO inferior to the much stricter checking that's performed today.
I don't think it really matters - if someone wants to pass a wierd config, so be it.
@@ -815,6 +816,11 @@ virFirewallApplyCmd(virFirewall *firewall, switch (virFirewallGetBackend(firewall)) { + case VIR_FIREWALL_BACKEND_NONE: + virReportError(VIR_ERR_NO_SUPPORT, + _("Firewall backend is not implemented")); + return -1;
This check (and the other ones you've introduced) are running too late, which leads to this behavior:
$ cat default-session.xml <network> <name>default-session</name> <forward mode='nat'/> <bridge name='virbr0' stp='on' delay='0'/> <ip address='192.168.123.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.123.2' end='192.168.123.254'/> </dhcp> </ip> </network>
$ virsh -c qemu:///session net-define default-session.xml Network default-session defined from default-session.xml
$ virsh -c qemu:///session net-start default-session error: Failed to start network default-session error: error creating bridge interface virbr0: Operation not permitted
Since <forward mode='nat'/> requires firewall rules, we shouldn't allow a network that uses it to be defined in the first place.
This is the behaviour we already had before the nftables backend was created, and its not been a problem. There's no functional reason why we should refuse to allow it to be defined, if the binaries aren't needed until startup.
diff --git a/po/POTFILES b/po/POTFILES index 4bfbb91164..1ed4086d2c 100644 --- a/po/POTFILES +++ b/po/POTFILES @@ -143,6 +143,7 @@ src/lxc/lxc_process.c src/network/bridge_driver.c src/network/bridge_driver_conf.c src/network/bridge_driver_linux.c +src/network/bridge_driver_nop.c src/network/leaseshelper.c src/network/network_iptables.c src/network/network_nftables.c diff --git a/src/network/bridge_driver_nop.c b/src/network/bridge_driver_nop.c index 2114d521d1..323990cf17 100644 --- a/src/network/bridge_driver_nop.c +++ b/src/network/bridge_driver_nop.c @@ -49,8 +49,8 @@ int networkAddFirewallRules(virNetworkDef *def G_GNUC_UNUSED, */ if (firewallBackend != VIR_FIREWALL_BACKEND_NONE) { virReportError(VIR_ERR_NO_SUPPORT, - _("Firewall backend '%s' not available on this platform"), - virFirewallBackendTypeToString(firewallBackend)); + _("Firewall backend '%1$s' not available on this platform"), + virFirewallBackendTypeToSTring(firewallBackend)); return -1; } return 0; diff --git a/src/util/virfirewall.c b/src/util/virfirewall.c index d374f54b64..090dbcdbed 100644 --- a/src/util/virfirewall.c +++ b/src/util/virfirewall.c @@ -817,7 +817,7 @@ virFirewallApplyCmd(virFirewall *firewall,
switch (virFirewallGetBackend(firewall)) { case VIR_FIREWALL_BACKEND_NONE: - virReportError(VIR_ERR_NO_SUPPORT, + virReportError(VIR_ERR_NO_SUPPORT, "%s", _("Firewall backend is not implemented")); return -1;
-- Andrea Bolognani / Red Hat / Virtualization
With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Fri, Jun 14, 2024 at 07:52:55PM GMT, Daniel P. Berrangé wrote:
On Fri, Jun 14, 2024 at 12:22:50PM -0400, Andrea Bolognani wrote:
On Fri, Jun 14, 2024 at 03:43:53PM GMT, Daniel P. Berrangé wrote:
@@ -815,6 +816,11 @@ virFirewallApplyCmd(virFirewall *firewall, switch (virFirewallGetBackend(firewall)) { + case VIR_FIREWALL_BACKEND_NONE: + virReportError(VIR_ERR_NO_SUPPORT, + _("Firewall backend is not implemented")); + return -1;
This check (and the other ones you've introduced) are running too late, which leads to this behavior:
$ cat default-session.xml <network> <name>default-session</name> <forward mode='nat'/> <bridge name='virbr0' stp='on' delay='0'/> <ip address='192.168.123.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.123.2' end='192.168.123.254'/> </dhcp> </ip> </network>
$ virsh -c qemu:///session net-define default-session.xml Network default-session defined from default-session.xml
$ virsh -c qemu:///session net-start default-session error: Failed to start network default-session error: error creating bridge interface virbr0: Operation not permitted
Since <forward mode='nat'/> requires firewall rules, we shouldn't allow a network that uses it to be defined in the first place.
This is the behaviour we already had before the nftables backend was created, and its not been a problem. There's no functional reason why we should refuse to allow it to be defined, if the binaries aren't needed until startup.
But in the case of qemu:///session, or when we're not on Linux, we already know that there is no way for the network that's being defined to ever work. To me that's the same as allowing a guest to be defined, when the corresponding QEMU binary doesn't support some of the devices. Or when a firmware binary satisfying the constraints isn't available. We reject such configurations for guests, and it would just be consistent to do the same for networks. Besides, even if we decide that we want to allow such networks to be defined, we should report a more meaningful error for the failed startup, one which points out that mode=nat will never work. It shouldn't be the same error message that I was getting on FreeBSD because of a kernel driver that wasn't loaded, as the user's ability to make the error go away is completely different in the two scenarios. -- Andrea Bolognani / Red Hat / Virtualization
On Mon, Jun 17, 2024 at 04:44:01AM -0400, Andrea Bolognani wrote:
On Fri, Jun 14, 2024 at 07:52:55PM GMT, Daniel P. Berrangé wrote:
On Fri, Jun 14, 2024 at 12:22:50PM -0400, Andrea Bolognani wrote:
On Fri, Jun 14, 2024 at 03:43:53PM GMT, Daniel P. Berrangé wrote:
@@ -815,6 +816,11 @@ virFirewallApplyCmd(virFirewall *firewall, switch (virFirewallGetBackend(firewall)) { + case VIR_FIREWALL_BACKEND_NONE: + virReportError(VIR_ERR_NO_SUPPORT, + _("Firewall backend is not implemented")); + return -1;
This check (and the other ones you've introduced) are running too late, which leads to this behavior:
$ cat default-session.xml <network> <name>default-session</name> <forward mode='nat'/> <bridge name='virbr0' stp='on' delay='0'/> <ip address='192.168.123.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.123.2' end='192.168.123.254'/> </dhcp> </ip> </network>
$ virsh -c qemu:///session net-define default-session.xml Network default-session defined from default-session.xml
$ virsh -c qemu:///session net-start default-session error: Failed to start network default-session error: error creating bridge interface virbr0: Operation not permitted
Since <forward mode='nat'/> requires firewall rules, we shouldn't allow a network that uses it to be defined in the first place.
This is the behaviour we already had before the nftables backend was created, and its not been a problem. There's no functional reason why we should refuse to allow it to be defined, if the binaries aren't needed until startup.
But in the case of qemu:///session, or when we're not on Linux, we already know that there is no way for the network that's being defined to ever work.
To me that's the same as allowing a guest to be defined, when the corresponding QEMU binary doesn't support some of the devices. Or when a firmware binary satisfying the constraints isn't available. We reject such configurations for guests, and it would just be consistent to do the same for networks.
FWIW, I find the QEMU behaviour really unpleasant / hostile, when I'm working with guests that reference a self-built QEMU, as the binary often "doesn't exist", if I've cleaned by build temporarily. We're forced into this practice in QEMU, because we need to expand the XML with defaults to fix the guest ABI. I don't like the idea of propogating this practice elsewhere though. Ultimately I think we should not provide the network driver at all when running as non-root, but my attempt todo that exposed wierd edgecases, as we've assumed the secondary drivers are always present and cleaning that up properly is quite a bit of work
Besides, even if we decide that we want to allow such networks to be defined, we should report a more meaningful error for the failed startup, one which points out that mode=nat will never work. It shouldn't be the same error message that I was getting on FreeBSD because of a kernel driver that wasn't loaded, as the user's ability to make the error go away is completely different in the two scenarios.
Overall, there's lots that could be improved in the network driver, but for this, I'd really just like to focus on fixing the regression, suc that we return to the behaviour we had historically. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Mon, Jun 17, 2024 at 01:40:16PM GMT, Daniel P. Berrangé wrote:
On Mon, Jun 17, 2024 at 04:44:01AM -0400, Andrea Bolognani wrote:
On Fri, Jun 14, 2024 at 07:52:55PM GMT, Daniel P. Berrangé wrote:
This is the behaviour we already had before the nftables backend was created, and its not been a problem. There's no functional reason why we should refuse to allow it to be defined, if the binaries aren't needed until startup.
But in the case of qemu:///session, or when we're not on Linux, we already know that there is no way for the network that's being defined to ever work.
To me that's the same as allowing a guest to be defined, when the corresponding QEMU binary doesn't support some of the devices. Or when a firmware binary satisfying the constraints isn't available. We reject such configurations for guests, and it would just be consistent to do the same for networks.
FWIW, I find the QEMU behaviour really unpleasant / hostile, when I'm working with guests that reference a self-built QEMU, as the binary often "doesn't exist", if I've cleaned by build temporarily.
That makes things slightly annoying for developers, sure, but it's much more friendly towards users IMO. Getting an error sooner (define) rather than later (start) for situations where we know in advance that things can't possibly ever work is a lot better, which is also why libvirt erroring out is more user friendly than running QEMU and let if fail instead.
Overall, there's lots that could be improved in the network driver, but for this, I'd really just like to focus on fixing the regression, suc that we return to the behaviour we had historically.
Clearly I'm not entirely happy with the behavior you're implementing with this, but I agree that it's still much better than leaving several common configurations completely broken. Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
On 6/14/24 12:22 PM, Andrea Bolognani wrote:
On Fri, Jun 14, 2024 at 03:43:53PM GMT, Daniel P. Berrangé wrote:
meson.build | 26 +++++++++++++++++++------- meson_options.txt | 2 +- src/network/bridge_driver_conf.c | 19 ++++++++++++++----- src/network/bridge_driver_linux.c | 10 ++++++++++ src/network/bridge_driver_nop.c | 15 ++++++++++++++- src/util/virfirewall.c | 6 ++++++ src/util/virfirewall.h | 1 + 7 files changed, 65 insertions(+), 14 deletions(-)
The test suite no longer passes after applying this. At the very least, you need to squash in the diff at the bottom of this message.
firewall_backend_priority = get_option('firewall_backend_priority') - if (not firewall_backend_priority.contains('nftables') or - not firewall_backend_priority.contains('iptables') or - firewall_backend_priority.length() != 2) - error('invalid value for firewall_backend_priority option') + if firewall_backend_priority.length() == 0 + if host_machine.system() == 'linux' + firewall_backend_priority = ['nftables', 'iptables'] + else + # No firewall impl on non-Linux so far, so force 'none' + # as placeholder + firewall_backend_priority = ['none'] + endif + else + if host_machine.system() != 'linux' + error('firewall backend priority only supported on linux hosts') + endif endif
I suppose this is true for now, but will hopefully some day become false :-)
This implementation allows things such as
-Dfirewall_backend_priority=nftables
and
-Dfirewall_backend_priority=iptables,iptables
At least
-Dfirewall_backend_priority=iptables,nftables,iptables
will be blocked, but only because it results in a compilation error: meson will happily accept it.
Are we okay with that? It's IMO inferior to the much stricter checking that's performed today.
I haven't had the time to test this, or even look at it in detail, since I'm leaving for a multi-week vacation on Sunday, but I thought I should at least give my opinion on how I think it *should* work - maybe it already *does* work as I describe) What I would most prefer would be if you could list 1 or more backends (i.e. not all backends need to be listed, and if a particular backend isn't listed then it isn't checked for automatically (maybe isn't even allowed to be set explicitly), and ideally wouldn't even be compiled in). It would be nice if it gave an error if the same backend was given more than once in the list, but since you can probably count the number of people who will ever mess with this setting with the fingers on your two hands (and maybe the addition of your toes), and they're hopefully mentally aware enough to realize that naming the same backend multiple time would be pointless, I don't think a check to prevent duplicates is absolutely necessary (if fixing it would mean that you can't list fewer than the total available backends - someone on Linux might want to only enable nftables). To say it by example, I think on a Linux system it should work to define firewall_backend_prioty as: nftables,iptables iptables,nftables nftables iptables null Currently on FreeBSD the only acceptable setting would be "null", but in the future it could be "ipfw,pf", or the opposite order, or just one or the other.
@@ -815,6 +816,11 @@ virFirewallApplyCmd(virFirewall *firewall, switch (virFirewallGetBackend(firewall)) { + case VIR_FIREWALL_BACKEND_NONE: + virReportError(VIR_ERR_NO_SUPPORT, + _("Firewall backend is not implemented")); + return -1;
This check (and the other ones you've introduced) are running too late, which leads to this behavior:
$ cat default-session.xml <network> <name>default-session</name> <forward mode='nat'/> <bridge name='virbr0' stp='on' delay='0'/> <ip address='192.168.123.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.123.2' end='192.168.123.254'/> </dhcp> </ip> </network>
$ virsh -c qemu:///session net-define default-session.xml Network default-session defined from default-session.xml
$ virsh -c qemu:///session net-start default-session error: Failed to start network default-session error: error creating bridge interface virbr0: Operation not permitted
Since <forward mode='nat'/> requires firewall rules, we shouldn't allow a network that uses it to be defined in the first place.
I agree with Dan that we shouldn't prevent anyone from *defining* such a network, we should just log an error and fail if they try to start such a network on a system that doesn't have a non-null firewall backend.
diff --git a/po/POTFILES b/po/POTFILES index 4bfbb91164..1ed4086d2c 100644 --- a/po/POTFILES +++ b/po/POTFILES @@ -143,6 +143,7 @@ src/lxc/lxc_process.c src/network/bridge_driver.c src/network/bridge_driver_conf.c src/network/bridge_driver_linux.c +src/network/bridge_driver_nop.c src/network/leaseshelper.c src/network/network_iptables.c src/network/network_nftables.c diff --git a/src/network/bridge_driver_nop.c b/src/network/bridge_driver_nop.c index 2114d521d1..323990cf17 100644 --- a/src/network/bridge_driver_nop.c +++ b/src/network/bridge_driver_nop.c @@ -49,8 +49,8 @@ int networkAddFirewallRules(virNetworkDef *def G_GNUC_UNUSED, */ if (firewallBackend != VIR_FIREWALL_BACKEND_NONE) { virReportError(VIR_ERR_NO_SUPPORT, - _("Firewall backend '%s' not available on this platform"), - virFirewallBackendTypeToString(firewallBackend)); + _("Firewall backend '%1$s' not available on this platform"), + virFirewallBackendTypeToSTring(firewallBackend)); return -1; } return 0; diff --git a/src/util/virfirewall.c b/src/util/virfirewall.c index d374f54b64..090dbcdbed 100644 --- a/src/util/virfirewall.c +++ b/src/util/virfirewall.c @@ -817,7 +817,7 @@ virFirewallApplyCmd(virFirewall *firewall,
switch (virFirewallGetBackend(firewall)) { case VIR_FIREWALL_BACKEND_NONE: - virReportError(VIR_ERR_NO_SUPPORT, + virReportError(VIR_ERR_NO_SUPPORT, "%s", _("Firewall backend is not implemented")); return -1;
On Fri, Jun 14, 2024 at 12:22:50PM -0400, Andrea Bolognani wrote:
On Fri, Jun 14, 2024 at 03:43:53PM GMT, Daniel P. Berrangé wrote:
meson.build | 26 +++++++++++++++++++------- meson_options.txt | 2 +- src/network/bridge_driver_conf.c | 19 ++++++++++++++----- src/network/bridge_driver_linux.c | 10 ++++++++++ src/network/bridge_driver_nop.c | 15 ++++++++++++++- src/util/virfirewall.c | 6 ++++++ src/util/virfirewall.h | 1 + 7 files changed, 65 insertions(+), 14 deletions(-)
The test suite no longer passes after applying this. At the very least, you need to squash in the diff at the bottom of this message.
firewall_backend_priority = get_option('firewall_backend_priority') - if (not firewall_backend_priority.contains('nftables') or - not firewall_backend_priority.contains('iptables') or - firewall_backend_priority.length() != 2) - error('invalid value for firewall_backend_priority option') + if firewall_backend_priority.length() == 0 + if host_machine.system() == 'linux' + firewall_backend_priority = ['nftables', 'iptables'] + else + # No firewall impl on non-Linux so far, so force 'none' + # as placeholder + firewall_backend_priority = ['none'] + endif + else + if host_machine.system() != 'linux' + error('firewall backend priority only supported on linux hosts') + endif endif
This implementation allows things such as
-Dfirewall_backend_priority=nftables
and
-Dfirewall_backend_priority=iptables,iptables
At least
-Dfirewall_backend_priority=iptables,nftables,iptables
will be blocked, but only because it results in a compilation error: meson will happily accept it.
Are we okay with that? It's IMO inferior to the much stricter checking that's performed today.
I found that if you try this with meson you'll see this DEPRECATION: Duplicated values in array option is deprecated. This will become a hard error in the future. I think we're fine to delegate this to Meson, given its intent to turn this into a hard error eventually, since duplication is harmless for us in the short term. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Mon, Jun 17, 2024 at 09:36:15AM GMT, Daniel P. Berrangé wrote:
On Fri, Jun 14, 2024 at 12:22:50PM -0400, Andrea Bolognani wrote:
This implementation allows things such as
-Dfirewall_backend_priority=nftables
and
-Dfirewall_backend_priority=iptables,iptables
At least
-Dfirewall_backend_priority=iptables,nftables,iptables
will be blocked, but only because it results in a compilation error: meson will happily accept it.
Are we okay with that? It's IMO inferior to the much stricter checking that's performed today.
I found that if you try this with meson you'll see this
DEPRECATION: Duplicated values in array option is deprecated. This will become a hard error in the future.
I think we're fine to delegate this to Meson, given its intent to turn this into a hard error eventually, since duplication is harmless for us in the short term.
I didn't realize that. Sounds good! -- Andrea Bolognani / Red Hat / Virtualization
participants (4)
-
Andrea Bolognani -
Daniel P. Berrangé -
Laine Stump -
Michal Prívozník