Libvirt virsh : Error starting network, cannot execute binary /usr/sbin/iptables
Hi, I am a bit lost and hope someone can help me. I am running Debian bookworm (testing) with last updates. $ sudo apt policy libvirt-daemon libvirt-daemon: Installé : 8.5.0-1 Candidat : 8.5.0-1 Table de version : *** 8.5.0-1 100 100 /var/lib/dpkg/status I am unable to start default network , and get an error related to iptables : $ sudo virsh net-start default erreur :Impossible de démarrer le réseau default erreur :internal error: Failed to apply firewall rules /usr/sbin/iptables -w --table filter --list-rules: libvirt: erreur : cannot execute binary /usr/sbin/iptables: Aucun fichier ou dossier de ce type Sorry for the french, it says "impossible to start default network" and "no such file or folder" at the end. It is true I removed iptables because I want to use only nftables (I removed both ufw and iptables packages (apt remove), and enabled the nftables service before error raises). Before this, all was fine, but when I enabled nftables, all VMs disapeared from virt-manager). I uninstalled KVM related packages and reinstalled, still the same. I also installed back iptables, but strangely I still get the same error, although binary /usr/sbin/iptables is there. I tried many things with no luck, restarted libvirtd service, recreated the network, etc... Has anyone some idea about what is happening here ? is there some incompatibility with nftables (firewalld service is disabled) and libvirt ? Thank you, Pascal
Hi,
I am a bit lost and hope someone can help me. I am running Debian bookworm (testing) with last updates.
$ sudo apt policy libvirt-daemon libvirt-daemon: Installé : 8.5.0-1 Candidat : 8.5.0-1 Table de version : *** 8.5.0-1 100 100 /var/lib/dpkg/status
I am unable to start default network , and get an error related to iptables :
$ sudo virsh net-start default erreur :Impossible de démarrer le réseau default erreur :internal error: Failed to apply firewall rules /usr/sbin/iptables -w --table filter --list-rules: libvirt: erreur : cannot execute binary /usr/sbin/iptables: Aucun fichier ou dossier de ce type
Sorry for the french, it says "impossible to start default network" and "no such file or folder" at the end.
On 8/15/22 1:00 PM, Pascal wrote: libvirt execs the iptables command to install packet filtering rules that are part of the setup of virtual networks with forward mode of "nat", "route", and for those virtual networks that have no forward mode at all (aka "isolated"). libvirt's default network uses NAT to make all virtual machines appear to be at the IP address of the host's external ethernet interface, so it requires the iptables command Although I've been working on patches to make it possible for libvirt to use /usr/sbin/nft to add & remove packet filter rules, libvirt still uses iptables to add its rules, so if have a virtual network that needs to add packet filtering rules (or a guest that has nwfilter rules), you'll need the iptables command, and if it's not found then libvirt will fail at runtime (as you've seen). Beyond that, if you install the libvirt-daemon-driver-nwfilter or libvirt-daemon-driver-network packages, then iptables *should* be a hard prerequisite listed in the package manifest; if your distro's packaging system is allowing you to remove the iptables package without also removing those libvirt packages (which is what you say you were able to do), or allowing you to install those libvirt packages without (semi)automatically pulling in iptables (which you also say you've done), then that is a bug in the distro's packaging files for libvirt. It's unclear from your original message if you really require libvirt's default network in any of your guests. If you don't, then you can just not install or start the libvirt-daemon-driver-network package, and don't try to start the default network, and you should be okay without the iptables package on your system. It's not really gaining you anything (other than a miniscule amount of disk space) to do that though. These days /usr/sbin/iptables is a symbolic link to either iptables-nft (which uses the nftables API to put all the rules into special nft tables named "filter" and "nat", so you actually see the iptables rules as nft rules when you run "nft list ruleset") or to /usr/sbin/iptables-legacy (which uses the old iptables API to install rules that *don't* show up in "nft list ruleset"). In either case though, packets are processed using the same nft packet matching code in the kernel. There is some amount of info about these two variants here: https://developers.redhat.com/blog/2020/08/18/iptables-the-two-variants-and-... The end of it all though is that even if you have some userspace programs that "use iptables", they are in the end using nftables in the kernel.
It is true I removed iptables because I want to use only nftables (I removed both ufw and iptables packages (apt remove), and enabled the nftables service before error raises). Before this, all was fine, but when I enabled nftables, all VMs disapeared from virt-manager).
Are the guests completely disappearing from the list? Or is it that you're unable to start the guests? A guest would only completely disappear spontaneously from virt-manager (i.e. be completely removed from the list of all guests) if something went wrong while parsing the guest config during libvirtd (or virtqemud) startup. Look in the system logs for any errors during libvirtd startup (or virtqemud startup if your debian has switched to modular daemons). I can't think of anything that could/should change in the parsing of the config due to enabling the nftables servive (which libvirt knows nothing about).
I uninstalled KVM related packages and reinstalled, still the same.
I also installed back iptables, but strangely I still get the same error, although binary /usr/sbin/iptables is there.
Now that certainly makes no sense! :-/ Possibly /usr/sbin/iptables is a symbolic link to a file that doesn't exist (because there was an additional iptables package, e.g. iptables-nft or iptables-legacy, that you didn't reinstall?)
I tried many things with no luck, restarted libvirtd service, recreated the network, etc...
Has anyone some idea about what is happening here ? is there some incompatibility with nftables (firewalld service is disabled) and libvirt ?
As I've said above, libvirt is still using iptables to add its rules, but there is no incompatibility with nftables, and in fact the rules under the hood end up being nftables rules.
Hi Laine, Thank you so much for your answer and the fully detailled explanation. It is much clear to me now. First I want to correct my error : after reinstalling iptables, I forgot to restart libvirtd.service, thus the same error raised. After restarting the service, I was able to start the virtual network. Also my guests are back in virt-manager, don't know exactly what happened (.xml files are in |/etc/libvirt/qemu/|) but issue was gone after reinstalling paskages. I can see /usr/sbin/iptables is a link to etc/alternatives/iptables which is a link to /usr/sbin/iptables-nft. So when I am able to start virtual network (with iptables installed), I can see rules created in nftables (filter, nat, mangles). So now all is fine, I can run my VMs (yes I need a network to access internet from them). If I need to keep iptables on the system, that's OK, just happy to understand why. -=- Now, regarding the dependency to iptables, I did check libvirtd before writing with no result. But that was not the right package to check. On my debian, the package is libvirt-daemon-system . libvirt-daemon-system depends adduser, gettext-base, iptables, firewalld, libvirt-clients, libvirt-daemon, libvirt-daemon-config-network, libvirt-daemon-config-nwfilter, libvirt-daemon-system-systemd, libvirt-daemon-system-sysv, logrotate, policykit-1, debconf, <debconf-2.0> Thank you for that too ! I will check if I need to create a defect on debian, as you say, it is not expected to be able to remove iptables without any warning (but I am running debian bookworn aka "testing", so such thing may happen). Will check with debian guys. Again, thank you very much for your help and explanations. It has been very useful, great support ! Best regards, Pascal Le 16/08/2022 à 21:49, Laine Stump a écrit :
Hi,
I am a bit lost and hope someone can help me. I am running Debian bookworm (testing) with last updates.
$ sudo apt policy libvirt-daemon libvirt-daemon: Installé : 8.5.0-1 Candidat : 8.5.0-1 Table de version : *** 8.5.0-1 100 100 /var/lib/dpkg/status
I am unable to start default network , and get an error related to iptables :
$ sudo virsh net-start default erreur :Impossible de démarrer le réseau default erreur :internal error: Failed to apply firewall rules /usr/sbin/iptables -w --table filter --list-rules: libvirt: erreur : cannot execute binary /usr/sbin/iptables: Aucun fichier ou dossier de ce type
Sorry for the french, it says "impossible to start default network" and "no such file or folder" at the end.
On 8/15/22 1:00 PM, Pascal wrote: libvirt execs the iptables command to install packet filtering rules that are part of the setup of virtual networks with forward mode of "nat", "route", and for those virtual networks that have no forward mode at all (aka "isolated"). libvirt's default network uses NAT to make all virtual machines appear to be at the IP address of the host's external ethernet interface, so it requires the iptables command
Although I've been working on patches to make it possible for libvirt to use /usr/sbin/nft to add & remove packet filter rules, libvirt still uses iptables to add its rules, so if have a virtual network that needs to add packet filtering rules (or a guest that has nwfilter rules), you'll need the iptables command, and if it's not found then libvirt will fail at runtime (as you've seen).
Beyond that, if you install the libvirt-daemon-driver-nwfilter or libvirt-daemon-driver-network packages, then iptables *should* be a hard prerequisite listed in the package manifest; if your distro's packaging system is allowing you to remove the iptables package without also removing those libvirt packages (which is what you say you were able to do), or allowing you to install those libvirt packages without (semi)automatically pulling in iptables (which you also say you've done), then that is a bug in the distro's packaging files for libvirt.
It's unclear from your original message if you really require libvirt's default network in any of your guests. If you don't, then you can just not install or start the libvirt-daemon-driver-network package, and don't try to start the default network, and you should be okay without the iptables package on your system. It's not really gaining you anything (other than a miniscule amount of disk space) to do that though.
These days /usr/sbin/iptables is a symbolic link to either iptables-nft (which uses the nftables API to put all the rules into special nft tables named "filter" and "nat", so you actually see the iptables rules as nft rules when you run "nft list ruleset") or to /usr/sbin/iptables-legacy (which uses the old iptables API to install rules that *don't* show up in "nft list ruleset"). In either case though, packets are processed using the same nft packet matching code in the kernel. There is some amount of info about these two variants here:
https://developers.redhat.com/blog/2020/08/18/iptables-the-two-variants-and-...
The end of it all though is that even if you have some userspace programs that "use iptables", they are in the end using nftables in the kernel.
It is true I removed iptables because I want to use only nftables (I removed both ufw and iptables packages (apt remove), and enabled the nftables service before error raises). Before this, all was fine, but when I enabled nftables, all VMs disapeared from virt-manager).
Are the guests completely disappearing from the list? Or is it that you're unable to start the guests? A guest would only completely disappear spontaneously from virt-manager (i.e. be completely removed from the list of all guests) if something went wrong while parsing the guest config during libvirtd (or virtqemud) startup. Look in the system logs for any errors during libvirtd startup (or virtqemud startup if your debian has switched to modular daemons). I can't think of anything that could/should change in the parsing of the config due to enabling the nftables servive (which libvirt knows nothing about).
I uninstalled KVM related packages and reinstalled, still the same.
I also installed back iptables, but strangely I still get the same error, although binary /usr/sbin/iptables is there.
Now that certainly makes no sense! :-/ Possibly /usr/sbin/iptables is a symbolic link to a file that doesn't exist (because there was an additional iptables package, e.g. iptables-nft or iptables-legacy, that you didn't reinstall?)
I tried many things with no luck, restarted libvirtd service, recreated the network, etc...
Has anyone some idea about what is happening here ? is there some incompatibility with nftables (firewalld service is disabled) and libvirt ?
As I've said above, libvirt is still using iptables to add its rules, but there is no incompatibility with nftables, and in fact the rules under the hood end up being nftables rules.
participants (2)
-
Laine Stump -
Pascal