[libvirt] bug: network lock-out
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, I'm sorry to report this, but network should start or stop regardless iptables status. virsh # net-start default error: Failed to start network default error: internal error '/usr/sbin/iptables --table filter --delete INPUT - --in-interface virbr0 --protocol udp --destination-port 69 --jump ACCEPT' exited with non-zero status 1 and signal 0: iptables: Bad rule (does a matching rule exist in that chain?). Ok, so I'm going to create this rule to make you happy. virsh # net-start default error: Failed to start network default error: internal error '/usr/sbin/iptables --table mangle --delete POSTROUTING --out-interface virbr0 --protocol udp --destination-port 68 - --jump CHECKSUM --checksum-fill' exited with non-zero status 2 and signal 0: iptables v1.4.7: unknown option `--checksum-fill' Try `iptables -h' or 'iptables --help' for more information. And Oops, because I don't have CHECKSUM support. And to note, it's not even in kernel. Rely on such features is unfriendly and bellow belt :) Once again, I'm locked by hard-coded features :( That's why I "fight" against these. btw it's strange to me that libvirt is deleting rules that shouldn't be present since I want to start network, not to stop it. Let's dump all nwfilters and hope for miracle...and nothing. Same errors, although nwfilter rules are gone. What the ...? :| libvirt-0.8.4 Regards, Zdenek - -- Zdenek Styblik Net/Linux admin OS TurnovFree.net email: stybla@turnovfree.net jabber: stybla@jabber.turnovfree.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkyrezwACgkQ8MreUbSH7ikbpQCdEDtbwG+PV6u0yvUZYzXlQas9 ohEAoIyy/HPZwtWlaOAgtx6jnOEFashR =u7st -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/05/10 21:23, Zdenek Styblik wrote:
Hello,
I'm sorry to report this, but network should start or stop regardless iptables status.
virsh # net-start default error: Failed to start network default error: internal error '/usr/sbin/iptables --table filter --delete INPUT --in-interface virbr0 --protocol udp --destination-port 69 --jump ACCEPT' exited with non-zero status 1 and signal 0: iptables: Bad rule (does a matching rule exist in that chain?).
Ok, so I'm going to create this rule to make you happy.
virsh # net-start default error: Failed to start network default error: internal error '/usr/sbin/iptables --table mangle --delete POSTROUTING --out-interface virbr0 --protocol udp --destination-port 68 --jump CHECKSUM --checksum-fill' exited with non-zero status 2 and signal 0: iptables v1.4.7: unknown option `--checksum-fill' Try `iptables -h' or 'iptables --help' for more information.
And Oops, because I don't have CHECKSUM support. And to note, it's not even in kernel. Rely on such features is unfriendly and bellow belt :)
Once again, I'm locked by hard-coded features :( That's why I "fight" against these.
btw it's strange to me that libvirt is deleting rules that shouldn't be present since I want to start network, not to stop it.
Let's dump all nwfilters and hope for miracle...and nothing. Same errors, although nwfilter rules are gone. What the ...? :|
libvirt-0.8.4
Regards, Zdenek
More info: - ---SNIP--- 21:31:09.298: error : virRunWithHook:857 : internal error '/usr/sbin/iptables --table mangle --insert POSTROUTING --out-interface virbr0 --protocol udp --destination-port 68 --jump CHECKSUM - --checksum-fill' exited with non-zero status 2 and signal 0: iptables v1.4.7: unknown option `--checksum-fill' Try `iptables -h' or 'iptables --help' for more information. 21:31:09.299: warning : networkAddIptablesRules:873 : Could not add rule to fixup DHCP response checksums on network 'default'. 21:31:09.299: warning : networkAddIptablesRules:874 : May need to update iptables package & kernel to support CHECKSUM rule. 21:31:09.301: error : virRunWithHook:857 : internal error '/usr/sbin/dnsmasq --strict-order --bind-interfaces - --pid-file=/var/run/libvirt/network/default.pid --conf-file= - --listen-address 10.117.9.1 --except-interface lo' exited with non-zero status 1 and signal 0: libvir: error : cannot execute binary /usr/sbin/dnsmasq: No such file or directory 21:31:09.305: error : virRunWithHook:857 : internal error '/usr/sbin/iptables --table mangle --delete POSTROUTING --out-interface virbr0 --protocol udp --destination-port 68 --jump CHECKSUM - --checksum-fill' exited with non-zero status 2 and signal 0: iptables v1.4.7: unknown option `--checksum-fill' Try `iptables -h' or 'iptables --help' for more information. 21:31:09.343: error : virRunWithHook:857 : internal error '/usr/sbin/iptables --table filter --delete INPUT --in-interface virbr0 - --protocol udp --destination-port 69 --jump ACCEPT' exited with non-zero status 1 and signal 0: iptables: Bad rule (does a matching rule exist in that chain?). - ---SNIP--- Yes, the networking used to work and actually, who cares about failed iptables anyway? That's not the reason for not bringing up iface :| Have a better evening than I'm having! :) Zdenek - -- Zdenek Styblik Net/Linux admin OS TurnovFree.net email: stybla@turnovfree.net jabber: stybla@jabber.turnovfree.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkyrfY8ACgkQ8MreUbSH7il/HwCg0ssizKjxjOWF2tEnO8IViIm0 RCsAnjqbHLh4Ag/1M64/Jqy3HPexOqvB =UZnf -----END PGP SIGNATURE-----
21:31:09.298: error : virRunWithHook:857 : internal error '/usr/sbin/iptables --table mangle --insert POSTROUTING --out-interface virbr0 --protocol udp --destination-port 68 --jump CHECKSUM --checksum-fill' exited with non-zero status 2 and signal 0: iptables v1.4.7: unknown option `--checksum-fill' Try `iptables -h' or 'iptables --help' for more information.
This is harmless and ignored by libvirt. If DHCP still works in your guests, you don't need worry about this feature. The warning below tries to suggest the error was ignored...
21:31:09.299: warning : networkAddIptablesRules:873 : Could not add rule to fixup DHCP response checksums on network 'default'. 21:31:09.299: warning : networkAddIptablesRules:874 : May need to update iptables package & kernel to support CHECKSUM rule. 21:31:09.301: error : virRunWithHook:857 : internal error '/usr/sbin/dnsmasq --strict-order --bind-interfaces --pid-file=/var/run/libvirt/network/default.pid --conf-file= --listen-address 10.117.9.1 --except-interface lo' exited with non-zero status 1 and signal 0: libvir: error : cannot execute binary /usr/sbin/dnsmasq: No such file or directory
This is the really important error for you; /usr/sbin/dnsmasq could not be found. Jirka
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jiri, thank you for very prompt reply. On 10/05/10 21:44, Jiri Denemark wrote:
21:31:09.298: error : virRunWithHook:857 : internal error '/usr/sbin/iptables --table mangle --insert POSTROUTING --out-interface virbr0 --protocol udp --destination-port 68 --jump CHECKSUM --checksum-fill' exited with non-zero status 2 and signal 0: iptables v1.4.7: unknown option `--checksum-fill' Try `iptables -h' or 'iptables --help' for more information.
This is harmless and ignored by libvirt. If DHCP still works in your guests, you don't need worry about this feature. The warning below tries to suggest the error was ignored...
Uh huh ok.
21:31:09.299: warning : networkAddIptablesRules:873 : Could not add rule to fixup DHCP response checksums on network 'default'. 21:31:09.299: warning : networkAddIptablesRules:874 : May need to update iptables package & kernel to support CHECKSUM rule. 21:31:09.301: error : virRunWithHook:857 : internal error '/usr/sbin/dnsmasq --strict-order --bind-interfaces --pid-file=/var/run/libvirt/network/default.pid --conf-file= --listen-address 10.117.9.1 --except-interface lo' exited with non-zero status 1 and signal 0: libvir: error : cannot execute binary /usr/sbin/dnsmasq: No such file or directory
This is the really important error for you; /usr/sbin/dnsmasq could not be found.
Well, so is it reason not to bring up an interface? (Answer is - YES) I've removed dnsmasq because it was giving away IP addresses, although DHCP has not been defined in XML. I've mentioned this some time ago in an e-mail with poetic subj: "dnsmasq, dhcp - bug or feature" :) Umm thanks for helping me to solve this one, yet I still think it's - odd. I mean ... you know ... oh well :| (how to put it - head down, walking away) :) Have a good night! Zdenek - -- Zdenek Styblik Net/Linux admin OS TurnovFree.net email: stybla@turnovfree.net jabber: stybla@jabber.turnovfree.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkyrgi4ACgkQ8MreUbSH7inx9gCguntYbSTtLMFu7RGO1Q3PWVWy hlAAoNRB48+11JXP4zkzp8+DylLKl7uA =byJC -----END PGP SIGNATURE-----
On 10/05/2010 03:53 PM, Zdenek Styblik wrote:
I've removed dnsmasq because it was giving away IP addresses, although DHCP has not been defined in XML. I've mentioned this some time ago in an e-mail with poetic subj: "dnsmasq, dhcp - bug or feature" :)
I tried changing my default network to remove the <dhcp> section, restarted the network, and rebooted a guest to try and acquire an address from DHCP - it failed. Can you do "ps -AlF | grep dnsmasq" at a time when you see this behavior, and send that. Perhaps there was another dnsmasq hanging around listening on that interface? (I think it's possible that dnsmasq sometimes isn't terminated if libvirtd crashes)
On 10/06/2010 05:23 PM, Laine Stump wrote:
On 10/05/2010 03:53 PM, Zdenek Styblik wrote:
I've removed dnsmasq because it was giving away IP addresses, although DHCP has not been defined in XML. I've mentioned this some time ago in an e-mail with poetic subj: "dnsmasq, dhcp - bug or feature" :)
I tried changing my default network to remove the <dhcp> section, restarted the network, and rebooted a guest to try and acquire an address from DHCP - it failed. Can you do "ps -AlF | grep dnsmasq" at a time when you see this behavior, and send that. Perhaps there was another dnsmasq hanging around listening on that interface? (I think it's possible that dnsmasq sometimes isn't terminated if libvirtd crashes)
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Hanging dnsmasq was either the case or it has been fixed in 0.8.4. I sometimes suspect dnsmasq doesn't get restarted at all, although libvirtd/network does. Another "workaround" seems to be eg. range 192.168.1.0/24; 192.168.1.1/24 at interface; leases 192.168.1.1-192.168.1.1. This is from ISC DHCPd where you have to provide leases for pool with static (IP:MAC) hosts. Thanks, Zdenek -- Zdenek Styblik Net/Linux admin OS TurnovFree.net email: stybla@turnovfree.net jabber: stybla@jabber.turnovfree.net
On 10/07/2010 09:56 AM, Zdenek Styblik wrote:
On 10/06/2010 05:23 PM, Laine Stump wrote:
On 10/05/2010 03:53 PM, Zdenek Styblik wrote:
I've removed dnsmasq because it was giving away IP addresses, although DHCP has not been defined in XML. I've mentioned this some time ago in an e-mail with poetic subj: "dnsmasq, dhcp - bug or feature" :)
I tried changing my default network to remove the <dhcp> section, restarted the network, and rebooted a guest to try and acquire an address from DHCP - it failed. Can you do "ps -AlF | grep dnsmasq" at a time when you see this behavior, and send that. Perhaps there was another dnsmasq hanging around listening on that interface? (I think it's possible that dnsmasq sometimes isn't terminated if libvirtd crashes)
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Hanging dnsmasq was either the case or it has been fixed in 0.8.4. I sometimes suspect dnsmasq doesn't get restarted at all, although libvirtd/network does.
Another "workaround" seems to be eg. range 192.168.1.0/24; 192.168.1.1/24 at interface; leases 192.168.1.1-192.168.1.1. This is from ISC DHCPd where you have to provide leases for pool with static (IP:MAC) hosts.
Thanks, Zdenek
I forgot to say it was connected to virt-manager, thus it might have been a virt-manager bug. I've verified most of this by hand and using virsh, so- Zdenek -- Zdenek Styblik Net/Linux admin OS TurnovFree.net email: stybla@turnovfree.net jabber: stybla@jabber.turnovfree.net
On 10/05/2010 03:44 PM, Jiri Denemark wrote:
21:31:09.298: error : virRunWithHook:857 : internal error '/usr/sbin/iptables --table mangle --insert POSTROUTING --out-interface virbr0 --protocol udp --destination-port 68 --jump CHECKSUM --checksum-fill' exited with non-zero status 2 and signal 0: iptables v1.4.7: unknown option `--checksum-fill' Try `iptables -h' or 'iptables --help' for more information. This is harmless and ignored by libvirt. If DHCP still works in your guests, you don't need worry about this feature. The warning below tries to suggest the error was ignored...
Correct. The reason for this is that the only way to determine whether or not iptables supports the new CHECKSUM target is to try the command and see if it fails. Since the CHECKSUM target is in upstream iptables, it will eventually be in all distro-specific versions, so the less-than-elegant warning was deemed sufficient. This particular rule is required to support guests that use the vhost-net module (ie kernel-based rather than userspace-based) for virtio network interfaces. Whether or not that will be needed depends on guest config, which can't be known at the time that the virtual networks are started, so we must always try to add it, then fail "semi-silently" (we continue, but first complain a little).
21:31:09.299: warning : networkAddIptablesRules:873 : Could not add rule to fixup DHCP response checksums on network 'default'. 21:31:09.299: warning : networkAddIptablesRules:874 : May need to update iptables package& kernel to support CHECKSUM rule. 21:31:09.301: error : virRunWithHook:857 : internal error '/usr/sbin/dnsmasq --strict-order --bind-interfaces --pid-file=/var/run/libvirt/network/default.pid --conf-file= --listen-address 10.117.9.1 --except-interface lo' exited with non-zero status 1 and signal 0: libvir: error : cannot execute binary /usr/sbin/dnsmasq: No such file or directory This is the really important error for you; /usr/sbin/dnsmasq could not be found.
That location comes from config.h, so it's determined at configure time. Apparently it found /usr/sbin/dnsmasq at configure time. Did you build on a different machine from where you're running (and maybe this new machine doesn't have dnsmasq installed? It should be in the prerequisites for your libvirt package to ensure that it's always installed when libvirt is installed).
On 10/05/2010 10:52 PM, Laine Stump wrote:
On 10/05/2010 03:44 PM, Jiri Denemark wrote:
21:31:09.298: error : virRunWithHook:857 : internal error '/usr/sbin/iptables --table mangle --insert POSTROUTING --out-interface virbr0 --protocol udp --destination-port 68 --jump CHECKSUM --checksum-fill' exited with non-zero status 2 and signal 0: iptables v1.4.7: unknown option `--checksum-fill' Try `iptables -h' or 'iptables --help' for more information. This is harmless and ignored by libvirt. If DHCP still works in your guests, you don't need worry about this feature. The warning below tries to suggest the error was ignored...
Correct. The reason for this is that the only way to determine whether or not iptables supports the new CHECKSUM target is to try the command and see if it fails. Since the CHECKSUM target is in upstream iptables, it will eventually be in all distro-specific versions, so the less-than-elegant warning was deemed sufficient.
If it's only warning then it's 'ok'. :)
[...]
21:31:09.299: warning : networkAddIptablesRules:873 : Could not add rule to fixup DHCP response checksums on network 'default'. 21:31:09.299: warning : networkAddIptablesRules:874 : May need to update iptables package& kernel to support CHECKSUM rule. 21:31:09.301: error : virRunWithHook:857 : internal error '/usr/sbin/dnsmasq --strict-order --bind-interfaces --pid-file=/var/run/libvirt/network/default.pid --conf-file= --listen-address 10.117.9.1 --except-interface lo' exited with non-zero status 1 and signal 0: libvir: error : cannot execute binary /usr/sbin/dnsmasq: No such file or directory This is the really important error for you; /usr/sbin/dnsmasq could not be found.
That location comes from config.h, so it's determined at configure time. Apparently it found /usr/sbin/dnsmasq at configure time. Did you build on a different machine from where you're running (and maybe this new machine doesn't have dnsmasq installed? It should be in the prerequisites for your libvirt package to ensure that it's always installed when libvirt is installed).
It's more complicated than that, but yeah - it has been compiled at different machine; dnsmasq used to be present and so on. Pkg prerequisites do not exist everywhere. Anyway, yes libvirt has been compiled with dnsmasq present. If it's not already, I would mention dnsmasq (and all other) optional dependency in some README. That would be great :) Thank you all, Zdenek -- Zdenek Styblik Net/Linux admin OS TurnovFree.net email: stybla@turnovfree.net jabber: stybla@jabber.turnovfree.net
participants (3)
-
Jiri Denemark -
Laine Stump -
Zdenek Styblik