[libvirt-users] libvirt-bin MOSTLY fails to create default virtual network
I have observed 3 scenarios after I reboot the host with 2 guests (which are supposed to autostart after boot/reboot of the host): 1. Default network is created but I can not ping guests. Gests don't have assigned IP address or they both have IP address 192.168.122.100. 2. Default network is not created. 3. Default netork is properly created and everything works fine. In each case guests are started and they work properly. Cases 1. and 2. happen a lot more often then case 3. My solutions: - When I restart libvirt-bin then everything is fine. - When I set 'sleep 40' in 'pre-start' section of /etc/init/libvirt-bin.conf and modify 'start on' section in the following way: start on (runlevel [2345] and net-device-up IFACE=br0) then everything works fine. However, none of the above solutions indeed solves the problem. Please, let me know if there is solution to this problem or what are your suggestions regarding this issue. **** HOST$ uname -a Linux ras 2.6.32-32-server #62-Ubuntu SMP Wed Apr 20 22:07:43 UTC 2011 x86_64 GNU/Linux GUEST1 and GUEST2 $ uname -a Linux server1-desktop 2.6.32-32-generic #62-Ubuntu SMP Wed Apr 20 21:54:21 UTC 2011 i686 GNU/Linux HOST$ libvirtd --version libvirtd (libvirt) 0.7.5 My idea was that some kernel modules sometimes are loaded before libvirt-bin starts, and sometimes libvirt-bin starts earlier, or in the middle of loading of these modules, that is why 3 scenarios can be observed. That is why I have added: cp /proc/modules /home/wsadmin2/modules1 cp /proc/modules /home/wsadmin2/modules2 to 'pre-start' and 'post-start' parts respectively of /etc/init/libvirt-bin.conf. Of course, beween execution of - pre-start part - exec /usr/sbin/libvirtd -d - post-start part other kernel modules can be loaded, so we can not be sure which additional modules in file 'modules2' were indeed loaded before libvirtd was actually started. Here are my observations: Scenatio 1. 33 (fixed) modules were logged - call them BASE MODULES. Scenatio 2. Beside BASE MODULES + the following modules were loaded:
iptable_filter 1841 0 - Live 0xffffffffa007c000 ip_tables 18201 1 iptable_filter, Live 0xffffffffa0172000 x_tables 22361 1 ip_tables, Live 0xffffffffa0164000
Scenatio 3. Beside BASE MODULES + the following modules were loaded: $ diff modules1 modules2 1c1,6 < iptable_filter 1841 0 - Live 0xffffffffa008b000 ---
xt_state 1490 14 - Live 0xffffffffa01ab000 xt_tcpudp 2667 12 - Live 0xffffffffa01a5000 nf_conntrack_ipv4 12742 14 - Live 0xffffffffa019b000 nf_conntrack 73326 2 xt_state,nf_conntrack_ipv4, Live 0xffffffffa017d000 nf_defrag_ipv4 1481 1 nf_conntrack_ipv4, Live 0xffffffffa0177000 iptable_filter 1841 1 - Live 0xffffffffa008b000 3c8 < x_tables 22361 1 ip_tables, Live 0xffffffffa0162000
x_tables 22361 3 xt_state,xt_tcpudp,ip_tables, Live 0xffffffffa0162000
Other observations: Scenatio 1. No errors in libvirtd.log Scenario 2. 10:06:06.980: error : networkAddIptablesRules:750 : failed to add iptables rule to allow DHCP requests from 'virbr0': Invalid argument (...) 10:06:08.630: error : qemudNetworkIfaceConnect:1445 : Failed to add tap interface to bridge 'virbr0': No such device 10:06:08.630: error : qemuAutostartDomain:584 : Failed to autostart VM 'cluster_server1': Failed to add tap interface to bridge 'virbr0': No such device 10:06:08.921: info : qemudDispatchSignalEvent:390 : Received unexpected signal 17 10:06:08.927: debug : virExecWithHook:617 : LC_ALL=C /usr/bin/kvm -help 10:06:08.938: info : qemudDispatchSignalEvent:390 : Received unexpected signal 17 10:06:09.000: error : qemudNetworkIfaceConnect:1445 : Failed to add tap interface to bridge 'virbr0': No such device 10:06:09.000: error : qemuAutostartDomain:584 : Failed to autostart VM 'cluster_server2': Failed to add tap interface to bridge 'virbr0': No such device (...) Scenatio 3. No errors in libvirtd.log I didn't see any valuable logs in /var/log/boot.log, /var/log/messages, /var/log/syslog. Please let me know if you want to see them anyway or have any suggestions right away.
On 06/06/2011 03:24 PM, m m wrote:
I have observed 3 scenarios after I reboot the host with 2 guests (which are supposed to autostart after boot/reboot of the host): 1. Default network is created but I can not ping guests. Gests don't have assigned IP address or they both have IP address 192.168.122.100. 2. Default network is not created. 3. Default netork is properly created and everything works fine. In each case guests are started and they work properly. Cases 1. and 2. happen a lot more often then case 3.
My solutions: - When I restart libvirt-bin then everything is fine. - When I set 'sleep 40' in 'pre-start' section of /etc/init/libvirt-bin.conf and modify 'start on' section in the following way:
start on (runlevel [2345] and net-device-up IFACE=br0)
then everything works fine.
However, none of the above solutions indeed solves the problem. Please, let me know if there is solution to this problem or what are your suggestions regarding this issue.
**** HOST$ uname -a Linux ras 2.6.32-32-server #62-Ubuntu SMP Wed Apr 20 22:07:43 UTC 2011 x86_64 GNU/Linux
GUEST1 and GUEST2 $ uname -a Linux server1-desktop 2.6.32-32-generic #62-Ubuntu SMP Wed Apr 20 21:54:21 UTC 2011 i686 GNU/Linux
HOST$ libvirtd --version libvirtd (libvirt) 0.7.5
My idea was that some kernel modules sometimes are loaded before libvirt-bin starts, and sometimes libvirt-bin starts earlier, or in the middle of loading of these modules, that is why 3 scenarios can be observed. That is why I have added: cp /proc/modules /home/wsadmin2/modules1 cp /proc/modules /home/wsadmin2/modules2 to 'pre-start' and 'post-start' parts respectively of /etc/init/libvirt-bin.conf.
Of course, beween execution of - pre-start part - exec /usr/sbin/libvirtd -d - post-start part other kernel modules can be loaded, so we can not be sure which additional modules in file 'modules2' were indeed loaded before libvirtd was actually started.
Here are my observations: Scenatio 1. 33 (fixed) modules were logged - call them BASE MODULES.
Scenatio 2. Beside BASE MODULES + the following modules were loaded:
iptable_filter 1841 0 - Live 0xffffffffa007c000 ip_tables 18201 1 iptable_filter, Live 0xffffffffa0172000 x_tables 22361 1 ip_tables, Live 0xffffffffa0164000
In your "scenario 1" the modules necessary for iptables to function haven't been loaded. Each virtual network requires a set of iptables rules to function properly, and if there is no iptables, the rules can't be added and network creation fails. Your libvirtd.log confirms that problem: 10:06:06.980: error : networkAddIptablesRules:750 : failed to add iptables rule to allow DHCP requests from 'virbr0': Invalid argument 0.7.5 is quite old (> 1 year), is there perhaps a newer pre-built version of libvirt available for your release of Ubuntu? It seems that Ubuntu's method of starting up libvirtd (at least on your machine) is missing a dependency to force iptables to be loaded before libvirtd starts. I'm unfamiliar with Ubuntu/debian startup stuff (upstart, I guess), but that's where you need to look for the solution. I found the following in the ubuntu bug database, which seems similar, but not identical (basically the system networking isn't completely ready by the time the autostart domains are brought up): https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/495394
Scenatio 3. Beside BASE MODULES + the following modules were loaded:
$ diff modules1 modules2 1c1,6 < iptable_filter 1841 0 - Live 0xffffffffa008b000 ---
xt_state 1490 14 - Live 0xffffffffa01ab000 xt_tcpudp 2667 12 - Live 0xffffffffa01a5000 nf_conntrack_ipv4 12742 14 - Live 0xffffffffa019b000 nf_conntrack 73326 2 xt_state,nf_conntrack_ipv4, Live 0xffffffffa017d000
conntrack is used by iptables. I haven't tried it, but wouldn't be surprised if adding rules that required conntrack failed if it hadn't been loaded, or something like that.
nf_defrag_ipv4 1481 1 nf_conntrack_ipv4, Live 0xffffffffa0177000 iptable_filter 1841 1 - Live 0xffffffffa008b000 3c8 < x_tables 22361 1 ip_tables, Live 0xffffffffa0162000
x_tables 22361 3 xt_state,xt_tcpudp,ip_tables, Live 0xffffffffa0162000
Other observations: Scenatio 1. No errors in libvirtd.log Scenario 2.
10:06:06.980: error : networkAddIptablesRules:750 : failed to add iptables rule to allow DHCP requests from 'virbr0': Invalid argument (...) 10:06:08.630: error : qemudNetworkIfaceConnect:1445 : Failed to add tap interface to bridge 'virbr0': No such device 10:06:08.630: error : qemuAutostartDomain:584 : Failed to autostart VM 'cluster_server1': Failed to add tap interface to bridge 'virbr0': No such device
10:06:08.921: info : qemudDispatchSignalEvent:390 : Received unexpected signal 17 10:06:08.927: debug : virExecWithHook:617 : LC_ALL=C /usr/bin/kvm -help 10:06:08.938: info : qemudDispatchSignalEvent:390 : Received unexpected signal 17 10:06:09.000: error : qemudNetworkIfaceConnect:1445 : Failed to add tap interface to bridge 'virbr0': No such device 10:06:09.000: error : qemuAutostartDomain:584 : Failed to autostart VM 'cluster_server2': Failed to add tap interface to bridge 'virbr0': No such device (...) Scenatio 3. No errors in libvirtd.log
I didn't see any valuable logs in /var/log/boot.log, /var/log/messages, /var/log/syslog. Please let me know if you want to see them anyway or have any suggestions right away.
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
participants (2)
-
Laine Stump -
m m