My system: Ubuntu Jaunty 9.04 libvirt 0.6.4 kvm 0.8.4 qemu 0.10.0 I'm not sure what triggered this, I was working with several VMs, and then found that virsh decided to hang: garry@kvr02:~$ virsh list Connecting to uri: qemu:///system <hang> I have to ^C out of it. If I 'force-stop' and then 'start' libvirt-bin: garry@kvr02:~$ sudo /etc/init.d/libvirt-bin force-stop * Forcefully stopping libvirt management daemon libvirtd ...done. garry@kvr02:~$ sudo /etc/init.d/libvirt-bin start * Starting libvirt management daemon libvirtd ...done. I can then get something: garry@kvr02:~$ virsh list Connecting to uri: qemu:///system Id Name State ---------------------------------- 1 vm1 running 4 s3-lax running 14 freebsd-test running 19 freebsd-2 running <hang> But it hangs after that 4th one. I must ^C it again. If I do 'virsh list' again, it'll then show nothing (hangs like it does above). Any suggestions? Thanks -- Garry Dolley ARP Networks, Inc. | http://www.arpnetworks.com | (818) 206-0181 Data center, VPS, and IP Transit solutions Member Los Angeles County REACT, Unit 336 | WQGK336 Blog http://scie.nti.st
On Tue, Jun 23, 2009 at 06:22:34AM -0700, Garry Dolley wrote:
My system:
Ubuntu Jaunty 9.04
libvirt 0.6.4 kvm 0.8.4 qemu 0.10.0
I'm not sure what triggered this, I was working with several VMs, and then found that virsh decided to hang:
garry@kvr02:~$ virsh list Connecting to uri: qemu:///system <hang>
I have to ^C out of it.
If I 'force-stop' and then 'start' libvirt-bin:
garry@kvr02:~$ sudo /etc/init.d/libvirt-bin force-stop * Forcefully stopping libvirt management daemon libvirtd ...done. garry@kvr02:~$ sudo /etc/init.d/libvirt-bin start * Starting libvirt management daemon libvirtd ...done.
I can then get something:
garry@kvr02:~$ virsh list Connecting to uri: qemu:///system Id Name State ---------------------------------- 1 vm1 running 4 s3-lax running 14 freebsd-test running 19 freebsd-2 running <hang>
But it hangs after that 4th one. I must ^C it again.
If I do 'virsh list' again, it'll then show nothing (hangs like it does above).
Any suggestions?
From playing with this, I'm led to believe libvirt "remembers" some VM that I may have killed manually w/ 'kill'.
Where does libvirt store what VMs it knows about across restarts? I think I may need to manually poke around there and take out the bad VM... -- Garry Dolley ARP Networks, Inc. | http://www.arpnetworks.com | (818) 206-0181 Data center, VPS, and IP Transit solutions Member Los Angeles County REACT, Unit 336 | WQGK336 Blog http://scie.nti.st
On Tue, Jun 23, 2009 at 12:05:35PM -0700, Garry Dolley wrote:
On Tue, Jun 23, 2009 at 06:22:34AM -0700, Garry Dolley wrote:
My system:
Ubuntu Jaunty 9.04
libvirt 0.6.4 kvm 0.8.4 qemu 0.10.0
I'm not sure what triggered this, I was working with several VMs, and then found that virsh decided to hang:
garry@kvr02:~$ virsh list Connecting to uri: qemu:///system <hang>
I have to ^C out of it.
If I 'force-stop' and then 'start' libvirt-bin:
garry@kvr02:~$ sudo /etc/init.d/libvirt-bin force-stop * Forcefully stopping libvirt management daemon libvirtd ...done. garry@kvr02:~$ sudo /etc/init.d/libvirt-bin start * Starting libvirt management daemon libvirtd ...done.
I can then get something:
garry@kvr02:~$ virsh list Connecting to uri: qemu:///system Id Name State ---------------------------------- 1 vm1 running 4 s3-lax running 14 freebsd-test running 19 freebsd-2 running <hang>
But it hangs after that 4th one. I must ^C it again.
If I do 'virsh list' again, it'll then show nothing (hangs like it does above).
Any suggestions?
From playing with this, I'm led to believe libvirt "remembers" some VM that I may have killed manually w/ 'kill'.
Where does libvirt store what VMs it knows about across restarts? I think I may need to manually poke around there and take out the bad VM...
crobinso in #virt (irc.oftc.net) helped me solve this. The info about running VMs is kept across restarts in: /var/run/libvirt/qemu There was a VM that went crazy, it totally hung, and I guess something about it didn't sit well with virsh. When I 'kill -9' this VM (yes, 'kill' by itself didn't even work), then virsh didn't hang anymore. Problem solved. (thankfully, the hanging VM was a throw-away one, so kill -9 was OK in this case) -- Garry Dolley ARP Networks, Inc. | http://www.arpnetworks.com | (818) 206-0181 Data center, VPS, and IP Transit solutions Member Los Angeles County REACT, Unit 336 | WQGK336 Blog http://scie.nti.st
On Tue, Jun 23, 2009 at 06:22:34AM -0700, Garry Dolley wrote:
My system:
Ubuntu Jaunty 9.04
libvirt 0.6.4 kvm 0.8.4 qemu 0.10.0
I'm not sure what triggered this, I was working with several VMs, and then found that virsh decided to hang:
garry@kvr02:~$ virsh list Connecting to uri: qemu:///system <hang>
I have to ^C out of it.
If I 'force-stop' and then 'start' libvirt-bin:
garry@kvr02:~$ sudo /etc/init.d/libvirt-bin force-stop * Forcefully stopping libvirt management daemon libvirtd ...done. garry@kvr02:~$ sudo /etc/init.d/libvirt-bin start * Starting libvirt management daemon libvirtd ...done.
I can then get something:
garry@kvr02:~$ virsh list Connecting to uri: qemu:///system Id Name State ---------------------------------- 1 vm1 running 4 s3-lax running 14 freebsd-test running 19 freebsd-2 running <hang>
But it hangs after that 4th one. I must ^C it again.
If I do 'virsh list' again, it'll then show nothing (hangs like it does above).
Any suggestions?
Install the -debug packages for libvirt, and get a trace of all its threads under GDB, eg (gdb) thread apply all backtrace Also, capture traces of the client & server, by setting LIBVIRT_DEBUG=1 before running each. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Tue, Jun 23, 2009 at 09:39:25PM +0100, Daniel P. Berrange wrote:
On Tue, Jun 23, 2009 at 06:22:34AM -0700, Garry Dolley wrote:
My system:
Ubuntu Jaunty 9.04
libvirt 0.6.4 kvm 0.8.4 qemu 0.10.0
I'm not sure what triggered this, I was working with several VMs, and then found that virsh decided to hang:
garry@kvr02:~$ virsh list Connecting to uri: qemu:///system <hang>
I have to ^C out of it.
If I 'force-stop' and then 'start' libvirt-bin:
garry@kvr02:~$ sudo /etc/init.d/libvirt-bin force-stop * Forcefully stopping libvirt management daemon libvirtd ...done. garry@kvr02:~$ sudo /etc/init.d/libvirt-bin start * Starting libvirt management daemon libvirtd ...done.
I can then get something:
garry@kvr02:~$ virsh list Connecting to uri: qemu:///system Id Name State ---------------------------------- 1 vm1 running 4 s3-lax running 14 freebsd-test running 19 freebsd-2 running <hang>
But it hangs after that 4th one. I must ^C it again.
If I do 'virsh list' again, it'll then show nothing (hangs like it does above).
Any suggestions?
Install the -debug packages for libvirt, and get a trace of all its threads under GDB, eg
(gdb) thread apply all backtrace
Also, capture traces of the client & server, by setting LIBVIRT_DEBUG=1 before running each.
I solved the problem, but thanks for this info. I'll use it next time. How do I attach gdb to a running process? (I'm a gdb noob) -- Garry Dolley ARP Networks, Inc. | http://www.arpnetworks.com | (818) 206-0181 Data center, VPS, and IP Transit solutions Member Los Angeles County REACT, Unit 336 | WQGK336 Blog http://scie.nti.st
On Tue, Jun 23, 2009 at 03:28:40PM -0700, Garry Dolley wrote:
On Tue, Jun 23, 2009 at 09:39:25PM +0100, Daniel P. Berrange wrote:
Install the -debug packages for libvirt, and get a trace of all its threads under GDB, eg
(gdb) thread apply all backtrace
Also, capture traces of the client & server, by setting LIBVIRT_DEBUG=1 before running each.
I solved the problem, but thanks for this info. I'll use it next time.
How do I attach gdb to a running process? (I'm a gdb noob)
gdb path_to_the_binary (gdb) attach pid_of_the_process this will attach to the process, load the debugging informations, stop the process and get back to the gdb prompt for commands. Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
On Wed, Jun 24, 2009 at 08:17:18AM +0200, Daniel Veillard wrote:
On Tue, Jun 23, 2009 at 03:28:40PM -0700, Garry Dolley wrote:
On Tue, Jun 23, 2009 at 09:39:25PM +0100, Daniel P. Berrange wrote:
Install the -debug packages for libvirt, and get a trace of all its threads under GDB, eg
(gdb) thread apply all backtrace
Also, capture traces of the client & server, by setting LIBVIRT_DEBUG=1 before running each.
I solved the problem, but thanks for this info. I'll use it next time.
How do I attach gdb to a running process? (I'm a gdb noob)
gdb path_to_the_binary
(gdb) attach pid_of_the_process
this will attach to the process, load the debugging informations, stop the process and get back to the gdb prompt for commands.
Awesome, thanks! -- Garry Dolley ARP Networks, Inc. | http://www.arpnetworks.com | (818) 206-0181 Data center, VPS, and IP Transit solutions Member Los Angeles County REACT, Unit 336 | WQGK336 Blog http://scie.nti.st
participants (3)
-
Daniel P. Berrange -
Daniel Veillard -
Garry Dolley