[libvirt] libvirtd cpu usage
hi there i'm playing on debian lenny with kvm-66/libvirt-0.4.2 . if i run top, i see libvirtd is top cpu consumer. if i connect with strace to libvirtd, i see a thousand times this message: poll([{fd=3, events=POLLIN}, {fd=6, events=POLLIN|POLLERR|POLLHUP}, {fd=7, events=POLLIN|POLLERR|POLLHUP}, {fd=8, events=POLLIN}, {fd=13, events=POLLIN|POLLERR|POLLHUP, revents=POLLHUP}, {fd=15, events=POLLIN|POLLERR|POLLHUP, revents=POLLHUP}, {fd=17, events=POLLIN|POLLERR|POLLHUP}, {fd=19, events=POLLIN|POLLERR|POLLHUP}], 8, -1) = 2 if i stop libvirtd and start the guest myself on cli, the guest is much faster. - Thomas
On Thu, 15 May 2008, Thomas Mueller wrote:
i'm playing on debian lenny with kvm-66/libvirt-0.4.2 .
if i run top, i see libvirtd is top cpu consumer. if i connect with strace to libvirtd, i see a thousand times this message:
poll([{fd=3, events=POLLIN}, {fd=6, events=POLLIN|POLLERR|POLLHUP}, {fd=7, events=POLLIN|POLLERR|POLLHUP}, {fd=8, events=POLLIN}, {fd=13, events=POLLIN|POLLERR|POLLHUP, revents=POLLHUP}, {fd=15, events=POLLIN|POLLERR|POLLHUP, revents=POLLHUP}, {fd=17, events=POLLIN|POLLERR|POLLHUP}, {fd=19, events=POLLIN|POLLERR|POLLHUP}], 8, -1) = 2
if i stop libvirtd and start the guest myself on cli, the guest is much faster.
root 21929 0.0 0.0 62584 4084 pts/2 S+ May14 0:00 libvirtd -l This is my libvirtd (cvs) nothing happens. Stefan
On Thu, May 15, 2008 at 11:47:44AM +0200, Thomas Mueller wrote:
hi there
i'm playing on debian lenny with kvm-66/libvirt-0.4.2 .
if i run top, i see libvirtd is top cpu consumer. if i connect with strace to libvirtd, i see a thousand times this message:
poll([{fd=3, events=POLLIN}, {fd=6, events=POLLIN|POLLERR|POLLHUP}, {fd=7, events=POLLIN|POLLERR|POLLHUP}, {fd=8, events=POLLIN}, {fd=13, events=POLLIN|POLLERR|POLLHUP, revents=POLLHUP}, {fd=15, events=POLLIN|POLLERR|POLLHUP, revents=POLLHUP}, {fd=17, events=POLLIN|POLLERR|POLLHUP}, {fd=19, events=POLLIN|POLLERR|POLLHUP}], 8, -1) = 2
So this is saying 2 of file descriptors have hit 'end of file' condition and for some bizarre reason libvirt is not cleaning them up. What do you do to cause trigger the high CPU usage ? Does it occurr the moment you start libvirtd ? Or only when you start a VM ? Or only when you shutdown a VM ? Or something else altogether... The output of 'lsof -p' on the libvirtd process might be helpful. You might also try running with --verbose and LIBVIRT_DEBUG=1 envirnoment variable set and capturing the output Regards, Dan. -- |: Red Hat, Engineering, Boston -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
hi daniel
i'm playing on debian lenny with kvm-66/libvirt-0.4.2 .
if i run top, i see libvirtd is top cpu consumer. if i connect with strace to libvirtd, i see a thousand times this message:
poll([{fd=3, events=POLLIN}, {fd=6, events=POLLIN|POLLERR|POLLHUP}, {fd=7, events=POLLIN|POLLERR|POLLHUP}, {fd=8, events=POLLIN}, {fd=13, events=POLLIN|POLLERR|POLLHUP, revents=POLLHUP}, {fd=15, events=POLLIN|POLLERR|POLLHUP, revents=POLLHUP}, {fd=17, events=POLLIN|POLLERR|POLLHUP}, {fd=19, events=POLLIN|POLLERR|POLLHUP}], 8, -1) = 2
So this is saying 2 of file descriptors have hit 'end of file' condition and for some bizarre reason libvirt is not cleaning them up. What do you do to cause trigger the high CPU usage ? Does it occurr the moment you start libvirtd ? Or only when you start a VM ? Or only when you shutdown a VM ? Or something else altogether...
The output of 'lsof -p' on the libvirtd process might be helpful. You might also try running with --verbose and LIBVIRT_DEBUG=1 envirnoment variable set and capturing the output
hmm.. ok, i can't reproduce it anymore. :/ but i found a second libvirtd process in defunct state. maybe this was the cause for this "bizzare reason libvirt not cleaning them up". after rebooting i tested again and libvirtd was not consuming all of the cpu... sorry for disturbing. - Thomas
On Thu, May 15, 2008 at 05:03:37PM +0200, Thomas Mueller wrote:
hi daniel
i'm playing on debian lenny with kvm-66/libvirt-0.4.2 .
if i run top, i see libvirtd is top cpu consumer. if i connect with strace to libvirtd, i see a thousand times this message:
poll([{fd=3, events=POLLIN}, {fd=6, events=POLLIN|POLLERR|POLLHUP}, {fd=7, events=POLLIN|POLLERR|POLLHUP}, {fd=8, events=POLLIN}, {fd=13, events=POLLIN|POLLERR|POLLHUP, revents=POLLHUP}, {fd=15, events=POLLIN|POLLERR|POLLHUP, revents=POLLHUP}, {fd=17, events=POLLIN|POLLERR|POLLHUP}, {fd=19, events=POLLIN|POLLERR|POLLHUP}], 8, -1) = 2
So this is saying 2 of file descriptors have hit 'end of file' condition and for some bizarre reason libvirt is not cleaning them up. What do you do to cause trigger the high CPU usage ? Does it occurr the moment you start libvirtd ? Or only when you start a VM ? Or only when you shutdown a VM ? Or something else altogether...
The output of 'lsof -p' on the libvirtd process might be helpful. You might also try running with --verbose and LIBVIRT_DEBUG=1 envirnoment variable set and capturing the output
hmm.. ok, i can't reproduce it anymore. :/ but i found a second libvirtd process in defunct state. maybe this was the cause for this "bizzare reason libvirt not cleaning them up".
after rebooting i tested again and libvirtd was not consuming all of the cpu...
Well if it hits you again let us know, because it sounds like there's a wierd edge case bug hiding in there somewhere... Dan. -- |: Red Hat, Engineering, Boston -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
participants (3)
-
Daniel P. Berrange -
Stefan de Konink -
Thomas Mueller