Hi, There is a quite annoying thing, don`t sure if I can call it a bug. How to reproduce: - start a bunch of VMs, - stop libvirt, - start libvirt and immediately issue any call, say, 'virsh version', - delay until call completion may be fitted nearly as quadratic function from number of running VMs, qemu-kvm in my case (see link below). After this delay passed, any calls executed without slowdown, so problem is only a 'dead gap' after restarting service.

On Mon, Sep 24, 2012 at 01:35:21PM +0100, Daniel P. Berrange wrote: > On Mon, Sep 24, 2012 at 11:48:44AM +0400, Andrey Korolyov wrote: > > Hi, > > > > There is a quite annoying thing, don`t sure if I can call it a bug. > > > > How to reproduce: > > > > - start a bunch of VMs, > > - stop libvirt, > > - start libvirt and immediately issue any call, say, 'virsh version', > > - delay until call completion may be fitted nearly as quadratic > > function from number of running VMs, qemu-kvm in my case (see link > > below). After this delay passed, any calls executed without slowdown, > > so problem is only a 'dead gap' after restarting service. > > When you restart the libvirtd service, while KVM guests are running, > the first thing libvirtd needs todo is to connect to the QEMU monitor > for each guest and figure out its status. This can take some time > depending on how loaded the host is in general. It will definitely > slow libvirt API calls which need to query individual VMs, but I'm > rather surprised that you saw any slowdown of the 'virsh version' > command, since that should not touch VMs at all. So I think I *would* > class this as a bug. It could be something silly like the main I/O > event loop having some badly written bit of non-scalable code. Please > do file a bug about this, providing as much raw data as you have > managed to collect. Hum, IIRC there was a time where libvirtd would not process any request until those reconnections were done, did that change ?

On Mon, Sep 24, 2012 at 4:35 PM, Daniel P. Berrange <berrange(a)redhat.com&gt; wrote: > On Mon, Sep 24, 2012 at 11:48:44AM +0400, Andrey Korolyov wrote: >> Hi, >> >> There is a quite annoying thing, don`t sure if I can call it a bug. >> >> How to reproduce: >> >> - start a bunch of VMs, >> - stop libvirt, >> - start libvirt and immediately issue any call, say, 'virsh version', >> - delay until call completion may be fitted nearly as quadratic >> function from number of running VMs, qemu-kvm in my case (see link >> below). After this delay passed, any calls executed without slowdown, >> so problem is only a 'dead gap' after restarting service. > > When you restart the libvirtd service, while KVM guests are running, > the first thing libvirtd needs todo is to connect to the QEMU monitor > for each guest and figure out its status. This can take some time > depending on how loaded the host is in general. It will definitely > slow libvirt API calls which need to query individual VMs, but I'm > rather surprised that you saw any slowdown of the 'virsh version' > command, since that should not touch VMs at all. So I think I *would* > class this as a bug. It could be something silly like the main I/O > event loop having some badly written bit of non-scalable code. Please > do file a bug about this, providing as much raw data as you have > managed to collect. > Here is a report, please take a look. https://bugzilla.redhat.com/show_bug.cgi?id=860053