On Mon, Sep 06, 2010 at 07:31:55PM +0100, Daniel P. Berrange wrote:
On Mon, Sep 06, 2010 at 07:14:47PM +0100, Richard W.M. Jones wrote:
On Mon, Sep 06, 2010 at 06:06:08PM +0100, Daniel P. Berrange wrote:
On Mon, Sep 06, 2010 at 04:58:42PM +0100, Richard W.M. Jones wrote:
On Mon, Sep 06, 2010 at 05:46:03PM +0200, Wolfram Schlich wrote:
* Richard W.M. Jones <rjones@redhat.com> [2010-09-06 17:23]:
On Mon, Sep 06, 2010 at 05:06:55PM +0200, Wolfram Schlich wrote: > We are experiencing a problem with virt-top on our RHEL5 virthosts. > > On a heavily loaded virthost, virt-top is damn slow -- it takes 4-5s > to update and imposes a noticeable load on the dom0 as well > (xenstored shows up in top eating all CPU for some seconds). > xentop seems to impose *much* less load on the dom0. > Is there something I can do about it? > > Another problem: 0.3.3.1 shows RDRQ/WRRQ/RXBY/TXBY after some seconds > whereas 1.0.4 does not...?
I bet both of these will be libvirt issues.
Try running:
virsh list --all
Takes around 2-4s depending on dom0 load.
CC-ing to libvir-list. There may be a better way now for virt-top to get the list of domains, but if 'virsh list --all' is also slow, then it's probably a generic libvirt problem.
virsh domblkstat DomainName hda
s/hda/xvda/ I guess :) This one is quite fast, below 0.2s.
virsh domifstat DomainName vnet0
This takes around 0.1s most of the time, but sometimes around 1.0s.
etc.
If those commands (done in a suitable loop) also cause load on the dom0, and if domblkstat/domifstat don't show stats, then it's down to libvirt.
virt-top is a simple little program that just exercises those libvirt APIs ...
Hmm :/ So I guess we're better off with xentop...
We should be able to have performance very close to xentop. After all, libvirt makes exactly the same direct hypervisor calls.
It depends on which API calls virt-top makes. The virConnectListDomains goes to XenStore (fast), virDomainLookupByID/UUID go to HV (fast). The possibly bad ones are virConnectDefinedDomains goes to /etc/xen (fast) or XenD (slow) and virDomainLookupByName goes to XenD (slow).
virsh list --all is slow because inactive domains requires using those latter two APIs which hit XenD. virt-top/xentop shouldn't care about inactive domains though, since they've obviously no interesting CPU time to report :-)
Unfortunately we do list them :-(
In a perfect world you'd also want to use the lifecycle events, so you only need call ListDomains/ListDefinedDomains once, and then get explicit notifications of new virDomainPtr instances without any need todo a lookup.
OK, we might need to move to this model. Is it workable now? In RHEL 5?
Yes but the caveat is that not all libvirt drivers support events, so you need to keep the polling code too, and detect whether the driver raises a VIR_ERR_NO_SUPPORT error when registering for the lifecycle events.
Thanks Dan. Wolfram, guess you should open a request for this feature at http://bugzilla.redhat.com/ if you want to see this. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones New in Fedora 11: Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 70 libraries supprt'd http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw