Re: [libvirt] virt-top version bump in EPEL5

On Mon, Sep 06, 2010 at 07:14:47PM +0100, Richard W.M. Jones wrote: > On Mon, Sep 06, 2010 at 06:06:08PM +0100, Daniel P. Berrange wrote: > > On Mon, Sep 06, 2010 at 04:58:42PM +0100, Richard W.M. Jones wrote: > > > On Mon, Sep 06, 2010 at 05:46:03PM +0200, Wolfram Schlich wrote: > > > > * Richard W.M. Jones <rjones(a)redhat.com&gt; [2010-09-06 17:23]: > > > > > On Mon, Sep 06, 2010 at 05:06:55PM +0200, Wolfram Schlich wrote: > > > > > > We are experiencing a problem with virt-top on our RHEL5 virthosts. > > > > > > > > > > > > On a heavily loaded virthost, virt-top is damn slow -- it takes 4-5s > > > > > > to update and imposes a noticeable load on the dom0 as well > > > > > > (xenstored shows up in top eating all CPU for some seconds). > > > > > > xentop seems to impose *much* less load on the dom0. > > > > > > Is there something I can do about it? > > > > > > > > > > > > Another problem: 0.3.3.1 shows RDRQ/WRRQ/RXBY/TXBY after some seconds > > > > > > whereas 1.0.4 does not...? > > > > > > > > > > I bet both of these will be libvirt issues. > > > > > > > > > > Try running: > > > > > > > > > > virsh list --all > > > > > > > > Takes around 2-4s depending on dom0 load. > > > > > > CC-ing to libvir-list. There may be a better way now for virt-top to > > > get the list of domains, but if 'virsh list --all' is also slow, then > > > it's probably a generic libvirt problem. > > > > > > > > virsh domblkstat DomainName hda > > > > > > > > s/hda/xvda/ I guess :) > > > > This one is quite fast, below 0.2s. > > > > > > > > > virsh domifstat DomainName vnet0 > > > > > > > > This takes around 0.1s most of the time, but sometimes around 1.0s. > > > > > > > > > etc. > > > > > > > > > > If those commands (done in a suitable loop) also cause load on the > > > > > dom0, and if domblkstat/domifstat don't show stats, then it's down to > > > > > libvirt. > > > > > > > > > > virt-top is a simple little program that just exercises those libvirt > > > > > APIs ... > > > > > > > > Hmm :/ So I guess we're better off with xentop... > > > > > > We should be able to have performance very close to xentop. After > > > all, libvirt makes exactly the same direct hypervisor calls. > > > > It depends on which API calls virt-top makes. The virConnectListDomains > > goes to XenStore (fast), virDomainLookupByID/UUID go to HV (fast). The > > possibly bad ones are virConnectDefinedDomains goes to /etc/xen (fast) > > or XenD (slow) and virDomainLookupByName goes to XenD (slow). > > > > virsh list --all is slow because inactive domains requires using those > > latter two APIs which hit XenD. virt-top/xentop shouldn't care about > > inactive domains though, since they've obviously no interesting CPU > > time to report :-) > > Unfortunately we do list them :-( > > > In a perfect world you'd also want to use the lifecycle events, so you > > only need call ListDomains/ListDefinedDomains once, and then get explicit > > notifications of new virDomainPtr instances without any need todo a lookup. > > OK, we might need to move to this model. Is it workable now? In RHEL 5? Yes but the caveat is that not all libvirt drivers support events, so you need to keep the polling code too, and detect whether the driver raises a VIR_ERR_NO_SUPPORT error when registering for the lifecycle events.