On Thu, Apr 13, 2006 at 06:49:33PM +0100, Daniel P. Berrange wrote: > The primary use of the virDomainInfo data is in calculating resource > utilization over time. The CPU runtime data, however, is represented > as a counter, so to calulate utilization requires some transformations. > Basically subtract the CPU runtime value for time X, from the value > at time Y, and divide the result by Y-X. Obviously this requires us to > know the times X & Y. Currently the virDomainInfo structure does not > contain this information though, so apps have to call something like > gettimeofday() before virDomainGetInfo(), to record the timestamp of each > virDomainInfo record. > > The problem is that this is a pretty poor approximation - particularly > if libvirt is talking via XenD, a non-trivial period of time can pass > between the call to gettimeofday() and the values for virDomainInfo > actually sampled. the fact that libvirt does it or the application does it generate the exact same incertitude. You won't get any added precision really... > Thus I would like to suggest that the virtDomainInfo structure be expanded > to add an extra data field of type 'struct timeval' to record a high > resolution timestamp. Now currently this wouldn't help accuracy all that > much, since libvirt would still be filling this in much the same way as > the app did. It does however, open the door to having Xen itself provide > an accurate timestamp back to libvirt, which would in turn benefit apps > without requiring further code change. In practice this forces libvir to make one extra syscall for all sampling even if the application is not interested in such a timestamp. I find this a bit dubious honnestly. It would make some sense only if the timestamp is actually added at the protocol level, best is to ask on xen-devel, if there is a positive feedback then okay why not. But to me this just won't work in the general case, if you extract the informations from the Xen Store this mean multiple RPCs to get back the informations which one should be used for the timestamp ? In practice what kind of deviation are you seeing ? If you're afraid of network induced delays ? Please explain with some concrete actual numbers because I'm not convinced myself, and it would help also convince people on xen-devel this is actually needed. Basically if doing the RPC(s) takes a millisecond, then you're in troubles anyway because that means 1ms of active CPU time, and the more you sample to get accuracy, the more you generate artefact. Adding the timestamp makes sense only if the time to process is large and you want accuracy, i.e. sampling often, but high latency, and in that case the simple fact of sampling often kills any precision you may hope to get, so my gut feeling is that this is not useful in practice, but doing so increase the cost of the operation even in the fast case or when not needed. Data would help convince me :-)

On Thu, Apr 13, 2006 at 10:37:17PM +0100, Daniel P. Berrange wrote: > > I've not really got any formal data on it at this time - it was just a random > afternoon thought. I'll see if there's any useful way to get some data on > the effects. if running as root locally with Xen, then getting the data is a simple hypercall, I would expect that to be nearly as fast as a gettimeofday(), and this won't increase precision. In the case of a non-root local process with an HTTP request to xend the time spent could potentially be quite large actually not bounded at all due to potential I/O, the quality of the data extracted then will be poor due to the tme of acquisition, would that be worth it ? The last corner case is a remote monitoring, and there the time spent is most likely to be due to network round trip, which in general is approximated by taking the medium time between emission and reception the time to do the 2 gettimeofday() are probably neglectible. So in those 3 kind of extreme scenario it's a bit unclear how adding the timestamp to the data would really help, except maybe as a convenience to the user layer. Actually getting some data about the costs of doing the call as root though the hypervisor versus the xend HTTP RPC would be an interesting datapoint in itself, I initially wanted to hack virsh to extract statistics about this but never took the time to do it :-)

On Tue, Apr 18, 2006 at 11:32:12PM +0100, Daniel P. Berrange wrote: > So, as to be expected, the XenD/XenStoreD approach has significantly higher > overhead that direct HV calls. The question is, is a x350 overhead for > unprivileged user's acceptable / can it be improved to just one order of > magnitude worse. with HTTP and no pipelining of requests it's gonna be hard to improve, but I'm afraid most of the time was spent in python code interpretation on the xend side. did you ran top at some point to check ?

> So this basic DBus service (written in Perl BTW) is approx x50 overhead > compared to HV calls, significantly better than the existing HTTP/SExpr > RPC method. It'll be interesting to see how the new XML-RPC method compares > in performance. Well it doesn't exist yet at least on the client side. There is certainly a number of optimization doable, hard to tell without a first full round trip to test.

So all options considered it seem virDomainInfo should be defined once and for all, so if we foresee more informations to be needed this should be added ASAP (not that network device informations should really be made part of a different probe call IMHO).