On Sat, Jan 05, 2008 at 01:06:52AM +0000, Daniel P. Berrange wrote:
We currently have a restriction that virConnectPtr objects must only be used by a single thread. At the same time we have a number of operations that can take a very long time. This means that applications are pretty much forced to use multi-threading if they want to continue to respond to user input, and thus are also forced to use multiple virConnectPtr objects.
This is sub-optimal because opening multiple virConnectPtr objects may well result in the user being prompted for auth credentials multiple times. It also doesn't provide any way of providing progess information on the long running jobs, nor a way to cancel them.
Thus the following series of patches introduces the concept of an 'asynchronous background job', represented by a virJobPtr object in the libvirt public API.
A job has an elapsed time, and may optionally include an estimated time of completion. If it is a job with a bounded amount of work, it will also have a percentage completion counter. There is the ability to request cancellation of an in progress job, and fetch any error associated with a completed job.
The primary reason I implemented this support, is that the storage APIs will make the current situation untennable. Allocating storage can take a seriously long time (many many minutes if cloning multi-GB files). This absolutely requires progress information and an ability to cancel an operation.
Okay, I understand the need for asynch operations. And I also agree that an API which forces to use threads in practice is a really annoying thing and should be avoided. I didn't really though about solutions for this until now, the 'job' approach is interesting, it sounds close to what OSes provides for some asynchronous I/O. I like the idea, the question is if we will provide any synchronization primitive or if the time estimate will be sufficient to handle to an application select/poll main loop.
There are a number of existing APIs which can benefit from this, hence I decided to work on this separately from the main storage APIs. The APIs which can make use of this are virDomainCreateLinux, virDomainCreate, virNetworkCreate, virNetworkCreateXML, virDomainSave, virDomainRestore, and virDomainDumpCore. For all of these we add a second variant postfixed with 'Job' in the name, returning a virJobPtr object
This work isn't finished, but I wanted to put these patches out for comment before going further. In particular I need to figure out how to hook this up to the remote driver and daemon, and deal with a few lifecycle issues. eg what todo if a virConnectPtr object is released while a background job is active.
Sounds to me that the creation of a Job should increate a reference counter for conn, which will be decreased when the Job object is destroyed. This shouldmap well with Python bindings too.
With all 5 patches applied you should be able to run the following test case:
virsh --connect test:///default save --verbose test foo
No other drivers / commands are hooked up aside from 'save' in the 'test' driver.
I will try to provide a finer grained review of your patches for Tuesday, not really urgent as this is a work in progress, right ? thanks a lot for looking at it, sounds a good idea, Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/