Re: [libvirt] [test-API] RFC: Stabilization of libvirt-test-API
On 03/29/2012 02:14 PM, Martin Kletzander wrote:
Hi everyone,
following our minutes, I'd like to start a discussion on what should be
done with libvirt-test-API so we can say it's stable and usable.
I would like to stress out that everything mentioned here is just an
opinion and I don't mean to talk down to someone as it may have seemed
earlier.
I think we should get some ideas from everyone, mostly QE as they will
be (are) the ones using this the most (if I understood correctly), and
then I'll be happy to help getting the code to the agreed status. I was
thinking about this from the wrong way probably and changing the angle
from what I look at it (and knowing there is some deadline) makes me
think of few levels of changes, which when introduced, could speed up
the test development and code understandability.
So here are the things I would like to do definitely (the optional
things follow later on):
- fix hard-coded options into real options (e.g. commit 65449e)
- fix some env_* and util* code (functions duplicated with
different
behavior)
- fix or remove harmful and pointless code (at this point, when
creating domain on remote machine, be prepared for the test to fail with
any other user then root and with root, have backup of both local and
remote '/root/.ssh' directories as the contents will be erased!)
- fix method names for the {connect,domain,etc.}API (get_host_name vs.
lookupByUUID etc.)
The optional things:
- get rid of classes in lib and make just few utility functions
covering *only* the methods that do something else than call the same
method in underlying class from the libvirt module.
Apart from actualy enabling all
functionality provided by the libvirt
python api this would actually increase the object orientation of the
code. The current api breaks the idea of objects in some places: eg.
looks up a guest by name, uses the domain object to call one API and
discards it instead of re-using it for further calls that have to look
the domain up again.
- get rid of the new exception (I don't see any other
difference than
in the name, which can make a difference in "except:" clause, but it's
converted everywhere)
Other useful thing would be to improve exception handling to
free the
test writer from handling exception that actualy signal that an error
has occured and free him from having to catch the exception and print
the error message in a nice way (not to have to read a backtrace ).
(On the other hand, handling exceptions will be needed if an error
actually should happen as a result of the test)
- be able to share variables between tests (connection object and
anything else)
This would enable to write really simple test cases that would not
require to create a separate hypervisor connection and to the complete
test separately, but you could combine these simple test cases into complex
- introduce new config file for tests (.ini format, can be parsed
by
ConfigParser, same as env.cfg, define variables used throughout the test
specifications
- update the documentation
This might speed up new
deployments of the test suite as some filenames
and other details have changed so the users have to figure them out by
themselves.
- use some python code style (PEP-8?), make use of True/False,
None
- eliminate duplicated (and x-plicated) code (append_path in all the
files, etc.)
Agreed.
I have all of these figured out, so I'm willing to discuss all of them,
but in most cases changing it in the current code seems very
time-consumable to me.
Writing a test now requires some redundant work to be done.
A common
test case (written in python) requires that the writer creates a
hypervisor connection, gets the domain object and then does all the
testing. This could be minimized, when these common tasks would have
utility tests (eg a test that just connects to the hypervisor and
returns the object) and then you'd just combine them in the test case file.
Please, feel free to comment on any of these, add yours, discuss, shout
at me, etc. =)
Regards,
Martin
Peter