[libvirt] [libvirt-test-API PATCH 2/2] Documentation: Update to current information

Update of all the things found to be outdated. The test described in the chapter "Writing a test case" was changed to simpler one, with only one source file included. It shows how to use the libvirt API (no need for connectAPI etc. anymore) and mainly libvirt test API. --- .../en-US/Understanding_libvirt-test-API.xml | 6 - .../en-US/Using_libvirt-test-API.xml | 26 +- .../en-US/Writing_a_test_case.xml | 325 ++++++++------------ .../libvirt-test-API_Guide/en-US/extras/log.txt | 53 ++-- 4 files changed, 164 insertions(+), 246 deletions(-) diff --git a/docs/User_Guide/libvirt-test-API_Guide/en-US/Understanding_libvirt-test-API.xml b/docs/User_Guide/libvirt-test-API_Guide/en-US/Understanding_libvirt-test-API.xml index 7fe1e97..446e9e2 100644 --- a/docs/User_Guide/libvirt-test-API_Guide/en-US/Understanding_libvirt-test-API.xml +++ b/docs/User_Guide/libvirt-test-API_Guide/en-US/Understanding_libvirt-test-API.xml @@ -186,12 +186,6 @@ repos </blockquote> - <bridgehead renderas="sect3">/lib</bridgehead> - <blockquote> - <para><filename>/lib</filename> is a directory that contains the basic subroutines that call the API functions of libvirt bindings languages to form basic unit classes. The test case initiates the first action by calling these subroutines.</para> - </blockquote> - - <bridgehead renderas="sect3">/utils</bridgehead> <blockquote> <para><filename>/utils</filename> is a directory which contains various scripts to assist in creating and verifying test cases.</para> diff --git a/docs/User_Guide/libvirt-test-API_Guide/en-US/Using_libvirt-test-API.xml b/docs/User_Guide/libvirt-test-API_Guide/en-US/Using_libvirt-test-API.xml index 65653e3..7fa14f5 100644 --- a/docs/User_Guide/libvirt-test-API_Guide/en-US/Using_libvirt-test-API.xml +++ b/docs/User_Guide/libvirt-test-API_Guide/en-US/Using_libvirt-test-API.xml @@ -13,21 +13,25 @@ <section id="Using_libvirt-test-API-Running_a_test"> <title>Running a test</title> - <para>The <filename>main.py</filename> file, located in the root directory of the test tool, is the file that runs the test and performs other operations in libvirt-test-API.</para> + <para>The <filename>libvirt-test-api.py</filename> file, located in the root directory of the test tool, is the file that runs the test and performs other operations in libvirt-test-API.</para> <para>To run a test, from the libvirt-test-API root directory enter:</para> <screen> -# python main.py +# python libvirt-test-api.py </screen> <para>The following switches can be used with the above command:</para> <example> - <title>python main.py switches</title> + <title>python libvirt-test-api.py options</title> <screen> --C, --configfile Specify the configuration file to parse, default is 'case.conf'. --D, --delete-log Delete a log item. --H, --help Display the command usage. --L, --xmlfile Specify the name of the log file, default is 'log.xml'. +-h, --help : Display usage information +-c, --casefile: Specify configuration file +-t, --template: Print testcase config file template +-f, --logxml: Specify log file with type xml +-l, --log-level: 0 or 1 currently +-d, --delete-log: Delete log items +-m, --merge: Merge two log xmlfiles +-r, --rerun: Rerun one or more test </screen> </example> @@ -53,25 +57,25 @@ <bridgehead renderas="sect3">Deleting log information</bridgehead> - <para>The switch <command>-D, --delete-log</command> in the <command>python main.py</command> command can be used to delete a test item, a test run, or all test runs in a log file.</para> + <para>The switch <command>-d, --delete-log</command> in the <command>python libvirt-test-api.py</command> command can be used to delete a test item, a test run, or all test runs in a log file.</para> <para>For example:</para> <itemizedlist> <listitem> <para>To delete the test item (testid) in the test run (testrunid):</para> <screen> -# python main.py -D log.xml testrunid testid +# python libvirt-test-api.py -d log.xml testrunid testid </screen> </listitem> <listitem> <para>To delete the test run (testrunid):</para> <screen> -# python main.py -D log.xml testrunid +# python libvirt-test-api.py -d log.xml testrunid </screen> </listitem> <listitem> <para>To delete all test runs:</para> <screen> -# python main.py -D log.xml all +# python libvirt-test-api.py -d log.xml all </screen> </listitem> </itemizedlist> diff --git a/docs/User_Guide/libvirt-test-API_Guide/en-US/Writing_a_test_case.xml b/docs/User_Guide/libvirt-test-API_Guide/en-US/Writing_a_test_case.xml index 3dd714e..49ad0b5 100644 --- a/docs/User_Guide/libvirt-test-API_Guide/en-US/Writing_a_test_case.xml +++ b/docs/User_Guide/libvirt-test-API_Guide/en-US/Writing_a_test_case.xml @@ -7,7 +7,7 @@ <para>Test cases form the foundation of the test tool. They are the building blocks from which test runs can be created. Test cases are written as Python scripts and define the functions that can be tested. Separating test cases by function allows test cases to be re-used and combined in the configuration file to create the desired test run.</para> - <para>The following section provides details on the test case file structure that must be used in order to be correctly accepted by the test tool.</para> + <para>The following section shows a simple screenshot test with its details on the test case file structure that must be used in order to be correctly accepted by the test tool.</para> <section id="Writing_a_test_case-Test_case_file_structure"> @@ -16,14 +16,14 @@ <para><application>Filename</application></para> <blockquote> <para>The name of the test case file must be the same as the name of the main function inside it.</para> - <para>For example, if the main function is <application>install_guest()</application>, then test case file must be named <filename>install_guest.py</filename>.</para> + <para>For example, if the main function is <application>screenshot()</application>, then test case file must be named <filename>screenshot.py</filename>.</para> <para>Save the test case to its corresponding directory in <filename>/repos</filename>. For example, if the test case is related to domain then save the file in the <filename>/repos/domain</filename> directory.</para> </blockquote> <para><application>Function</application></para> <blockquote> - <para>The function requires a dictionary (dict) as the argument. The dict uses the function arguments defined in the configuration file.</para> - <para>To write log information to a file, the function also requires a key value pair with the keyname 'logger' and the value is a log object.</para> + <para>The function requires a dictionary (dict) as the argument. The dictionary includes function arguments defined in the configuration file.</para> + <para>To write log information to a file, the function also requires a logger saved in the dictionary under key 'logger'.</para> </blockquote> <para><application>Return code</application></para> @@ -39,28 +39,95 @@ </itemizedlist> </blockquote> + <para><application>Parameters</application></para> + <blockquote> + <para>There are two global variables needed for the test case to be valid:</para> + <itemizedlist> + <listitem> + <para><application>required_params</application> - a tuple of all required parameters</para> + </listitem> + <listitem> + <para><application>optional_params</application> - a tuple of all optional parameters</para> + </listitem> + </itemizedlist> + <para>Thanks to these variables, all the parameters passed to the test are checked prior to running the test in order to make the test writing simpler.</para> + </blockquote> + <example> <title>Test case file</title> <programlisting> -# install_guest.py -import time -import sys -import os +#!/usr/bin/env python +# To test domain screenshot, the screenshot format is +# hypervisor specific. -def install_guest(dict): - logger = dict['logger'] - print "this is from testcase_repos:domain" - for eachvargs in dict.keys(): - logger.info("the argu is %s" % eachvargs) - time.sleep(1) - logger.info("the corresponding value is %s" % dict[eachvargs]) - logger.info("I am from install_guest log info") - logger.debug("I am from install_guest log debug") - logger.warning("I am from install_guest log warning") - logger.error("I am from install_guest log error") - logger.critical("I am from install_guest log critical") - return 0 +import os +import mimetypes +import libvirt + +# "sharedmod" is needed in order to get the shared connection object +import sharedmod + +# These variables are mandatory for running the test +# The test won't be run unless both "guestname" and "filename" are specified. +required_params = ('guestname', 'filename',) +optional_params = ('screen',) + +# In this filename we store the last filename that has a screenshot +# saved inside in order to be able to clean it after +filename = None + +# This is just a helper function that will be used later. +def saver(stream, data, file_): + return file_.write(data) + +# This is the main test +def screenshot(params): + """This method takes a screenshot of a running machine and saves + it in a filename""" + + # Shared logger to be used for messages output + logger = params['logger'] + + # Shared connection object to be used for messages output + conn = sharedmod.libvirtobj['conn'] + guestname = params['guestname'] + + logger.info('Looking for domain %s' % guestname) + dom = conn.lookupByName(guestname) + logger.debug('Domain %s found' % guestname) + + st = conn.newStream(0) + screen = params.get('screen', 0) + logger.info('Taking a screenshot') + mime = dom.screenshot(st, int(screen), 0) + logger.debug('Screenshot taken') + + # As mentioned before, the screenshot format is + # hypervisor-specific + ext = mimetypes.guess_extension(mime) or '.ppm' + filename = params['filename'] + ext + + # Here we create a file object used later + f = file(filename, 'w') + + logger.info('Saving screenshot into %s' % filename) + logger.info('Mimetype of the file is %s' % mime) + # Here is the use of the "saver" function + st.recvAll(saver, f) + logger.debug('Screenshot saved') + + # The test fails (return is different from 0) if the stream is not + # properly ended + return st.finish() + +# This cleanup function is called if specified by the configuration +# file. In this file we remove the saved screenshot if the filename is +# known. +def cleanup(params): + if last_filename: + os.remove(last_filename) </programlisting> +<para>Notice that for working with libvirt, the default <application>libvirt</application> module is used with all of its functionality.</para> </example> </section><!--End Test case file structure--> @@ -72,212 +139,68 @@ def install_guest(dict): <para>In this example, the test objective is to:</para> <orderedlist> <listitem> - <para>Create a new NFS based storage pool and create a volume.</para> + <para>Start a guest and wait 3 seconds</para> + </listitem> + <listitem> + <para>Make a screenshot of its screen</para> </listitem> <listitem> - <para>Install a new rhel5u4 guest machine on the volume with multiprocess disabled and repeat the test once.</para> + <para>Destroy the guest</para> </listitem> </orderedlist> - <para>To achieve the test objective, two test cases are required:</para> + <para>To achieve the test objective, three test cases are required:</para> <orderedlist> <listitem> - <para>Initialize the NFS based storage pool, the information of nfsserver is defined in the env.ini file.</para> - <para>The storage test case is called 'initialize_storage' and is located in <filename>/repos/storage/initialize_storage.py</filename></para> + <para>Start of the domain is in file <filename>/repos/domain/start.py</filename>, its parameters are <application>guestname</application> and optional <application>flags</application>. Without <application>flags</application> the testcase tries to ping the host and thanks to that it waits until (at least) the network on the host is up. Because we want a screenshot of the host while booting, we will disable this by adding the <application>noping</application> flag.</para> + </listitem> + <listitem> + <para>The test taking a screenshot of the domain, defined in file <filename>/repos/domain/screenshot.py</filename>, with its parameters <application>guestname</application>, <application>filename</application> and optional <application>screen</application>.</para> </listitem> <listitem> - <para>Install the guest on the volume.</para> - <para>The install test case is called 'domain_install' and is located in <filename>/repos/domain/install_guest.py</filename></para> + <para>Domain destruction is done by test from file <filename>/repos/domain/destroy.py</filename> and its parameters are the same as with the first test case.</para> </listitem> </orderedlist> - <para>The two test case are independent of each other, which allows the test cases to be re-used and combined with other test cases to create different test runs.</para> + <para>Waiting can be specified without a test as it is handled by the framework.</para> + <para>The three test cases are independent of each other. That allows the test cases to be re-used and combined with each other in order to create different test runs.</para> - <para>Below is the configuration file, which includes the two test cases.</para> + <para>The next thing needed is a configuration file with all three test cases included.</para> <example> <title>Configuration file <filename>case.conf</filename></title> <programlisting> -storage:initialize_storage - poolname - nfspool - pooltype - netfs - volname - rhel5u4.img - volformat - raw - -domain:install_guest +domain:start guestname - rhel5u4 - guesttype - xenfv - memory - 1048576 - vcpu - 1 - -options multiprocess=disable times=1 -</programlisting> - </example> + $defaultname + flags + noping - <para>Below are the two test case scripts.</para> - <example> - <title>Initialize storage test case <filename>/repos/storage/initialize_storage.py</filename></title> -<programlisting> -#!/usr/bin/env python +sleep 3 -import sys -import os -import time -import copy -import shutil -import urllib -import commands - -dir = os.path.dirname(sys.modules[__name__].__file__) -absdir = os.path.abspath(dir) -rootdir = os.path.split(os.path.split(absdir)[0])[0] -sys.path.append(rootdir) - -import exception -from lib import connectAPI -from lib import storageAPI -from utils import env_parser -from utils import xmlbuilder - -envfile = 'env.ini' - -def initialize_storage(dict): - logger = dict['logger'] - dict['hypertype'] = 'xen' - envparser = env_parser.Envpaser(envfile) - dict['sourcename'] = envparser.get_value('storage', 'sourcename') - dict['sourcepath'] = envparser.get_value('storage', 'sourcepath') - - logger.info('prepare create storage pool') - xmlobj = xmlbuilder.XmlBuilder() - poolxml = xmlobj.build_pool(dict) - logger.debug('dump create storage pool xml:\n%s' %poolxml) - - conn = connectAPI.ConnectAPI.open("xen:///") - stgobj = storageAPI.StorageAPI(conn) - - logger.info('list current storage pool: %s' %stgobj.storage_pool_list()) - logger.info('define pool from xml description') - stgobj.define_storage_pool(poolxml) - - logger.info('active storage pool') - stgobj.active_storage_pool(dict['poolname']) - logger.info('list current storage pool: %s' %stgobj.storage_pool_list()) - - volxml = xmlobj.build_volume(dict) - logger.debug('dump create storage volume xml:\n%s' %volxml) - logger.info('prepare create storage volume') - stgobj.create_storage_volume(dict['poolname'], volxml) - logger.info('list current storage volume: %s' %stgobj.get_volume_list(dict['poolname'])) - - return 0 -</programlisting> - </example> +domain:screenshot + guestname + $defaultname + filename + screenshot - <example> - <title>Install guest test case <filename>/repos/domain/install_guest.py</filename></title> -<programlisting> -#!/usr/bin/env python +domain:destroy + guestname + $defaultname + flags + noping -import sys -import os -import time -import copy -import shutil -import urllib -import commands - -dir = os.path.dirname(sys.modules[__name__].__file__) -absdir = os.path.abspath(dir) -rootdir = os.path.split(os.path.split(absdir)[0])[0] -sys.path.append(rootdir) - -import exception -from lib import connectAPI -from lib import domainAPI -from utils import env_parser -from utils import xmlbuilder - -envfile = 'env.ini' - -def prepare_cdrom(*args): - url, ks, gname, logger = args - ks_name = os.path.basename(ks) - - new_dir = os.path.join('/tmp', gname) - os.makedirs(new_dir) - - boot_path = os.path.join(url, 'images/boot.iso') - boot_iso = urllib.urlretrieve(boot_path, 'boot.iso')[0] - time.sleep(10) - shutil.move(boot_iso, new_dir) - - ks_file = urllib.urlretrieve(ks, ks_name)[0] - shutil.move(ks_file, new_dir) - - shutil.copy('utils/ksiso.sh', new_dir) - src_path = os.getcwd() - os.chdir(new_dir) - shell_cmd = 'sh ksiso.sh %s' %ks_file - (status, text) = commands.getstatusoutput(shell_cmd) - if status != 0: - logger.debug(text) - os.chdir(src_path) - -def install_guest(dict): - logger = dict['logger'] - gname = dict['guestname'] - dict['ifacetype'] = 'bridge' - dict['bridge'] = 'xenbr0' - dict['bootcd'] = '/tmp/%s/custom.iso' %gname - - logger.info('get system environment information') - envparser = env_parser.Envpaser(envfile) - url = envparser.get_value("guest", gname + "src") - dict['kickstart'] = envparser.get_value("guest", gname + "ks") - logger.debug('install source: \n %s' %url) - logger.debug('kisckstart file: \n %s' %dict['kickstart']) - - logger.info('prepare installation booting cdrom') - prepare_cdrom(url, dict['kickstart'], gname, logger) - - xmlobj = xmlbuilder.XmlBuilder() - guestinstxml = xmlobj.build_domain_install(dict) - logger.debug('dump installation guest xml:\n%s' %guestinstxml) - - conn = connectAPI.ConnectAPI.open("xen:///") - domobj = domainAPI.DomainAPI(conn) - logger.info('define guest from xml description') - domobj.define_domain(guestinstxml) - - logger.info('start installation guest ...') - domobj.start_domain(gname) - - state = domobj.get_domain_state(gname) - logger.debug('current guest state: %s' %state) +options multiprocess=disable times=1 </programlisting> </example> - <para>Below is the environment configuration file.</para> + <para>Below is part of the environment configuration file.</para> <example> - <title>Environment configuration file <filename>env.ini</filename></title> + <title>Environment configuration file <filename>env.cfg</filename></title> <programlisting> -[guest] -rhel5u4src = http://redhat.com/pub/rhel/rel-eng/RHEL5.4-Server-latest/tree-x86_64 -rhel5u4ks = http://10.00.00.01/ks-rhel-5.4-x86_64-noxen-smp-minimal.cfg - -[storage] -sourcename = 10.00.00.02 -sourcepath = /media/share +[variables] +defaultname = fedora </programlisting> </example> - <para>Below is the log file that is generated. It is stored in <filename>/log</filename>.</para> + <para>Below is the output of the test, more information can be found in the log that is stored in <filename>/log</filename> directory.</para> <example> <title>Test case log file</title> <programlisting> @@ -285,7 +208,7 @@ sourcepath = /media/share </programlisting> </example> - <para>This example shows how simple it is to create two test cases to achieve a complex test. The advantage of combining test cases to produce a complex test is that the test cases can be used repeatedly across other tests.</para> + <para>This example shows how simple it is to create a complete test by combining few test cases.</para> </section><!-- End Test case example--> diff --git a/docs/User_Guide/libvirt-test-API_Guide/en-US/extras/log.txt b/docs/User_Guide/libvirt-test-API_Guide/en-US/extras/log.txt index 682aa6b..5edd384 100644 --- a/docs/User_Guide/libvirt-test-API_Guide/en-US/extras/log.txt +++ b/docs/User_Guide/libvirt-test-API_Guide/en-US/extras/log.txt @@ -1,31 +1,28 @@ +Start Testing: + Case Count: 4 + Log File: log/20120416154731/libvirt_test001 ---------------- initialize_storage --------------- -[2009-09-18 15:38:50] 5141 INFO (initialize_storage:31) prepare create storage pool -[2009-09-18 15:38:50] 5141 DEBUG (initialize_storage:34) dump create storage pool xml: -<?xml version="1.0" ?> -<pool type="netfs"><name>nfspool</name><source><host name="10.66.70.85"/><dir path="/media/share"/></source><target><path>/var/lib/xen/images</path></target></pool> -[2009-09-18 15:38:50] 5141 INFO (initialize_storage:39) list current storage pool: [] -[2009-09-18 15:38:50] 5141 INFO (initialize_storage:40) define pool from xml description -[2009-09-18 15:38:50] 5141 INFO (initialize_storage:43) active storage pool -[2009-09-18 15:38:50] 5141 INFO (initialize_storage:45) list current storage pool: ['nfspool'] -[2009-09-18 15:38:50] 5141 DEBUG (initialize_storage:48) dump create storage volume xml: -<?xml version="1.0" ?> -<volume><name>rhel5u4.img</name><capacity unit="G">10</capacity><allocation>0</allocation><target><path>/var/lib/xen/images</path><format type="raw"/></target></volume> -[2009-09-18 15:38:50] 5141 INFO (initialize_storage:49) prepare create storage volume -[2009-09-18 15:38:50] 5141 INFO (initialize_storage:51) list current storage volume: ['rhel5u4.img'] -------------- initialize_storage pass ------------- + domain:start + 15:47:32|INFO |start domain + 15:47:32|INFO |Guest started + 15:47:32|INFO |PASS + Result: OK + sleep + 15:47:32|INFO |sleep 3 seconds + Result: OK ------------------- install_guest ------------------ -[2009-09-18 15:38:50] 5141 INFO (install_guest:55) get system environment information -[2009-09-18 15:38:50] 5141 DEBUG (install_guest:59) install source: - http://download.redhat.com/pub/rhel/rel-eng/RHEL5.4-Server-latest/tree-x8... -[2009-09-18 15:38:50] 5141 DEBUG (install_guest:60) kisckstart file: - http://10.00.00.01/ks-rhel-5.4-x86_64-noxen-smp-minimal.cfg -[2009-09-18 15:38:50] 5141 INFO (install_guest:62) prepare installation booting cdrom -[2009-09-18 15:39:01] 5141 DEBUG (install_guest:67) dump installation guest xml: -<?xml version="1.0" ?> -<domain type="xen"><name>rhel5u4</name><memory>1048576</memory><vcpu>1</vcpu><os><type>hvm</type><loader>/usr/lib/xen/boot/hvmloader</loader><boot dev="cdrom"/></os><features><acpi/><apic/><pae/></features><clock offset="utc"/><on_poweroff>destroy</on_poweroff><on_reboot>restart</on_reboot><on_crash>restart</on_crash><devices><emulator>/usr/lib64/xen/bin/qemu-dm</emulator><disk device="disk" type="file"><driver name="file"/><source file="/var/lib/xen/images/rhel5u4.img"/><target bus="ide" dev="hda"/></disk><disk device="cdrom" type="file"><driver name="file"/><source file="/tmp/rhel5u4/custom.iso"/><target bus="ide" dev="hdc"/><readonly/></disk><interface type="bridge"><source bridge="xenbr0"/><script path="vif-bridge"/></interface><console/><input bus="ps2" type="mouse"/><graphics keymap="en-us" port="-1" type="vnc"/></devices></domain> -[2009-09-18 15:39:01] 5141 INFO (install_guest:71) define guest from xml description -[2009-09-18 15:39:01] 5141 INFO (install_guest:74) start installation guest ... -[2009-09-18 15:39:02] 5141 DEBUG (install_guest:78) current guest state: running + domain:screenshot + 15:47:35|INFO |Looking for domain fedora + 15:47:35|INFO |Taking a screenshot + 15:47:36|INFO |Saving screenshot into screenshot.ppm + 15:47:36|INFO |Mimetype of the file is image/x-portable-pixmap + Result: OK + + domain:destroy + 15:47:36|INFO |destroy domain + Result: OK + + +Summary: + Total:4 [Pass:4 Fail:0] -- 1.7.8.5