November 2010 - Devel - libvirt List Archives

[libvirt] [PATCH] docs: removed outdated reference to virt-mem
by Justin Clift 22 Nov '10

22 Nov '10

The virt-mem program is no longer shipped, but was still being referenced at the bottom of the virsh and libvirtd man pages. This patch removes it from those man pages, addressing BZ# 639603: https://bugzilla.redhat.com/show_bug.cgi?id=639603 --- daemon/libvirtd.pod.in | 2 +- tools/virsh.pod | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/daemon/libvirtd.pod.in b/daemon/libvirtd.pod.in index 04a4cb4..6e699b8 100644 --- a/daemon/libvirtd.pod.in +++ b/daemon/libvirtd.pod.in @@ -163,6 +163,6 @@ PURPOSE =head1 SEE ALSO L<virsh(1)>, L<virt-install(1)>, L<virt-xml-validate(1)>, L<virt-top(1)>, -L<virt-mem(1)>, L<virt-df(1)>, L<http://www.libvirt.org/> +L<virt-df(1)>, L<http://www.libvirt.org/> =cut diff --git a/tools/virsh.pod b/tools/virsh.pod index 5a96116..c97786a 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -1259,6 +1259,7 @@ PURPOSE =head1 SEE ALSO -L<virt-install(1)>, L<virt-xml-validate(1)>, L<virt-top(1)>, L<virt-mem(1)>, L<virt-df(1)>, L<http://www.libvirt.org/> +L<virt-install(1)>, L<virt-xml-validate(1)>, L<virt-top(1)>, L<virt-df(1)>, +L<http://www.libvirt.org/> =cut -- 1.7.3.2

2 2

[libvirt] arbitrary qemu command feature
by Bruce Hohl 22 Nov '10

22 Nov '10

Hello List, I had a problem with the recently added qemu command feature (no one on user list could help): This feature: http://libvirt.org/news.html 0.8.3: Aug 4 2010 Improvements: Handle arbitrary qemu command-lines in qemuParseCommandLine. Qemu arbitrary command-line arguments. Also described here: http://www.spinics.net/linux/fedora/libvir/msg26687.html [PATCH v4 0/8]: Add arbitrary qemu command-line and monitor commands I am using the following libvirt 0.8.3 packages from unbuntu 10.10: libvirt-bin 0.8.3-1ubuntu14 libvirt0 0.8.3-1ubuntu14 python-libvirt 0.8.3-1ubuntu14 Example of problem: "$ virsh edit my.xml" refuses to add / will not save the following to a domain (for this domain graphics type='sdl') using the specified format: <qemu:commandline> <qemu:arg value='-ctrl-grab'/> </qemu:commandline> If I edit and save the xml directly the change is not recognized upon start-up of the domain. That is, I must use ctrl+alt instead of the right ctrl for mouse capture. Am I missing a library or package needed? Am I using this feature correctly? Is there a working example or additional documentation somewhere? Any suggestions?

2 1

[libvirt] [PATCH] support sheepdog volumes
by MORITA Kazutaka 21 Nov '10

21 Nov '10

Sheepdog is a distributed storage system for QEMU. It provides highly available block level storage volumes to VMs like Amazon EBS. This patch adds support for Sheepdog; we can create VMs with sheepdog volumes, and attach sheepdog volumes to running machines via the attach-device command. Sheepdog volumes can be declared like this: <disk type='sheepdog' device='disk'> <driver name='qemu' type='raw' /> <source vdi='volume_name' host='hostname' port='7000'/> <target dev='vda' bus='virtio' /> </disk> 'host' and 'port' in the source element are optional. If they are not specified, sheepdog clients use the default value (localhost:7000). Signed-off-by: MORITA Kazutaka <morita.kazutaka(a)lab.ntt.co.jp> --- docs/schemas/domain.rng | 24 ++++++++++++++++++ src/conf/domain_conf.c | 26 ++++++++++++++++++- src/conf/domain_conf.h | 3 ++ src/qemu/qemu_conf.c | 63 +++++++++++++++++++++++++++++++++++++++++++++- 4 files changed, 113 insertions(+), 3 deletions(-) diff --git a/docs/schemas/domain.rng b/docs/schemas/domain.rng index bbbc846..5802993 100644 --- a/docs/schemas/domain.rng +++ b/docs/schemas/domain.rng @@ -612,6 +612,30 @@ <ref name="diskspec"/> </interleave> </group> + <group> + <attribute name="type"> + <value>sheepdog</value> + </attribute> + <interleave> + <optional> + <element name="source"> + <attribute name="vdi"> + <ref name="genericName"/> + </attribute> + <optional> + <attribute name="host"> + <ref name="genericName"/> + </attribute> + <attribute name="port"> + <ref name="unsignedInt"/> + </attribute> + </optional> + <empty/> + </element> + </optional> + <ref name="diskspec"/> + </interleave> + </group> <ref name="diskspec"/> </choice> </element> diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 30c27db..d0ea0bb 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -113,7 +113,8 @@ VIR_ENUM_IMPL(virDomainDeviceAddress, VIR_DOMAIN_DEVICE_ADDRESS_TYPE_LAST, VIR_ENUM_IMPL(virDomainDisk, VIR_DOMAIN_DISK_TYPE_LAST, "block", "file", - "dir") + "dir", + "sheepdog") VIR_ENUM_IMPL(virDomainDiskDevice, VIR_DOMAIN_DISK_DEVICE_LAST, "disk", @@ -505,6 +506,8 @@ void virDomainDiskDefFree(virDomainDiskDefPtr def) VIR_FREE(def->serial); VIR_FREE(def->src); + VIR_FREE(def->host); + VIR_FREE(def->port); VIR_FREE(def->dst); VIR_FREE(def->driverName); VIR_FREE(def->driverType); @@ -1517,6 +1520,8 @@ virDomainDiskDefParseXML(virCapsPtr caps, char *driverName = NULL; char *driverType = NULL; char *source = NULL; + char *host = NULL; + char *port = NULL; char *target = NULL; char *bus = NULL; char *cachetag = NULL; @@ -1557,6 +1562,11 @@ virDomainDiskDefParseXML(virCapsPtr caps, case VIR_DOMAIN_DISK_TYPE_DIR: source = virXMLPropString(cur, "dir"); break; + case VIR_DOMAIN_DISK_TYPE_SHEEPDOG: + source = virXMLPropString(cur, "vdi"); + host = virXMLPropString(cur, "host"); + port = virXMLPropString(cur, "port"); + break; default: virDomainReportError(VIR_ERR_INTERNAL_ERROR, _("unexpected disk type %s"), @@ -1726,6 +1736,10 @@ virDomainDiskDefParseXML(virCapsPtr caps, def->src = source; source = NULL; + def->host = host; + host = NULL; + def->port = port; + port = NULL; def->dst = target; target = NULL; def->driverName = driverName; @@ -1756,6 +1770,8 @@ cleanup: VIR_FREE(type); VIR_FREE(target); VIR_FREE(source); + VIR_FREE(host); + VIR_FREE(port); VIR_FREE(device); VIR_FREE(driverType); VIR_FREE(driverName); @@ -5831,6 +5847,14 @@ virDomainDiskDefFormat(virBufferPtr buf, virBufferEscapeString(buf, " <source dir='%s'/>\n", def->src); break; + case VIR_DOMAIN_DISK_TYPE_SHEEPDOG: + virBufferEscapeString(buf, " <source vdi='%s'", def->src); + if (def->host && def->port) { + virBufferEscapeString(buf, " host='%s'", def->host); + virBufferEscapeString(buf, " port='%s'", def->port); + } + virBufferVSprintf(buf, "/>\n"); + break; default: virDomainReportError(VIR_ERR_INTERNAL_ERROR, _("unexpected disk type %s"), diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 7d2d6f5..9c929c4 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -120,6 +120,7 @@ enum virDomainDiskType { VIR_DOMAIN_DISK_TYPE_BLOCK, VIR_DOMAIN_DISK_TYPE_FILE, VIR_DOMAIN_DISK_TYPE_DIR, + VIR_DOMAIN_DISK_TYPE_SHEEPDOG, VIR_DOMAIN_DISK_TYPE_LAST }; @@ -172,6 +173,8 @@ struct _virDomainDiskDef { int device; int bus; char *src; + char *host; + char *port; char *dst; char *driverName; char *driverType; diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index 83a117a..84f55cb 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -2731,6 +2731,11 @@ qemuBuildDriveStr(virDomainDiskDefPtr disk, virBufferVSprintf(&opt, "file=fat:floppy:%s,", disk->src); else virBufferVSprintf(&opt, "file=fat:%s,", disk->src); + } else if (disk->type == VIR_DOMAIN_DISK_TYPE_SHEEPDOG) { + if (disk->host && disk->port) + virBufferVSprintf(&opt, "file=sheepdog:%s:%s:%s,", disk->host, disk->port, disk->src); + else + virBufferVSprintf(&opt, "file=sheepdog:%s,", disk->src); } else { virBufferVSprintf(&opt, "file=%s,", disk->src); } @@ -4720,6 +4725,11 @@ int qemudBuildCommandLine(virConnectPtr conn, snprintf(file, PATH_MAX, "fat:floppy:%s", disk->src); else snprintf(file, PATH_MAX, "fat:%s", disk->src); + } else if (disk->type == VIR_DOMAIN_DISK_TYPE_SHEEPDOG) { + if (disk->host && disk->port) + snprintf(file, PATH_MAX, "sheepdog:%s:%s:%s", disk->host, disk->port, disk->src); + else + snprintf(file, PATH_MAX, "sheepdog:%s", disk->src); } else { snprintf(file, PATH_MAX, "%s", disk->src); } @@ -5792,7 +5802,32 @@ qemuParseCommandLineDisk(virCapsPtr caps, values[i] = NULL; if (STRPREFIX(def->src, "/dev/")) def->type = VIR_DOMAIN_DISK_TYPE_BLOCK; - else + else if (STRPREFIX(def->src, "sheepdog:")) { + char *p = def->src; + char *port, *vdi; + + def->type = VIR_DOMAIN_DISK_TYPE_SHEEPDOG; + def->src = strdup(p + strlen("sheepdog:")); + + /* def->src must be [vdiname] or [host]:[port]:[vdiname] */ + port = strchr(def->src, ':'); + if (port) { + *port++ = '\0'; + vdi = strchr(port, ':'); + if (!vdi) { + def = NULL; + qemuReportError(VIR_ERR_INTERNAL_ERROR, + _("cannot parse sheepdog filename '%s'"), p); + goto cleanup; + } + *vdi++ = '\0'; + def->host = def->src; + def->port = strdup(port); + def->src = strdup(vdi); + } + + VIR_FREE(p); + } else def->type = VIR_DOMAIN_DISK_TYPE_FILE; } else { def->type = VIR_DOMAIN_DISK_TYPE_FILE; @@ -6729,7 +6764,10 @@ virDomainDefPtr qemuParseCommandLine(virCapsPtr caps, if (STRPREFIX(val, "/dev/")) disk->type = VIR_DOMAIN_DISK_TYPE_BLOCK; - else + else if (STRPREFIX(val, "sheepdog:")) { + disk->type = VIR_DOMAIN_DISK_TYPE_SHEEPDOG; + val = STRSKIP(val, "sheepdog:"); + } else disk->type = VIR_DOMAIN_DISK_TYPE_FILE; if (STREQ(arg, "-cdrom")) { disk->device = VIR_DOMAIN_DISK_DEVICE_CDROM; @@ -6749,6 +6787,27 @@ virDomainDefPtr qemuParseCommandLine(virCapsPtr caps, disk->dst = strdup(arg + 1); } disk->src = strdup(val); + + if (disk->type == VIR_DOMAIN_DISK_TYPE_SHEEPDOG) { + char *port, *vdi; + + /* disk->src must be [vdiname] or [host]:[port]:[vdiname] */ + port = strchr(disk->src, ':'); + if (port) { + *port++ = '\0'; + vdi = strchr(port, ':'); + if (!vdi) { + qemuReportError(VIR_ERR_INTERNAL_ERROR, + _("cannot parse sheepdog filename '%s'"), val); + goto error; + } + *vdi++ = '\0'; + disk->host = disk->src; + disk->port = strdup(port); + disk->src = strdup(vdi); + } + } + if (!disk->src || !disk->dst) { virDomainDiskDefFree(disk); -- 1.7.1

3 4

[libvirt] [PATCH] allow '#' as valid character for domain name
by Osier Yang 20 Nov '10

20 Nov '10

* docs/schemas/domain.rng --- docs/schemas/domain.rng | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/docs/schemas/domain.rng b/docs/schemas/domain.rng index bbbc846..815134d 100644 --- a/docs/schemas/domain.rng +++ b/docs/schemas/domain.rng @@ -2003,7 +2003,7 @@ </define> <define name="domainName"> <data type="string"> - <param name="pattern">[A-Za-z0-9_\.\+\-&:/]+</param> + <param name="pattern">[A-Za-z0-9_\.\+\-\\#&:/]+</param> </data> </define> <define name="diskSerial"> -- 1.7.3.2

3 4

[libvirt] [PATCH] nwfilter: re-order lock grabbed by IP addr. learn thread
by Stefan Berger 20 Nov '10

20 Nov '10

The IP address learning thread was causing a deadlock when it instantiated a filter while a filter update/change was ongoing. The reason for this was the ordering of locks due to the following calls virNWFilterUnlockFilterUpdates() virNWFilterPoolObjFindByName() The below patch now puts the order of the locks in the above shown order when instantiating the filter from the IP address learning thread. Signed-off-by: Stefan Berger <stefanb(a)us.ibm.com> --- src/nwfilter/nwfilter_gentech_driver.c | 45 ++++++++++++++++++++------------- 1 file changed, 28 insertions(+), 17 deletions(-) Index: libvirt-acl/src/nwfilter/nwfilter_gentech_driver.c =================================================================== --- libvirt-acl.orig/src/nwfilter/nwfilter_gentech_driver.c +++ libvirt-acl/src/nwfilter/nwfilter_gentech_driver.c @@ -552,6 +552,8 @@ virNWFilterRuleInstancesToArray(int nEnt * all its subfilters in a depth-first traversal of the tree of referenced * filters. The name of the interface to which the rules belong must be * provided. Apply the values of variables as needed. + * + * Call this function while holding the NWFilter filter update lock */ static int virNWFilterInstantiate(virConnectPtr conn, @@ -575,8 +577,6 @@ virNWFilterInstantiate(virConnectPtr con void **ptrs = NULL; int instantiate = 1; - virNWFilterLockFilterUpdates(); - virNWFilterHashTablePtr missing_vars = virNWFilterHashTableCreate(0); if (!missing_vars) { virReportOOMError(); @@ -668,8 +668,6 @@ virNWFilterInstantiate(virConnectPtr con err_exit: - virNWFilterUnlockFilterUpdates(); - for (j = 0; j < nEntries; j++) virNWFilterRuleInstFree(insts[j]); @@ -681,6 +679,7 @@ err_exit: } + * Call this function while holding the NWFilter filter update lock static int __virNWFilterInstantiateFilter(virConnectPtr conn, bool teardownOld, @@ -823,23 +822,30 @@ _virNWFilterInstantiateFilter(virConnect ? net->data.direct.linkdev : NULL; int ifindex; + int rc; if (ifaceGetIndex(true, net->ifname, &ifindex)) return 1; - return __virNWFilterInstantiateFilter(conn, - teardownOld, - net->ifname, - ifindex, - linkdev, - net->type, - net->mac, - net->filter, - net->filterparams, - useNewFilter, - conn->nwfilterPrivateData, - false, - foundNewFilter); + virNWFilterLockFilterUpdates(); + + rc = __virNWFilterInstantiateFilter(conn, + teardownOld, + net->ifname, + ifindex, + linkdev, + net->type, + net->mac, + net->filter, + net->filterparams, + useNewFilter, + conn->nwfilterPrivateData, + false, + foundNewFilter); + + virNWFilterUnlockFilterUpdates(); + + return rc; } @@ -857,6 +863,8 @@ virNWFilterInstantiateFilterLate(virConn int rc; bool foundNewFilter = false; + virNWFilterLockFilterUpdates(); + rc = __virNWFilterInstantiateFilter(conn, 1, ifname, @@ -878,6 +886,9 @@ virNWFilterInstantiateFilterLate(virConn _virNWFilterTeardownFilter(ifname); } } + + virNWFilterUnlockFilterUpdates(); + return rc; }

2 2

[libvirt] [PATCH v2] [TCK] nwfilter: add a test case using concurrency
by Stefan Berger 20 Nov '10

20 Nov '10

V2: - following Eric's suggestions from review of V1 - all scripts started by the main script trap on signal 2 and clean up - the main program waits for all child processes to have terminated - the test now requires each child process to do 10 steps: - 1 VM start-destroy cycle is 1 step - 50 filter changes are considered 1 step - the test terminates after 3 minutes (in case the test system is very slow / busy) with a bail out message Now that the existing scripts are cleaned up and my POSIX compliancy shell-scripting skills have temporarily:-) improved, I am now adding a test case that exercises concurrency. The test case creates and destroys 2 VMs concurrently as well as changes their referenced filters in a tight loop. This kind of test previously uncovered - deadlocks in libvirt due to lock-ordering in the nwfilter subsystem - libvirt termination bug in libaugeas due to doubly closed file descriptors and the side effects All of these have been fixed recently. The test script is known to run in bash, dash and ksh shells. Signed-off-by: Stefan Berger<stefanb(a)us.ibm.com> --- scripts/nwfilter/060-concurrency.t | 5 scripts/nwfilter/concurrency/chg-vm-filter.sh | 29 + scripts/nwfilter/concurrency/start-destroy-vm.sh | 31 + scripts/nwfilter/concurrency/tck-vm1-filter1.xml | 26 + scripts/nwfilter/concurrency/tck-vm1-filter2.xml | 26 + scripts/nwfilter/concurrency/tck-vm1.xml | 30 + scripts/nwfilter/concurrency/tck-vm2-filter1.xml | 26 + scripts/nwfilter/concurrency/tck-vm2-filter2.xml | 26 + scripts/nwfilter/concurrency/tck-vm2.xml | 30 + scripts/nwfilter/nwfilter_concurrent.sh | 406 +++++++++++++++++++++++ 10 files changed, 635 insertions(+) Index: libvirt-tck/scripts/nwfilter/060-concurrency.t =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/060-concurrency.t @@ -0,0 +1,5 @@ +#!/bin/sh + +pwd=$(dirname -- "$0") + +(cd -- "${pwd}"&& sh ./nwfilter_concurrent.sh --tap-test) Index: libvirt-tck/scripts/nwfilter/nwfilter_concurrent.sh =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/nwfilter_concurrent.sh @@ -0,0 +1,406 @@ +#!/bin/sh + +VIRSH=virsh + +# For each line starting with uri=, remove the prefix and set the hold +# space to the rest of the line. Then at file end, print the hold +# space, which is effectively the last uri= line encountered. +uri=$(sed -n '/^uri[ ]*=[ ]*/ { + s/// + h +} +$ { + x + p +}'< "$LIBVIRT_TCK_CONFIG") +: "${uri:=qemu:///system}" + +LIBVIRT_URI=${uri} + +FLAG_WAIT="$((1<<0))" +FLAG_ATTACH="$((1<<1))" +FLAG_VERBOSE="$((1<<2))" +FLAG_LIBVIRT_TEST="$((1<<3))" +FLAG_TAP_TEST="$((1<<4))" +FLAG_FORCE_CLEAN="$((1<<5))" + +failctr=0 +passctr=0 +attachfailctr=0 +attachctr=0 + +TAP_FAIL_LIST="" +TAP_FAIL_CTR=0 +TAP_TOT_CTR=0 + +usage() { + cmd="$0" +cat<<EOF +Usage: ${cmd} [--help|-h|-?] [--noattach] [--wait] [--verbose] + [--libvirt-test] [--tap-test] + +Options: + --help,-h,-? : Display this help screen. + --noattach : Skip tests that attach and detach a network interface + --wait : Wait for the user to press the enter key once an error + was detected + --verbose : Verbose output + --libvirt-test : Use the libvirt test output format + --tap-test : TAP format output + --force : Allow the automatic cleaning of VMs and nwfilters + previously created by the TCK test suite + +This test will create two virtual machines. The one virtual machine +will use a filter called '${TESTFILTERNAME}', and reference the filter +'clean-traffic' which should be available by default with every install. +The other virtual machine will reference the filter 'tck-testcase' and will +have its filter permanently updated. +EOF +} + + +tap_fail() { + txt=$(echo "$2" | gawk '{print substr($0,1,66)}') + echo "not ok $1 - ${txt}" + TAP_FAIL_LIST="$TAP_FAIL_LIST $1 " + TAP_FAIL_CTR=$(($TAP_FAIL_CTR + 1)) + TAP_TOT_CTR=$(($TAP_TOT_CTR + 1)) +} + +tap_pass() { + txt=$(echo "$2" | gawk '{print substr($0,1,70)}') + echo "ok $1 - ${txt}" + TAP_TOT_CTR=$(($TAP_TOT_CTR + 1)) +} + +tap_final() { + [ -n "${TAP_FAIL_LIST}" ]&& echo "FAILED tests ${TAP_FAIL_LIST}" + + okay=`echo "($TAP_TOT_CTR-$TAP_FAIL_CTR)*100/$TAP_TOT_CTR" | bc -l` + txt=$(echo $okay | gawk '{print substr($0,1,5)}') + echo "Failed ${TAP_FAIL_CTR}/${TAP_TOT_CTR} tests, ${txt}% okay" +} + +# A wrapper for mktemp in case it does not exist +# Echos the name of a temporary file. +mktmpdir() { + { + tmp=$( (umask 077&& mktemp -d ./nwfvmtest.XXXXXX) 2>/dev/null)&& + test -n "$tmp"&& test -d "$tmp" + } || + { + tmp=./nwfvmtest$$-$RANDOM + (umask 077&& mkdir "$tmp") + } || { echo "failed to create secure temporary directory">&2; exit 1; } + echo "${tmp}" + return 0 +} + + +PRGDIR="./concurrency" +CREATE_DES_VM1="${PRGDIR}/start-destroy-vm.sh 1" +CREATE_DES_VM2="${PRGDIR}/start-destroy-vm.sh 2" +CHG_FILTER_VM1="${PRGDIR}/chg-vm-filter.sh 1" +CHG_FILTER_VM2="${PRGDIR}/chg-vm-filter.sh 2" + + +startPrgs() +{ + flags="$1" + + sh ${CHG_FILTER_VM1} "$4" 2>&1>/dev/null& + [ $? -ne 0 ]&& \ + ( killPrgs "${flags}" "Could not start program 3" ; return 1; ) + CHG_FILTER_VM1_THR=$! + + sh ${CHG_FILTER_VM2} "$5" 2>&1>/dev/null& + [ $? -ne 0 ]&& \ + ( killPrgs "${flags}" "Could not start program 4" ; return 1; ) + CHG_FILTER_VM2_THR=$! + + # Give some time for the filters to be created + sleep 2 + + sh ${CREATE_DES_VM1} "$2" 2>&1>/dev/null& + [ $? -ne 0 ]&& \ + ( killPrgs "${flags}" "Could not start program 1" ; return 1; ) + CREATE_DES_VM1_THR=$! + + sh ${CREATE_DES_VM2} "$3" 2>&1>/dev/null& + [ $? -ne 0 ]&& \ + ( killPrgs "${flags}" "Could not start program 2" ; return 1; ) + CREATE_DES_VM2_THR=$! + +} + + +killPrgs() +{ + msg="$1" + + # terminate all process + [ "x${CREATE_DES_VM1_THR}x" != "xx" ]&& \ + kill -2 ${CREATE_DES_VM1_THR} + [ "x${CREATE_DES_VM2_THR}x" != "xx" ]&& \ + kill -2 ${CREATE_DES_VM2_THR} + [ "x${CHG_FILTER_VM1_THR}x" != "xx" ]&& \ + kill -2 ${CHG_FILTER_VM1_THR} + [ "x${CHG_FILTER_VM2_THR}x" != "xx" ]&& \ + kill -2 ${CHG_FILTER_VM2_THR} + + wait +} + + +testFail() +{ + flags="$1" + msg="$2" + + failctr=$(($failctr + 1)) + if [ $(($flags& $FLAG_VERBOSE)) -ne 0 ]; then + echo "FAIL : ${msg}" + fi + if [ $(($flags& $FLAG_WAIT)) -ne 0 ]; then + echo "Press enter" + read enter + fi + [ $(($flags& $FLAG_LIBVIRT_TEST)) -ne 0 ]&& \ + test_result $(($passctr + $failctr)) "" 1 + [ $(($flags& $FLAG_TAP_TEST)) -ne 0 ]&& \ + tap_fail $(($passctr + $failctr)) "${msg}" +} + + +testPass() +{ + flags="$1" + msg="$2" + + passctr=$(($passctr + 1)) + if [ $(($flags& $FLAG_VERBOSE)) -ne 0 ]; then + echo "PASS : ${msg}" + fi + [ $(($flags& $FLAG_LIBVIRT_TEST)) -ne 0 ]&& \ + test_result $(($passctr + $failctr)) "" 0 + [ $(($flags& $FLAG_TAP_TEST)) -ne 0 ]&& \ + tap_pass $(($passctr + $failctr)) "${msg}" +} + +cleanup() +{ + rm -rf "$1" + killPrgs + exit $2 +} + +missedSteps() +{ + cur="$1" + exp="$2" + flags="$3" + + while [ $cur -lt $exp ]; do + cur=$(($cur + 1)) + testFail "${flags}" "$4 ${cur}" + done +} + +runTest() +{ + flags="$1" + + passctr=0 + failctr=0 + + tmpdir=`mktmpdir` + failctr=0 + passctr=0 + logvm1="${PWD}/${tmpdir}/logvm1" + logvm2="${PWD}/${tmpdir}/logvm2" + logfivm1="${PWD}/${tmpdir}/logfivm1" + logfivm2="${PWD}/${tmpdir}/logfivm2" + + # exp. number of steps each 'thread' has to do + steps=10 + + steps_vm1=0 + steps_vm2=0 + steps_fivm1=0 + steps_fivm2=0 + + trap "cleanup ${tmpdir} 1" INT + + if [ $(($flags& $FLAG_TAP_TEST)) -ne 0 ]; then + # Need to display the number of total tests + tap_total=$((4 * $steps)) + echo "1..${tap_total}" + fi + + startPrgs "${flags}" "${logvm1}" "${logvm2}" \ + "${logfivm1}" "${logfivm2}" + + [ $? -ne 0 ]&& rm -rf "${tmpdir}"&& return 1; + + # Test runs for a maximum of 3 minutes + now=`date +%s` + test_end=$(($now + 3 * 60)) + + while :; + do + # The logs give us the number of cycles the VMs were created + # and destroyed. + val=$(cat "${logvm1}" 2>/dev/null | tail -n 1) + while [ -n "${val}" ]&& \ + [ $steps_vm1 -lt $steps ]&& [ $steps_vm1 -lt $val ]; do + steps_vm1=$(($steps_vm1 + 1)) + testPass "${flags}" \ + "VM1 log - step ${steps_vm1}" + done + + val=$(cat "${logvm2}" 2>/dev/null | tail -n 1) + while [ -n "${val}" ]&& \ + [ $steps_vm2 -lt $steps ]&& [ $steps_vm2 -lt $val ]; do + steps_vm2=$(($steps_vm2 + 1)) + testPass "${flags}" \ + "VM2 log - step ${steps_vm2}" + done + + # The changing of the filters is expected to work much faster + val=$(cat "${logfivm1}" 2>/dev/null | tail -n 1) + while [ -n "${val}" ]&& \ + [ $steps_fivm1 -lt $steps ]&& [ $steps_fivm1 -lt $(($val / 50)) ]; + do + steps_fivm1=$(($steps_fivm1 + 1)) + testPass "${flags}" \ + "VM1 filter log - step ${steps_fivm1}" + done + + val=$(cat "${logfivm2}" 2>/dev/null | tail -n 1) + while [ -n "${val}" ]&& \ + [ $steps_fivm2 -lt $steps ]&& [ $steps_fivm2 -lt $(($val / 50)) ]; + do + steps_fivm2=$(($steps_fivm2 + 1)) + testPass "${flags}" \ + "VM2 filter log - step ${steps_fivm2}" + done + + [ $steps_vm1 -ge $steps ]&& \ + [ $steps_vm2 -ge $steps ]&& \ + [ $steps_fivm1 -ge $steps ]&& \ + [ $steps_fivm2 -ge $steps ]&& \ + break + + now=`date +%s` + [ $now -gt $test_end ]&& break + + sleep 4 + done + + missedSteps $steps_vm1 $steps $flags "VM1 log - step " + missedSteps $steps_vm2 $steps $flags "VM2 log - step " + missedSteps $steps_fivm1 $steps $flags "VM1 filter log - step " + missedSteps $steps_fivm1 $steps $flags "VM2 filter log - step " + + [ $now -gt $test_end ]&& \ + echo "Bail out! Not all tests finished before test expired. Busy system?" + + cleanup "${tmpdir}" 0 +} + + +main() { + prgname="$0" + xmldir="nwfilterxml2xmlin" + fwalldir="nwfilterxml2fwallout" + found=0 + filtername="tck-testcase" + libvirtdpid=-1 + + flags=${FLAG_ATTACH} + + while [ $# -ne 0 ]; do + case "$1" in + --help|-h|-\?) usage ${prgname}; exit 0;; + --noattach) flags=$(($flags& ~$FLAG_ATTACH));; + --wait) flags=$(($flags | $FLAG_WAIT ));; + --verbose) flags=$(($flags | $FLAG_VERBOSE ));; + --libvirt-test) flags=$(($flags | $FLAG_LIBVIRT_TEST ));; + --tap-test) flags=$(($flags | $FLAG_TAP_TEST ));; + --force) flags=$(($flags | $FLAG_FORCE_CLEAN ));; + *) usage ${prgname}; exit 1;; + esac + shift 1 + done + + if [ `uname` != "Linux" ]; then + if [ $(($flags& $FLAG_TAP_TEST)) -ne 0 ]; then + echo "1..0 # Skipped: Only valid on Linux hosts" + else + echo "This script will only run on Linux." + fi + exit 1; + fi + + if [ $(($flags& $FLAG_TAP_TEST)) -ne 0 ]; then + if [ "${LIBVIRT_URI}" != "qemu:///system" ]; then + echo "1..0 # Skipped: Only valid for Qemu system driver" + exit 0 + fi + + for name in `virsh list --all | awk '{print $2}'` + do + case ${name} in + tck*) + if [ "x${LIBVIRT_TCK_AUTOCLEAN}" = "x1" ] || \ + [ $(($flags& $FLAG_FORCE_CLEAN)) -ne 0 ]; then + res=$(virsh destroy ${name} 2>&1) + rc1=$? + res=$(virsh undefine ${name} 2>&1) + rc2=$? + if [ $rc1 -ne 0 ]&& [ $rc2 -ne 0 ]; then + echo "Bail out! Could not destroy VM ${name}: ${res}" + exit 0 + fi + else + echo "Bail out! VM ${name} already exists, use --force to clean" + exit 1 + fi + esac + done + + for name in `virsh nwfilter-list | awk '{print $2}'` + do + case ${name} in + tck*) + if [ "x${LIBVIRT_TCK_AUTOCLEAN}" = "x1" ] || \ + [ $(($flags& $FLAG_FORCE_CLEAN)) -ne 0 ]; then + res=$(virsh nwfilter-undefine ${name} 2>&1) + if [ $? -ne 0 ]; then + echo "Bail out! Could not undefine filter ${name}: ${res}" + exit 1 + fi + else + echo "Bail out! Filter ${name} already exists, use --force to clean" + exit 1 + fi + esac + done + fi + + if [ $(($flags& $FLAG_LIBVIRT_TEST)) -ne 0 ]; then + curdir="${PWD}" + . test-lib.sh + if [ $? -ne 0 ]; then + exit 1 + fi + test_intro $this_test + cd "${curdir}" || { echo "cd failed">&2; exit 1;} + fi + + runTest "${flags}" + + return 0 +} + +main "$@" Index: libvirt-tck/scripts/nwfilter/concurrency/chg-vm-filter.sh =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/concurrency/chg-vm-filter.sh @@ -0,0 +1,29 @@ +#!/bin/sh +cleanup() +{ + virsh nwfilter-undefine tck-vm${idx}-filter 2>/dev/null +} + +cd $(dirname "$0") +ctr=0 +[ -z "$2" ]&& exit 1 +idx="$1" +logfile="$2" +rm -f "${logfile}" +touch "${logfile}" + +trap cleanup 2 + +while :; +do + virsh nwfilter-define tck-vm${idx}-filter1.xml + [ $? -ne 0 ]&& break + [ ! -w "${logfile}" ]&& break + virsh nwfilter-define tck-vm${idx}-filter2.xml + [ $? -ne 0 ]&& break + ctr=$(($ctr + 1)) + [ ! -w "${logfile}" ]&& break + echo "${ctr}">> "${logfile}" +done + +cleanup Index: libvirt-tck/scripts/nwfilter/concurrency/start-destroy-vm.sh =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/concurrency/start-destroy-vm.sh @@ -0,0 +1,31 @@ +#!/bin/sh + +cleanup() +{ + virsh destroy tck-vm${idx} 2>/dev/null + virsh nwfilter-undefine tck-vm${idx}-filter 2>/dev/null +} + +cd $(dirname "$0") +ctr=0 +[ -z "$2" ]&& exit 1 +idx="$1" +logfile="$2" +rm -f "${logfile}" +touch "${logfile}" + +trap cleanup 2 + +while :; +do + virsh create tck-vm${idx}.xml + [ $? -ne 0 ]&& break + sleep 2 + virsh destroy tck-vm${idx} + [ $? -ne 0 ]&& break + ctr=$(($ctr + 1)) + [ ! -w "${logfile}" ]&& break + echo "${ctr}">> "${logfile}" +done + +cleanup Index: libvirt-tck/scripts/nwfilter/concurrency/tck-vm1-filter1.xml =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/concurrency/tck-vm1-filter1.xml @@ -0,0 +1,26 @@ +<filter name='tck-vm1-filter' chain='root'> +<uuid>464d2617-43d0-7694-b479-320b72dac187</uuid> +<filterref filter='clean-traffic'/> +<rule action='accept' direction='in' priority='500'> +<all comment='test'/> +</rule> +<rule action='accept' direction='out' priority='500'> +<all/> +</rule> +<rule action='accept' direction='in' priority='500'> +<tcp dstportstart='21' dstportend='22'/> +</rule> +<rule action='accept' direction='in' priority='500'> +<tcp dstportstart='80'/> +</rule> +<rule action='accept' direction='out' priority='500'> +<icmp/> +</rule> +<rule action='accept' direction='out' priority='500'> +<udp dstportstart='53'/> +</rule> +<rule action='drop' direction='inout' priority='500'> +<all/> +</rule> +</filter> + Index: libvirt-tck/scripts/nwfilter/concurrency/tck-vm1-filter2.xml =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/concurrency/tck-vm1-filter2.xml @@ -0,0 +1,26 @@ +<filter name='tck-vm1-filter' chain='root'> +<uuid>464d2617-43d0-7694-b479-320b72dac187</uuid> +<filterref filter='clean-traffic'/> +<rule action='accept' direction='in' priority='500'> +<all comment='test again'/> +</rule> +<rule action='accept' direction='out' priority='500'> +<all/> +</rule> +<rule action='accept' direction='in' priority='500'> +<tcp dstportstart='21' dstportend='22'/> +</rule> +<rule action='accept' direction='in' priority='500'> +<tcp dstportstart='80'/> +</rule> +<rule action='accept' direction='out' priority='500'> +<icmp/> +</rule> +<rule action='accept' direction='out' priority='500'> +<udp dstportstart='53'/> +</rule> +<rule action='drop' direction='inout' priority='500'> +<all/> +</rule> +</filter> + Index: libvirt-tck/scripts/nwfilter/concurrency/tck-vm1.xml =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/concurrency/tck-vm1.xml @@ -0,0 +1,30 @@ +<domain type='kvm'> +<name>tck-vm1</name> +<memory>32768</memory> +<currentMemory>32768</currentMemory> +<vcpu>1</vcpu> +<os> +<type>hvm</type> +<boot dev='hd'/> +</os> +<features> +<acpi/> +<apic/> +</features> +<clock offset='utc'/> +<on_poweroff>destroy</on_poweroff> +<on_reboot>restart</on_reboot> +<on_crash>destroy</on_crash> +<devices> +<interface type='bridge'> +<source bridge='virbr0'/> +<filterref filter='tck-vm1-filter'> +</filterref> +<target dev='tck-vm1-if0'/> +</interface> +<console type='pty'> +</console> +<input type='mouse' bus='ps2'/> +<graphics type='vnc' port='-1' autoport='yes'/> +</devices> +</domain> Index: libvirt-tck/scripts/nwfilter/concurrency/tck-vm2-filter1.xml =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/concurrency/tck-vm2-filter1.xml @@ -0,0 +1,26 @@ +<filter name='tck-vm2-filter' chain='root'> +<uuid>364d2617-43d0-7694-b479-320b72dac187</uuid> +<filterref filter='clean-traffic'/> +<rule action='accept' direction='in' priority='500'> +<all comment='another test'/> +</rule> +<rule action='accept' direction='out' priority='500'> +<all/> +</rule> +<rule action='accept' direction='in' priority='500'> +<tcp dstportstart='21' dstportend='22'/> +</rule> +<rule action='accept' direction='in' priority='500'> +<tcp dstportstart='80'/> +</rule> +<rule action='accept' direction='out' priority='500'> +<icmp/> +</rule> +<rule action='accept' direction='out' priority='500'> +<udp dstportstart='53'/> +</rule> +<rule action='drop' direction='inout' priority='500'> +<all/> +</rule> +</filter> + Index: libvirt-tck/scripts/nwfilter/concurrency/tck-vm2-filter2.xml =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/concurrency/tck-vm2-filter2.xml @@ -0,0 +1,26 @@ +<filter name='tck-vm2-filter' chain='root'> +<uuid>364d2617-43d0-7694-b479-320b72dac187</uuid> +<filterref filter='clean-traffic'/> +<rule action='accept' direction='in' priority='500'> +<all comment='another test again'/> +</rule> +<rule action='accept' direction='out' priority='500'> +<all/> +</rule> +<rule action='accept' direction='in' priority='500'> +<tcp dstportstart='21' dstportend='22'/> +</rule> +<rule action='accept' direction='in' priority='500'> +<tcp dstportstart='80'/> +</rule> +<rule action='accept' direction='out' priority='500'> +<icmp/> +</rule> +<rule action='accept' direction='out' priority='500'> +<udp dstportstart='53'/> +</rule> +<rule action='drop' direction='inout' priority='500'> +<all/> +</rule> +</filter> + Index: libvirt-tck/scripts/nwfilter/concurrency/tck-vm2.xml =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/concurrency/tck-vm2.xml @@ -0,0 +1,30 @@ +<domain type='kvm'> +<name>tck-vm2</name> +<memory>32768</memory> +<currentMemory>32768</currentMemory> +<vcpu>1</vcpu> +<os> +<type>hvm</type> +<boot dev='hd'/> +</os> +<features> +<acpi/> +<apic/> +</features> +<clock offset='utc'/> +<on_poweroff>destroy</on_poweroff> +<on_reboot>restart</on_reboot> +<on_crash>destroy</on_crash> +<devices> +<interface type='bridge'> +<source bridge='virbr0'/> +<filterref filter='tck-vm2-filter'> +</filterref> +<target dev='tck-vm2-if0'/> +</interface> +<console type='pty'> +</console> +<input type='mouse' bus='ps2'/> +<graphics type='vnc' port='-1' autoport='yes'/> +</devices> +</domain>

2 2

[libvirt] rbd storage pool support for libvirt
by Sage Weil 19 Nov '10

19 Nov '10

Hi, We've been working on RBD, a distributed block device backed by the Ceph distributed object store. (Ceph is a highly scalable, fault tolerant distributed storage and file system; see http://ceph.newdream.net.) Although the Ceph file system client has been in Linux since 2.6.34, the RBD block device was just merged for 2.6.37. We also have patches pending for Qemu that use librados to natively talk to the Ceph storage backend, avoiding any kernel dependency. To support disks backed by RBD in libvirt, we originally proposed a 'virtual' type that simply passed the configuration information through to qemu, but that idea was shot down for a variety of reasons: http://www.redhat.com/archives/libvir-list/2010-June/thread.html#00257 It sounds like the "right" approach is to create a storage pool type. Ceph also has a 'pool' concept that contains some number of RBD images and a command line tool to manipulate (create, destroy, resize, rename, snapshot, etc.) those images, which seems to map nicely onto the storage pool abstraction. For example, $ rbd create foo -s 1000 rbd image 'foo': size 1000 MB in 250 objects order 22 (4096 KB objects) adding rbd image to directory... creating rbd image... done. $ rbd create bar -s 10000 [...] $ rbd list bar foo Something along the lines of <pool type="rbd"> <name>virtimages</name> <source mode="kernel"> <host monitor="ceph-mon1.domain.com:6789"/> <host monitor="ceph-mon2.domain.com:6789"/> <host monitor="ceph-mon3.domain.com:6789"/> <pool name="rbd"/> </source> </pool> or whatever (I'm not too familiar with the libvirt schema)? One difference between the existing pool types listed at libvirt.org/storage.html is that RBD does not necessarily associate itself with a path in the local file system. If the native qemu driver is used, there is no path involved, just a magic string passed to qemu (rbd:poolname/imagename). If the kernel RBD driver is used, it gets mapped to a /dev/rbd/$n (or similar, depending on the udev rule), but $n is not static across reboots. In any case, before someone goes off and implements something, does this look like the right general approach to adding rbd support to libvirt? Thanks! sage

4 11

[libvirt] [PATCH] doc: Add doc for missed options of migrate
by Osier Yang 19 Nov '10

19 Nov '10

* tools/virsh.pod (add doc for options like "--p2p", "--direct", "--copy-storage-all", "dname", etc.) --- tools/virsh.pod | 23 ++++++++++++++++------- 1 files changed, 16 insertions(+), 7 deletions(-) diff --git a/tools/virsh.pod b/tools/virsh.pod index c4a5803..036848d 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -456,13 +456,22 @@ Provide the maximum number of virtual CPUs supported for a guest VM on this connection. If provided, the I<type> parameter must be a valid type attribute for the <domain> element of XML. -=item B<migrate> optional I<--live> I<--suspend> I<domain-id> I<desturi> -I<migrateuri> - -Migrate domain to another host. Add --live for live migration; --suspend -leaves the domain paused on the destination host. The I<desturi> is the -connection URI of the destination host, and I<migrateuri> is the -migration URI, which usually can be omitted. +=item B<migrate> optional I<--live> I<--p2p> I<--direct> I<--tunnelled> +I<--persistent> I<--undefinesource> I<--suspend> I<--copy-storage-all> +I<--copy-storage-inc> I<domain-id> I<desturi> I<migrateuri> I<dname> + +Migrate domain to another host. Add I<--live> for live migration; I<--p2p> +for peer-2-peer migration; I<--direct> for direct migration; I<--tunnelled> +for tunnelled migration; I<--persistent> leaves the domain persistent on +destination host, I<--undefinesource> undefine the domain on source host, +I<--suspend> leaves the domain paused on the destination host. +I<--copy-storage-all> indicates migration with non-shared storage with full +disk copy, I<--copy-storage-inc> indicates migration with non-shared storage +with incremental copy (same base image shared between source and destination). +The I<desturi> is the connection URI of the destination host, and +I<migrateuri> is the migration URI, which usually can be omitted. +I<dname> is used for renaming the domain to new name during migration, which +also usually can be ommited. =item B<migrate-setmaxdowntime> I<domain-id> I<downtime> -- 1.7.3.2

2 2

[libvirt] [PATCH 0/2] Support of auto-dump on watchdog event in libvirtd
by Hu Tao 19 Nov '10

19 Nov '10

This patch series adds support of auto-dump on watchdog event in libvirtd. If a qemu guest is configured with a watchdog device and libvirtd receives a watchdog event from the guest, it can do something on the guest, currently `dump' action is defined. This patch series adds a new watchdog action `managed', it has a subaction wich indicates the action libvirtd will take on a watchdog event. The meaning of subaction_arg depends on subaction. If subaction is `dump', then subaction_arg defines a folder path into which libvirtd saves dumpfile. An example: <watchdog model='i6300esb' action='managed' subaction='dump' subaction_arg='/mnt/data/dumps'> In order to make the function work, there must be a watchdog device added to guest, and guest must have a watchdog daemon running, for example, /etc/init.d/watchdog start or auto-started on boot. Suggestions are welcome! Hu Tao (2): Add a thread pool implementation Add a watchdog action `managed'. src/Makefile.am | 3 +- src/conf/domain_conf.c | 27 ++++++++- src/conf/domain_conf.h | 11 ++++- src/qemu/qemu_conf.c | 2 +- src/qemu/qemu_conf.h | 6 ++ src/qemu/qemu_driver.c | 138 ++++++++++++++++++++++++++++++++++++++++++----- src/util/threadpool.c | 119 +++++++++++++++++++++++++++++++++++++++++ src/util/threadpool.h | 34 ++++++++++++ 8 files changed, 319 insertions(+), 21 deletions(-) create mode 100644 src/util/threadpool.c create mode 100644 src/util/threadpool.h -- 1.7.3 -- Thanks, Hu Tao

2 6

[libvirt] [PATCH] doc: add doc for missed parameters of attach-disk
by Osier Yang 18 Nov '10

18 Nov '10

* tools/virsh.pod (add docs for --persistent and --sourcetype of attach-disk, break the long lines in the meantime) --- tools/virsh.pod | 10 ++++++++-- 1 files changed, 8 insertions(+), 2 deletions(-) diff --git a/tools/virsh.pod b/tools/virsh.pod index b4644d8..c4a5803 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -645,7 +645,9 @@ See the documentation to learn about libvirt XML format for a device. For cdrom and floppy devices, this command only replaces the media within the single existing device; consider using B<update-device> for this usage. -=item B<attach-disk> I<domain-id> I<source> I<target> optional I<--driver driver> I<--subdriver subdriver> I<--type type> I<--mode mode> +=item B<attach-disk> I<domain-id> I<source> I<target> optional +I<--driver driver> I<--subdriver subdriver> I<--type type> +I<--mode mode> I<--persistent> I<--sourcetype soucetype> Attach a new disk device to the domain. I<source> and I<target> are paths for the files and devices. @@ -654,8 +656,12 @@ I<type> can indicate I<cdrom> or I<floppy> as alternative to the disk default, although this use only replaces the media within the existing virtual cdrom or floppy device; consider using B<update-device> for this usage instead. I<mode> can specify the two specific mode I<readonly> or I<shareable>. +I<persistent> indicates the changes will affect the next boot of the domain. +I<sourcetype> can indicate the type of source (block|file) -=item B<attach-interface> I<domain-id> I<type> I<source> optional I<--target target> I<--mac mac> I<--script script> I<--model model> I<--persistent> +=item B<attach-interface> I<domain-id> I<type> I<source> optional +I<--target target> I<--mac mac> I<--script script> I<--model model> +I<--persistent> Attach a new network interface to the domain. I<type> can be either I<network> to indicate a physical network device or I<bridge> to indicate a bridge to a device. -- 1.7.3.2

2 1