- Devel - libvirt List Archives

[libvirt] [PATCH] Ensure virExec preserves logging environment
by Daniel P. Berrange 23 Nov '10

23 Nov '10

The virFork call resets all logging handlers that may have been set. Re-enable them after fork in virExec, so that env variables fir LIBVIRT_LOG_OUTPUTS and LIBVIRT_LOG_FILTERS take effect until the execve() * src/util/util.c: Preserve logging in child in virExec --- src/util/util.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/src/util/util.c b/src/util/util.c index f2fe58a..a0849e1 100644 --- a/src/util/util.c +++ b/src/util/util.c @@ -602,6 +602,9 @@ __virExec(const char *const*argv, childout = -1; } + /* Initialize full logging for a while */ + virLogSetFromEnv(); + /* Daemonize as late as possible, so the parent process can detect * the above errors with wait* */ if (flags & VIR_EXEC_DAEMON) { @@ -650,6 +653,9 @@ __virExec(const char *const*argv, virClearCapabilities() < 0) goto fork_error; + /* Close logging again to ensure no FDs leak to child */ + virLogReset(); + if (envp) execve(argv[0], (char **) argv, (char**)envp); else -- 1.7.2.3

2 3

[libvirt] Misc improvements to the storage driver
by Daniel P. Berrange 23 Nov '10

23 Nov '10

This patch series started out as an attempt to fix the seriously painful problem with iSCSI targets automatically logging in whenever the network came online. In the process of fixing this alot of other cruft is cleaned up docs/schemas/storagepool.rng | 5 src/conf/storage_conf.c | 21 + src/conf/storage_conf.h | 1 src/storage/storage_backend.h | 3 src/storage/storage_backend_fs.c | 29 ++ src/storage/storage_backend_iscsi.c | 427 +++++++++++++++++++++++++--------- src/storage/storage_backend_logical.c | 24 + src/storage/storage_backend_mpath.c | 20 + src/storage/storage_backend_scsi.c | 96 +++++++ src/storage/storage_driver.c | 65 +++-- tools/virsh.c | 51 ++-- 11 files changed, 573 insertions(+), 169 deletions(-) Daniel

5 40

[libvirt] New features : cloning
by Clément Valentin 23 Nov '10

23 Nov '10

Hi everyone ! I'm currently developing a big project using libvirt to manage ESXi and Xen hypervisor. I was wondering to know if someday libvirt will propose a "cloning" function ? Thanks for taking time to consider my request. Valentin

1 0

[libvirt] [PATCH] Don't catch SIGCHLD in libvirtd
by Daniel P. Berrange 23 Nov '10

23 Nov '10

libvirtd no longer deals with SIGCHLD in its signal handler since the QEMU driver switched to always daemonize processes. Thus remove the sigaction for it, to avoid warning log messages * daemon/libvirtd.c: Don't catch SIGCHLD --- daemon/libvirtd.c | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/daemon/libvirtd.c b/daemon/libvirtd.c index e544c48..2eb2374 100644 --- a/daemon/libvirtd.c +++ b/daemon/libvirtd.c @@ -2973,7 +2973,6 @@ daemonSetupSignals(struct qemud_server *server) sigaction(SIGINT, &sig_action, NULL); sigaction(SIGQUIT, &sig_action, NULL); sigaction(SIGTERM, &sig_action, NULL); - sigaction(SIGCHLD, &sig_action, NULL); sig_action.sa_handler = SIG_IGN; sigaction(SIGPIPE, &sig_action, NULL); -- 1.7.2.3

2 1

[libvirt] [PATCH] Include a thread identifier in log messages
by Daniel P. Berrange 23 Nov '10

23 Nov '10

To allow messages from different threads to be untangled, include an integer thread identifier in log messages. * src/util/logging.c: Include thread ID * src/util/threads.h, src/util/threads.h, src/util/threads-pthread.c: Add new virThreadSelfID() function * configure.ac: Check for sys/syscall.h --- configure.ac | 2 +- src/libvirt_private.syms | 1 + src/util/logging.c | 6 ++++-- src/util/threads-pthread.c | 16 ++++++++++++++++ src/util/threads-win32.c | 16 ++++++++++++++++ src/util/threads.h | 1 + 6 files changed, 39 insertions(+), 3 deletions(-) diff --git a/configure.ac b/configure.ac index b912a7a..af1dfec 100644 --- a/configure.ac +++ b/configure.ac @@ -110,7 +110,7 @@ LIBS=$old_libs dnl Availability of various common headers (non-fatal if missing). AC_CHECK_HEADERS([pwd.h paths.h regex.h sys/syslimits.h sys/un.h \ sys/poll.h syslog.h mntent.h net/ethernet.h linux/magic.h \ - sys/un.h]) + sys/un.h sys/syscall.h]) AC_CHECK_LIB([intl],[gettext],[]) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index e808375..3bb1762 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -781,6 +781,7 @@ virThreadCreate; virThreadIsSelf; virThreadJoin; virThreadSelf; +virThreadSelfID; # usb.h diff --git a/src/util/logging.c b/src/util/logging.c index ac38d4d..d65dec0 100644 --- a/src/util/logging.c +++ b/src/util/logging.c @@ -548,14 +548,16 @@ void virLogMessage(const char *category, int priority, const char *funcname, localtime_r(&cur_time.tv_sec, &time_info); if ((funcname != NULL)) { - ret = virAsprintf(&msg, "%02d:%02d:%02d.%03d: %s : %s:%lld : %s\n", + ret = virAsprintf(&msg, "%02d:%02d:%02d.%03d: %d: %s : %s:%lld : %s\n", time_info.tm_hour, time_info.tm_min, time_info.tm_sec, (int) cur_time.tv_usec / 1000, + virThreadSelfID(), virLogPriorityString(priority), funcname, linenr, str); } else { - ret = virAsprintf(&msg, "%02d:%02d:%02d.%03d: %s : %s\n", + ret = virAsprintf(&msg, "%02d:%02d:%02d.%03d: %d: %s : %s\n", time_info.tm_hour, time_info.tm_min, time_info.tm_sec, (int) cur_time.tv_usec / 1000, + virThreadSelfID(), virLogPriorityString(priority), str); } VIR_FREE(str); diff --git a/src/util/threads-pthread.c b/src/util/threads-pthread.c index 34d9ce8..02070ae 100644 --- a/src/util/threads-pthread.c +++ b/src/util/threads-pthread.c @@ -21,6 +21,11 @@ #include <config.h> +#include <unistd.h> +#if HAVE_SYS_SYSCALL_H +# include <sys/syscall.h> +#endif + /* Nothing special required for pthreads */ int virThreadInitialize(void) @@ -170,6 +175,17 @@ bool virThreadIsSelf(virThreadPtr thread) return pthread_equal(pthread_self(), thread->thread) ? true : false; } +int virThreadSelfID(void) +{ +#if defined(HAVE_SYS_SYSCALL_H) && defined(SYS_gettid) + pid_t tid; + tid = syscall(SYS_gettid); + return (int)tid; +#else + return (int)pthread_self(); +#endif +} + void virThreadJoin(virThreadPtr thread) { pthread_join(thread->thread, NULL); diff --git a/src/util/threads-win32.c b/src/util/threads-win32.c index de73fd5..45c7eef 100644 --- a/src/util/threads-win32.c +++ b/src/util/threads-win32.c @@ -288,6 +288,22 @@ bool virThreadIsSelf(virThreadPtr thread) return self.thread == thread->thread ? true : false; } +int virThreadSelfID(void) +{ + HANDLE handle = GetCurrentThread(); + HANDLE process = GetCurrentProcess(); + HANDLE thread; + int id; + + DuplicateHandle(process, handle, process, + &thread, 0, FALSE, + DUPLICATE_SAME_ACCESS); + id = (int)thread; + CloseHandle(thread); + return id; +} + + void virThreadJoin(virThreadPtr thread) { if (thread->joinable) { diff --git a/src/util/threads.h b/src/util/threads.h index b3b827d..fa00a91 100644 --- a/src/util/threads.h +++ b/src/util/threads.h @@ -51,6 +51,7 @@ int virThreadCreate(virThreadPtr thread, void virThreadSelf(virThreadPtr thread); bool virThreadIsSelf(virThreadPtr thread); void virThreadJoin(virThreadPtr thread); +int virThreadSelfID(void); int virMutexInit(virMutexPtr m) ATTRIBUTE_RETURN_CHECK; int virMutexInitRecursive(virMutexPtr m) ATTRIBUTE_RETURN_CHECK; -- 1.7.2.3

3 4

[libvirt] [PATCH 0/6 v2] xen: parsing and sexpr escaping fixes
by Cole Robinson 22 Nov '10

22 Nov '10

This series fixes some xen XM parsing and sexpr generation issues. The first three patches fix parsing /etc/xen files that use python style triple quotes (which libvirt will actually generate in certain situations). The last three patches fix generating xend sexpr with the reserved characters & \ or ' v2: Tweak domain schema path usage Cleanup virBufferEscape* Small improvements in xend sexpr formatting More sensible values in sexpr escape tests Cole Robinson (6): schemas: domain: Add more valid file path chars conf: Convert ParseString to use STRPREFIX conf: Fix parsing python style triple quotes xend: urlencode: Properly escape '&' buf: Simplify virBufferEscapeString xend: Escape reserved sexpr characters docs/schemas/domain.rng | 23 ++---- src/util/buf.c | 79 ++++++++++++++------ src/util/buf.h | 1 + src/util/conf.c | 20 +++-- src/xen/xend_internal.c | 113 +++++++++++++++------------ tests/xmconfigdata/test-escape-paths.cfg | 2 +- tests/xmconfigdata/test-escape-paths.xml | 5 + tests/xml2sexprdata/xml2sexpr-escape.sexpr | 1 + tests/xml2sexprdata/xml2sexpr-escape.xml | 24 ++++++ tests/xml2sexprtest.c | 1 + 10 files changed, 172 insertions(+), 97 deletions(-) create mode 100644 tests/xml2sexprdata/xml2sexpr-escape.sexpr create mode 100644 tests/xml2sexprdata/xml2sexpr-escape.xml -- 1.7.3.2

2 8

[libvirt] [PATCH] qed: Minor updates to QED support patches
by Adam Litke 22 Nov '10

22 Nov '10

This patch makes two corrections to the newly-added QED support patch series: - Correct the QED header field offsets - Remove XML parsing for VIR_STORAGE_FILE_AUTO_SAFE Signed-off-by: Adam Litke <agl(a)us.ibm.com> --- src/util/storage_file.c | 6 +++--- src/util/storage_file.h | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/util/storage_file.c b/src/util/storage_file.c index aa117e7..2612eb6 100644 --- a/src/util/storage_file.c +++ b/src/util/storage_file.c @@ -42,7 +42,7 @@ VIR_ENUM_IMPL(virStorageFileFormat, VIR_STORAGE_FILE_LAST, - "raw", "probe", "dir", "bochs", + "raw", "dir", "bochs", "cloop", "cow", "dmg", "iso", "qcow", "qcow2", "qed", "vmdk", "vpc") @@ -108,8 +108,8 @@ qedGetBackingStore(char **, int *, const unsigned char *, size_t); #define QCOW2_HDR_EXTENSION_BACKING_FORMAT 0xE2792ACA #define QED_HDR_FEATURES_OFFSET (4+4+4+4) -#define QED_HDR_IMAGE_SIZE (QED_HDR_FEATURES_OFFSET+8+8+8) -#define QED_HDR_BACKING_FILE_OFFSET (QED_HDR_IMAGE_SIZE+8+8) +#define QED_HDR_IMAGE_SIZE (QED_HDR_FEATURES_OFFSET+8+8+8+8) +#define QED_HDR_BACKING_FILE_OFFSET (QED_HDR_IMAGE_SIZE+8) #define QED_HDR_BACKING_FILE_SIZE (QED_HDR_BACKING_FILE_OFFSET+4) #define QED_F_BACKING_FILE 0x01 #define QED_F_BACKING_FORMAT_NO_PROBE 0x04 diff --git a/src/util/storage_file.h b/src/util/storage_file.h index 1b91830..04c1bb2 100644 --- a/src/util/storage_file.h +++ b/src/util/storage_file.h @@ -28,9 +28,9 @@ # include <stdbool.h> enum virStorageFileFormat { + VIR_STORAGE_FILE_AUTO_SAFE = -2, VIR_STORAGE_FILE_AUTO = -1, VIR_STORAGE_FILE_RAW = 0, - VIR_STORAGE_FILE_AUTO_SAFE, VIR_STORAGE_FILE_DIR, VIR_STORAGE_FILE_BOCHS, VIR_STORAGE_FILE_CLOOP, -- 1.7.3.2.164.g6f10c

2 1

[libvirt] [PATCH] Remove trailing ':' from timestamp
by Daniel P. Berrange 22 Nov '10

22 Nov '10

The QEMU logger appends a ':' to the timestamp when it deems it neccessary, so the virTimestamp API should not duplicate this * src/util/util.c: Remove trailing ':' from timestamp --- src/util/util.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/src/util/util.c b/src/util/util.c index a0849e1..a2582aa 100644 --- a/src/util/util.c +++ b/src/util/util.c @@ -2930,7 +2930,7 @@ virTimestamp(void) strftime(timestr, sizeof(timestr), "%Y-%m-%d %H:%M:%S", &time_info); - if (virAsprintf(&timestamp, "%s.%03d: ", + if (virAsprintf(&timestamp, "%s.%03d", timestr, (int) cur_time.tv_usec / 1000) < 0) { return NULL; } -- 1.7.2.3

2 1

[libvirt] [PATCH] Ensure logfile isn't truncated by shutdown message.
by Daniel P. Berrange 22 Nov '10

22 Nov '10

When running non-root, the QEMU log file is usually opened with truncation, since there is no logrotate for non-root usage. This means that when libvirt logs the shutdown timestamp, the log is accidentally truncated * src/qemu/qemu_driver.c: Never truncate log file with shutdown message --- src/qemu/qemu_driver.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index bf6cd0e..2a686dc 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -741,7 +741,7 @@ static int qemuCgroupControllerActive(struct qemud_driver *driver, } static int -qemudLogFD(struct qemud_driver *driver, const char* name) +qemudLogFD(struct qemud_driver *driver, const char* name, bool append) { char logfile[PATH_MAX]; mode_t logmode; @@ -756,7 +756,7 @@ qemudLogFD(struct qemud_driver *driver, const char* name) logmode = O_CREAT | O_WRONLY; /* Only logrotate files in /var/log, so only append if running privileged */ - if (driver->privileged) + if (driver->privileged || append) logmode |= O_APPEND; else logmode |= O_TRUNC; @@ -3964,7 +3964,7 @@ static int qemudStartVMDaemon(virConnectPtr conn, } DEBUG0("Creating domain log file"); - if ((logfile = qemudLogFD(driver, vm->def->name)) < 0) + if ((logfile = qemudLogFD(driver, vm->def->name, false)) < 0) goto cleanup; DEBUG0("Determing emulator version"); @@ -4245,7 +4245,7 @@ static void qemudShutdownVMDaemon(struct qemud_driver *driver, VIR_DEBUG("Shutting down VM '%s' pid=%d migrated=%d", vm->def->name, vm->pid, migrated); - if ((logfile = qemudLogFD(driver, vm->def->name)) < 0) { + if ((logfile = qemudLogFD(driver, vm->def->name, true)) < 0) { /* To not break the normal domain shutdown process, skip the * timestamp log writing if failed on opening log file. */ VIR_WARN("Unable to open logfile: %s", -- 1.7.2.3

2 1

[libvirt] [PATCH 0/4] Support the QED disk image format (V3)
by Adam Litke 22 Nov '10

22 Nov '10

Changes since V2: - VIR_STORAGE_FILE_AUTO_SAFE is not XML parsable - Break out QED header defines by field and add link to QED spec - Removed redundant overflow check in qedGetBackingStore Changes since V1: - Fix virStorageFileMatchesVersion() for formats without version info - Allow backingStore format probing for QED images since it is safe Qemu is about to gain support for a new disk image format: QED. Details on this format (including specification) can be found here: http://wiki.qemu.org/Features/QED. This short series of patches allows QED images to be used with libvirt. Adam Litke (4): Allow probing of image formats without version information QED: Basic support for QED images storage_file: Add a new flag to mark backing files that are safe to probe Support for probing qed image metadata src/conf/domain_conf.c | 4 ++ src/util/storage_file.c | 86 +++++++++++++++++++++++++++++++++++++++++++++- src/util/storage_file.h | 2 + 3 files changed, 90 insertions(+), 2 deletions(-) -- 1.7.3.2.164.g6f10c

2 6

[libvirt] [PATCH 0/4] xen: parsing and sexpr escaping fixes
by Cole Robinson 22 Nov '10

22 Nov '10

This series fixes some xen XM parsing and sexpr generation issues. The first two patches fix parsing /etc/xen files that use python style triple quotes (which libvirt will actually generate in certain situations). The last two patches fix generating xend sexpr with the reserved characters & \ or ' Cole Robinson (4): conf: Convert ParseString to use STRPREFIX conf: Fix parsing python style triple quotes xend: urlencode: Properly escape '&' xend: Escape reserved sexpr characters docs/schemas/domain.rng | 10 +- src/util/buf.c | 79 +++++++++++++++++++ src/util/buf.h | 1 + src/util/conf.c | 20 +++-- src/xen/xend_internal.c | 116 +++++++++++++++------------ tests/xmconfigdata/test-escape-paths.cfg | 2 +- tests/xmconfigdata/test-escape-paths.xml | 5 + tests/xml2sexprdata/xml2sexpr-escape.sexpr | 1 + tests/xml2sexprdata/xml2sexpr-escape.xml | 24 ++++++ tests/xml2sexprtest.c | 1 + 10 files changed, 193 insertions(+), 66 deletions(-) create mode 100644 tests/xml2sexprdata/xml2sexpr-escape.sexpr create mode 100644 tests/xml2sexprdata/xml2sexpr-escape.xml -- 1.7.3.2

2 13

[libvirt] [PATCH] docs: removed outdated reference to virt-mem
by Justin Clift 22 Nov '10

22 Nov '10

The virt-mem program is no longer shipped, but was still being referenced at the bottom of the virsh and libvirtd man pages. This patch removes it from those man pages, addressing BZ# 639603: https://bugzilla.redhat.com/show_bug.cgi?id=639603 --- daemon/libvirtd.pod.in | 2 +- tools/virsh.pod | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/daemon/libvirtd.pod.in b/daemon/libvirtd.pod.in index 04a4cb4..6e699b8 100644 --- a/daemon/libvirtd.pod.in +++ b/daemon/libvirtd.pod.in @@ -163,6 +163,6 @@ PURPOSE =head1 SEE ALSO L<virsh(1)>, L<virt-install(1)>, L<virt-xml-validate(1)>, L<virt-top(1)>, -L<virt-mem(1)>, L<virt-df(1)>, L<http://www.libvirt.org/> +L<virt-df(1)>, L<http://www.libvirt.org/> =cut diff --git a/tools/virsh.pod b/tools/virsh.pod index 5a96116..c97786a 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -1259,6 +1259,7 @@ PURPOSE =head1 SEE ALSO -L<virt-install(1)>, L<virt-xml-validate(1)>, L<virt-top(1)>, L<virt-mem(1)>, L<virt-df(1)>, L<http://www.libvirt.org/> +L<virt-install(1)>, L<virt-xml-validate(1)>, L<virt-top(1)>, L<virt-df(1)>, +L<http://www.libvirt.org/> =cut -- 1.7.3.2

2 2

[libvirt] arbitrary qemu command feature
by Bruce Hohl 22 Nov '10

22 Nov '10

Hello List, I had a problem with the recently added qemu command feature (no one on user list could help): This feature: http://libvirt.org/news.html 0.8.3: Aug 4 2010 Improvements: Handle arbitrary qemu command-lines in qemuParseCommandLine. Qemu arbitrary command-line arguments. Also described here: http://www.spinics.net/linux/fedora/libvir/msg26687.html [PATCH v4 0/8]: Add arbitrary qemu command-line and monitor commands I am using the following libvirt 0.8.3 packages from unbuntu 10.10: libvirt-bin 0.8.3-1ubuntu14 libvirt0 0.8.3-1ubuntu14 python-libvirt 0.8.3-1ubuntu14 Example of problem: "$ virsh edit my.xml" refuses to add / will not save the following to a domain (for this domain graphics type='sdl') using the specified format: <qemu:commandline> <qemu:arg value='-ctrl-grab'/> </qemu:commandline> If I edit and save the xml directly the change is not recognized upon start-up of the domain. That is, I must use ctrl+alt instead of the right ctrl for mouse capture. Am I missing a library or package needed? Am I using this feature correctly? Is there a working example or additional documentation somewhere? Any suggestions?

2 1

[libvirt] [PATCH] support sheepdog volumes
by MORITA Kazutaka 21 Nov '10

21 Nov '10

Sheepdog is a distributed storage system for QEMU. It provides highly available block level storage volumes to VMs like Amazon EBS. This patch adds support for Sheepdog; we can create VMs with sheepdog volumes, and attach sheepdog volumes to running machines via the attach-device command. Sheepdog volumes can be declared like this: <disk type='sheepdog' device='disk'> <driver name='qemu' type='raw' /> <source vdi='volume_name' host='hostname' port='7000'/> <target dev='vda' bus='virtio' /> </disk> 'host' and 'port' in the source element are optional. If they are not specified, sheepdog clients use the default value (localhost:7000). Signed-off-by: MORITA Kazutaka <morita.kazutaka(a)lab.ntt.co.jp> --- docs/schemas/domain.rng | 24 ++++++++++++++++++ src/conf/domain_conf.c | 26 ++++++++++++++++++- src/conf/domain_conf.h | 3 ++ src/qemu/qemu_conf.c | 63 +++++++++++++++++++++++++++++++++++++++++++++- 4 files changed, 113 insertions(+), 3 deletions(-) diff --git a/docs/schemas/domain.rng b/docs/schemas/domain.rng index bbbc846..5802993 100644 --- a/docs/schemas/domain.rng +++ b/docs/schemas/domain.rng @@ -612,6 +612,30 @@ <ref name="diskspec"/> </interleave> </group> + <group> + <attribute name="type"> + <value>sheepdog</value> + </attribute> + <interleave> + <optional> + <element name="source"> + <attribute name="vdi"> + <ref name="genericName"/> + </attribute> + <optional> + <attribute name="host"> + <ref name="genericName"/> + </attribute> + <attribute name="port"> + <ref name="unsignedInt"/> + </attribute> + </optional> + <empty/> + </element> + </optional> + <ref name="diskspec"/> + </interleave> + </group> <ref name="diskspec"/> </choice> </element> diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 30c27db..d0ea0bb 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -113,7 +113,8 @@ VIR_ENUM_IMPL(virDomainDeviceAddress, VIR_DOMAIN_DEVICE_ADDRESS_TYPE_LAST, VIR_ENUM_IMPL(virDomainDisk, VIR_DOMAIN_DISK_TYPE_LAST, "block", "file", - "dir") + "dir", + "sheepdog") VIR_ENUM_IMPL(virDomainDiskDevice, VIR_DOMAIN_DISK_DEVICE_LAST, "disk", @@ -505,6 +506,8 @@ void virDomainDiskDefFree(virDomainDiskDefPtr def) VIR_FREE(def->serial); VIR_FREE(def->src); + VIR_FREE(def->host); + VIR_FREE(def->port); VIR_FREE(def->dst); VIR_FREE(def->driverName); VIR_FREE(def->driverType); @@ -1517,6 +1520,8 @@ virDomainDiskDefParseXML(virCapsPtr caps, char *driverName = NULL; char *driverType = NULL; char *source = NULL; + char *host = NULL; + char *port = NULL; char *target = NULL; char *bus = NULL; char *cachetag = NULL; @@ -1557,6 +1562,11 @@ virDomainDiskDefParseXML(virCapsPtr caps, case VIR_DOMAIN_DISK_TYPE_DIR: source = virXMLPropString(cur, "dir"); break; + case VIR_DOMAIN_DISK_TYPE_SHEEPDOG: + source = virXMLPropString(cur, "vdi"); + host = virXMLPropString(cur, "host"); + port = virXMLPropString(cur, "port"); + break; default: virDomainReportError(VIR_ERR_INTERNAL_ERROR, _("unexpected disk type %s"), @@ -1726,6 +1736,10 @@ virDomainDiskDefParseXML(virCapsPtr caps, def->src = source; source = NULL; + def->host = host; + host = NULL; + def->port = port; + port = NULL; def->dst = target; target = NULL; def->driverName = driverName; @@ -1756,6 +1770,8 @@ cleanup: VIR_FREE(type); VIR_FREE(target); VIR_FREE(source); + VIR_FREE(host); + VIR_FREE(port); VIR_FREE(device); VIR_FREE(driverType); VIR_FREE(driverName); @@ -5831,6 +5847,14 @@ virDomainDiskDefFormat(virBufferPtr buf, virBufferEscapeString(buf, " <source dir='%s'/>\n", def->src); break; + case VIR_DOMAIN_DISK_TYPE_SHEEPDOG: + virBufferEscapeString(buf, " <source vdi='%s'", def->src); + if (def->host && def->port) { + virBufferEscapeString(buf, " host='%s'", def->host); + virBufferEscapeString(buf, " port='%s'", def->port); + } + virBufferVSprintf(buf, "/>\n"); + break; default: virDomainReportError(VIR_ERR_INTERNAL_ERROR, _("unexpected disk type %s"), diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 7d2d6f5..9c929c4 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -120,6 +120,7 @@ enum virDomainDiskType { VIR_DOMAIN_DISK_TYPE_BLOCK, VIR_DOMAIN_DISK_TYPE_FILE, VIR_DOMAIN_DISK_TYPE_DIR, + VIR_DOMAIN_DISK_TYPE_SHEEPDOG, VIR_DOMAIN_DISK_TYPE_LAST }; @@ -172,6 +173,8 @@ struct _virDomainDiskDef { int device; int bus; char *src; + char *host; + char *port; char *dst; char *driverName; char *driverType; diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index 83a117a..84f55cb 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -2731,6 +2731,11 @@ qemuBuildDriveStr(virDomainDiskDefPtr disk, virBufferVSprintf(&opt, "file=fat:floppy:%s,", disk->src); else virBufferVSprintf(&opt, "file=fat:%s,", disk->src); + } else if (disk->type == VIR_DOMAIN_DISK_TYPE_SHEEPDOG) { + if (disk->host && disk->port) + virBufferVSprintf(&opt, "file=sheepdog:%s:%s:%s,", disk->host, disk->port, disk->src); + else + virBufferVSprintf(&opt, "file=sheepdog:%s,", disk->src); } else { virBufferVSprintf(&opt, "file=%s,", disk->src); } @@ -4720,6 +4725,11 @@ int qemudBuildCommandLine(virConnectPtr conn, snprintf(file, PATH_MAX, "fat:floppy:%s", disk->src); else snprintf(file, PATH_MAX, "fat:%s", disk->src); + } else if (disk->type == VIR_DOMAIN_DISK_TYPE_SHEEPDOG) { + if (disk->host && disk->port) + snprintf(file, PATH_MAX, "sheepdog:%s:%s:%s", disk->host, disk->port, disk->src); + else + snprintf(file, PATH_MAX, "sheepdog:%s", disk->src); } else { snprintf(file, PATH_MAX, "%s", disk->src); } @@ -5792,7 +5802,32 @@ qemuParseCommandLineDisk(virCapsPtr caps, values[i] = NULL; if (STRPREFIX(def->src, "/dev/")) def->type = VIR_DOMAIN_DISK_TYPE_BLOCK; - else + else if (STRPREFIX(def->src, "sheepdog:")) { + char *p = def->src; + char *port, *vdi; + + def->type = VIR_DOMAIN_DISK_TYPE_SHEEPDOG; + def->src = strdup(p + strlen("sheepdog:")); + + /* def->src must be [vdiname] or [host]:[port]:[vdiname] */ + port = strchr(def->src, ':'); + if (port) { + *port++ = '\0'; + vdi = strchr(port, ':'); + if (!vdi) { + def = NULL; + qemuReportError(VIR_ERR_INTERNAL_ERROR, + _("cannot parse sheepdog filename '%s'"), p); + goto cleanup; + } + *vdi++ = '\0'; + def->host = def->src; + def->port = strdup(port); + def->src = strdup(vdi); + } + + VIR_FREE(p); + } else def->type = VIR_DOMAIN_DISK_TYPE_FILE; } else { def->type = VIR_DOMAIN_DISK_TYPE_FILE; @@ -6729,7 +6764,10 @@ virDomainDefPtr qemuParseCommandLine(virCapsPtr caps, if (STRPREFIX(val, "/dev/")) disk->type = VIR_DOMAIN_DISK_TYPE_BLOCK; - else + else if (STRPREFIX(val, "sheepdog:")) { + disk->type = VIR_DOMAIN_DISK_TYPE_SHEEPDOG; + val = STRSKIP(val, "sheepdog:"); + } else disk->type = VIR_DOMAIN_DISK_TYPE_FILE; if (STREQ(arg, "-cdrom")) { disk->device = VIR_DOMAIN_DISK_DEVICE_CDROM; @@ -6749,6 +6787,27 @@ virDomainDefPtr qemuParseCommandLine(virCapsPtr caps, disk->dst = strdup(arg + 1); } disk->src = strdup(val); + + if (disk->type == VIR_DOMAIN_DISK_TYPE_SHEEPDOG) { + char *port, *vdi; + + /* disk->src must be [vdiname] or [host]:[port]:[vdiname] */ + port = strchr(disk->src, ':'); + if (port) { + *port++ = '\0'; + vdi = strchr(port, ':'); + if (!vdi) { + qemuReportError(VIR_ERR_INTERNAL_ERROR, + _("cannot parse sheepdog filename '%s'"), val); + goto error; + } + *vdi++ = '\0'; + disk->host = disk->src; + disk->port = strdup(port); + disk->src = strdup(vdi); + } + } + if (!disk->src || !disk->dst) { virDomainDiskDefFree(disk); -- 1.7.1

3 4

[libvirt] [PATCH] allow '#' as valid character for domain name
by Osier Yang 20 Nov '10

20 Nov '10

* docs/schemas/domain.rng --- docs/schemas/domain.rng | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/docs/schemas/domain.rng b/docs/schemas/domain.rng index bbbc846..815134d 100644 --- a/docs/schemas/domain.rng +++ b/docs/schemas/domain.rng @@ -2003,7 +2003,7 @@ </define> <define name="domainName"> <data type="string"> - <param name="pattern">[A-Za-z0-9_\.\+\-&:/]+</param> + <param name="pattern">[A-Za-z0-9_\.\+\-\\#&:/]+</param> </data> </define> <define name="diskSerial"> -- 1.7.3.2

3 4

[libvirt] [PATCH] nwfilter: re-order lock grabbed by IP addr. learn thread
by Stefan Berger 20 Nov '10

20 Nov '10

The IP address learning thread was causing a deadlock when it instantiated a filter while a filter update/change was ongoing. The reason for this was the ordering of locks due to the following calls virNWFilterUnlockFilterUpdates() virNWFilterPoolObjFindByName() The below patch now puts the order of the locks in the above shown order when instantiating the filter from the IP address learning thread. Signed-off-by: Stefan Berger <stefanb(a)us.ibm.com> --- src/nwfilter/nwfilter_gentech_driver.c | 45 ++++++++++++++++++++------------- 1 file changed, 28 insertions(+), 17 deletions(-) Index: libvirt-acl/src/nwfilter/nwfilter_gentech_driver.c =================================================================== --- libvirt-acl.orig/src/nwfilter/nwfilter_gentech_driver.c +++ libvirt-acl/src/nwfilter/nwfilter_gentech_driver.c @@ -552,6 +552,8 @@ virNWFilterRuleInstancesToArray(int nEnt * all its subfilters in a depth-first traversal of the tree of referenced * filters. The name of the interface to which the rules belong must be * provided. Apply the values of variables as needed. + * + * Call this function while holding the NWFilter filter update lock */ static int virNWFilterInstantiate(virConnectPtr conn, @@ -575,8 +577,6 @@ virNWFilterInstantiate(virConnectPtr con void **ptrs = NULL; int instantiate = 1; - virNWFilterLockFilterUpdates(); - virNWFilterHashTablePtr missing_vars = virNWFilterHashTableCreate(0); if (!missing_vars) { virReportOOMError(); @@ -668,8 +668,6 @@ virNWFilterInstantiate(virConnectPtr con err_exit: - virNWFilterUnlockFilterUpdates(); - for (j = 0; j < nEntries; j++) virNWFilterRuleInstFree(insts[j]); @@ -681,6 +679,7 @@ err_exit: } + * Call this function while holding the NWFilter filter update lock static int __virNWFilterInstantiateFilter(virConnectPtr conn, bool teardownOld, @@ -823,23 +822,30 @@ _virNWFilterInstantiateFilter(virConnect ? net->data.direct.linkdev : NULL; int ifindex; + int rc; if (ifaceGetIndex(true, net->ifname, &ifindex)) return 1; - return __virNWFilterInstantiateFilter(conn, - teardownOld, - net->ifname, - ifindex, - linkdev, - net->type, - net->mac, - net->filter, - net->filterparams, - useNewFilter, - conn->nwfilterPrivateData, - false, - foundNewFilter); + virNWFilterLockFilterUpdates(); + + rc = __virNWFilterInstantiateFilter(conn, + teardownOld, + net->ifname, + ifindex, + linkdev, + net->type, + net->mac, + net->filter, + net->filterparams, + useNewFilter, + conn->nwfilterPrivateData, + false, + foundNewFilter); + + virNWFilterUnlockFilterUpdates(); + + return rc; } @@ -857,6 +863,8 @@ virNWFilterInstantiateFilterLate(virConn int rc; bool foundNewFilter = false; + virNWFilterLockFilterUpdates(); + rc = __virNWFilterInstantiateFilter(conn, 1, ifname, @@ -878,6 +886,9 @@ virNWFilterInstantiateFilterLate(virConn _virNWFilterTeardownFilter(ifname); } } + + virNWFilterUnlockFilterUpdates(); + return rc; }

2 2

[libvirt] [PATCH v2] [TCK] nwfilter: add a test case using concurrency
by Stefan Berger 20 Nov '10

20 Nov '10

V2: - following Eric's suggestions from review of V1 - all scripts started by the main script trap on signal 2 and clean up - the main program waits for all child processes to have terminated - the test now requires each child process to do 10 steps: - 1 VM start-destroy cycle is 1 step - 50 filter changes are considered 1 step - the test terminates after 3 minutes (in case the test system is very slow / busy) with a bail out message Now that the existing scripts are cleaned up and my POSIX compliancy shell-scripting skills have temporarily:-) improved, I am now adding a test case that exercises concurrency. The test case creates and destroys 2 VMs concurrently as well as changes their referenced filters in a tight loop. This kind of test previously uncovered - deadlocks in libvirt due to lock-ordering in the nwfilter subsystem - libvirt termination bug in libaugeas due to doubly closed file descriptors and the side effects All of these have been fixed recently. The test script is known to run in bash, dash and ksh shells. Signed-off-by: Stefan Berger<stefanb(a)us.ibm.com> --- scripts/nwfilter/060-concurrency.t | 5 scripts/nwfilter/concurrency/chg-vm-filter.sh | 29 + scripts/nwfilter/concurrency/start-destroy-vm.sh | 31 + scripts/nwfilter/concurrency/tck-vm1-filter1.xml | 26 + scripts/nwfilter/concurrency/tck-vm1-filter2.xml | 26 + scripts/nwfilter/concurrency/tck-vm1.xml | 30 + scripts/nwfilter/concurrency/tck-vm2-filter1.xml | 26 + scripts/nwfilter/concurrency/tck-vm2-filter2.xml | 26 + scripts/nwfilter/concurrency/tck-vm2.xml | 30 + scripts/nwfilter/nwfilter_concurrent.sh | 406 +++++++++++++++++++++++ 10 files changed, 635 insertions(+) Index: libvirt-tck/scripts/nwfilter/060-concurrency.t =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/060-concurrency.t @@ -0,0 +1,5 @@ +#!/bin/sh + +pwd=$(dirname -- "$0") + +(cd -- "${pwd}"&& sh ./nwfilter_concurrent.sh --tap-test) Index: libvirt-tck/scripts/nwfilter/nwfilter_concurrent.sh =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/nwfilter_concurrent.sh @@ -0,0 +1,406 @@ +#!/bin/sh + +VIRSH=virsh + +# For each line starting with uri=, remove the prefix and set the hold +# space to the rest of the line. Then at file end, print the hold +# space, which is effectively the last uri= line encountered. +uri=$(sed -n '/^uri[ ]*=[ ]*/ { + s/// + h +} +$ { + x + p +}'< "$LIBVIRT_TCK_CONFIG") +: "${uri:=qemu:///system}" + +LIBVIRT_URI=${uri} + +FLAG_WAIT="$((1<<0))" +FLAG_ATTACH="$((1<<1))" +FLAG_VERBOSE="$((1<<2))" +FLAG_LIBVIRT_TEST="$((1<<3))" +FLAG_TAP_TEST="$((1<<4))" +FLAG_FORCE_CLEAN="$((1<<5))" + +failctr=0 +passctr=0 +attachfailctr=0 +attachctr=0 + +TAP_FAIL_LIST="" +TAP_FAIL_CTR=0 +TAP_TOT_CTR=0 + +usage() { + cmd="$0" +cat<<EOF +Usage: ${cmd} [--help|-h|-?] [--noattach] [--wait] [--verbose] + [--libvirt-test] [--tap-test] + +Options: + --help,-h,-? : Display this help screen. + --noattach : Skip tests that attach and detach a network interface + --wait : Wait for the user to press the enter key once an error + was detected + --verbose : Verbose output + --libvirt-test : Use the libvirt test output format + --tap-test : TAP format output + --force : Allow the automatic cleaning of VMs and nwfilters + previously created by the TCK test suite + +This test will create two virtual machines. The one virtual machine +will use a filter called '${TESTFILTERNAME}', and reference the filter +'clean-traffic' which should be available by default with every install. +The other virtual machine will reference the filter 'tck-testcase' and will +have its filter permanently updated. +EOF +} + + +tap_fail() { + txt=$(echo "$2" | gawk '{print substr($0,1,66)}') + echo "not ok $1 - ${txt}" + TAP_FAIL_LIST="$TAP_FAIL_LIST $1 " + TAP_FAIL_CTR=$(($TAP_FAIL_CTR + 1)) + TAP_TOT_CTR=$(($TAP_TOT_CTR + 1)) +} + +tap_pass() { + txt=$(echo "$2" | gawk '{print substr($0,1,70)}') + echo "ok $1 - ${txt}" + TAP_TOT_CTR=$(($TAP_TOT_CTR + 1)) +} + +tap_final() { + [ -n "${TAP_FAIL_LIST}" ]&& echo "FAILED tests ${TAP_FAIL_LIST}" + + okay=`echo "($TAP_TOT_CTR-$TAP_FAIL_CTR)*100/$TAP_TOT_CTR" | bc -l` + txt=$(echo $okay | gawk '{print substr($0,1,5)}') + echo "Failed ${TAP_FAIL_CTR}/${TAP_TOT_CTR} tests, ${txt}% okay" +} + +# A wrapper for mktemp in case it does not exist +# Echos the name of a temporary file. +mktmpdir() { + { + tmp=$( (umask 077&& mktemp -d ./nwfvmtest.XXXXXX) 2>/dev/null)&& + test -n "$tmp"&& test -d "$tmp" + } || + { + tmp=./nwfvmtest$$-$RANDOM + (umask 077&& mkdir "$tmp") + } || { echo "failed to create secure temporary directory">&2; exit 1; } + echo "${tmp}" + return 0 +} + + +PRGDIR="./concurrency" +CREATE_DES_VM1="${PRGDIR}/start-destroy-vm.sh 1" +CREATE_DES_VM2="${PRGDIR}/start-destroy-vm.sh 2" +CHG_FILTER_VM1="${PRGDIR}/chg-vm-filter.sh 1" +CHG_FILTER_VM2="${PRGDIR}/chg-vm-filter.sh 2" + + +startPrgs() +{ + flags="$1" + + sh ${CHG_FILTER_VM1} "$4" 2>&1>/dev/null& + [ $? -ne 0 ]&& \ + ( killPrgs "${flags}" "Could not start program 3" ; return 1; ) + CHG_FILTER_VM1_THR=$! + + sh ${CHG_FILTER_VM2} "$5" 2>&1>/dev/null& + [ $? -ne 0 ]&& \ + ( killPrgs "${flags}" "Could not start program 4" ; return 1; ) + CHG_FILTER_VM2_THR=$! + + # Give some time for the filters to be created + sleep 2 + + sh ${CREATE_DES_VM1} "$2" 2>&1>/dev/null& + [ $? -ne 0 ]&& \ + ( killPrgs "${flags}" "Could not start program 1" ; return 1; ) + CREATE_DES_VM1_THR=$! + + sh ${CREATE_DES_VM2} "$3" 2>&1>/dev/null& + [ $? -ne 0 ]&& \ + ( killPrgs "${flags}" "Could not start program 2" ; return 1; ) + CREATE_DES_VM2_THR=$! + +} + + +killPrgs() +{ + msg="$1" + + # terminate all process + [ "x${CREATE_DES_VM1_THR}x" != "xx" ]&& \ + kill -2 ${CREATE_DES_VM1_THR} + [ "x${CREATE_DES_VM2_THR}x" != "xx" ]&& \ + kill -2 ${CREATE_DES_VM2_THR} + [ "x${CHG_FILTER_VM1_THR}x" != "xx" ]&& \ + kill -2 ${CHG_FILTER_VM1_THR} + [ "x${CHG_FILTER_VM2_THR}x" != "xx" ]&& \ + kill -2 ${CHG_FILTER_VM2_THR} + + wait +} + + +testFail() +{ + flags="$1" + msg="$2" + + failctr=$(($failctr + 1)) + if [ $(($flags& $FLAG_VERBOSE)) -ne 0 ]; then + echo "FAIL : ${msg}" + fi + if [ $(($flags& $FLAG_WAIT)) -ne 0 ]; then + echo "Press enter" + read enter + fi + [ $(($flags& $FLAG_LIBVIRT_TEST)) -ne 0 ]&& \ + test_result $(($passctr + $failctr)) "" 1 + [ $(($flags& $FLAG_TAP_TEST)) -ne 0 ]&& \ + tap_fail $(($passctr + $failctr)) "${msg}" +} + + +testPass() +{ + flags="$1" + msg="$2" + + passctr=$(($passctr + 1)) + if [ $(($flags& $FLAG_VERBOSE)) -ne 0 ]; then + echo "PASS : ${msg}" + fi + [ $(($flags& $FLAG_LIBVIRT_TEST)) -ne 0 ]&& \ + test_result $(($passctr + $failctr)) "" 0 + [ $(($flags& $FLAG_TAP_TEST)) -ne 0 ]&& \ + tap_pass $(($passctr + $failctr)) "${msg}" +} + +cleanup() +{ + rm -rf "$1" + killPrgs + exit $2 +} + +missedSteps() +{ + cur="$1" + exp="$2" + flags="$3" + + while [ $cur -lt $exp ]; do + cur=$(($cur + 1)) + testFail "${flags}" "$4 ${cur}" + done +} + +runTest() +{ + flags="$1" + + passctr=0 + failctr=0 + + tmpdir=`mktmpdir` + failctr=0 + passctr=0 + logvm1="${PWD}/${tmpdir}/logvm1" + logvm2="${PWD}/${tmpdir}/logvm2" + logfivm1="${PWD}/${tmpdir}/logfivm1" + logfivm2="${PWD}/${tmpdir}/logfivm2" + + # exp. number of steps each 'thread' has to do + steps=10 + + steps_vm1=0 + steps_vm2=0 + steps_fivm1=0 + steps_fivm2=0 + + trap "cleanup ${tmpdir} 1" INT + + if [ $(($flags& $FLAG_TAP_TEST)) -ne 0 ]; then + # Need to display the number of total tests + tap_total=$((4 * $steps)) + echo "1..${tap_total}" + fi + + startPrgs "${flags}" "${logvm1}" "${logvm2}" \ + "${logfivm1}" "${logfivm2}" + + [ $? -ne 0 ]&& rm -rf "${tmpdir}"&& return 1; + + # Test runs for a maximum of 3 minutes + now=`date +%s` + test_end=$(($now + 3 * 60)) + + while :; + do + # The logs give us the number of cycles the VMs were created + # and destroyed. + val=$(cat "${logvm1}" 2>/dev/null | tail -n 1) + while [ -n "${val}" ]&& \ + [ $steps_vm1 -lt $steps ]&& [ $steps_vm1 -lt $val ]; do + steps_vm1=$(($steps_vm1 + 1)) + testPass "${flags}" \ + "VM1 log - step ${steps_vm1}" + done + + val=$(cat "${logvm2}" 2>/dev/null | tail -n 1) + while [ -n "${val}" ]&& \ + [ $steps_vm2 -lt $steps ]&& [ $steps_vm2 -lt $val ]; do + steps_vm2=$(($steps_vm2 + 1)) + testPass "${flags}" \ + "VM2 log - step ${steps_vm2}" + done + + # The changing of the filters is expected to work much faster + val=$(cat "${logfivm1}" 2>/dev/null | tail -n 1) + while [ -n "${val}" ]&& \ + [ $steps_fivm1 -lt $steps ]&& [ $steps_fivm1 -lt $(($val / 50)) ]; + do + steps_fivm1=$(($steps_fivm1 + 1)) + testPass "${flags}" \ + "VM1 filter log - step ${steps_fivm1}" + done + + val=$(cat "${logfivm2}" 2>/dev/null | tail -n 1) + while [ -n "${val}" ]&& \ + [ $steps_fivm2 -lt $steps ]&& [ $steps_fivm2 -lt $(($val / 50)) ]; + do + steps_fivm2=$(($steps_fivm2 + 1)) + testPass "${flags}" \ + "VM2 filter log - step ${steps_fivm2}" + done + + [ $steps_vm1 -ge $steps ]&& \ + [ $steps_vm2 -ge $steps ]&& \ + [ $steps_fivm1 -ge $steps ]&& \ + [ $steps_fivm2 -ge $steps ]&& \ + break + + now=`date +%s` + [ $now -gt $test_end ]&& break + + sleep 4 + done + + missedSteps $steps_vm1 $steps $flags "VM1 log - step " + missedSteps $steps_vm2 $steps $flags "VM2 log - step " + missedSteps $steps_fivm1 $steps $flags "VM1 filter log - step " + missedSteps $steps_fivm1 $steps $flags "VM2 filter log - step " + + [ $now -gt $test_end ]&& \ + echo "Bail out! Not all tests finished before test expired. Busy system?" + + cleanup "${tmpdir}" 0 +} + + +main() { + prgname="$0" + xmldir="nwfilterxml2xmlin" + fwalldir="nwfilterxml2fwallout" + found=0 + filtername="tck-testcase" + libvirtdpid=-1 + + flags=${FLAG_ATTACH} + + while [ $# -ne 0 ]; do + case "$1" in + --help|-h|-\?) usage ${prgname}; exit 0;; + --noattach) flags=$(($flags& ~$FLAG_ATTACH));; + --wait) flags=$(($flags | $FLAG_WAIT ));; + --verbose) flags=$(($flags | $FLAG_VERBOSE ));; + --libvirt-test) flags=$(($flags | $FLAG_LIBVIRT_TEST ));; + --tap-test) flags=$(($flags | $FLAG_TAP_TEST ));; + --force) flags=$(($flags | $FLAG_FORCE_CLEAN ));; + *) usage ${prgname}; exit 1;; + esac + shift 1 + done + + if [ `uname` != "Linux" ]; then + if [ $(($flags& $FLAG_TAP_TEST)) -ne 0 ]; then + echo "1..0 # Skipped: Only valid on Linux hosts" + else + echo "This script will only run on Linux." + fi + exit 1; + fi + + if [ $(($flags& $FLAG_TAP_TEST)) -ne 0 ]; then + if [ "${LIBVIRT_URI}" != "qemu:///system" ]; then + echo "1..0 # Skipped: Only valid for Qemu system driver" + exit 0 + fi + + for name in `virsh list --all | awk '{print $2}'` + do + case ${name} in + tck*) + if [ "x${LIBVIRT_TCK_AUTOCLEAN}" = "x1" ] || \ + [ $(($flags& $FLAG_FORCE_CLEAN)) -ne 0 ]; then + res=$(virsh destroy ${name} 2>&1) + rc1=$? + res=$(virsh undefine ${name} 2>&1) + rc2=$? + if [ $rc1 -ne 0 ]&& [ $rc2 -ne 0 ]; then + echo "Bail out! Could not destroy VM ${name}: ${res}" + exit 0 + fi + else + echo "Bail out! VM ${name} already exists, use --force to clean" + exit 1 + fi + esac + done + + for name in `virsh nwfilter-list | awk '{print $2}'` + do + case ${name} in + tck*) + if [ "x${LIBVIRT_TCK_AUTOCLEAN}" = "x1" ] || \ + [ $(($flags& $FLAG_FORCE_CLEAN)) -ne 0 ]; then + res=$(virsh nwfilter-undefine ${name} 2>&1) + if [ $? -ne 0 ]; then + echo "Bail out! Could not undefine filter ${name}: ${res}" + exit 1 + fi + else + echo "Bail out! Filter ${name} already exists, use --force to clean" + exit 1 + fi + esac + done + fi + + if [ $(($flags& $FLAG_LIBVIRT_TEST)) -ne 0 ]; then + curdir="${PWD}" + . test-lib.sh + if [ $? -ne 0 ]; then + exit 1 + fi + test_intro $this_test + cd "${curdir}" || { echo "cd failed">&2; exit 1;} + fi + + runTest "${flags}" + + return 0 +} + +main "$@" Index: libvirt-tck/scripts/nwfilter/concurrency/chg-vm-filter.sh =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/concurrency/chg-vm-filter.sh @@ -0,0 +1,29 @@ +#!/bin/sh +cleanup() +{ + virsh nwfilter-undefine tck-vm${idx}-filter 2>/dev/null +} + +cd $(dirname "$0") +ctr=0 +[ -z "$2" ]&& exit 1 +idx="$1" +logfile="$2" +rm -f "${logfile}" +touch "${logfile}" + +trap cleanup 2 + +while :; +do + virsh nwfilter-define tck-vm${idx}-filter1.xml + [ $? -ne 0 ]&& break + [ ! -w "${logfile}" ]&& break + virsh nwfilter-define tck-vm${idx}-filter2.xml + [ $? -ne 0 ]&& break + ctr=$(($ctr + 1)) + [ ! -w "${logfile}" ]&& break + echo "${ctr}">> "${logfile}" +done + +cleanup Index: libvirt-tck/scripts/nwfilter/concurrency/start-destroy-vm.sh =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/concurrency/start-destroy-vm.sh @@ -0,0 +1,31 @@ +#!/bin/sh + +cleanup() +{ + virsh destroy tck-vm${idx} 2>/dev/null + virsh nwfilter-undefine tck-vm${idx}-filter 2>/dev/null +} + +cd $(dirname "$0") +ctr=0 +[ -z "$2" ]&& exit 1 +idx="$1" +logfile="$2" +rm -f "${logfile}" +touch "${logfile}" + +trap cleanup 2 + +while :; +do + virsh create tck-vm${idx}.xml + [ $? -ne 0 ]&& break + sleep 2 + virsh destroy tck-vm${idx} + [ $? -ne 0 ]&& break + ctr=$(($ctr + 1)) + [ ! -w "${logfile}" ]&& break + echo "${ctr}">> "${logfile}" +done + +cleanup Index: libvirt-tck/scripts/nwfilter/concurrency/tck-vm1-filter1.xml =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/concurrency/tck-vm1-filter1.xml @@ -0,0 +1,26 @@ +<filter name='tck-vm1-filter' chain='root'> +<uuid>464d2617-43d0-7694-b479-320b72dac187</uuid> +<filterref filter='clean-traffic'/> +<rule action='accept' direction='in' priority='500'> +<all comment='test'/> +</rule> +<rule action='accept' direction='out' priority='500'> +<all/> +</rule> +<rule action='accept' direction='in' priority='500'> +<tcp dstportstart='21' dstportend='22'/> +</rule> +<rule action='accept' direction='in' priority='500'> +<tcp dstportstart='80'/> +</rule> +<rule action='accept' direction='out' priority='500'> +<icmp/> +</rule> +<rule action='accept' direction='out' priority='500'> +<udp dstportstart='53'/> +</rule> +<rule action='drop' direction='inout' priority='500'> +<all/> +</rule> +</filter> + Index: libvirt-tck/scripts/nwfilter/concurrency/tck-vm1-filter2.xml =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/concurrency/tck-vm1-filter2.xml @@ -0,0 +1,26 @@ +<filter name='tck-vm1-filter' chain='root'> +<uuid>464d2617-43d0-7694-b479-320b72dac187</uuid> +<filterref filter='clean-traffic'/> +<rule action='accept' direction='in' priority='500'> +<all comment='test again'/> +</rule> +<rule action='accept' direction='out' priority='500'> +<all/> +</rule> +<rule action='accept' direction='in' priority='500'> +<tcp dstportstart='21' dstportend='22'/> +</rule> +<rule action='accept' direction='in' priority='500'> +<tcp dstportstart='80'/> +</rule> +<rule action='accept' direction='out' priority='500'> +<icmp/> +</rule> +<rule action='accept' direction='out' priority='500'> +<udp dstportstart='53'/> +</rule> +<rule action='drop' direction='inout' priority='500'> +<all/> +</rule> +</filter> + Index: libvirt-tck/scripts/nwfilter/concurrency/tck-vm1.xml =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/concurrency/tck-vm1.xml @@ -0,0 +1,30 @@ +<domain type='kvm'> +<name>tck-vm1</name> +<memory>32768</memory> +<currentMemory>32768</currentMemory> +<vcpu>1</vcpu> +<os> +<type>hvm</type> +<boot dev='hd'/> +</os> +<features> +<acpi/> +<apic/> +</features> +<clock offset='utc'/> +<on_poweroff>destroy</on_poweroff> +<on_reboot>restart</on_reboot> +<on_crash>destroy</on_crash> +<devices> +<interface type='bridge'> +<source bridge='virbr0'/> +<filterref filter='tck-vm1-filter'> +</filterref> +<target dev='tck-vm1-if0'/> +</interface> +<console type='pty'> +</console> +<input type='mouse' bus='ps2'/> +<graphics type='vnc' port='-1' autoport='yes'/> +</devices> +</domain> Index: libvirt-tck/scripts/nwfilter/concurrency/tck-vm2-filter1.xml =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/concurrency/tck-vm2-filter1.xml @@ -0,0 +1,26 @@ +<filter name='tck-vm2-filter' chain='root'> +<uuid>364d2617-43d0-7694-b479-320b72dac187</uuid> +<filterref filter='clean-traffic'/> +<rule action='accept' direction='in' priority='500'> +<all comment='another test'/> +</rule> +<rule action='accept' direction='out' priority='500'> +<all/> +</rule> +<rule action='accept' direction='in' priority='500'> +<tcp dstportstart='21' dstportend='22'/> +</rule> +<rule action='accept' direction='in' priority='500'> +<tcp dstportstart='80'/> +</rule> +<rule action='accept' direction='out' priority='500'> +<icmp/> +</rule> +<rule action='accept' direction='out' priority='500'> +<udp dstportstart='53'/> +</rule> +<rule action='drop' direction='inout' priority='500'> +<all/> +</rule> +</filter> + Index: libvirt-tck/scripts/nwfilter/concurrency/tck-vm2-filter2.xml =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/concurrency/tck-vm2-filter2.xml @@ -0,0 +1,26 @@ +<filter name='tck-vm2-filter' chain='root'> +<uuid>364d2617-43d0-7694-b479-320b72dac187</uuid> +<filterref filter='clean-traffic'/> +<rule action='accept' direction='in' priority='500'> +<all comment='another test again'/> +</rule> +<rule action='accept' direction='out' priority='500'> +<all/> +</rule> +<rule action='accept' direction='in' priority='500'> +<tcp dstportstart='21' dstportend='22'/> +</rule> +<rule action='accept' direction='in' priority='500'> +<tcp dstportstart='80'/> +</rule> +<rule action='accept' direction='out' priority='500'> +<icmp/> +</rule> +<rule action='accept' direction='out' priority='500'> +<udp dstportstart='53'/> +</rule> +<rule action='drop' direction='inout' priority='500'> +<all/> +</rule> +</filter> + Index: libvirt-tck/scripts/nwfilter/concurrency/tck-vm2.xml =================================================================== --- /dev/null +++ libvirt-tck/scripts/nwfilter/concurrency/tck-vm2.xml @@ -0,0 +1,30 @@ +<domain type='kvm'> +<name>tck-vm2</name> +<memory>32768</memory> +<currentMemory>32768</currentMemory> +<vcpu>1</vcpu> +<os> +<type>hvm</type> +<boot dev='hd'/> +</os> +<features> +<acpi/> +<apic/> +</features> +<clock offset='utc'/> +<on_poweroff>destroy</on_poweroff> +<on_reboot>restart</on_reboot> +<on_crash>destroy</on_crash> +<devices> +<interface type='bridge'> +<source bridge='virbr0'/> +<filterref filter='tck-vm2-filter'> +</filterref> +<target dev='tck-vm2-if0'/> +</interface> +<console type='pty'> +</console> +<input type='mouse' bus='ps2'/> +<graphics type='vnc' port='-1' autoport='yes'/> +</devices> +</domain>

2 2

[libvirt] rbd storage pool support for libvirt
by Sage Weil 19 Nov '10

19 Nov '10

Hi, We've been working on RBD, a distributed block device backed by the Ceph distributed object store. (Ceph is a highly scalable, fault tolerant distributed storage and file system; see http://ceph.newdream.net.) Although the Ceph file system client has been in Linux since 2.6.34, the RBD block device was just merged for 2.6.37. We also have patches pending for Qemu that use librados to natively talk to the Ceph storage backend, avoiding any kernel dependency. To support disks backed by RBD in libvirt, we originally proposed a 'virtual' type that simply passed the configuration information through to qemu, but that idea was shot down for a variety of reasons: http://www.redhat.com/archives/libvir-list/2010-June/thread.html#00257 It sounds like the "right" approach is to create a storage pool type. Ceph also has a 'pool' concept that contains some number of RBD images and a command line tool to manipulate (create, destroy, resize, rename, snapshot, etc.) those images, which seems to map nicely onto the storage pool abstraction. For example, $ rbd create foo -s 1000 rbd image 'foo': size 1000 MB in 250 objects order 22 (4096 KB objects) adding rbd image to directory... creating rbd image... done. $ rbd create bar -s 10000 [...] $ rbd list bar foo Something along the lines of <pool type="rbd"> <name>virtimages</name> <source mode="kernel"> <host monitor="ceph-mon1.domain.com:6789"/> <host monitor="ceph-mon2.domain.com:6789"/> <host monitor="ceph-mon3.domain.com:6789"/> <pool name="rbd"/> </source> </pool> or whatever (I'm not too familiar with the libvirt schema)? One difference between the existing pool types listed at libvirt.org/storage.html is that RBD does not necessarily associate itself with a path in the local file system. If the native qemu driver is used, there is no path involved, just a magic string passed to qemu (rbd:poolname/imagename). If the kernel RBD driver is used, it gets mapped to a /dev/rbd/$n (or similar, depending on the udev rule), but $n is not static across reboots. In any case, before someone goes off and implements something, does this look like the right general approach to adding rbd support to libvirt? Thanks! sage

4 11

[libvirt] [PATCH] doc: Add doc for missed options of migrate
by Osier Yang 19 Nov '10

19 Nov '10

* tools/virsh.pod (add doc for options like "--p2p", "--direct", "--copy-storage-all", "dname", etc.) --- tools/virsh.pod | 23 ++++++++++++++++------- 1 files changed, 16 insertions(+), 7 deletions(-) diff --git a/tools/virsh.pod b/tools/virsh.pod index c4a5803..036848d 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -456,13 +456,22 @@ Provide the maximum number of virtual CPUs supported for a guest VM on this connection. If provided, the I<type> parameter must be a valid type attribute for the <domain> element of XML. -=item B<migrate> optional I<--live> I<--suspend> I<domain-id> I<desturi> -I<migrateuri> - -Migrate domain to another host. Add --live for live migration; --suspend -leaves the domain paused on the destination host. The I<desturi> is the -connection URI of the destination host, and I<migrateuri> is the -migration URI, which usually can be omitted. +=item B<migrate> optional I<--live> I<--p2p> I<--direct> I<--tunnelled> +I<--persistent> I<--undefinesource> I<--suspend> I<--copy-storage-all> +I<--copy-storage-inc> I<domain-id> I<desturi> I<migrateuri> I<dname> + +Migrate domain to another host. Add I<--live> for live migration; I<--p2p> +for peer-2-peer migration; I<--direct> for direct migration; I<--tunnelled> +for tunnelled migration; I<--persistent> leaves the domain persistent on +destination host, I<--undefinesource> undefine the domain on source host, +I<--suspend> leaves the domain paused on the destination host. +I<--copy-storage-all> indicates migration with non-shared storage with full +disk copy, I<--copy-storage-inc> indicates migration with non-shared storage +with incremental copy (same base image shared between source and destination). +The I<desturi> is the connection URI of the destination host, and +I<migrateuri> is the migration URI, which usually can be omitted. +I<dname> is used for renaming the domain to new name during migration, which +also usually can be ommited. =item B<migrate-setmaxdowntime> I<domain-id> I<downtime> -- 1.7.3.2

2 2

[libvirt] [PATCH 0/2] Support of auto-dump on watchdog event in libvirtd
by Hu Tao 19 Nov '10

19 Nov '10

This patch series adds support of auto-dump on watchdog event in libvirtd. If a qemu guest is configured with a watchdog device and libvirtd receives a watchdog event from the guest, it can do something on the guest, currently `dump' action is defined. This patch series adds a new watchdog action `managed', it has a subaction wich indicates the action libvirtd will take on a watchdog event. The meaning of subaction_arg depends on subaction. If subaction is `dump', then subaction_arg defines a folder path into which libvirtd saves dumpfile. An example: <watchdog model='i6300esb' action='managed' subaction='dump' subaction_arg='/mnt/data/dumps'> In order to make the function work, there must be a watchdog device added to guest, and guest must have a watchdog daemon running, for example, /etc/init.d/watchdog start or auto-started on boot. Suggestions are welcome! Hu Tao (2): Add a thread pool implementation Add a watchdog action `managed'. src/Makefile.am | 3 +- src/conf/domain_conf.c | 27 ++++++++- src/conf/domain_conf.h | 11 ++++- src/qemu/qemu_conf.c | 2 +- src/qemu/qemu_conf.h | 6 ++ src/qemu/qemu_driver.c | 138 ++++++++++++++++++++++++++++++++++++++++++----- src/util/threadpool.c | 119 +++++++++++++++++++++++++++++++++++++++++ src/util/threadpool.h | 34 ++++++++++++ 8 files changed, 319 insertions(+), 21 deletions(-) create mode 100644 src/util/threadpool.c create mode 100644 src/util/threadpool.h -- 1.7.3 -- Thanks, Hu Tao

2 6

[libvirt] [PATCH] doc: add doc for missed parameters of attach-disk
by Osier Yang 18 Nov '10

18 Nov '10

* tools/virsh.pod (add docs for --persistent and --sourcetype of attach-disk, break the long lines in the meantime) --- tools/virsh.pod | 10 ++++++++-- 1 files changed, 8 insertions(+), 2 deletions(-) diff --git a/tools/virsh.pod b/tools/virsh.pod index b4644d8..c4a5803 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -645,7 +645,9 @@ See the documentation to learn about libvirt XML format for a device. For cdrom and floppy devices, this command only replaces the media within the single existing device; consider using B<update-device> for this usage. -=item B<attach-disk> I<domain-id> I<source> I<target> optional I<--driver driver> I<--subdriver subdriver> I<--type type> I<--mode mode> +=item B<attach-disk> I<domain-id> I<source> I<target> optional +I<--driver driver> I<--subdriver subdriver> I<--type type> +I<--mode mode> I<--persistent> I<--sourcetype soucetype> Attach a new disk device to the domain. I<source> and I<target> are paths for the files and devices. @@ -654,8 +656,12 @@ I<type> can indicate I<cdrom> or I<floppy> as alternative to the disk default, although this use only replaces the media within the existing virtual cdrom or floppy device; consider using B<update-device> for this usage instead. I<mode> can specify the two specific mode I<readonly> or I<shareable>. +I<persistent> indicates the changes will affect the next boot of the domain. +I<sourcetype> can indicate the type of source (block|file) -=item B<attach-interface> I<domain-id> I<type> I<source> optional I<--target target> I<--mac mac> I<--script script> I<--model model> I<--persistent> +=item B<attach-interface> I<domain-id> I<type> I<source> optional +I<--target target> I<--mac mac> I<--script script> I<--model model> +I<--persistent> Attach a new network interface to the domain. I<type> can be either I<network> to indicate a physical network device or I<bridge> to indicate a bridge to a device. -- 1.7.3.2

2 1

[libvirt] Fix a failure to restore SELinux label for character devices
by Daniel Veillard 18 Nov '10

18 Nov '10

The code in SELinuxRestoreSecurityChardevLabel() was trying to use SELinuxSetFilecon directly for devices or file types while it should really use SELinuxRestoreSecurityFileLabel encapsulating routine, which avoid various problems like resolving symlinks, making sure he file exists and work around NFS problems Daniel diff --git a/src/security/security_selinux.c b/src/security/security_selinux.c index 936a1a6..996177a 100644 --- a/src/security/security_selinux.c +++ b/src/security/security_selinux.c @@ -694,9 +694,10 @@ SELinuxRestoreSecurityChardevLabel(virDomainObjPtr vm, switch (dev->type) { case VIR_DOMAIN_CHR_TYPE_DEV: case VIR_DOMAIN_CHR_TYPE_FILE: - ret = SELinuxSetFilecon(dev->data.file.path, secdef->imagelabel); + if (SELinuxRestoreSecurityFileLabel(dev->data.file.path) < 0) + goto done; + ret = 0; break; - case VIR_DOMAIN_CHR_TYPE_PIPE: if ((virAsprintf(&out, "%s.out", dev->data.file.path) < 0) || (virAsprintf(&in, "%s.in", dev->data.file.path) < 0)) { -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel(a)veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/

3 3

[libvirt] [PATCH 0/2] Support the QED disk image format
by Adam Litke 18 Nov '10

18 Nov '10

Qemu is about to gain support for a new disk image format: QED. Details on this format (including specification) can be found here: http://wiki.qemu.org/Features/QED. This short series of patches allows QED images to be used with libvirt. Adam Litke (2): QED: Basic support for QED images Support for probing qed image metadata src/util/storage_file.c | 85 ++++++++++++++++++++++++++++++++++++++++++++++- src/util/storage_file.h | 1 + 2 files changed, 85 insertions(+), 1 deletions(-) -- 1.7.3.2.164.g6f10c

2 4

[libvirt] Interesting "Mobile apps" food for thought
by Justin Clift 18 Nov '10

18 Nov '10

Just came across this. There's a Linux distribution based off of Centos, called ClearOS, that's targeted at a router/gateway/vpn/etc. They've designed their own API for managing things (nice looking site), and people are creating nifty looking visual mobile apps for it. http://www.clearfoundation.com/docs/developer/api/start Here's screenshot of an Android app, monitoring bandwidth on a remote host (worth looking at): http://www.clearfoundation.com/docs/_detail/developer/api/android_monitor02… This is the kind of thing I can see people making virt apps for, as we get our doc coverage done properly, mobile friendly API's well documented, and our Community really built. :) Regards and best wishes, Justin Clift

2 2

[libvirt] [PATCH] virsh: tell if domain is updated in dominfo
by Osier Yang 18 Nov '10

18 Nov '10

* tools/virsh.c (introduce new information 'Updated' for 'dominfo', standsfor if a domain is updated via operations like 'update-device', 'update-disk', etc.) --- tools/virsh.c | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/tools/virsh.c b/tools/virsh.c index ae88cc0..d260443 100644 --- a/tools/virsh.c +++ b/tools/virsh.c @@ -1992,6 +1992,7 @@ cmdDominfo(vshControl *ctl, const vshCmd *cmd) virSecurityModel secmodel; virSecurityLabel seclabel; int persistent = 0; + int updated = -1; int ret = TRUE, autostart; unsigned int id; char *str, uuid[VIR_UUID_STRING_BUFLEN]; @@ -2053,6 +2054,12 @@ cmdDominfo(vshControl *ctl, const vshCmd *cmd) else vshPrint(ctl, "%-15s %s\n", _("Persistent:"), persistent ? _("yes") : _("no")); + updated = virDomainIsUpdated(dom); + if (updated < 0) + vshPrint(ctl, "%-15s %s\n", _("Updated:"), _("unknown")); + else + vshPrint(ctl, "%-15s %s\n", _("Updated:"), updated ? _("yes") : _("no")); + /* Check and display whether the domain autostarts or not */ if (!virDomainGetAutostart(dom, &autostart)) { vshPrint(ctl, "%-15s %s\n", _("Autostart:"), -- 1.7.3.2

2 2