- Devel - libvirt List Archives

[libvirt] [PATCHv2 00/33] Gluster backing chains and relative block commit/pull
by Peter Krempa 28 May '14

28 May '14

Hi, this is my second take on this topic. Some of the patches are still in a RFC status as the last six patches require qemu support which is still on review upstream. The rest should be fine as is even without the qemu bits. Peter Krempa (33): qemu: process: Refresh backing chain info when reconnecting to qemu qemu: Make qemuDomainPrepareDiskChainElement aware of remote storage storage: Store gluster volume name separately storage: Rework debugging of storage file access through storage driver conf: Fix domain disk path iterator to work with networked storage storage: Add NONE protocol type for network disks storage: Add support for access to files using provided uid/gid storage: Add storage file API to read file headers storage: backend: Add unique id retrieval API storage: Add API to check accessibility of storage volumes storage: Move virStorageFileGetMetadata to the storage driver storage: Determine the local storage type right away test: storage: Initialize storage source to correct type storage: backend: Add possibility to suppress errors from backend lookup storage: Switch metadata crawler to use storage driver to get unique path storage: Switch metadata crawler to use storage driver to read headers storage: Switch metadata crawler to use storage driver file access check storage: Add infrastructure to parse remote network backing names storage: Change to new backing store parser storage: Traverse backing chains of network disks util: string: Return element count from virStringSplit util: string: Add helper to free non-NULL terminated string arrays util: storagefile: Add helper to resolve "../", "./" and "////" in paths util: storage: Add helper to resolve relative path difference util: storagefile: Add canonicalization to virStorageFileSimplifyPath storage: gluster: Add backend to return unique storage file path qemu: json: Add format strings for optional command arguments qemu: monitor: Add argument for specifying backing name for block commit qemu: monitor: Add support for backing name specification for block-stream lib: Introduce flag VIR_DOMAIN_BLOCK_COMMIT_RELATIVE lib: Introduce flag VIR_DOMAIN_BLOCK_REBASE_RELATIVE qemu: Add support for networked disks for block commit qemu: Add support for networked disks for block pull/block rebase cfg.mk | 2 +- include/libvirt/libvirt.h.in | 6 + src/Makefile.am | 2 + src/conf/domain_conf.c | 72 ++- src/libvirt_private.syms | 7 +- src/qemu/qemu_capabilities.c | 8 + src/qemu/qemu_capabilities.h | 2 + src/qemu/qemu_command.c | 24 +- src/qemu/qemu_domain.c | 10 +- src/qemu/qemu_driver.c | 160 +++++-- src/qemu/qemu_migration.c | 6 +- src/qemu/qemu_monitor.c | 21 +- src/qemu/qemu_monitor.h | 4 +- src/qemu/qemu_monitor_json.c | 139 ++++-- src/qemu/qemu_monitor_json.h | 2 + src/qemu/qemu_process.c | 5 + src/security/virt-aa-helper.c | 2 + src/storage/storage_backend.c | 16 +- src/storage/storage_backend.h | 22 +- src/storage/storage_backend_fs.c | 127 +++++- src/storage/storage_backend_gluster.c | 203 +++++++-- src/storage/storage_driver.c | 329 +++++++++++++- src/storage/storage_driver.h | 15 +- src/util/virstoragefile.c | 818 ++++++++++++++++++++++++---------- src/util/virstoragefile.h | 31 +- src/util/virstring.c | 44 +- src/util/virstring.h | 7 + tests/Makefile.am | 7 +- tests/qemumonitorjsontest.c | 2 +- tests/virstoragetest.c | 290 +++++++++++- tools/virsh-domain.c | 29 +- 31 files changed, 1988 insertions(+), 424 deletions(-) -- 1.9.3

2 59

[libvirt] [bug] python-libvirt vcpus mismatch
by Chris Friesen 28 May '14

28 May '14

I've got a libvirt-created instance where I've been messing with affinity, and now something is strange. I did the following in python: >>> import libvirt >>> conn=libvirt.open("qemu:///system") >>> dom = conn.lookupByName('instance-00000027') >>> dom.vcpus() ([(0, 1, 528150000000L, 2), (1, 1, 548070000000L, 3)], [(False, False, True, False), (False, False, True, False)]) I'm totally confused by that "3". It's supposed to represent the physical cpu that virtual cpu 1 is running on. But cpu 3 isn't even in the allowable affinity map for vcpu 1. If I query the data other ways, I get both cpus running on physical cpu 2: root@compute-0:~# virsh vcpupin instance-00000027 VCPU: CPU Affinity ---------------------------------- 0: 2 1: 2 root@compute-0:~# virsh emulatorpin instance-00000027 emulator: CPU Affinity ---------------------------------- *: 2 root@compute-0:~# taskset -pac 15072 pid 15072's current affinity list: 2 pid 15073's current affinity list: 1-3 pid 15075's current affinity list: 2 pid 15076's current affinity list: 0 So I'm left with the conclusion that there is something strange going on with libvirt-python. Anyone got any ideas? Chris

2 1

[libvirt] [PATCH RFC] build: fix build with libselinux 2.3
by Jim Fehlig 28 May '14

28 May '14

The attached patch is an attempt to fix recent build failures I've noticed with libselinux 2.3 CC securityselinuxhelper.lo securityselinuxhelper.c:159:5: error: conflicting types for 'setcon_raw' int setcon_raw(security_context_t context) ^ In file included from securityselinuxhelper.c:30:0: /usr/include/selinux/selinux.h:41:12: note: previous declaration of 'setcon_raw' was here extern int setcon_raw(const char * con); ^ securityselinuxhelper.c:168:5: error: conflicting types for 'setcon' int setcon(security_context_t context) ^ In file included from securityselinuxhelper.c:30:0: /usr/include/selinux/selinux.h:40:12: note: previous declaration of 'setcon' was here extern int setcon(const char * con); ^ securityselinuxhelper.c:174:5: error: conflicting types for 'setfilecon_raw' int setfilecon_raw(const char *path, security_context_t con) ^ In file included from securityselinuxhelper.c:30:0: /usr/include/selinux/selinux.h:110:12: note: previous declaration of 'setfilecon_raw' was here extern int setfilecon_raw(const char *path, const char * con); ^ securityselinuxhelper.c:185:5: error: conflicting types for 'setfilecon' int setfilecon(const char *path, security_context_t con) ^ In file included from securityselinuxhelper.c:30:0: /usr/include/selinux/selinux.h:109:12: note: previous declaration of 'setfilecon' was here extern int setfilecon(const char *path, const char * con); ^ Noticing that security_context_t changed to 'const char *', my first thought was to use AC_CHECK_TYPE to check for security_conext_t, but alas the typedef remains in 2.3 with the comment "No longer used; here for compatibility with legacy callers". I then pursued the approach in this patch of defining a config var based on 'pkg-config --modversion', which works in a test script, but not in the context of the LIBVIRT_CHECK_SELINUX macro. Probably due to some missed quoting, but I'm reaching the m4 knowledge barrier. Before attempting to bypass that, I'd like to see what others think of this approach. Is there a simpler solution? Regards, Jim

2 2

[libvirt] [PATCH] util: fix virTimeLocalOffsetFromUTC DST processing
by Laine Stump 27 May '14

27 May '14

The original version of virTimeLocalOffsetFromUTC() (commit 1cddaea7aeca441b733c31990a3f139dd2d346f6) would fail for certain times of the day if daylight savings time was active. This could most easily be seen by uncommenting the TEST_LOCALOFFSET() cases that invlude a DST setting. After a lot of experimenting, I found that the way to solve it in almost all test cases is to set tm_isdst = -1 in the stuct tm prior to calling mktime(). Once this is done, the correct offset is returned for all test cases at all times except the two hours just after 00:00:00 Jan 1 UTC - during that time, any timezone that is *behind* UTC, and that is supposed to always be in DST will not have DST accounted for in its offset. I still do not know the source of this problem, but it appears to be either a bug in glibc, or my improper specification of a TZ setting, so I am leaving the offending tests listed in virtimetest.c, but disabling them for now. --- See https://www.redhat.com/archives/libvir-list/2014-May/msg00898.html for my earlier comments on this problem. src/util/virtime.c | 3 +++ tests/virtimetest.c | 24 +++++++++++++++++------- 2 files changed, 20 insertions(+), 7 deletions(-) diff --git a/src/util/virtime.c b/src/util/virtime.c index 3a56400..c69dff1 100644 --- a/src/util/virtime.c +++ b/src/util/virtime.c @@ -377,6 +377,9 @@ virTimeLocalOffsetFromUTC(long *offset) return -1; } + /* tell mktime to figure out itself whether or not DST is in effect */ + gmtimeinfo.tm_isdst = -1; + /* mktime() also obeys current timezone rules */ if ((utc = mktime(&gmtimeinfo)) == (time_t)-1) { virReportSystemError(errno, "%s", diff --git a/tests/virtimetest.c b/tests/virtimetest.c index bf27682..35551ea 100644 --- a/tests/virtimetest.c +++ b/tests/virtimetest.c @@ -161,20 +161,30 @@ mymain(void) TEST_LOCALOFFSET("VIR00:30", -30 * 60); TEST_LOCALOFFSET("VIR01:30", -90 * 60); + TEST_LOCALOFFSET("VIR05:00", (-5 * 60) * 60); TEST_LOCALOFFSET("UTC", 0); TEST_LOCALOFFSET("VIR-00:30", 30 * 60); TEST_LOCALOFFSET("VIR-01:30", 90 * 60); -#if __TEST_DST + /* test DST processing with timezones that always * have DST in effect; what's more, cover a zone with * with an unusual DST different than a usual one hour */ - /* NB: These tests fail at certain times of the day, so - * must be disabled until we figure out why - */ - TEST_LOCALOFFSET("VIR-00:30VID,0,365", 90 * 60); - TEST_LOCALOFFSET("VIR-02:30VID,0,365", 210 * 60); - TEST_LOCALOFFSET("VIR-02:30VID-04:30,0,365", 270 * 60); + TEST_LOCALOFFSET("VIR-00:30VID,0/00:00:00,366/23:59:59", + ((1 * 60) + 30) * 60); + TEST_LOCALOFFSET("VIR-02:30VID,0/00:00:00,366/23:59:59", + ((3 * 60) + 30) * 60); + TEST_LOCALOFFSET("VIR-02:30VID-04:30,0/00:00:00,366/23:59:59", + ((4 * 60) + 30) * 60); + TEST_LOCALOFFSET("VIR-12:00VID-13:00,0/00:00:00,366/23:59:59", + ((13 * 60) + 0) * 60); +#ifdef __BROKEN_DST_TESTS + TEST_LOCALOFFSET("VIR02:45VID00:45,0/00:00:00,366/23:59:59", + -45 * 60); + TEST_LOCALOFFSET("VIR05:00VID04:00,0/00:00:00,366/23:59:59", + ((-4 * 60) + 0) * 60); + TEST_LOCALOFFSET("VIR11:00VID10:00,0/00:00:00,366/23:59:59", + ((-10 * 60) + 0) * 60); #endif return ret == 0 ? EXIT_SUCCESS : EXIT_FAILURE; -- 1.9.3

2 2

[libvirt] Python setBlkioParameters function broken
by Qiang Guan 27 May '14

27 May '14

I've tried to use domain.setBlkioParameters with ipython, and I'm finding it's throwing the mysterious error: ubuntu@host-192-168-0-7:~$ dpkg -l | grep libvirt ii libvirt-bin 1.2.2-0ubuntu13.1 amd64 programs for the libvirt library ii libvirt0 1.2.2-0ubuntu13.1 amd64 library for interfacing with different virtualization systems ii python-libvirt 1.2.2-0ubuntu1 amd64 libvirt Python bindings ubuntu@host-192-168-0-7:~$ sudo ipython ...... In [85]: doms[0] Out[85]: <libvirt.virDomain at 0x7fe688879250> In [86]: params Out[86]: {'weight': 250} In [87]: doms[0].setBlkioParameters(params, 0) libvirt: error : argument unsupported: parameter '' not supported --------------------------------------------------------------------------- libvirtError Traceback (most recent call last) <ipython-input-87-026c35502a5c> in <module>() ----> 1 doms[0].setBlkioParameters(params, 0) /usr/lib/python2.7/dist-packages/libvirt.pyc in setBlkioParameters(self, params, flags) 1961 """Change the blkio tunables """ 1962 ret = libvirtmod.virDomainSetBlkioParameters(self._o, params, flags) -> 1963 if ret == -1: raise libvirtError ('virDomainSetBlkioParameters() failed', dom=self) 1964 return ret 1965 libvirtError: argument unsupported: parameter '' not supported The libvirt debug log info are as following: 2014-05-27 08:37:27.752+0000: 7559: debug : remoteDispatchDomainSetBlkioParametersHelper:7328 : server=0x7fd81deec930 client=0x7fd81df05170 msg=0x7fd81df06eb0 rerr=0x7fd816059c80 args=0x7fd7e40009a0 ret=0x7fd7e4000910 2014-05-27 08:37:27.752+0000: 7559: debug : virObjectNew:199 : OBJECT_NEW: obj=0x7fd7e4000c60 classname=virDomain 2014-05-27 08:37:27.752+0000: 7559: debug : virObjectRef:293 : OBJECT_REF: obj=0x7fd7f80011f0 2014-05-27 08:37:27.752+0000: 7559: debug : virDomainSetBlkioParameters:3985 : dom=0x7fd7e4000c60, (VM: name=instance-00000050, uuid=f844cf42-07d1-4b73-8de7-7437bf32aab0), params=0x7fd7e4000930, nparams=1, flags=0 2014-05-27 08:37:27.752+0000: 7559: debug : virDomainSetBlkioParameters:3986 : params[""]=(uint)250 2014-05-27 08:37:27.752+0000: 7559: debug : virObjectRef:293 : OBJECT_REF: obj=0x7fd81deec300 2014-05-27 08:37:27.752+0000: 7559: debug : virAccessManagerCheckConnect:215 : manager=0x7fd81deec300(name=stack) driver=QEMU perm=0 2014-05-27 08:37:27.752+0000: 7559: debug : virAccessManagerCheckConnect:215 : manager=0x7fd81deec370(name=none) driver=QEMU perm=0 2014-05-27 08:37:27.752+0000: 7559: debug : virObjectUnref:256 : OBJECT_UNREF: obj=0x7fd81deec300 2014-05-27 08:37:27.752+0000: 7559: error : virTypedParamsValidate:97 : argument unsupported: parameter '' not supported -- ------------ Jackie Best Regards

1 0

[libvirt] [Patch v2 0/3] use -serial for ppce500 board and add test case
by Olivia Yin 27 May '14

27 May '14

Machine name ppce500 is used to replace ppce500v2 supported by QEMU. QEMU ppce500 board uses the legacy -serial option. Test case ppce500-serial is used to verify this change. Olivia Yin (3): change machine name ppce500v2 as ppce500 qemu: Fix specifying char devs for PPC tests: add test case for -serial option for ppce500 docs/schemas/domaincommon.rng | 2 +- src/qemu/qemu_capabilities.c | 9 +++++++- tests/qemuxml2argvdata/qemuxml2argv-ppc-dtb.args | 2 +- tests/qemuxml2argvdata/qemuxml2argv-ppc-dtb.xml | 2 +- .../qemuxml2argv-ppce500-serial.args | 7 ++++++ .../qemuxml2argv-ppce500-serial.xml | 26 ++++++++++++++++++++++ tests/qemuxml2argvtest.c | 1 + tests/testutilsqemu.c | 2 +- 8 files changed, 46 insertions(+), 5 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-ppce500-serial.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-ppce500-serial.xml -- 1.8.5

2 4

[libvirt] [PATCH 0/3] use -serial for ppce500 board and add test case
by Olivia Yin 27 May '14

27 May '14

Machine name ppce500 is used to replace ppce500v2 supported by QEMU. QEMU ppce500 board uses the old style -serial options. Test case ppce500-serial is used to verify this change. Olivia Yin (3): change machine name ppce500v2 as ppce500 qemu: Fix specifying char devs for PPC tests: add test case for -serial option for ppce500 docs/schemas/domaincommon.rng | 2 +- src/qemu/qemu_capabilities.c | 10 ++++++--- tests/qemuxml2argvdata/qemuxml2argv-ppc-dtb.args | 2 +- tests/qemuxml2argvdata/qemuxml2argv-ppc-dtb.xml | 2 +- .../qemuxml2argv-ppce500-serial.args | 7 ++++++ .../qemuxml2argv-ppce500-serial.xml | 26 ++++++++++++++++++++++ tests/qemuxml2argvtest.c | 1 + tests/testutilsqemu.c | 2 +- 8 files changed, 45 insertions(+), 7 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-ppce500-serial.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-ppce500-serial.xml -- 1.8.5

5 11

[libvirt] Job Control API [RFC]
by Tucker DiNapoli 26 May '14

26 May '14

My name is Tucker DiNapoli and I am working on implementing job control for the storage driver for the google summer of code, the first step in doing this is creating and implementing a unified api for job control. Currently there are several places where various aspects of job control are implemented. The qemu and libxl drivers both contain internal implementations for job control on domain level jobs, with the qemu driver containing support for asynchronous jobs. There is also code in the libvirt.c file for running block jobs and for querying domain jobs for information. I would like for the job control api to be as independent of different drivers as possible since it will need to be used with storage drivers as well as different virtualization drivers. I imagine most of the api will revolve around a job object, and I think it's important to decide what exactly should go in this job object. This is a response from my first post on the mailing list and I think this is a good idea. >>I'd _really_ like to see a common notion of a 'job id' that EVERY job >>(whether domain-level, like migration; or block-level, like >>commit/pull/rebase; or storage-level, like your new proposed storage >>jobs) shares a common job namespace. The job id is a positive integer. >> Existing APIs will have to be retrofitted into the new job id notion; >>any action that starts a long-running job that currently returns 0 on >>success could be changed to return a positive job id; or we may need a >>new API that queries the notion of the 'current job' (the job most >>recently started) or even to set the 'current job' to a different job >>id. We'll need new API for querying a job by id, and to be most >>portable, we should do job reporting via virTypedParameter >>(virDomainGetJobInfo and virDomainGetBlockJobInfo are hardcoded into >>returning a struct, so they are non-extensible; virDomainGetJobStats >>almost did it right, except that the user has to call it twice, once to >>learn how large to allocate, and again to pass in pre-allocated memory - >>the ideal API would allocate the memory on a single call). Currently there are separate types for block job info and job info, if possible I would like to merge these into a common job info type, and perhaps make this a part of the job object itself. Currently (in libxl and qemu) jobs are a part of the domain struct, I think that jobs should be moved out of the domain struct instead using the idea of job ids for domains to keep track of currently running jobs. I'm still new to libvirt so it this doesn't make sense and the idea of keeping job objects attached to domains makes sense that's fine. I think at the minimum each job object should contain: the id of the thread running the job, the type of job, the job id, a condition variable to coordinate jobs, and information about the job, either as a separate job info object or as part of the job object itself. The job should also contain a reference to the domain or storage it is associated with. There are a few basic functions that should definitely be part of the api: initialize a job, free a job, start a job, end a job, abort a job and get info on a job. It would be nice to be able to suspend a job and to change the currently running job as well. That's what I can come up with, but I don't have much experience in libvirt so if there are other features that make sense they can be added as well. Finally (as far as I can think of right now) is the idea of parallel jobs. Currently the qemu driver allows some jobs to be run in parallel by allowing a job to be run asynchronously, this async job has a mask of job types associated with it that determine what types of regular jobs can be run during it. However I would like to allow an arbitrary number of jobs to be run at once (I'm not sure how useful this would be, but it seems best not to impose hard limits on things). The easiest way to deal with this is to just ignore it and put the burden of synchronizing jobs on the drivers. This is obviously a bad solution. Another way would be the way it is currently done it the qemu driver, have a mask of job types associated with each domain/storage which is updated when a job is started or ended which dictates what types of jobs can be started. Regardless of how this is done it will require support from the driver/domain/storage that each job is associated with. Tucker DINapoli

2 1

[libvirt] Fix an extra ' in a translated string
by Daniel Veillard 26 May '14

26 May '14

Raised by ukrainian translator Yuri Chornoivan https://fedora.transifex.com/projects/p/libvirt/translate/#uk/strings/25483… Pushed as trivial fix, Daniel diff --git a/src/storage/storage_driver.c b/src/storage/storage_driver.c index 9575344..cd6babe 100644 --- a/src/storage/storage_driver.c +++ b/src/storage/storage_driver.c @@ -2975,7 +2975,7 @@ virStorageFileReadHeader(virStorageSourcePtr src, if (!src->drv->backend->storageFileReadHeader) { virReportError(VIR_ERR_INTERNAL_ERROR, _("storage file header reading is not supported for " - "storage type %s (protocol: %s)'"), + "storage type %s (protocol: %s)"), virStorageTypeToString(src->type), virStorageNetProtocolTypeToString(src->protocol)); return -2; -- Daniel Veillard | Open Source and Standards, Red Hat veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

1 0

[libvirt] [PATCH] Clean up chardev sockets on QEMU shutdown
by Ján Tomko 26 May '14

26 May '14

https://bugzilla.redhat.com/show_bug.cgi?id=1088787 Clean up unix socket files for chardevs using mode='bind', like we clean up the monitor socket. They are created by QEMU on startup and not really useful after shutting it down. --- src/qemu/qemu_process.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 592e3b7..f3ec246 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -2655,6 +2655,19 @@ qemuProcessPrepareChardevDevice(virDomainDefPtr def ATTRIBUTE_UNUSED, } +static int +qemuProcessCleanupChardevDevice(virDomainDefPtr def ATTRIBUTE_UNUSED, + virDomainChrDefPtr dev, + void *opaque ATTRIBUTE_UNUSED) +{ + if (dev->source.type == VIR_DOMAIN_CHR_TYPE_UNIX && + dev->source.data.nix.listen) + unlink(dev->source.data.nix.path); + + return 0; +} + + struct qemuProcessHookData { virConnectPtr conn; virDomainObjPtr vm; @@ -4339,6 +4352,12 @@ void qemuProcessStop(virQEMUDriverPtr driver, priv->monConfig = NULL; } + ignore_value(virDomainChrDefForeach(vm->def, + false, + qemuProcessCleanupChardevDevice, + NULL)); + + /* shut it off for sure */ ignore_value(qemuProcessKill(vm, VIR_QEMU_PROCESS_KILL_FORCE| -- 1.8.3.2

2 3