January 2016 - Devel - libvirt List Archives

[libvirt] [libvirt-perl][PATCH] Add VIR_STORAGE_VOL_WIPE_ALG_TRIM constant
by Michal Privoznik 01 Feb '16

01 Feb '16

Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- Changes | 1 + Virt.xs | 1 + lib/Sys/Virt/StorageVol.pm | 4 ++++ 3 files changed, 6 insertions(+) diff --git a/Changes b/Changes index cfcc640..cb57a4d 100644 --- a/Changes +++ b/Changes @@ -2,6 +2,7 @@ Revision history for perl module Sys::Virt 1.3.2 2016-02-00 + - Add VIR_STORAGE_VOL_WIPE_ALG_TRIM constant - Add VIR_FROM_XENXL constant - Add VIR_DOMAIN_EVENT_ID_MIGRATION_ITERATION event handling callback diff --git a/Virt.xs b/Virt.xs index e47d52e..06111b4 100644 --- a/Virt.xs +++ b/Virt.xs @@ -8187,6 +8187,7 @@ BOOT: REGISTER_CONSTANT(VIR_STORAGE_VOL_WIPE_ALG_PFITZNER7, WIPE_ALG_PFITZNER7); REGISTER_CONSTANT(VIR_STORAGE_VOL_WIPE_ALG_PFITZNER33, WIPE_ALG_PFITZNER33); REGISTER_CONSTANT(VIR_STORAGE_VOL_WIPE_ALG_RANDOM, WIPE_ALG_RANDOM); + REGISTER_CONSTANT(VIR_STORAGE_VOL_WIPE_ALG_TRIM, WIPE_ALG_TRIM); REGISTER_CONSTANT(VIR_STORAGE_VOL_RESIZE_ALLOCATE, RESIZE_ALLOCATE); REGISTER_CONSTANT(VIR_STORAGE_VOL_RESIZE_DELTA, RESIZE_DELTA); diff --git a/lib/Sys/Virt/StorageVol.pm b/lib/Sys/Virt/StorageVol.pm index 84ba4e0..461a9fe 100644 --- a/lib/Sys/Virt/StorageVol.pm +++ b/lib/Sys/Virt/StorageVol.pm @@ -260,6 +260,10 @@ Cryptography" (1996) 1-pass, all zeroes +=item Sys::Virt::StorageVol::WIPE_ALG_TRIM + +1-pass, trim all data on the volume by using TRIM or DISCARD + =back VOLUME RESIZE CONSTANTS -- 2.4.10

2 1

[libvirt] ABI compatibility report
by Ponomarenko Andrey 01 Feb '16

01 Feb '16

Hello, I've started to maintain ABI compatibility report for the library here: http://abi-laboratory.pro/tracker/timeline/libvirt/ Seems that ABI compatibility is excellent from the beginning of time) Thank you.

2 1

[libvirt] [PATCH] includes: Install libvirt-common.h
by Michal Privoznik 01 Feb '16

01 Feb '16

The libvirt-common.h is build time generated file from .in. Obviously, it's generated into builddir and not srcdir. Problem is, the list of header files to install, virinc_HEADERS contains only $(srcdir)/*.h and this misses libvirt-common.h. This problem is pretty obvious when doing a VPATH build. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- Steps to reproduce: 1) make distclean 2) mkdir _build _install 3) cd _build 4) ../configure --prefix=$(pwd)/../_install 5) make && make install 6) find ../_install -name libvirt-common.h include/Makefile.am | 1 + 1 file changed, 1 insertion(+) diff --git a/include/Makefile.am b/include/Makefile.am index d626963..6b0b41e 100644 --- a/include/Makefile.am +++ b/include/Makefile.am @@ -20,6 +20,7 @@ virincdir = $(includedir)/libvirt allheaders = $(wildcard $(srcdir)/libvirt/*.h) virinc_HEADERS = $(filter-out $(srcdir)/libvirt/libvirt-admin.h, $(allheaders)) +virinc_HEADERS += libvirt/libvirt-common.h EXTRA_DIST = libvirt/libvirt-common.h.in -- 2.4.10

2 1

[libvirt] [libvirt-php] add flags support to virStorageVolCreateXML
by Vasiliy Tolstov 31 Jan '16

31 Jan '16

Signed-off-by: Vasiliy Tolstov <v.tolstov(a)selfip.ru> --- src/libvirt-php.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/src/libvirt-php.c b/src/libvirt-php.c index f3b3f9f81e6d..a0c960957edc 100644 --- a/src/libvirt-php.c +++ b/src/libvirt-php.c @@ -1238,6 +1238,8 @@ PHP_MINIT_FUNCTION(libvirt) REGISTER_LONG_CONSTANT("VIR_STORAGE_VOL_RESIZE_ALLOCATE", 1, CONST_CS | CONST_PERSISTENT); REGISTER_LONG_CONSTANT("VIR_STORAGE_VOL_RESIZE_DELTA", 2, CONST_CS | CONST_PERSISTENT); REGISTER_LONG_CONSTANT("VIR_STORAGE_VOL_RESIZE_SHRINK", 4, CONST_CS | CONST_PERSISTENT); + REGISTER_LONG_CONSTANT("VIR_STORAGE_VOL_CREATE_PREALLOC_METADATA", VIR_STORAGE_VOL_CREATE_PREALLOC_METADATA, CONST_CS | CONST_PERSISTENT); + REGISTER_LONG_CONSTANT("VIR_STORAGE_VOL_CREATE_REFLINK", VIR_STORAGE_VOL_CREATE_REFLINK, CONST_CS | CONST_PERSISTENT); /* Domain vCPU flags */ REGISTER_LONG_CONSTANT("VIR_DOMAIN_VCPU_CONFIG", VIR_DOMAIN_VCPU_CONFIG, CONST_CS | CONST_PERSISTENT); @@ -7105,6 +7107,7 @@ PHP_FUNCTION(libvirt_storagevolume_get_xml_desc) * Description: Function is used to create the new storage pool and return the handle to new storage pool * Arguments: @res [resource]: libvirt storagepool resource * @xml [string]: XML string to create the storage volume in the storage pool + * @flags [int]: virStorageVolCreateXML flags * Returns: libvirt storagevolume resource */ PHP_FUNCTION(libvirt_storagevolume_create_xml) @@ -7114,11 +7117,12 @@ PHP_FUNCTION(libvirt_storagevolume_create_xml) zval *zpool; virStorageVolPtr volume=NULL; char *xml; + long flags = 0; int xml_len; - GET_STORAGEPOOL_FROM_ARGS("rs",&zpool,&xml,&xml_len); + GET_STORAGEPOOL_FROM_ARGS("rs|l",&zpool,&xml,&xml_len, &flags); - volume=virStorageVolCreateXML(pool->pool,xml,0); + volume=virStorageVolCreateXML(pool->pool, xml, flags); DPRINTF("%s: virStorageVolCreateXML(%p, <xml>, 0) returned %p\n", PHPFUNC, pool->pool, volume); if (volume==NULL) RETURN_FALSE; -- 2.7.0

2 1

[libvirt] [PATCH v2] qemu: qemuDomainRename and virDomainObjListNumOfDomains ABBA deadlock fix
by Nikolay Shirokovskiy 30 Jan '16

30 Jan '16

A pretty nasty deadlock occurs while trying to rename a VM in parallel with virDomainObjListNumOfDomains. The short description of the problem is as follows: Thread #1: qemuDomainRename: ------> aquires domain lock by qemuDomObjFromDomain ---------> waits for domain list lock in any of the listed functions: - virDomainObjListFindByName - virDomainObjListRenameAddNew - virDomainObjListRenameRemove Thread #2: virDomainObjListNumOfDomains: ------> aquires domain list lock ---------> waits for domain lock in virDomainObjListCount Consider another solution for the deadlock besides those proposed by Maxim and Michael. Simply unlock before calling any function that internally grabs domains list lock. First it looks correct. We use qemuDomainObjBeginJob/qemuDomainObjEndJob to wrap the operation so no one will run another operation on that domain in parallel. In THREADS.txt this unlocking/locking is described explicitly for the cases when we need to wait/sleep. Here we use same means just to different purpose. Also called functions do not use domain internals in any way. Second I find approach proposed by Michael difficult to implement correctly. Imagine we write such a generic rename function with callbacks. Then it would look like this: lock domains list ... add new name to list unlock domain list call callback to change driver state lock domain list remove old or new name (depends on callback result) unlock domain list We need to unlock domain list or we block operations on all domains while we wait for job conditions in the callback. But now we are in potential trouble as we leaving list in an inconsistent state. We can't move remove operation before calling callback or we will be in trouble on callback failures - we would need to add the old name back and this is not always possible. Current rename implementation leaves config with new name on disk in worst case on rollback but does not break the list. --- src/qemu/qemu_driver.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index abcdbe6..26a8fb0 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -19987,6 +19987,7 @@ static int qemuDomainRename(virDomainPtr dom, virObjectEventPtr event_new = NULL; virObjectEventPtr event_old = NULL; int ret = -1; + int rc; char *new_dom_name = NULL; char *old_dom_name = NULL; char *old_dom_cfg_file = NULL; @@ -20036,10 +20037,11 @@ static int qemuDomainRename(virDomainPtr dom, /* * This is a rather racy check, but still better than reporting - * internal error. And since new_name != name here, there's no - * deadlock imminent. + * internal error. */ + virObjectUnlock(vm); tmp_dom = virDomainObjListFindByName(driver->domains, new_name); + virObjectLock(vm); if (tmp_dom) { virObjectUnlock(tmp_dom); virObjectUnref(tmp_dom); @@ -20057,7 +20059,10 @@ static int qemuDomainRename(virDomainPtr dom, goto endjob; } - if (virDomainObjListRenameAddNew(driver->domains, vm, new_name) < 0) + virObjectUnlock(vm); + rc = virDomainObjListRenameAddNew(driver->domains, vm, new_name); + virObjectLock(vm); + if (rc < 0) goto endjob; event_old = virDomainEventLifecycleNewFromObj(vm, @@ -20081,7 +20086,9 @@ static int qemuDomainRename(virDomainPtr dom, } /* Remove old domain name from table. */ + virObjectUnlock(vm); virDomainObjListRenameRemove(driver->domains, old_dom_name); + virObjectLock(vm); event_new = virDomainEventLifecycleNewFromObj(vm, VIR_DOMAIN_EVENT_DEFINED, @@ -20110,7 +20117,9 @@ static int qemuDomainRename(virDomainPtr dom, old_dom_name = NULL; } + virObjectUnlock(vm); virDomainObjListRenameRemove(driver->domains, new_name); + virObjectLock(vm); goto endjob; } -- 1.8.3.1

1 1

[libvirt] [PATCH v3 0/8] add close callback for drivers with persistent connection
by Nikolay Shirokovskiy 30 Jan '16

30 Jan '16

Currently close callback API can only inform us of closing the connection between remote driver and daemon. But what if a driver running in the daemon itself can have another persistent connection? In this case we want to be informed of that connection changes state too. This patch series extends meaning of current close callback API so that now it notifies of closing of any internal persistent connection. The overall approach is to move close callback support to drivers. Changes from v2: Split patches further to make it more comprehensible. Nikolay Shirokovskiy (8): factor out virConnectCloseCallbackDataPtr methods virConnectCloseCallbackData: fix connection object refcount virConnectCloseCallbackData: factor out callback disarming close callback API: remove unnecessary locks virConnectCloseCallbackDataDispose: remove unnecessary locks close callback: move it to driver daemon: add connection close rpc vz: implement connection close notification daemon/libvirtd.h | 1 + daemon/remote.c | 84 ++++++++++++++++++++++++++++++++ po/POTFILES.in | 1 + src/datatypes.c | 113 +++++++++++++++++++++++++++++++++---------- src/datatypes.h | 11 +++++ src/driver-hypervisor.h | 12 +++++ src/libvirt-host.c | 46 ++---------------- src/remote/remote_driver.c | 106 ++++++++++++++++++++++++++++++++-------- src/remote/remote_protocol.x | 24 ++++++++- src/remote_protocol-structs | 6 +++ src/vz/vz_driver.c | 39 +++++++++++++++ src/vz/vz_sdk.c | 4 ++ src/vz/vz_utils.h | 3 ++ 13 files changed, 363 insertions(+), 87 deletions(-) -- 1.8.3.1

2 9

[libvirt] [PATCH] rbd: Open in Read-Only mode when refreshing a volume
by Wido den Hollander 29 Jan '16

29 Jan '16

By opening a RBD volume in Read-Only we do not register a watcher on the header object inside the Ceph cluster. Refreshing a volume only calls rbd_stat() which is a operation which does not write to a RBD image. This allows us to use a cephx user which has no write permissions if we would want to use the libvirt storage pool for informational purposes only. It also saves us a write into the Ceph cluster which should speed up refreshing a RBD pool. rbd_open_read_only() is available in all librbd versions which also support rbd_open(). Signed-off-by: Wido den Hollander <wido(a)widodh.nl> --- src/storage/storage_backend_rbd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/storage/storage_backend_rbd.c b/src/storage/storage_backend_rbd.c index 8c7a80d..3ab7912 100644 --- a/src/storage/storage_backend_rbd.c +++ b/src/storage/storage_backend_rbd.c @@ -283,7 +283,7 @@ static int volStorageBackendRBDRefreshVolInfo(virStorageVolDefPtr vol, int r = 0; rbd_image_t image = NULL; - r = rbd_open(ptr->ioctx, vol->name, &image, NULL); + r = rbd_open_read_only(ptr->ioctx, vol->name, &image, NULL); if (r < 0) { ret = -r; virReportSystemError(-r, _("failed to open the RBD image '%s'"), -- 1.9.1

2 1

[libvirt] [PATCH] Simplify virDomainParseMemory
by Ján Tomko 29 Jan '16

29 Jan '16

Do not store the return value of virDomainParseScaledValue, it was overwritten anyway. Delete the cleanup label, there is nothing to clean up. --- src/conf/domain_conf.c | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 1ea74a6..f663969 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -7547,28 +7547,22 @@ virDomainParseMemory(const char *xpath, bool required, bool capped) { - int ret = -1; unsigned long long bytes, max; max = virMemoryMaxValue(capped); - ret = virDomainParseScaledValue(xpath, units_xpath, ctxt, - &bytes, 1024, max, required); - if (ret < 0) - goto cleanup; + if (virDomainParseScaledValue(xpath, units_xpath, ctxt, + &bytes, 1024, max, required) < 0) + return -1; /* Yes, we really do use kibibytes for our internal sizing. */ *mem = VIR_DIV_UP(bytes, 1024); if (*mem >= VIR_DIV_UP(max, 1024)) { virReportError(VIR_ERR_OVERFLOW, "%s", _("size value too large")); - ret = -1; - goto cleanup; + return -1; } - - ret = 0; - cleanup: - return ret; + return 0; } -- 2.4.10

2 1

[libvirt] [PATCH 0/4 v2] vz: rework domain creation
by Mikhail Feoktistov 29 Jan '16

29 Jan '16

Patches 1 and 2 make preparation for patch 4 Patch 3 fixes notification subscription To get domain info we should receive and handle notification from hypervisor Patch 4 fixes race condition when adding domain to domains list Race condition: User calls defineXML to create new instance. The main thread from vzDomainDefineXMLFlags() creates new instance by prlsdkCreateVm. Then this thread calls prlsdkAddDomain to add new domain to domains list. The second thread receives notification from hypervisor that new VM was created. It calls prlsdkHandleVmAddedEvent() and also tries to add new domain to domains list. These two threads call virDomainObjListFindByUUID() from prlsdkAddDomain() and don't find new domain. So they add two domains with the same uuid to domains list. Mikhail Feoktistov (4): vz: make output arguments in prlsdkGetDomainIds as optional vz: remove unused struct field vz: fix notification subscription vz: fix race condition when adding domain to domains list src/vz/vz_driver.c | 13 ++- src/vz/vz_sdk.c | 293 +++++++++++++++++++++++++++-------------------------- src/vz/vz_sdk.h | 9 +- src/vz/vz_utils.h | 1 - 4 files changed, 165 insertions(+), 151 deletions(-) -- 1.8.3.1

3 15

[libvirt] [PATCH 00/34] vcpu info storage refactors - part 2
by Peter Krempa 29 Jan '16

29 Jan '16

This is a continuation of the previous patches that change how we store info related to vcpus. In this series the vcpupin and vcpusched info is moved to the vcpu structure which makes for a much simpler storage and handling of the data. This series also fixes numerous bugs in the vcpu code. Peter Krempa (34): qemu: process: refactor and rename qemuValidateCpuMax to qemuValidateCpuCount qemu: process: Disallow VMs with 0 vcpus vz: Fix invalid iteration of def->cputune.vcpupin qemu: don't iterate vcpus using priv->nvcpupids in qemuProcessSetSchedParams (qemu|lxc)DomainGetCPUStats: Clean up cgroup: Clean up virCgroupGetPercpuStats conf: Add helper to retrieve bitmap of active vcpus for a definition virsh: cpu-stats: Extract common printing code into a function virsh: cpu-stats: Remove unneeded flags qemu: Don't use priv->ncpus to iterate cgroup setting cgroup: Prepare for sparse vCPU topologies in virCgroupGetPercpuStats qemu: cpu hotplug: Set vcpu state directly in the new structure qemu: Move and rename qemuProcessDetectVcpuPIDs to qemuDomainDetectVcpuPids qemu: domain: Prepare qemuDomainDetectVcpuPids for reuse qemu: Reuse qemuDomainDetectVcpuPids in cpu hot(un)plug conf: Don't copy def->cpumask into cpu pinning info tests: qemuxml2xml: Order pinning information numerically conf: Split out logic to determine whether cpupin was provided test: Touch up error message when attempting to pin invalid vCPU conf: Store cpu pinning data in def->vcpus conf: remove unused cpu pinning helpers and data structures conf: disallow empty cpusets for vcpu pinning when parsing XML conf: disallow empty cpuset for emulatorpin conf: Extract code that formats <cputune> util: buffer: Sanitize comment for virBufferAddBuffer util: bitmap: Introduce bitmap subtraction conf: Add helper to return a bitmap of active iothread ids conf: Extract code for parsing thread resource scheduler info conf: Don't store vcpusched orthogonally to other vcpu info conf: Fix how iothread scheduler info is stored qemu: vcpu: Aggregate code to set vCPU tuning qemu: vcpu: Reuse qemuProcessSetupVcpu in vcpu hotplug qemu: iothread: Aggregate code to set IOTrhead tuning qemu: iothread: Reuse qemuProcessSetupIOThread in iothread hotplug src/conf/domain_conf.c | 948 +++++++++++---------- src/conf/domain_conf.h | 48 +- src/libvirt_private.syms | 10 +- src/libxl/libxl_domain.c | 20 +- src/libxl/libxl_driver.c | 41 +- src/lxc/lxc_driver.c | 5 +- src/qemu/qemu_cgroup.c | 198 ----- src/qemu/qemu_cgroup.h | 2 - src/qemu/qemu_domain.c | 81 ++ src/qemu/qemu_domain.h | 2 + src/qemu/qemu_driver.c | 417 +++------ src/qemu/qemu_process.c | 479 ++++++----- src/qemu/qemu_process.h | 6 + src/test/test_driver.c | 50 +- src/util/virbitmap.c | 21 + src/util/virbitmap.h | 3 + src/util/virbuffer.c | 3 +- src/util/vircgroup.c | 52 +- src/util/vircgroup.h | 3 +- src/vz/vz_sdk.c | 6 +- .../qemuxml2argv-cputune-iothreads.xml | 2 +- .../qemuxml2xmlout-cputune-iothreads.xml | 2 +- .../qemuxml2xmlout-cputune-iothreadsched.xml | 3 +- tests/vcpupin | 2 +- tests/virbitmaptest.c | 55 ++ tests/vircgrouptest.c | 2 +- tools/virsh-domain.c | 66 +- 27 files changed, 1165 insertions(+), 1362 deletions(-) -- 2.6.2

4 79