- Devel - libvirt List Archives

[libvirt] [PATCH] Bypass caching in saving VM memory upon external memory snapshot.
by fuweiwei 25 Aug '16

25 Aug '16

From: Fuweiwei <fuweiwei2(a)huawei.com> Currently in qemu-kvm platform, the process of making an external memory snapshot is based on the "migration-to-file" sheme. It will use the system cache to speed up dumping. However, it will make external disk snapshots afterwards, which must wait for the completion of flushing the dirty pages to the snapshot file. i.e. In virFileWrapperFdClose() after qemuMigrationToFile(), it should wait until the libvirt_iohelper thread finishes fdatasync and exits. During this time, the VM is paused (since it is suspended from the last iteration of migration-to-file, to the completion of disk snapshots). Assuming saving 4GB dirty memory at 200MB/s fdatasync speed, the VM will pause for up to 20s, which is unfriendly to guests. So I propose that it may be better to bypass caching upon external memory snapshot, via the VIR_DOMAIN_SAVE_BYPASS_CACHE flag. As a result, it may avoid long-term fdatasync in libvirt_iohelper thread and achieve seemless VM suspend. Signed-off-by: Fuweiwei <fuweiwei2(a)huawei.com> --- src/qemu/qemu_driver.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 2089359..f954c23 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -14117,7 +14117,8 @@ qemuDomainSnapshotCreateActiveExternal(virConnectPtr conn, goto cleanup; if ((ret = qemuDomainSaveMemory(driver, vm, snap->def->file, - xml, compressed, resume, 0, + xml, compressed, resume, + VIR_DOMAIN_SAVE_BYPASS_CACHE, QEMU_ASYNC_JOB_SNAPSHOT)) < 0) goto cleanup; -- 1.8.3.1

2 1

[libvirt] [PATCH 0/7] fspool: backend directory
by Olga Krishtal 25 Aug '16

25 Aug '16

Hi everyone, we would like to propose the first implementation of fspool with directory backend. Filesystem pools is a facility to manage filesystems resources similar to how storage pools manages volume resources. Furthermore new API follows storage API closely where it makes sense. Uploading/downloading operations are not defined yet as it is not obvious how to make it properly. I guess we can use some kind of tar to make a stream from a filesystem. Please share you thoughts on this particular issue. The patchset provides 'dir' backend which simply expose directories in some directory in host filesystem. The virsh commands are provided too. So it is ready to play with, just replace 'pool' in xml descriptions and virsh commands to 'fspool' and 'volume' to 'item'. Examle and usage: Define: virsh -c qemu:///system fspool-define-as fs_pool_name dir --target /path/on/host Build virsh -c qemu:///system fspool-build fs_pool_name Start virsh -c qemu:///system fspool-start fs_pool_name Look inside virsh -c qemu:///system fspool-list (--all) fspool_name Fspool called POOL, on the host fs uses /fs_driver to hold items. virsh -c qemu:///system fspool-dumpxml POOL <fspool type='dir'> <name>POOL</name> <uuid>c57c9d7c-b1d5-4c45-ba9c-67f03d4da160</uuid> <capacity unit='bytes'>733722615808</capacity> <allocation unit='bytes'>1331486720</allocation> <available unit='bytes'>534810800128</available> <source> </source> <target> <path>/fs_driver</path> <permissions> <mode>0755</mode> <owner>0</owner> <group>0</group> </permissions> </target> </fspool> virsh -c qemu:///system fspool-info POOL Name: POOL UUID: c57c9d7c-b1d5-4c45-ba9c-67f03d4da160 State: running Persistent: yes Autostart: no autostart Capacity: 683.33 GiB Allocation: 1.24 GiB Available: 498.08 GiB virsh -c qemu+unix:///system item-list POOL Name Path ------------------------------------------------------------------------------ item1 /fs_driver/item1 item10 /fs_driver/item10 item11 /fs_driver/item11 item12 /fs_driver/item12 item15 /fs_driver/item15 Fspool of directory type is some directory on host fs that holds items (subdirs). Example of usage for items: virsh -c vz+unix:///system item-create-as POOL item1 1g - create item virsh -c qemu+unix:///system item-dumpxml item1 POOL <fsitem> <name>item1</name> <key>/fs_driver/item1</key> <source> fspoo ='POOL' </source> <capacity unit='bytes'>0</capacity> <allocation unit='bytes'>0</allocation> <target> <format type='dir'/> </target> </fsitem> virsh -c qemu+unix:///system item-info item1 POOL Name: item1 Type: dir Capacity: 683.33 GiB Allocation: 634.87 MiB Autostart: no autostart Capacity: 683.33 GiB Allocation: 1.24 GiB Available: 498.08 GiB virsh -c qemu+unix:///system item-list POOL Name Path ------------------------------------------------------------------------------ item1 /fs_driver/item1 item10 /fs_driver/item10 item11 /fs_driver/item11 item12 /fs_driver/item12 item15 /fs_driver/item15 Olga Krishtal (7): fspool: introduce filesystem pools API fspool: usual driver based implementation of filesystem pools API fspools: configuration and internal representation fspools: acl support for filesystem pools remote: filesystem pools driver implementation fspool: default implementation of filesystem pools virsh: filesystem pools commands configure.ac | 33 + daemon/Makefile.am | 4 + daemon/libvirtd.c | 10 + daemon/remote.c | 35 + include/libvirt/libvirt-fs.h | 273 +++++ include/libvirt/libvirt.h | 1 + include/libvirt/virterror.h | 8 + po/POTFILES.in | 6 + src/Makefile.am | 46 + src/access/viraccessdriver.h | 12 + src/access/viraccessdrivernop.c | 19 + src/access/viraccessdriverpolkit.c | 47 + src/access/viraccessdriverstack.c | 49 + src/access/viraccessmanager.c | 31 + src/access/viraccessmanager.h | 11 + src/access/viraccessperm.c | 15 +- src/access/viraccessperm.h | 124 +++ src/conf/fs_conf.c | 1624 +++++++++++++++++++++++++++ src/conf/fs_conf.h | 310 ++++++ src/datatypes.c | 154 +++ src/datatypes.h | 94 ++ src/driver-fs.h | 210 ++++ src/driver.h | 3 + src/fs/fs_backend.h | 85 ++ src/fs/fs_backend_dir.c | 334 ++++++ src/fs/fs_backend_dir.h | 8 + src/fs/fs_driver.c | 2164 ++++++++++++++++++++++++++++++++++++ src/fs/fs_driver.h | 10 + src/libvirt-fs.c | 1715 ++++++++++++++++++++++++++++ src/libvirt.c | 28 + src/libvirt_private.syms | 53 + src/libvirt_public.syms | 46 + src/remote/remote_driver.c | 72 +- src/remote/remote_protocol.x | 522 ++++++++- src/rpc/gendispatch.pl | 19 +- src/util/virerror.c | 37 + tools/Makefile.am | 4 + tools/virsh-fspool.c | 1728 ++++++++++++++++++++++++++++ tools/virsh-fspool.h | 36 + tools/virsh-item.c | 1274 +++++++++++++++++++++ tools/virsh-item.h | 37 + tools/virsh.c | 4 + tools/virsh.h | 9 + 43 files changed, 11294 insertions(+), 10 deletions(-) create mode 100644 include/libvirt/libvirt-fs.h create mode 100644 src/conf/fs_conf.c create mode 100644 src/conf/fs_conf.h create mode 100644 src/driver-fs.h create mode 100644 src/fs/fs_backend.h create mode 100644 src/fs/fs_backend_dir.c create mode 100644 src/fs/fs_backend_dir.h create mode 100644 src/fs/fs_driver.c create mode 100644 src/fs/fs_driver.h create mode 100644 src/libvirt-fs.c create mode 100644 tools/virsh-fspool.c create mode 100644 tools/virsh-fspool.h create mode 100644 tools/virsh-item.c create mode 100644 tools/virsh-item.h -- 1.8.3.1

2 20

[libvirt] [PATCH] remove the dead code this patch is to remove the dead code Signed-off-by: JieWang <wangjie88@huawei.com>
by JieWang 25 Aug '16

25 Aug '16

Signed-off-by: JieWang <wangjie88(a)huawei.com> --- src/qemu/qemu_migration.c | 1 - 1 file changed, 1 deletion(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 6a683f7..759e15a 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -1602,7 +1602,6 @@ qemuMigrationPrecreateDisk(virConnectPtr conn, _("cannot precreate storage for disk type '%s'"), virStorageTypeToString(disk->src->type)); goto cleanup; - break; } if ((vol = virStorageVolLookupByName(pool, volName))) { -- 1.9.5.msysgit.1

4 3

[libvirt] [PATCH v3 00/24] qemu: Add support for new vcpu hotplug and unplug
by Peter Krempa 24 Aug '16

24 Aug '16

v3 fixes stuff pointed out in reviews for v2: - more than 10 vcpus problem (patch 8 and new patch 10 adding tests) - few typos and other problems and stuff found while testing: - ordering function for qsort being broken (patch 21) You can fetch the changes at: git fetch git://pipo.sk/pipo/libvirt.git vcpu-unplug-3 Peter Krempa (24): qemu: monitor: Return structures from qemuMonitorGetCPUInfo qemu: monitor: Return struct from qemuMonitor(Text|Json)QueryCPUs qemu: caps: Add capability for query-hotpluggable-cpus command qemu: Forbid config when topology based cpu count doesn't match the config qemu: capabilities: Extract availability of new cpu hotplug for machine types qemu: monitor: Extract QOM path from query-cpus reply qemu: monitor: Add support for calling query-hotpluggable-cpus qemu: monitor: Add algorithm for combining query-(hotpluggable-)-cpus data tests: Add test infrastructure for qemuMonitorGetCPUInfo tests: cpu-hotplug: Add data for x86 hotplug with 11+ vcpus tests: cpu-hotplug: Add data for ppc64 platform including hotplug tests: cpu-hotplug: Add data for ppc64 out-of-order hotplug tests: cpu-hotplug: Add data for ppc64 without threads enabled qemu: domain: Extract cpu-hotplug related data qemu: domain: Prepare for VCPUs vanishing while libvirt is not running util: Extract and rename qemuDomainDelCgroupForThread to virCgroupDelThread conf: Add XML for individual vCPU hotplug qemu: migration: Prepare for non-contiguous vcpu configurations qemu: command: Add helper to convert vcpu definition to JSON props qemu: process: Copy final vcpu order information into the vcpu definition qemu: command: Add support for sparse vcpu topologies qemu: Use modern vcpu hotplug approach if possible qemu: hotplug: Allow marking unplugged devices by alias qemu: hotplug: Add support for VCPU unplug docs/formatdomain.html.in | 45 +++ docs/schemas/domaincommon.rng | 25 ++ src/conf/domain_conf.c | 154 +++++++++- src/conf/domain_conf.h | 6 + src/libvirt_private.syms | 1 + src/qemu/qemu_capabilities.c | 31 +- src/qemu/qemu_capabilities.h | 3 + src/qemu/qemu_command.c | 50 +++- src/qemu/qemu_command.h | 3 + src/qemu/qemu_domain.c | 312 +++++++++++++++++---- src/qemu/qemu_domain.h | 19 +- src/qemu/qemu_driver.c | 247 +++++++++------- src/qemu/qemu_hotplug.c | 124 +++++++- src/qemu/qemu_hotplug.h | 7 + src/qemu/qemu_migration.c | 16 +- src/qemu/qemu_monitor.c | 268 +++++++++++++++++- src/qemu/qemu_monitor.h | 58 +++- src/qemu/qemu_monitor_json.c | 266 +++++++++++++++--- src/qemu/qemu_monitor_json.h | 8 +- src/qemu/qemu_monitor_text.c | 41 +-- src/qemu/qemu_monitor_text.h | 3 +- src/qemu/qemu_process.c | 187 +++++++++++- src/util/vircgroup.c | 20 ++ src/util/vircgroup.h | 4 + .../generic-vcpus-individual.xml | 23 ++ tests/genericxml2xmltest.c | 2 + tests/qemucapabilitiesdata/caps_2.7.0.x86_64.xml | 55 ++-- .../qemumonitorjson-cpuinfo-ppc64-basic-cpus.json | 77 +++++ ...emumonitorjson-cpuinfo-ppc64-basic-hotplug.json | 27 ++ .../qemumonitorjson-cpuinfo-ppc64-basic.data | 40 +++ ...mumonitorjson-cpuinfo-ppc64-hotplug-1-cpus.json | 149 ++++++++++ ...onitorjson-cpuinfo-ppc64-hotplug-1-hotplug.json | 28 ++ .../qemumonitorjson-cpuinfo-ppc64-hotplug-1.data | 51 ++++ ...mumonitorjson-cpuinfo-ppc64-hotplug-2-cpus.json | 221 +++++++++++++++ ...onitorjson-cpuinfo-ppc64-hotplug-2-hotplug.json | 29 ++ .../qemumonitorjson-cpuinfo-ppc64-hotplug-2.data | 62 ++++ ...mumonitorjson-cpuinfo-ppc64-hotplug-4-cpus.json | 221 +++++++++++++++ ...onitorjson-cpuinfo-ppc64-hotplug-4-hotplug.json | 29 ++ .../qemumonitorjson-cpuinfo-ppc64-hotplug-4.data | 62 ++++ ...umonitorjson-cpuinfo-ppc64-no-threads-cpus.json | 77 +++++ ...nitorjson-cpuinfo-ppc64-no-threads-hotplug.json | 125 +++++++++ .../qemumonitorjson-cpuinfo-ppc64-no-threads.data | 72 +++++ ...nitorjson-cpuinfo-x86-basic-pluggable-cpus.json | 50 ++++ ...orjson-cpuinfo-x86-basic-pluggable-hotplug.json | 82 ++++++ ...emumonitorjson-cpuinfo-x86-basic-pluggable.data | 39 +++ .../qemumonitorjson-cpuinfo-x86-full-cpus.json | 104 +++++++ .../qemumonitorjson-cpuinfo-x86-full-hotplug.json | 115 ++++++++ .../qemumonitorjson-cpuinfo-x86-full.data | 76 +++++ tests/qemumonitorjsontest.c | 184 +++++++++++- .../qemuxml2argv-cpu-hotplug-startup.args | 20 ++ .../qemuxml2argv-cpu-hotplug-startup.xml | 29 ++ tests/qemuxml2argvtest.c | 2 + tests/testutils.c | 4 +- 53 files changed, 3677 insertions(+), 276 deletions(-) create mode 100644 tests/genericxml2xmlindata/generic-vcpus-individual.xml create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-basic-cpus.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-basic-hotplug.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-basic.data create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-1-cpus.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-1-hotplug.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-1.data create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-2-cpus.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-2-hotplug.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-2.data create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-4-cpus.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-4-hotplug.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-hotplug-4.data create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-no-threads-cpus.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-no-threads-hotplug.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-ppc64-no-threads.data create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-x86-basic-pluggable-cpus.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-x86-basic-pluggable-hotplug.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-x86-basic-pluggable.data create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-x86-full-cpus.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-x86-full-hotplug.json create mode 100644 tests/qemumonitorjsondata/qemumonitorjson-cpuinfo-x86-full.data create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-cpu-hotplug-startup.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-cpu-hotplug-startup.xml -- 2.8.2

3 38

[libvirt] [PATCH] storage_backend_rbd: fix typos
by Chen Hanxiao 24 Aug '16

24 Aug '16

From: Chen Hanxiao <chenhanxiao(a)gmail.com> s/failed/failed to Signed-off-by: Chen Hanxiao <chenhanxiao(a)gmail.com> --- src/storage/storage_backend_rbd.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/storage/storage_backend_rbd.c b/src/storage/storage_backend_rbd.c index 9665fbc..4dd4b24 100644 --- a/src/storage/storage_backend_rbd.c +++ b/src/storage/storage_backend_rbd.c @@ -894,7 +894,7 @@ virStorageBackendRBDSnapshotProtect(rbd_image_t image, VIR_DEBUG("Querying if RBD snapshot %s@%s is protected", imgname, snapname); if ((r = rbd_snap_is_protected(image, snapname, &protected)) < 0) { - virReportSystemError(-r, _("failed verify if RBD snapshot %s@%s " + virReportSystemError(-r, _("failed to verify if RBD snapshot %s@%s " "is protected"), imgname, snapname); goto cleanup; } @@ -904,7 +904,7 @@ virStorageBackendRBDSnapshotProtect(rbd_image_t image, imgname, snapname); if ((r = rbd_snap_protect(image, snapname)) < 0) { - virReportSystemError(-r, _("failed protect RBD snapshot %s@%s"), + virReportSystemError(-r, _("failed to protect RBD snapshot %s@%s"), imgname, snapname); goto cleanup; } -- 1.8.3.1

2 1

[libvirt] Plans for next release
by Daniel Veillard 24 Aug '16

24 Aug '16

So if we want a 2.2.0 release around Sep 1st, I suggest entering freeze this Friday probably at the end of the day (european time), then push the rc2 next Tuesday for a release on Thursday, hope this works for everybody, Daniel -- Daniel Veillard | Open Source and Standards, Red Hat veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

3 3

[libvirt] [PATCH 0/3] virsh: Option completers and small improvements/fixes for autocomplete
by Nishith Shah 24 Aug '16

24 Aug '16

This series introduces option completers and adds some minor improvements and fixes(not bugs per se, just better/sane behavior) in vshReadlineParse. The first patch introduces the usage of option completers to auto-complete arguments for a particular option. The second and third patches provide small improvements like completing the options of type VSH_OT_ARGV or VSH_OT_DATA, and to complete multiple options as well, if a previous option requires an argument, and that argument has been provided. Nishith Shah (3): virsh: Introduce usage of option completers to auto-complete arguments virsh: Allow data or argument options to be completed as well virsh: Complete multiple options when any one option requires data tools/vsh.c | 75 ++++++++++++++++++++++++++++++++++++++++--------------------- 1 file changed, 49 insertions(+), 26 deletions(-) -- 2.1.4

2 5

[libvirt] [PATCH 0/3] Misc virsh improvements
by Pino Toscano 24 Aug '16

24 Aug '16

Hi, this series provides small improvements for virsh. The patches are sent as single series as some of the changes would conflict if sent as different patches. Thanks, Pino Toscano (3): virsh: respect -q/--quiet more virsh: avoid i18n puzzle virsh: use vshError consistently after virBufferError checks tests/virsh-undefine | 4 -- tools/virsh-domain.c | 130 ++++++++++++++++++++++++------------------------ tools/virsh-interface.c | 16 +++--- tools/virsh-network.c | 36 +++++++------- tools/virsh-nodedev.c | 12 ++--- tools/virsh-nwfilter.c | 6 +-- tools/virsh-pool.c | 30 +++++------ tools/virsh-secret.c | 6 +-- tools/virsh-snapshot.c | 16 +++--- tools/virsh-volume.c | 20 ++++---- 10 files changed, 137 insertions(+), 139 deletions(-) -- 2.7.4

3 5

[libvirt] [PATCH] tests: fix segault in objecteventtest
by Roman Bogorodskiy 24 Aug '16

24 Aug '16

Test 12 from objecteventtest (createXML add event) segaults on FreeBSD with bus error. At some point it calls testNodeDeviceDestroy() from the test driver. And it fails when it tries to unlock the device in the "out:" label of this function. Unlocking fails because the previous step was a call to virNodeDeviceObjRemove from conf/node_device_conf.c. This function removes the given device from the device list and cleans up the object, including destroying of its mutex. However, it does not nullify the pointer that was given to it. As a result, we end up in testNodeDeviceDestroy() here: out: if (obj) virNodeDeviceObjUnlock(obj); And instead of skipping this, we try to do Unlock and fail because of malformed mutex. Fix this by nullifying obj passed to virNodeDeviceObjRemove. --- src/conf/node_device_conf.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/conf/node_device_conf.c b/src/conf/node_device_conf.c index a23d8ef..16b9d93 100644 --- a/src/conf/node_device_conf.c +++ b/src/conf/node_device_conf.c @@ -218,6 +218,7 @@ void virNodeDeviceObjRemove(virNodeDeviceObjListPtr devs, if (devs->objs[i] == dev) { virNodeDeviceObjUnlock(dev); virNodeDeviceObjFree(devs->objs[i]); + *(void**)dev = NULL; VIR_DELETE_ELEMENT(devs->objs, i, devs->count); break; -- 2.7.4

2 2

[libvirt] [PATCH] Fix remote_protocol-structs after recent commit
by Peter Krempa 24 Aug '16

24 Aug '16

Commit 0adc9d26ae0cfbb4aa227e89424f3d48bb977da0 broke the build since remote_protocol-structs was not updated. --- Pushed under the build-breaker rule. src/remote_protocol-structs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/remote_protocol-structs b/src/remote_protocol-structs index 4552913..b71accc 100644 --- a/src/remote_protocol-structs +++ b/src/remote_protocol-structs @@ -3151,8 +3151,8 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_SET_USER_PASSWORD = 357, REMOTE_PROC_DOMAIN_RENAME = 358, REMOTE_PROC_DOMAIN_EVENT_CALLBACK_MIGRATION_ITERATION = 359, - REMOTE_PROC_CONNECT_CLOSE_CALLBACK_REGISTER = 360, - REMOTE_PROC_CONNECT_CLOSE_CALLBACK_UNREGISTER = 361, + REMOTE_PROC_CONNECT_REGISTER_CLOSE_CALLBACK = 360, + REMOTE_PROC_CONNECT_UNREGISTER_CLOSE_CALLBACK = 361, REMOTE_PROC_CONNECT_EVENT_CONNECTION_CLOSED = 362, REMOTE_PROC_DOMAIN_EVENT_CALLBACK_JOB_COMPLETED = 363, REMOTE_PROC_DOMAIN_MIGRATE_START_POST_COPY = 364, -- 2.8.2

2 2