- Devel - libvirt List Archives

[libvirt] [PATCH v2 00/12] Continue altering storage pool for privatization
by John Ferlan 19 Sep '17

19 Sep '17

Consider this round 1 of 2.... The next series will be 18 patches, but the majority of those deal with change every {pool|obj}->def->X to use the accessor virStoragePoolObjGetDef. v1: https://www.redhat.com/archives/libvir-list/2017-May/msg00218.html Probably not even worth looking at the v1, but this picks up where v1 left off somewhere around patch 10, but adding smaller steps between patches. John Ferlan (12): storage: Create accessor API's for virStoragePoolObj storage: Introduce virStoragePoolObjNew storage: Fill in storage pool @active properly storage: Introduce storage volume add, delete, count APIs storage: Introduce APIs to search/scan storage pool volumes list storage: Use virStoragePoolObj{Get|Set}ConfigFile storage: Use virStoragePoolObjGetAutostartLink storage: Use virStoragePoolObj{Is|Set}Active storage: Use virStoragePoolObj{Is|Set}Autostart storage: Internally represent @autostart to bool storage: Use virStoragePoolObj{Get|Incr}Decr}Asyncjobs storage: Use virStoragePoolObjDefUseNewDef src/conf/virstorageobj.c | 211 +++++++++++++++++++++++++++++++-- src/conf/virstorageobj.h | 84 ++++++++++++- src/libvirt_private.syms | 20 ++++ src/storage/storage_backend_disk.c | 93 +++++++++------ src/storage/storage_backend_gluster.c | 5 +- src/storage/storage_backend_logical.c | 4 +- src/storage/storage_backend_mpath.c | 3 +- src/storage/storage_backend_rbd.c | 4 +- src/storage/storage_backend_scsi.c | 4 +- src/storage/storage_backend_sheepdog.c | 4 +- src/storage/storage_backend_zfs.c | 6 +- src/storage/storage_driver.c | 142 +++++++++------------- src/storage/storage_util.c | 8 +- src/test/test_driver.c | 54 ++++----- tests/storagevolxml2argvtest.c | 20 ++-- 15 files changed, 465 insertions(+), 197 deletions(-) -- 2.9.5

2 19

[libvirt] [PATCH] check-spacing: Don't hardcode Perl interpreter path
by Andrea Bolognani 19 Sep '17

19 Sep '17

This is particularly useful on operating systems that don't ship Perl as part of the base system (eg. FreeBSD) while still working just as well as it did before on Linux. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- build-aux/check-spacing.pl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build-aux/check-spacing.pl b/build-aux/check-spacing.pl index 448acf234..ca8b43491 100755 --- a/build-aux/check-spacing.pl +++ b/build-aux/check-spacing.pl @@ -1,4 +1,4 @@ -#!/usr/bin/perl +#!/usr/bin/env perl # # check-spacing.pl: Report any usage of 'function (..args..)' # Also check for other syntax issues, such as correct use of ';' -- 2.13.5

2 2

[libvirt] [PATCH] virsh: man: Describe the 'create' command a bit more
by Erik Skultety 19 Sep '17

19 Sep '17

So we refer to the terms 'persistent' and 'transient' across the whole man page, without describing it further, but more importantly, how the create command affects it, i.e. explicitly stating that domain created via the 'create' command are going to be transient or persistent, depending on whether there is an existing persistent domain with a matching <name> and <uuid>, in which case it will remain persistent, but will run using a one-time configuration, otherwise it's going to be transient and will vanish once destroyed. Signed-off-by: Erik Skultety <eskultet(a)redhat.com> --- tools/virsh.pod | 32 ++++++++++++++++++++++++++------ 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/tools/virsh.pod b/tools/virsh.pod index 01453be60..c4c76fcb1 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -640,9 +640,14 @@ sessions, such as in a case of a broken connection. Create a domain from an XML <file>. Optionally, I<--validate> option can be passed to validate the format of the input XML file against an internal RNG -schema (identical to using L<virt-xml-validate(1)> tool). An easy way to create -the XML <file> is to use the B<dumpxml> command to obtain the definition of a -pre-existing guest. The domain will be paused if the I<--paused> option is used +schema (identical to using L<virt-xml-validate(1)> tool). Domains created using +this command are going to be either transient (temporary ones that will vanish +once destroyed) or existing persistent domains that will run with one-time use +configuration, leaving the persistent XML untouched (this can come handy during +an automated testing of various configurations all based on the original XML). +See the B<Example> section for usage demonstration. + +The domain will be paused if the I<--paused> option is used and supported by the driver; otherwise it will be running. If I<--console> is requested, attach to the console after creation. If I<--autodestroy> is requested, then the guest will be automatically @@ -656,9 +661,24 @@ is only supported with container based virtualization. B<Example> - virsh dumpxml <domain> > domain.xml - vi domain.xml (or make changes with your other text editor) - virsh create domain.xml + 1) prepare a template from an existing domain (skip directly to 3a if writing + one from scratch) + + # virsh dumpxml <domain> > domain.xml + + 2) edit the template using an editor of your choice and: + a) DO CHANGE! <name> and <uuid> (<uuid> can also be removed), or + b) DON'T CHANGE! either <name> or <uuid> + + # $EDITOR domain.xml + + 3) create a domain from domain.xml, depending on whether following 2a or 2b + respectively: + a) the domain is going to be transient + b) an existing persistent domain will run with a modified one-time + configuration + + # virsh create domain.xml =item B<define> I<FILE> [I<--validate>] -- 2.13.3

2 2

[libvirt] [PATCH 0/4] qemu: fix restore domain with bypass cache flag
by Nikolay Shirokovskiy 19 Sep '17

19 Sep '17

It is not working currently. Error message is (with convinient wraps): error: Failed to restore domain from tst6.sav error: internal error: Child process (LIBVIRT_LOG_OUTPUTS=1:stderr \ /usr/libexec/libvirt_iohelper /vz/dev/libvirt/tst6.sav 0 0) unexpected exit status 1: \ /usr/libexec/libvirt_iohelper: failure with /vz/dev/libvirt/tst6.sav : \ Unable to read /vz/dev/libvirt/tst6.sav: Invalid argument The problem is that saferead function being used by iohelper is not quite suitable for direct reads. It's last read that is supposed to read EOF is not properly aligned thus EINVAL is returned. Nikolay Shirokovskiy (4): iohelper: drop unused operation length limit iohelper: simplify last direct write alignment iohelper: reduce zero-out in align case iohelper: fix reading with O_DIRECT src/util/iohelper.c | 98 +++++++++++++++++++++++++++++++---------------------- src/util/virfile.c | 2 +- 2 files changed, 59 insertions(+), 41 deletions(-) -- 1.8.3.1

2 6

[libvirt] How to implement pool support in virt-aa-helper?
by Christian Ehrhardt 19 Sep '17

19 Sep '17

Hi, Currently virt-aa-helper has no support for pools, so if you use a volume from a pool like: <disk type='volume' device='disk'> <driver name='qemu' type='raw' cache='none'/> <source pool='internal' volume='foo'/> <target dev='vdc' bus='virtio'/> </disk> Then there is no matching apparmor rule generated to allow qemu to access the related devices. That goes along a simple pool like: # create a zpool to use $ for i in $(seq 1 3); do dd if=/dev/zero of=/tmp/fdisk${i} bs=1M count=1024; done $ sudo zpool create internal /tmp/fdisk* # make pool in libvirt and guest disk foo $ virsh pool-define-as internal zfs $ virsh pool-start internal $ virsh vol-create-as internal foo 2G When libvirt actually builds up the command for qemu what to use based on the pools information it uses the function "virStorageTranslateDiskSourcePool" to learn about the actual path of the disk. Usually virt-aa-helper tries to stay close to what libvirt does to spawn the guest, because only then the generated rules match what will be used. But in this case this isn't as easy. The lookup by "virStorageTranslateDiskSourcePool" is done on a virConnectPtr via: conn->storageDriver->storagePoolLookupByName That is fine from the code that sets up the guest, but virt-aa-helper has no virConnectPtr context. Surely virt-aa-helper could connect via e.g. "virConnectOpenReadOnly" but the lifecycle of virt-aa-helper doesn't match that. It is invoked as external program by libvirtd local to the execution of the guest e.g. the qemu. So adding a connection via a socket makes not much sense. Also in this environment the connect URI is not set and also not passed to virt-aa-helper so we can't be sure we have "the same". I found that while trying to implement that, but I don't like the preliminary change [1] for the reasons I identified and outlined above. It initializes the storage drivers correctly, but then fails to look up [2] and since I doubt it is the right way for the environment and lifecycle of virt-aa-helper I don't want to go down that way before having a discussion about it. Therefore I wanted to ask for some guidance: 1. is there a reasonable way to be able to call virStorageTranslateDiskSourcePool from virt-aa-helper 2. maybe I overlook a path to get to a valid virConnectPtr from the virDomainDefPtr or any other in [3] that virt-aa-helper aready has. 3. what would be an alternative to be able to translate pool/volume info to real paths from virt-aa-helper [1]: http://paste.ubuntu.com/25570670/ [2]: http://paste.ubuntu.com/25570673/ [3]: http://paste.ubuntu.com/25570720/ -- Christian Ehrhardt Software Engineer, Ubuntu Server Canonical Ltd

2 1

[libvirt] [PATCH 0/2] qemu: blockPeek: Fix invalid buffer usage and
by Peter Krempa 19 Sep '17

19 Sep '17

See 1/2 for the bugfix. Peter Krempa (2): qemu: blockPeek: Fix filling of the return buffer qemu: blockPeek: Enforce buffer filling src/qemu/qemu_driver.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) -- 2.14.1

2 4

[libvirt] [PATCH] qemu: Introduce a wrapper over virFileWrapperFdClose
by Michal Privoznik 19 Sep '17

19 Sep '17

https://bugzilla.redhat.com/show_bug.cgi?id=1448268 When migrating to a file (e.g. when doing 'virsh save file'), couple of things are happening in the thread that is executing the API: 1) the domain obj is locked 2) iohelper is spawned as a separate process to handle all I/O 3) the thread waits for iohelper to finish 4) the domain obj is unlocked Now, the problem is that while the thread waits in step 3 for iohelper to finish this may take ages because iohelper calls fdatasync(). And unfortunately, we are waiting the whole time with the domain locked. So if another thread wants to jump in and say copy the domain name ('virsh list' for instance), they are stuck. The solution is to unlock the domain whenever waiting for I/O and lock it back again when it finished. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/qemu/qemu_driver.c | 27 +++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index b334cf20b..f81d3def4 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -3216,6 +3216,25 @@ qemuOpenFileAs(uid_t fallback_uid, gid_t fallback_gid, goto cleanup; } + +static int +qemuFileWrapperFDClose(virDomainObjPtr vm, + virFileWrapperFdPtr fd) +{ + int ret; + + /* virFileWrapperFd uses iohelper to write data onto disk. + * However, iohelper calls fdatasync() which may take ages to + * finish. Therefore, we shouldn't be waiting with the domain + * object locked. */ + + virObjectUnlock(vm); + ret = virFileWrapperFdClose(fd); + virObjectLock(vm); + return ret; +} + + /* Helper function to execute a migration to file with a correct save header * the caller needs to make sure that the processors are stopped and do all other * actions besides saving memory */ @@ -3276,7 +3295,7 @@ qemuDomainSaveMemory(virQEMUDriverPtr driver, goto cleanup; } - if (virFileWrapperFdClose(wrapperFd) < 0) + if (qemuFileWrapperFDClose(vm, wrapperFd) < 0) goto cleanup; if ((fd = qemuOpenFile(driver, vm, path, O_WRONLY, NULL, NULL)) < 0 || @@ -3827,7 +3846,7 @@ doCoreDump(virQEMUDriverPtr driver, path); goto cleanup; } - if (virFileWrapperFdClose(wrapperFd) < 0) + if (qemuFileWrapperFDClose(vm, wrapperFd) < 0) goto cleanup; ret = 0; @@ -6687,7 +6706,7 @@ qemuDomainRestoreFlags(virConnectPtr conn, ret = qemuDomainSaveImageStartVM(conn, driver, vm, &fd, data, path, false, QEMU_ASYNC_JOB_START); - if (virFileWrapperFdClose(wrapperFd) < 0) + if (qemuFileWrapperFDClose(vm, wrapperFd) < 0) VIR_WARN("Failed to close %s", path); qemuProcessEndJob(driver, vm); @@ -6962,7 +6981,7 @@ qemuDomainObjRestore(virConnectPtr conn, ret = qemuDomainSaveImageStartVM(conn, driver, vm, &fd, data, path, start_paused, asyncJob); - if (virFileWrapperFdClose(wrapperFd) < 0) + if (qemuFileWrapperFDClose(vm, wrapperFd) < 0) VIR_WARN("Failed to close %s", path); cleanup: -- 2.13.5

2 2

[libvirt] [PATCH v5 0/2] Add support for qcow2 cache
by Liu Qing 19 Sep '17

19 Sep '17

Qcow2 small IO random write performance will drop dramatically if the l2 cache table could not cover the whole disk. This will be a lot of l2 cache table RW operations if cache miss happens frequently. This patch exports the qcow2 driver parameter l2-cache-size/refcount-cache-size, first added in Qemu 2.2, and cache-clean-interval, first added in Qemu 2.5, in libvirt. change since v4: updated doc and did code adjustment based on John Ferlan's review. Liu Qing (2): conf, docs: Add qcow2 cache configuration support qemu: add capability checking for qcow2 cache configuration docs/formatdomain.html.in | 41 +++++++++ docs/schemas/domaincommon.rng | 35 ++++++++ src/conf/domain_conf.c | 96 ++++++++++++++++++++-- src/qemu/qemu_capabilities.c | 11 +++ src/qemu/qemu_capabilities.h | 5 ++ src/qemu/qemu_command.c | 33 ++++++++ src/qemu/qemu_driver.c | 5 ++ src/util/virstoragefile.c | 3 + src/util/virstoragefile.h | 6 ++ tests/qemucapabilitiesdata/caps_2.5.0.x86_64.xml | 3 + .../caps_2.6.0-gicv2.aarch64.xml | 3 + .../caps_2.6.0-gicv3.aarch64.xml | 3 + tests/qemucapabilitiesdata/caps_2.6.0.ppc64le.xml | 3 + tests/qemucapabilitiesdata/caps_2.6.0.x86_64.xml | 3 + tests/qemucapabilitiesdata/caps_2.7.0.s390x.xml | 3 + tests/qemucapabilitiesdata/caps_2.7.0.x86_64.xml | 3 + tests/qemucapabilitiesdata/caps_2.8.0.s390x.xml | 3 + tests/qemucapabilitiesdata/caps_2.8.0.x86_64.xml | 3 + tests/qemucapabilitiesdata/caps_2.9.0.ppc64le.xml | 3 + tests/qemucapabilitiesdata/caps_2.9.0.s390x.xml | 3 + tests/qemucapabilitiesdata/caps_2.9.0.x86_64.xml | 3 + .../qemuxml2argv-disk-drive-qcow2-cache.args | 28 +++++++ .../qemuxml2argv-disk-drive-qcow2-cache.xml | 43 ++++++++++ tests/qemuxml2argvtest.c | 4 + .../qemuxml2xmlout-disk-drive-qcow2-cache.xml | 43 ++++++++++ tests/qemuxml2xmltest.c | 1 + 26 files changed, 385 insertions(+), 5 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-qcow2-cache.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-qcow2-cache.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-disk-drive-qcow2-cache.xml -- 1.8.3.1

3 5

[libvirt] [PATCH 0/2] [V2] Implement LVM volume resize via lvresize
by apolyakov＠beget.ru 19 Sep '17

19 Sep '17

From: Alexander Polyakov <apolyakov(a)beget.com> Includes suggestions by John Ferlan & Peter Krempa Alexander Polyakov (2): logical: implement volume resize news: Document lvm volume resize docs/news.xml | 8 ++++++++ m4/virt-storage-lvm.m4 | 4 ++++ src/storage/storage_backend_logical.c | 37 +++++++++++++++++++++++++++++++++++ 3 files changed, 49 insertions(+) -- 2.13.5

2 4

[libvirt] [PATCH] apparmor: cater for new AAVMF image location
by Guido Günther 18 Sep '17

18 Sep '17

Things moved again, sigh. --- src/security/virt-aa-helper.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/security/virt-aa-helper.c b/src/security/virt-aa-helper.c index 55a686a59c..0b43c8e391 100644 --- a/src/security/virt-aa-helper.c +++ b/src/security/virt-aa-helper.c @@ -516,7 +516,8 @@ valid_path(const char *path, const bool readonly) "/usr/share/OVMF/", /* for OVMF images */ "/usr/share/ovmf/", /* for OVMF images */ "/usr/share/AAVMF/", /* for AAVMF images */ - "/usr/share/qemu-efi/" /* for AAVMF images */ + "/usr/share/qemu-efi/", /* for AAVMF images */ + "/usr/share/qemu-efi-aarch64/" /* for AAVMF images */ }; /* override the above with these */ const char * const override[] = { -- 2.14.1

4 4