July 2020 - Devel - libvirt List Archives

[PATCH] docs/index.html.in: Add knowledge base link and description on index page
by Jianan Gao 08 Jul '20

08 Jul '20

Add link and description of libvirt knowledge base to make it easier for users and testers to understand libvirt. Signed-off-by: Jianan Gao <jgao(a)redhat.com> --- docs/index.html.in | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/index.html.in b/docs/index.html.in index 26e8406917..586defff54 100644 --- a/docs/index.html.in +++ b/docs/index.html.in @@ -63,6 +63,9 @@ <a href="formatbackup.html">backup jobs</a></dd> <dt><a href="http://wiki.libvirt.org">Wiki</a></dt> <dd>Read further community contributed content</dd> + + <dt><a href="https://libvirt.org/kbase.html">Knowledge base</a></dt> + <dd>Learn more about libvirt through knowledge base</dd> </dl> </div> -- 2.21.3

2 1

[PATCH 00/13] Allow HMAT configuration
by Michal Privoznik 08 Jul '20

08 Jul '20

Heterogeneous Memory Attribute Table describes links between NUMA nodes (what's the bandwidth and/or latency between two nodes and/or their side caches). This enables guests to fine tune their performance. Michal Prívozník (13): qemuxml2xmltest: Add "numatune-distance" test case conf: Move and rename virDomainParseScaledValue() numa_conf: Drop CPU from name of two functions qemu_command: Rename qemuBuildNumaArgStr() qemuBuildMachineCommandLine: Drop needless check numa_conf: Make virDomainNumaSetNodeCpumask() return void Allow NUMA nodes without vCPUs conf: Parse and format HMAT conf: Validate NUMA HMAT configuration numa: expose HMAT APIs qemu: Introduce QEMU_CAPS_NUMA_HMAT capability qemu: Build HMAT command line news: Document HMAT addition NEWS.rst | 5 + docs/formatdomain.html.in | 90 +++ docs/schemas/cputypes.rng | 118 +++- src/conf/cpu_conf.c | 2 +- src/conf/domain_conf.c | 140 +--- src/conf/numa_conf.c | 664 ++++++++++++++++-- src/conf/numa_conf.h | 72 +- src/libvirt_private.syms | 13 + src/libxl/xen_xl.c | 14 +- src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 226 +++++- src/qemu/qemu_validate.c | 22 +- src/util/virxml.c | 72 ++ src/util/virxml.h | 8 + .../caps_5.0.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_5.0.0.ppc64.xml | 1 + .../caps_5.0.0.riscv64.xml | 1 + .../caps_5.0.0.x86_64.xml | 1 + .../caps_5.1.0.x86_64.xml | 1 + .../numatune-hmat.x86_64-latest.args | 53 ++ tests/qemuxml2argvdata/numatune-hmat.xml | 52 ++ tests/qemuxml2argvdata/numatune-no-vcpu.args | 33 + tests/qemuxml2argvdata/numatune-no-vcpu.xml | 42 ++ tests/qemuxml2argvtest.c | 2 + .../qemuxml2xmloutdata/numatune-distances.xml | 96 +++ tests/qemuxml2xmloutdata/numatune-hmat.xml | 1 + tests/qemuxml2xmloutdata/numatune-no-vcpu.xml | 1 + tests/qemuxml2xmltest.c | 3 + 29 files changed, 1536 insertions(+), 201 deletions(-) create mode 100644 tests/qemuxml2argvdata/numatune-hmat.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/numatune-hmat.xml create mode 100644 tests/qemuxml2argvdata/numatune-no-vcpu.args create mode 100644 tests/qemuxml2argvdata/numatune-no-vcpu.xml create mode 100644 tests/qemuxml2xmloutdata/numatune-distances.xml create mode 120000 tests/qemuxml2xmloutdata/numatune-hmat.xml create mode 120000 tests/qemuxml2xmloutdata/numatune-no-vcpu.xml -- 2.26.2

3 31

[PATCH v2 0/4] backup: Allow configuring incremental backup per-disk individually
by Peter Krempa 07 Jul '20

07 Jul '20

This series addresses some of the testing issues raised in the review of 4/4. Patches 1-3 are new adding testing of virDomainBackupAlignDisks. Peter Krempa (4): backupxml2xmltest: Remove output symlink of 'backup-pull-internal-invalid' virDomainBackupDiskDefFormat: Format internal disk state only when valid backupxml2xmltest: Call 'virDomainBackupAlignDisks' before formatting output backup: Allow configuring incremental backup per-disk individually docs/formatbackup.rst | 11 ++++ docs/schemas/domainbackup.rng | 23 +++++++++ src/conf/backup_conf.c | 51 ++++++++++++++++++- src/conf/backup_conf.h | 11 ++++ tests/domainbackupxml2xmlin/backup-pull.xml | 12 +++++ .../backup-pull-encrypted.xml | 7 +-- .../backup-pull-internal-invalid.xml | 38 +++++++++++++- .../backup-pull-seclabel.xml | 5 +- tests/domainbackupxml2xmlout/backup-pull.xml | 15 +++++- .../backup-push-encrypted.xml | 7 +-- .../backup-push-seclabel.xml | 5 +- tests/domainbackupxml2xmlout/backup-push.xml | 3 +- tests/domainbackupxml2xmlout/empty.xml | 8 ++- tests/genericxml2xmltest.c | 36 +++++++++++++ 14 files changed, 216 insertions(+), 16 deletions(-) mode change 120000 => 100644 tests/domainbackupxml2xmlout/backup-pull-internal-invalid.xml -- 2.26.2

2 8

[PATCH 00/24] qemu: Incremental backup and TLS handling fixes
by Peter Krempa 07 Jul '20

07 Jul '20

This series consists of multiple parts fixing the following bugs. Some of them depend on previous so I'm sending it as one to prevent conflicts. - Patches 1 - 11: https://bugzilla.redhat.com/show_bug.cgi?id=1602328 [RFE] Add support for encrypted TLS client keys for disks - Patch 12: https://bugzilla.redhat.com/show_bug.cgi?id=1840053 [incremental_backup] cannot do FULL backup for a READONLY disk - Patches 13 - 14: https://bugzilla.redhat.com/show_bug.cgi?id=1829829 [incremental backup] Creating incremental backup that includes a new VM disk that requires full backup is impossible - Patch 15: https://bugzilla.redhat.com/show_bug.cgi?id=1799010 incremental-backup: RFE: Handle backup bitmaps during virDomainBlockPull - Patches 16 - 24: https://bugzilla.redhat.com/show_bug.cgi?id=1822631 [incremental backup] RFE: Support TLS for NBD connections for pull mode backup Peter Krempa (24): qemu: domain: Introduce helper for always fetching virStorageSource private data qemuDomainDiskHasEncryptionSecret: unexport qemu.conf: Remove misleading mention of 'migrate_tls' qemu: conf: Move 'nbd' and 'vxhs' tls config variables together with rest of tls setup virQEMUDriverConfigLoadSpecificTLSEntry: Move fetching of 'chardev_tls' above macro virQEMUDriverConfigLoadSpecificTLSEntry: Split up fetching of server-only config options qemu: domain: Add infrastructure passing in TLS key's decryption key via 'secret' qemu block: Add internals for handling 'secret' corresponding to TLS key qemu: conf: Add configuration of TLS key encryption for 'vxhs' and 'nbd' disks qemu: domain: Setup secret for TLS key for nbd/vxhs disks tests: qemuxml2argv: Test encrypted TLS key for nbd/vxhs disks conf: backup: Don't explicitly forbid backup of read-only disk docs: backup: Convert XML documentation to RST backup: Allow configuring incremental backup per-disk individually qemu: backup: integrate with blockpull docs: checkpoint: Convert XML documentation to RST conf: checkpoint: Add a flag storing whether disk 'size' is valid qemu: checkpoint: Implement VIR_DOMAIN_CHECKPOINT_XML_SIZE checkpoint: Mention that VIR_DOMAIN_CHECKPOINT_XML_SIZE is expensive and stale testCompareBackupXML: Add infrastructure for testing internal fields conf: backup: Store 'tlsAlias' and 'tlsSecretAlias' as internals of a backup qemu: conf: Add configuration of TLS environment for NBD transport of pull-backups conf: backup: Add 'tls' attribute for 'server' element qemu: backup: Setup TLS environment for pull-mode backup jobs docs/formatbackup.html.in | 191 ----------------- docs/formatbackup.rst | 164 +++++++++++++++ docs/formatcheckpoint.html.in | 198 ------------------ docs/formatcheckpoint.rst | 166 +++++++++++++++ docs/schemas/domainbackup.rng | 25 ++- src/conf/backup_conf.c | 123 ++++++++++- src/conf/backup_conf.h | 17 ++ src/conf/checkpoint_conf.c | 2 +- src/conf/checkpoint_conf.h | 1 + src/libvirt-domain-checkpoint.c | 3 +- src/qemu/libvirtd_qemu.aug | 19 +- src/qemu/qemu.conf | 63 +++++- src/qemu/qemu_backup.c | 80 ++++++- src/qemu/qemu_block.c | 12 ++ src/qemu/qemu_block.h | 2 + src/qemu/qemu_blockjob.c | 37 ++++ src/qemu/qemu_checkpoint.c | 143 ++++++++++++- src/qemu/qemu_command.c | 11 +- src/qemu/qemu_conf.c | 57 +++-- src/qemu/qemu_conf.h | 19 +- src/qemu/qemu_domain.c | 66 ++++-- src/qemu/qemu_domain.h | 8 +- src/qemu/test_libvirtd_qemu.aug.in | 5 + .../backup-pull-encrypted.xml | 2 +- .../backup-pull-internal-invalid.xml | 36 ++++ tests/domainbackupxml2xmlin/backup-pull.xml | 12 ++ .../backup-pull-encrypted.xml | 2 +- .../backup-pull-internal-invalid.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 12 ++ tests/genericxml2xmltest.c | 32 ++- tests/qemudomaincheckpointxml2xmltest.c | 1 + tests/qemustatusxml2xmldata/modern-in.xml | 1 + .../disk-network-tlsx509.x86_64-2.12.0.args | 15 +- .../disk-network-tlsx509.x86_64-latest.args | 18 +- tests/qemuxml2argvtest.c | 2 + 35 files changed, 1079 insertions(+), 467 deletions(-) delete mode 100644 docs/formatbackup.html.in create mode 100644 docs/formatbackup.rst delete mode 100644 docs/formatcheckpoint.html.in create mode 100644 docs/formatcheckpoint.rst create mode 100644 tests/domainbackupxml2xmlin/backup-pull-internal-invalid.xml create mode 120000 tests/domainbackupxml2xmlout/backup-pull-internal-invalid.xml -- 2.26.2

4 55

[libvirt PATCH] util: fix accessibility check for hook directory
by Ján Tomko 07 Jul '20

07 Jul '20

virFileIsAccessible does not return true on accessible directories. Check whether it set EISDIR and only then assume the directory is inaccessible. Return 0 (not found) instead of 1 (found), since the bridge driver taints the network based on this return value, not whether the hook actually ran. Remove the bogus check from virHookCall, since it already checks the virHooksFound bitmap that was filled before by virHookCheck. Signed-off-by: Ján Tomko <jtomko(a)redhat.com> Fixes: 7fa7f7eeb6e969e002845928e155914da2fc8cd0 Closes: https://gitlab.com/libvirt/libvirt/-/issues/47 --- src/util/virhook.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/src/util/virhook.c b/src/util/virhook.c index d1ac518c24..119ad1aae6 100644 --- a/src/util/virhook.c +++ b/src/util/virhook.c @@ -172,9 +172,9 @@ virHookCheck(int no, const char *driver) dir_path = g_strdup_printf("%s.d", path); - if (!virFileIsExecutable(dir_path)) { + if (!virFileIsExecutable(dir_path) && errno != EISDIR) { VIR_DEBUG("Hook dir %s is not accessible", dir_path); - return 1; + return 0; } if ((ret = virDirOpenIfExists(&dir, dir_path)) < 0) @@ -422,9 +422,6 @@ virHookCall(int driver, dir_path = g_strdup_printf("%s.d", path); - if (!virFileIsExecutable(dir_path)) - return script_ret; - if ((ret = virDirOpenIfExists(&dir, dir_path)) < 0) return -1; -- 2.25.4

2 1

[libvirt PATCH v2 0/2] docs: Point to pkg.go.dev instead of godoc.org
by Andrea Bolognani 07 Jul '20

07 Jul '20

Changes from [v1]: * fix a few instances where we were still using github.com instead of libvirt.org for Go imports; * replace all uses of godoc.org, not just a subset. [v1] https://www.redhat.com/archives/libvir-list/2020-July/msg00270.html Andrea Bolognani (2): docs: Use libvirt.org namespace for Go bindings docs: Point to pkg.go.dev instead of godoc.org docs/bindings.html.in | 2 +- docs/docs.html.in | 2 +- docs/downloads.html.in | 4 ++-- docs/libvirt-go-xml.rst | 2 +- docs/libvirt-go.rst | 2 +- 5 files changed, 6 insertions(+), 6 deletions(-) -- 2.25.4

2 3

[libvirt PATCH] docs: Point to pkg.go.dev instead of godoc.org
by Andrea Bolognani 07 Jul '20

07 Jul '20

The former is the new recommended frontend for browsing Go API documentation online. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- docs/libvirt-go-xml.rst | 2 +- docs/libvirt-go.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/libvirt-go-xml.rst b/docs/libvirt-go-xml.rst index ee3b878068..995fdd2b07 100644 --- a/docs/libvirt-go-xml.rst +++ b/docs/libvirt-go-xml.rst @@ -7,4 +7,4 @@ annotated Go struct definitions for parsing (and formatting) XML documents used with libvirt APIs. For details of Go specific behaviour consult the -`Go package documentation <https://godoc.org/libvirt.org/libvirt-go-xml>`__ +`Go package documentation <https://pkg.go.dev/libvirt.org/libvirt-go-xml>`__. diff --git a/docs/libvirt-go.rst b/docs/libvirt-go.rst index 6ec5c0c123..f2d3f80fb2 100644 --- a/docs/libvirt-go.rst +++ b/docs/libvirt-go.rst @@ -10,4 +10,4 @@ concepts to Go, so the native API documentation should serve as a reference for most behaviour. For details of Go specific behaviour consult the -`Go package documentation <https://godoc.org/libvirt.org/libvirt-go>`__ +`Go package documentation <https://pkg.go.dev/libvirt.org/libvirt-go>`__. -- 2.25.4

2 1

[question]qemu: any plan to support ivshmem-plain migration?
by Wangxin (Alexander, Cloud Infrastructure Service Product Dept.) 07 Jul '20

07 Jul '20

Hi, Martin Ivshmem-plain device support property role with 'master'(master=on) or 'peer'(master=off, default mode), which controls to copy the shared memory on migration to the destination host or not. https://git.qemu.org/?p=qemu.git;a=blob;f=docs/system/ivshmem.rst;h=b03a48a… And libvirt does not support to configure 'role' yet, see msg https://www.redhat.com/archives/libvir-list/2016-August/msg00591.html Qemu use 'master=off' as ivshmem-plain default property, the guest create by libvirt api can't be migrated now. Do you have an plan to support the property 'role' and ivshmem-palin migration? Regards, Xin

2 2

[PATCH] news.html: Add 3 new features
by Yanqiu Zhang 04 Jul '20

04 Jul '20

Add 'virtio packed' in 6.3.0, 'virDomainGetHostnameFlags' and 'Panic Crashloaded event' for 6.1.0. Signed-off-by: Yanqiu Zhang <yanqzhan(a)redhat.com> --- NEWS.rst | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/NEWS.rst b/NEWS.rst index 0e9822cd03..eb72b1be00 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -173,6 +173,11 @@ v6.3.0 (2020-05-05) (Cache Flush on Privilege Change), SBBC (Speculation Barrier Bounds Checking) and IBS (Indirect Branch Speculation). + * qemu: Add support for virtio packed option + + The attribute ``packed`` controls if QEMU should try to use packed + virtqueues. Possible values are ``on`` or ``off``. + * **Improvements** * qemu: Allow checkpoint redefine for offline VMs @@ -365,6 +370,17 @@ v6.1.0 (2020-03-03) This device, available starting from QEMU 5.0, is limited to pSeries guests. + * qemu: support Panic Crashloaded event handling + + Pvpanic device supports bit 1 as crashloaded event, it means that guest + actually panicked and run kexec to handle error by guest side. + + * qemu: Implement virDomainGetHostnameFlags + + The --source argument can be used to specify what data source to use for + the domain hostnames. Currently, except 'agent', libvirt could also use + 'lease' information from dnsmasq to get current hostname. + * **Improvements** * qemu: Image format probing is allowed in certain cases -- 2.25.4

2 1

[PATCH 0/9] Allow sparse streams for block devices
by Michal Privoznik 04 Jul '20

04 Jul '20

The way our sparse streams are implemented is: 1) user facing APIs (virStreamSparseRecvAll() and virStreamSparseSendAll()) take callbacks as arguments. These callbacks read/write data or determine if there is a hole in the underlying file and big it is. 2) libvirtd has something similar - virFDStream, except here the functions for read/write of data and hole handling are called directly. Sparse streams were originally implemented for regular files only => both ends of stream has to be regular files. This limitation exists because the callbacks from 1) (implemented in virsh for vol-download/vol-upload commands) and also from 2) (which is basically the same code) uses lseek(..., SEEK_DATA) and/or lseek(..., SEEK_HOLE) to get map of allocated file blocks. They also take a shortcut (valid for regular files) - when one side of the stream is asked to create a hole it merely lseek() + ftruncate(). For regular files this creates a hole and later, when somebody reads it all they get is zeroes. Neither of these two approaches work for block devices. Block devices have no notion of data/hole sections [1], nor can they be truncated. What we can do instead is read data from the block device and check if its full of zeroes. And for "creating a hole" we just write zeroes of requested size. There is a follow up patch that I am working on: this implementation I'm posting here has one disadvantage: after some blocks are read from the block device and they are found to contain data, the whole buffer is freed only to be read again. For instance, when uploading volume, virshStreamInData() is called at the beginning to check if the file containing data to upload doesn't start with a hole. If the file is a block device, then virFileInDataDetectZeroes() is called which reads 1MiB from it, finds (say) data and throws the buffer away. Then virshStreamSource() is called, which reads the 1MiB buffer again. The patch is still under development though. 1: Okay, SSDs keep list of free blocks for wear levelling, but the list is kept private by the SSD controller and I am not aware of any way of getting it. Michal Prívozník (9): libvirt-storage: Document volume upload/download stream format virstring: Introduce virStringIsNull() virfile: Introduce virFileInDataDetectZeroes() virsh: Pass virshStreamCallbackDataPtr to virshStreamSink() and virshStreamSkip() virsh: Track if vol-upload or vol-download work over a block device virshStreamSkip: Emulate skip for block devices virfdstream: Allow sparse stream vol-download virshStreamInData: Handle block devices virfdstream: Emulate skip for block devices src/libvirt-storage.c | 8 +++-- src/libvirt_private.syms | 2 ++ src/util/virfdstream.c | 74 ++++++++++++++++++++++++++++++---------- src/util/virfile.c | 67 ++++++++++++++++++++++++++++++++++++ src/util/virfile.h | 4 +++ src/util/virstring.c | 40 ++++++++++++++++++++++ src/util/virstring.h | 2 ++ tools/virsh-util.c | 52 ++++++++++++++++++++++------ tools/virsh-util.h | 1 + tools/virsh-volume.c | 20 ++++++++++- 10 files changed, 238 insertions(+), 32 deletions(-) -- 2.26.2

4 19