- Devel - libvirt List Archives

[PATCH v5 0/5] migration/dirtyrate: Introduce APIs for getting domain memory dirty rate
by Hao Wang 01 Feb '21

01 Feb '21

V4 -> V5: squash 1/7 and bits of 5/7 and 6/7 into 2/7 in v4 (to be 1/5 in v5) squash left of 5/7 into 4/7 in v4 (to be 3/5 in v5) add VIR_DOMAIN_DIRTYRATE_DEFAULT flag remove redundant error report rename virsh api to "domdirtyrate" use vshTablePtr for virsh api output add description in docs/manpages/virsh.rst other format optimize V3 -> V4: define flags to unsigned int fix some compile warnings V2 -> V3: reorganize patchset to fix compile warning V1 -> V2: replace QEMU_JOB_ASYNC with QEMU_JOB_QUERY Sometimes domain's memory dirty rate is expected by user in order to decide whether it's proper to be migrated out or not. We have already completed the QEMU part of the capability: https://patchew.org/QEMU/1600237327-33618-1-git-send-email-zhengchuan@huawe… And this serial of patches introduce the corresponding LIBVIRT part -- DomainGetDirtyRateInfo API and corresponding virsh api -- "domdirtyrate". instructions: bash# virsh domdirtyrate --help NAME domdirtyrate - Get a vm's memory dirty rate SYNOPSIS domdirtyrate <domain> [--seconds <number>] [--calculate] [--query] DESCRIPTION Get memory dirty rate of a domain in order to decide whether it's proper to be migrated out or not. OPTIONS [--domain] <string> domain name, id or uuid --seconds <number> calculate memory dirty rate within specified seconds, the supported value range is [1, 60], default to 1s. --calculate calculate dirty rate only, can be used together with --query, either or both is expected, otherwise would default to both. --query query dirty rate only, can be used together with --calculate, either or both is expected, otherwise would default to both. example: bash# virsh domdirtyrate vm0 --calculate --query --seconds 1 Item Value ----------------------------- Status: measured Start time: 51585 Calculate time: 1 s Dirty rate: 18 MB/s Hao Wang (5): migration/dirtyrate: Introduce DomainGetDirtyRateInfo API migration/dirtyrate: Implement qemuDomainCalculateDirtyRate migration/dirtyrate: Implement qemuDomainQueryDirtyRate migration/dirtyrate: Implement qemuDomainGetDirtyRateInfo migration/dirtyrate: Introduce domdirtyrate virsh api docs/manpages/virsh.rst | 17 +++++ include/libvirt/libvirt-domain.h | 59 ++++++++++++++ src/driver-hypervisor.h | 7 ++ src/libvirt-domain.c | 56 ++++++++++++++ src/libvirt_public.syms | 5 ++ src/qemu/qemu_driver.c | 67 ++++++++++++++++ src/qemu/qemu_migration.c | 48 ++++++++++++ src/qemu/qemu_migration.h | 10 +++ src/qemu/qemu_monitor.c | 24 ++++++ src/qemu/qemu_monitor.h | 8 ++ src/qemu/qemu_monitor_json.c | 99 ++++++++++++++++++++++++ src/qemu/qemu_monitor_json.h | 8 ++ src/remote/remote_driver.c | 1 + src/remote/remote_protocol.x | 21 ++++- tools/virsh-domain.c | 127 +++++++++++++++++++++++++++++++ 15 files changed, 556 insertions(+), 1 deletion(-) -- 2.23.0

3 12

[PATCH 00/14] util: Remove VIR_DISPOSE(_N)
by Peter Krempa 01 Feb '21

01 Feb '21

Most callers are way better off using memset directly additionally few places didn't even use it to clear sensitive data in the first place since the name probably sounded as the right thing to use. Peter Krempa (14): hypervFreeInvokeParams: Don't use VIR_DISPOSE_N for freeing 'params' libxlMakeDomBuildInfo: Don't use VIR_DISPOSE_N for USB device list storage_backend_iscsi(_direct): Properly clear secrets libxlMakeNetworkDiskSrc: Avoid use of VIR_DISPOSE_N qemuDomainMasterKeyCreate: Don't use VIR_DISPOSE_N on failure qemu: domain: Use memset for clearing secrets instead of VIR_DISPOSE_N cmdSecretSetValue: Make it obvious that --file, --base64 and --interactive are exlcusive virsh: cmdSecretSetValue: Rework handling of the secret value virsh: cmdSecretGetValue: Use memset instead of VIR_DISPOSE_N virStorageBackendRBDOpenRADOSConn: Use memset instead of VIR_DISPOSE_N virCryptoEncryptDataAESgnutls: Use memset instead of VIR_DISPOSE_N storageBackendCreateQemuImgSecretPath: Use memset instead of VIR_DISPOSE_N tests: viralloc: Remove testDispose case util: viralloc: Remove VIR_DISPOSE(_N) src/hyperv/hyperv_wmi.c | 4 +- src/libvirt_private.syms | 1 - src/libxl/libxl_conf.c | 8 +-- src/qemu/qemu_domain.c | 24 ++++++--- src/storage/storage_backend_iscsi.c | 16 +++--- src/storage/storage_backend_iscsi_direct.c | 17 ++++--- src/storage/storage_backend_rbd.c | 5 +- src/storage/storage_util.c | 5 +- src/util/viralloc.c | 39 +------------- src/util/viralloc.h | 27 ---------- src/util/vircrypto.c | 3 +- tests/viralloctest.c | 34 ------------- tools/virsh-secret.c | 59 ++++++++++------------ 13 files changed, 76 insertions(+), 166 deletions(-) -- 2.29.2

2 16

[PATCH] util: virstring: Remove unused prototypes for virStr(n)dup
by Peter Krempa 01 Feb '21

01 Feb '21

The headers weren't removed after use of VIR_STRDUP was removed. Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- src/util/virstring.h | 7 ------- 1 file changed, 7 deletions(-) diff --git a/src/util/virstring.h b/src/util/virstring.h index 55547b040a..cfd24f0b74 100644 --- a/src/util/virstring.h +++ b/src/util/virstring.h @@ -123,13 +123,6 @@ int virStrcpy(char *dest, const char *src, size_t destbytes) G_GNUC_WARN_UNUSED_RESULT; #define virStrcpyStatic(dest, src) virStrcpy((dest), (src), sizeof(dest)) -/* Don't call these directly - use the macros below */ -int virStrdup(char **dest, const char *src) - G_GNUC_WARN_UNUSED_RESULT ATTRIBUTE_NONNULL(1); - -int virStrndup(char **dest, const char *src, ssize_t n) - G_GNUC_WARN_UNUSED_RESULT ATTRIBUTE_NONNULL(1); - size_t virStringListLength(const char * const *strings); int virStringSortCompare(const void *a, const void *b); -- 2.29.2

2 1

[libvirt PATCH v2 00/10] Random bits found by clang-tidy
by Tim Wiederhake 01 Feb '21

01 Feb '21

clang-tidy is a static code analysis tool under the llvm umbrella. It is primarily meant to be used on C++ code bases, but some of the checks it provides also apply to C. The findings vary in severity and contain pseudo-false-positives, i.e. clang-tidy is flagging potential execution flows that could happen in theory but are virtually impossible in real life: In function `virGetUnprivSGIOSysfsPath`, variables `maj` and `min` would be read unintialized if `stat()` failed and set `errno` to a negative value, to name just one example. The main source of false positive findings is the lack of support for `__attribute__((cleanup))` in clang-tidy, which is heavily used in libvirt through glib's `g_autofree` and `g_auto()` macros: #include <stdlib.h> void freeptr(int** p) { if (*p) free(*p); } int main() { __attribute__((cleanup(freeptr))) int *ptr = NULL; ptr = calloc(sizeof(int), 1); return 0; /* flagged as memory leak of `ptr` */ } This sadly renders clang-tidy's analysis of dynamic memory useless, hiding all real issues that it could otherwise find. Meson provides excellent integration for clang-tidy (a "clang-tidy" target is automatically generated if a ".clang-tidy" configuration file is present in the project's root directory). The amount of false-positives (many of which present in a group of checks that cannot be disabled), random segfaults in memory constraint environments such as the containers in the CI, and the slow analysis (triggering time-outs in the CI), make this tool unfit for inclusion in libvirt's GitLab CI though. The patches in this series are the result of fixing some of the issues reported by running CC=clang meson build ninja -C build # generate sources and header files ninja -C build clang-tidy with the following `.clang-tidy` configuration file: --- Checks: > *, -abseil-*, -android-*, -boost-*, -cppcoreguidelines-*, -fuchsia-*, -google-*, -hicpp-*, -llvm-*, -modernize-*, -mpi-, -objc-, -openmp-, -zircon-*, -readability-braces-around-statements, -readability-magic-numbers WarningsAsErrors: '*' HeaderFilterRegex: '' FormatStyle: none ... V1: https://www.redhat.com/archives/libvir-list/2021-January/msg01152.html Changes since V1: * Expanded the justification for the "Replace bzero() with memset()" patch. * Rewrote "udevProcessCCW: Initialize variable". * Rewrote "tests: Prevent mallo with size 0". * Dropped "vircommand: Remove NULL check in virCommandAddArg". Note that this series now depends on the rewrite of tests/commandhelper.c, which can be found here: https://www.redhat.com/archives/libvir-list/2021-February/msg00034.html Regards, Tim Tim Wiederhake (10): virfile: Remove redundant #ifndef xen: Fix indentation in xenParseXLSpice qemu_tpm: Fix indentation in qemuTPMEmulatorBuildCommand virsh-domain: Fix error handling of pthread_sigmask Replace bzero() with memset() udevProcessCCW: Initialize variable virhostuptime: Fix rounding in uptime calculation tests: Prevent malloc with size 0 vircryptotest: Directly assign string to avoid memcpy vircommand: Simplify virCommandAddArg src/libxl/xen_xl.c | 5 ++--- src/node_device/node_device_udev.c | 2 +- src/qemu/qemu_tpm.c | 8 +++++--- src/util/virarptable.c | 2 +- src/util/vircommand.c | 7 +------ src/util/virfile.c | 4 ++-- src/util/virhostuptime.c | 4 +++- tests/commandhelper.c | 3 +++ tests/vircryptotest.c | 5 +---- tests/virpcimock.c | 2 +- tools/virsh-domain.c | 8 ++++---- 11 files changed, 24 insertions(+), 26 deletions(-) -- 2.26.2

2 13

[PATCH] virfile: workaround for when posix_fallocate() is not supported by FS
by Roman Bogorodskiy 01 Feb '21

01 Feb '21

posix_fallocate() might be not supported by a filesystem, for example, it's not supported by ZFS. In that case it fails with return code 22 (EINVAL), and thus safezero_posix_fallocate() returns -1. As safezero_posix_fallocate() is the first function tried by safezero() and it tries other functions only when it returns -2, it fails immediately without falling back to other methods, such as safezero_slow(). Fix that by returning -2 if posix_fallocate() returns EINVAL, to give safezero() a chance to try other functions. Signed-off-by: Roman Bogorodskiy <bogorodskiy(a)gmail.com> --- src/util/virfile.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/util/virfile.c b/src/util/virfile.c index 609cafdd34..67a350deb3 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -1119,6 +1119,16 @@ safezero_posix_fallocate(int fd, off_t offset, off_t len) int ret = posix_fallocate(fd, offset, len); if (ret == 0) return 0; + else if (ret == EINVAL) + /* EINVAL is returned when either: + - Operation is not supported by the underlying filesystem, + - offset or len argument values are invalid. + Assuming that offset and len are valid, this error means + the operation is not supported, and we need to fall back + to other methods. + */ + return -2; + errno = ret; return -1; } -- 2.30.0

2 1

[PULL 00/13] Misc patches
by Daniel P. Berrangé 30 Jan '21

30 Jan '21

The following changes since commit 5101d00d2f1138a73344dc4833587f76d7a5fa5c: Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-for-6.0-p= ull-request' into staging (2021-01-29 10:10:43 +0000) are available in the Git repository at: https://gitlab.com/berrange/qemu tags/misc-fixes-pull-request for you to fetch changes up to ecb98f5c7589ba8ecd15c8b1baa2ec7192e47c75: tests: Replace deprecated ASN1 code (2021-01-29 17:07:53 +0000) ---------------------------------------------------------------- * Replace --enable/disable-git-update with --with-git-submodules to allow improved control over use of git submodules * Deprecate the -enable-fips option * Ensure docs use prefer format for bool options * Clarify platform support rules * Misc fixes to keymap conversions * Fix misc problems on macOS ---------------------------------------------------------------- Dan Streetman (1): configure: replace --enable/disable-git-update with --with-git-submodules Daniel P. Berrang=C3=A9 (5): os: deprecate the -enable-fips option and QEMU's FIPS enforcement Prefer 'on' | 'off' over 'yes' | 'no' for bool options docs: simplify and clarify the platform support rules docs: fix missing backslash in certtool shell example ui: update keycodemapdb submodule commit Kevin Wolf (3): crypto: Move USER_CREATABLE to secret_common base class crypto: Forbid broken unloading of secrets crypto: Fix memory leaks in set_loaded for tls-* Stefan Weil (2): tests: Fix runtime error in test-authz-pam tests: Replace deprecated ASN1 code shiliyang (2): crypto: Fix some code style problems, add spaces around operator crypto: Add spaces around operator Makefile | 24 ++----------- configure | 51 +++++++++++++++++++------- crypto/aes.c | 6 ++-- crypto/desrfb.c | 4 +-- crypto/secret.c | 14 -------- crypto/secret_common.c | 21 +++++++++-- crypto/secret_keyring.c | 14 -------- crypto/tlscredsanon.c | 3 +- crypto/tlscredspsk.c | 3 +- crypto/tlscredsx509.c | 5 ++- docs/system/build-platforms.rst | 63 ++++++++++++--------------------- docs/system/deprecated.rst | 12 +++++++ docs/system/tls.rst | 2 +- docs/system/vnc-security.rst | 10 +++--- include/authz/listfile.h | 2 +- os-posix.c | 3 ++ qemu-options.hx | 4 +-- scripts/git-submodule.sh | 34 +++++++++++++----- tests/crypto-tls-x509-helpers.c | 10 +++--- tests/crypto-tls-x509-helpers.h | 2 +- tests/pkix_asn1_tab.c | 2 +- tests/qemu-iotests/233 | 4 +-- tests/test-authz-pam.c | 10 +++++- ui/keycodemapdb | 2 +- ui/meson.build | 3 +- 25 files changed, 161 insertions(+), 147 deletions(-) --=20 2.29.2

3 15

[libvirt PATCH 00/19] Overhaul test/commandhelper.c
by Tim Wiederhake 29 Jan '21

29 Jan '21

I stumbled upon a buffer overflow / stack smash present in "test/commandhelper.c" that could be triggered by e.g. $ ./tests/commandhelper --readfd 0 --readfd 0 --readfd 0 --readfd x Could not parse fd x *** stack smashing detected ***: terminated Aborted (core dumped) This series cleans up the file, fixes the buffer overflow and converts (most) memory handling to g_auto*. Note that it does not touch the "prevent malloc with zero size" issue discussed in https://www.redhat.com/archives/libvir-list/2021-January/msg01160.html, this will be done in the other series. Please feel free to comment on whether the copyright year in the file's header should be updated and whether a prefix for the function names and the new type is required. Cheers, Tim Tim Wiederhake (19): commandhelper: Remove origenv variable commandhelper: Remove numpollfds variable commandhelper: Simplify envsort commandhelper: Consolidate error paths commandhelper: Consolidate argument parsing commandhelper: Split argument parsing and printing commandhelper: Factor out parseArguments commandhelper: Factor out printArguments commandhelper: Factor out printEnvironment commandhelper: Factor out printFds commandhelper: Factor out printDaemonization commandhelper: Factor out printCwd commandhelper: Factor out printInput commandhelper: Make number of fds variable in printInput commandhelper: Make number of fds variable in parseArguments commandhelper: Convert parseArguments to g_auto* commandhelper: Convert printEnvironment to g_auto* commandhelper: Convert printCwd to g_auto* commandhelper: Convert main to g_auto* tests/commandhelper.c | 295 +++++++++++++++++++++++++++--------------- 1 file changed, 188 insertions(+), 107 deletions(-) -- 2.26.2

4 28

[libvirt PATCH v2 0/2] stop using netcf for the interface driver backend
by Laine Stump 29 Jan '21

29 Jan '21

V1 here: https://www.redhat.com/archives/libvir-list/2021-January/msg00922.html A short version of the cover letter from V1: this is a followup to my proposal to stop using netcf for the interface driver backend in Fedora/RHEL/CentOS builds and use the udev backend instead (it turns out Ubuntu already disabled netcf in 2018). Changes in V2: * removed the patch that made the default netcf=disabled even when netcf-devel was found on the host. If someone has netcf-devel installed and still wants to build without netcf, then can add "-Dnetcf=disabled" on the meson commandline. * Made the specfile changes more intelligent: * instead of hardcoding -Dnetcf=disabled, we now have a variable %{with_netcf} that is set to 1 for current Fedora (< 34) and current RHEL (< 9) but will be set to 0 for future Fedora/RHEL. This way the behavior on current OS releases will remain the same even for future libvirt. * it is possible to for netcf support off even in current/older OS releases by adding "--without netcf" to the rpmbuild commandline. I think at this point I would be comfortable pushing these patches, unless someone has misgivings about it... Laine Stump (2): build: support explicitly disabling netcf rpm: disable netcf for the interface driver in rpm build on new targets libvirt.spec.in | 22 +++++++++++++++++----- meson.build | 10 ++++++---- 2 files changed, 23 insertions(+), 9 deletions(-) -- 2.29.2

4 8

[PATCH] Increase timeout for tests and syntax-check
by Michal Privoznik 29 Jan '21

29 Jan '21

Since we've switched to meson our tests run with a timeout (meson uses 30 seconds as the default). However, not every machine that builds libvirt is fast enough to run every test under 30 seconds (each test binary has its own timeout, but still). For instance when building a package for distro on a farm that's under load. Or on a generally slow ARM hardware. While each developer can tune their command line for building by adding --timeout-multiplier=10, this is hard to do for aforementioned build farms. It's time to admit that not everybody has the latest, top shelf CPU and increase the timeout. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- Successfully tested on my RasPi 2B, where the top three longest test were: qemuxml2argvtest OK 142.43s virschematest OK 40.82s virnettlssessiontest OK 38.64s and syntax-check: sc_spacing-check OK 50.26 sc_avoid_if_before_free OK 42.30s sc_prohibit_cross_inclusion OK 40.51s build-aux/meson.build | 1 + tests/meson.build | 14 ++++++-------- 2 files changed, 7 insertions(+), 8 deletions(-) diff --git a/build-aux/meson.build b/build-aux/meson.build index c506feefd2..e7b2db8759 100644 --- a/build-aux/meson.build +++ b/build-aux/meson.build @@ -44,6 +44,7 @@ if git potfiles_dep, ], suite: 'syntax-check', + timeout: 120, ) endforeach endif diff --git a/tests/meson.build b/tests/meson.build index 0de0783839..702090d594 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -546,6 +546,10 @@ if conf.has('WITH_YAJL') ] endif +# Increase the default timeout because some tests may run longer, +# esp. on slower systems. +timeout = 180 + foreach data : tests test_sources = '@0@.c'.format(data['name']) test_bin = executable( @@ -577,12 +581,6 @@ foreach data : tests ], export_dynamic: true, ) - if data['name'] == 'qemuxml2argvtest' - timeout = 90 - else - # default meson timeout - timeout = 30 - endif test(data['name'], test_bin, env: tests_env, timeout: timeout) endforeach @@ -683,7 +681,7 @@ endif foreach name : test_scripts script = find_program(name) - test(name, script, env: tests_env) + test(name, script, env: tests_env, timeout: timeout) endforeach add_test_setup( @@ -703,6 +701,6 @@ add_test_setup( '--suppressions=@0@'.format(meson.current_source_dir() / '.valgrind.supp'), '--error-exitcode=1', ], - # default timeout in meson is 30s + # Tests take a lot longer when run under Valgrind timeout_multiplier: 4, ) -- 2.26.2

5 17

[libvirt PATCH] docs: compiling: mention build dir
by Ján Tomko 29 Jan '21

29 Jan '21

Our docs have not been fully updated to reflect the separate build directory. Suggested-by: Laszlo Ersek <lersek(a)redhat.com> Signed-off-by: Ján Tomko <jtomko(a)redhat.com> --- docs/compiling.html.in | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/compiling.html.in b/docs/compiling.html.in index c2c9d9f0ed..df6074780f 100644 --- a/docs/compiling.html.in +++ b/docs/compiling.html.in @@ -98,15 +98,17 @@ $ ninja -C build <pre> $ su - # service libvirtd stop (or systemctl stop libvirtd.service) -# /home/to/your/checkout/src/libvirtd +# /home/to/your/checkout/build/src/libvirtd </pre> <p> - It is also possible to run virsh directly from the source tree + It is also possible to run virsh directly from the build tree using the ./run script (which sets some environment variables): </p> <pre> +$ pwd +/home/to/your/checkout/build $ ./run ./tools/virsh .... </pre> </body> -- 2.29.2

2 1