- Devel - libvirt List Archives

[PATCH 00/24] chardev handling cleanups (chardev refactors part 2)
by Peter Krempa 13 Dec '21

13 Dec '21

In this part I'm cleaning up the command line generators and unifying them. Until now there were multiple custom -chardev generators in places where it was simpler to use it than the common code. This series fixes the common code to have sane API and refactors virtiofs, vhost-user-blk vhost-user-gpu and TPM code to use it. After this series we are left with 3 distinct generators for chardev properties: - commandline - commandline for capabilties probing - QMP After the refactors here we can finally also add DO_TEST_CAPS_LATEST of chardev tests without actually touching the host as part of the testsuite which was impossible before. In further parts the generators will be put into one place and prepared for upcoming qemu refactors. Peter Krempa (24): virDomainDeviceInfoIterateFlags: Export the symbol qemu: domain: Introduce helpers for initializing chardev backend of devices qemu: Move creation and opening of chardev backend FDs to host prepare step qemu: command: Remove unused QEMU_BUILD_CHARDEV_FILE_LOGD and QEMU_BUILD_CHARDEV_UNIX_FD_PASS qemu: command: remove the unused 'chardevStdioLogd' argument/variable qemu: domain: Refactor chardev definition preparing qemu: Store chardev 'wait' flag in chardev source private data qemuBuildChrChardevStr: Remove 'cdevflags' argument qemuBuildChrChardevStr: Remove unused arguments and clean up callers qemuxml2argvtest: Add DO_TEST_CAPS_LATEST versions for all character device tests qemuxml2argvtest: Add _LATEST version for 'name-escape' case qemuBuildConsoleCommandLine: Remove identical code qemuBuildChrChardevStr: Directly generate command line qemuBuildChardevCommand: Move earlier qemuBuildChardevCommand: Don't prefix alias qemuBuildChardevCommand: Split creation of the command and setup of other objects qemu: Store TLS config options for chardevs in qemuDomainChrSourcePrivate qemuBuildChardevCommand: Remove unused 'cfg' parameter qemu: Implement chardev source setup for disk qemu: Use qemuBuildChrChardevCommand for vhost-user disk backend qemu: Use qemuBuildChrChardevCommand for virtiofs backend chardev qemu: Use qemuBuildChrChardevCommand for vhost-user-gpu backend chardev qemu: Implement chardev source setup for tpm qemu: command: Use 'qemuBuildChrChardevCommand' to build TPM backend src/libvirt_private.syms | 1 + src/qemu/qemu_block.c | 1 - src/qemu/qemu_block.h | 1 - src/qemu/qemu_command.c | 1095 +++++++---------- src/qemu/qemu_command.h | 2 - src/qemu/qemu_domain.c | 258 +++- src/qemu/qemu_domain.h | 38 +- src/qemu/qemu_driver.c | 8 + src/qemu/qemu_hotplug.c | 3 - src/qemu/qemu_process.c | 264 +++- .../aarch64-tpm.aarch64-latest.args | 2 +- .../console-compat-auto.x86_64-latest.args | 39 + .../console-compat-chardev.x86_64-latest.args | 39 + .../console-compat.x86_64-latest.args | 38 + .../name-escape.x86_64-latest.args | 51 + ...arallel-parport-chardev.x86_64-latest.args | 39 + .../parallel-tcp-chardev.x86_64-latest.args | 39 + .../pci-serial-dev-chardev.x86_64-latest.args | 39 + .../ppc64-tpmproxy-with-tpm.ppc64-latest.args | 2 +- ...rial-dev-chardev-iobase.x86_64-latest.args | 39 + .../serial-dev-chardev.x86_64-latest.args | 39 + .../serial-file-chardev.x86_64-latest.args | 40 + .../serial-file-log.x86_64-latest.args | 40 + .../serial-many-chardev.x86_64-latest.args | 42 + .../serial-pty-chardev.x86_64-latest.args | 39 + ...erial-spiceport-nospice.x86_64-latest.args | 37 + .../serial-spiceport.x86_64-latest.args | 41 + .../serial-tcp-chardev.x86_64-latest.args | 39 + ...rial-tcp-telnet-chardev.x86_64-latest.args | 39 + ...p-tlsx509-chardev-notls.x86_64-latest.args | 41 + ...-tlsx509-chardev-verify.x86_64-latest.args | 42 + ...ial-tcp-tlsx509-chardev.x86_64-latest.args | 42 + ...-tlsx509-secret-chardev.x86_64-latest.args | 43 + .../serial-udp-chardev.x86_64-latest.args | 41 + ...rial-unix-missing-source.x86_64-latest.err | 1 + .../serial-vc-chardev.x86_64-latest.args | 39 + .../tpm-emulator-spapr.ppc64-latest.args | 2 +- .../tpm-emulator-tpm2-enc.x86_64-latest.args | 2 +- ...pm-emulator-tpm2-pstate.x86_64-latest.args | 2 +- .../tpm-emulator-tpm2.x86_64-latest.args | 2 +- .../tpm-emulator.x86_64-latest.args | 2 +- ...host-user-gpu-secondary.x86_64-latest.args | 2 +- ...irtio-options-video-ats.x86_64-latest.args | 2 +- ...tio-options-video-iommu.x86_64-latest.args | 2 +- ...io-options-video-packed.x86_64-latest.args | 2 +- tests/qemuxml2argvtest.c | 114 ++ 46 files changed, 1959 insertions(+), 776 deletions(-) create mode 100644 tests/qemuxml2argvdata/console-compat-auto.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/console-compat-chardev.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/console-compat.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/name-escape.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/parallel-parport-chardev.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/parallel-tcp-chardev.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/pci-serial-dev-chardev.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/serial-dev-chardev-iobase.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/serial-dev-chardev.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/serial-file-chardev.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/serial-file-log.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/serial-many-chardev.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/serial-pty-chardev.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/serial-spiceport-nospice.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/serial-spiceport.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/serial-tcp-chardev.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/serial-tcp-telnet-chardev.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/serial-tcp-tlsx509-chardev-notls.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/serial-tcp-tlsx509-chardev-verify.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/serial-tcp-tlsx509-chardev.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/serial-tcp-tlsx509-secret-chardev.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/serial-udp-chardev.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/serial-unix-missing-source.x86_64-latest.err create mode 100644 tests/qemuxml2argvdata/serial-vc-chardev.x86_64-latest.args -- 2.31.1

3 29

[libvirt PATCH 00/12] Clean up cruft in firewall/iptables code (in preparation for nftables)
by Laine Stump 13 Dec '21

13 Dec '21

These patches make no functional change, they just remove a bunch of cruft that accumulated over the years and is no longer needed. This is all in advance of adding support for native nftable support, but there is nothing nftables-specific being added here; I just wanted to get these cleanups out of way now so that the eventual nftables support patchset is smaller and less complicated. (NB: the concept of a "firewall backend" is being removed here, implying that it will no longer exist. This is not true, but the way that it will exist in the future will be different (per-firewall object rather than per-process) so almost all of the existing code won't be applicable anyway.) Laine Stump (12): network: eliminate code that uses default iptables chains util: rename/move iptablesFormatNetwork to virSocketAddrFormatWithPrefix util: rename iptables operators to something less generic tests: remove firewalld backend tests from virfirewalltest.c tests: remove unnecessary ret variables and cleanup labels tests: document why virgdbus must be mocked in networkxml2firewalltest.c util: eliminate pointless switch in virFirewallApplyRule util: simplify virFirewallBackendSynchronize() util: move and rename virFirewallBackendSynchronize() util: remove check for iptables binary during virFirewallInit util: remove currentBackend from virfirewall.c util: remove virFirewallOnceInit() src/libvirt_private.syms | 5 +- src/network/bridge_driver_linux.c | 37 +-- src/util/virfirewall.c | 143 +---------- src/util/virfirewall.h | 2 - src/util/virfirewalld.c | 43 ++++ src/util/virfirewalld.h | 2 + src/util/virfirewallpriv.h | 37 --- src/util/viriptables.c | 207 +++++++--------- src/util/viriptables.h | 2 - src/util/virsocketaddr.c | 44 ++++ src/util/virsocketaddr.h | 3 + tests/networkxml2firewalltest.c | 14 +- tests/nwfilterebiptablestest.c | 7 - tests/nwfilterxml2firewalltest.c | 8 +- tests/virfirewalltest.c | 390 ++++-------------------------- 15 files changed, 247 insertions(+), 697 deletions(-) delete mode 100644 src/util/virfirewallpriv.h -- 2.33.1

2 13

[PATCH] Fixes: #253
by JorhsonDeng 13 Dec '21

13 Dec '21

To resolve the bug: #253. The restore method should call the qemuProcessRefreshState method to refreash the state of the devices. --- src/qemu/qemu_process.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 6b83a571b9..ebd60a7b84 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -7703,14 +7703,11 @@ qemuProcessStart(virConnectPtr conn, if (incoming->deferredURI && qemuMigrationDstRun(driver, vm, incoming->deferredURI, asyncJob) < 0) goto stop; - } else { - /* Refresh state of devices from QEMU. During migration this happens - * in qemuMigrationDstFinish to ensure that state information is fully - * transferred. */ - if (qemuProcessRefreshState(driver, vm, asyncJob) < 0) - goto stop; } + if (qemuProcessRefreshState(driver, vm, asyncJob) < 0) + goto stop; + if (qemuProcessFinishStartup(driver, vm, asyncJob, !(flags & VIR_QEMU_PROCESS_START_PAUSED), incoming ? -- 2.27.0

3 2

[libvirt PATCH 0/6] Improve usage information and manual pages
by Andrea Bolognani 10 Dec '21

10 Dec '21

Each of the tools we ship as part of libvirt should display reasonable usage information when called without arguments and have an associated manual page. Andrea Bolognani (6): virt-ssh-helper: Don't use optind virt-ssh-helper: Improve usage information virt-ssh-helper: Add manual page virt-qemu-run: Improve usage information virt-qemu-run: Improve manual page virt-pki-query-dn: Add manual page docs/manpages/meson.build | 2 + docs/manpages/virt-pki-query-dn.rst | 91 +++++++++++++++++++++++++++ docs/manpages/virt-qemu-run.rst | 53 +++++++++------- docs/manpages/virt-ssh-helper.rst | 96 +++++++++++++++++++++++++++++ libvirt.spec.in | 2 + src/qemu/qemu_shim.c | 2 +- src/remote/remote_ssh_helper.c | 18 +++--- 7 files changed, 234 insertions(+), 30 deletions(-) create mode 100644 docs/manpages/virt-pki-query-dn.rst create mode 100644 docs/manpages/virt-ssh-helper.rst -- 2.31.1

2 15

[libvirt PATCH v2 00/17] cgroup and thread management in ch driver
by Praveen K Paladugu 10 Dec '21

10 Dec '21

I addressed all the open comments on the v1 patch set. In places where indentaion was hard to manage, I created explicit commits to apply correct indentation on files. Compared to v1, this version applies 2 code refactors: 1) some cgroup methods in src/qemu/qemu_cgroup.c were moved to src/hypervisor and shared with ch driver. 2) virProcessGetStatInfo, virProcessGetSchedInfo which were copied from qemu driver are moved to util and shared between qemu and ch drviers. Praveen K Paladugu (9): util: fix indentation in virprocess.c util: Helper functions to get process info ch_domain: fix indentation in ch_domain ch_driver: fix indentation in ch_driver ch_driver,ch_domain: vcpu info getter callbacks ch_monitor: fix indentation in ch_monitor.c qemu,hypervisor: refactor some cgroup mgmt methods ch_process: Setup emulator and iothread settings ch_driver: emulator threadinfo & pinning callbacks Vineeth Pillai (8): ch_domain: add virCHDomainGetMonitor helper method ch_domain: add methods to manage private vcpu data ch_driver: domainGetVcpuPinInfo and nodeGetCPUMap ch_monitor: Get nicindexes in prep for cgroup mgmt ch: methods for cgroup mgmt in ch driver ch_driver,ch_domain: vcpupin callback in ch driver ch_driver: enable typed param string for numatune ch_driver: add numatune callbacks for CH driver src/ch/ch_conf.c | 2 + src/ch/ch_conf.h | 6 +- src/ch/ch_domain.c | 302 ++++++--- src/ch/ch_domain.h | 32 +- src/ch/ch_driver.c | 1045 ++++++++++++++++++++++++++++---- src/ch/ch_monitor.c | 533 +++++++++++----- src/ch/ch_monitor.h | 60 +- src/ch/ch_process.c | 386 +++++++++++- src/ch/ch_process.h | 3 + src/ch/meson.build | 1 + src/hypervisor/domain_cgroup.c | 426 ++++++++++++- src/hypervisor/domain_cgroup.h | 52 ++ src/libvirt_private.syms | 12 + src/qemu/qemu_cgroup.c | 410 +------------ src/qemu/qemu_cgroup.h | 11 - src/qemu/qemu_driver.c | 130 +--- src/qemu/qemu_hotplug.c | 4 +- src/qemu/qemu_process.c | 17 +- src/util/virprocess.c | 609 +++++++++++-------- src/util/virprocess.h | 5 + 20 files changed, 2905 insertions(+), 1141 deletions(-) -- 2.27.0

2 28

[libvirt PATCH v3] qemu: Enable unprivileged userfaultfd for post-copy migration
by Jiri Denemark 10 Dec '21

10 Dec '21

Userfaultfd is by default allowed only for privileged processes. Since libvirt runs QEMU unprivileged, we need to enable unprivileged access to userfaultfd to enable post-copy migration. https://bugzilla.redhat.com/show_bug.cgi?id=1945420 Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- Notes: If you wonder about the 60-* prefix of the installed sysctl file, we already install 60-libvirtd.conf so I think it makes sense to use the same prefix for all sysctl configuration installed by libvirt. Version 2: - setting unprivileged_userfaultfd only when it is not already enabled - virReportSystemError replaced with VIR_WARN Version 3: - set the knob via a sysctl conf file instead libvirt.spec.in | 1 + src/qemu/meson.build | 8 ++++++++ src/qemu/postcopy-migration.sysctl | 6 ++++++ 3 files changed, 15 insertions(+) create mode 100644 src/qemu/postcopy-migration.sysctl diff --git a/libvirt.spec.in b/libvirt.spec.in index 5a079cdaf3..3aa4cc75b5 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -1859,6 +1859,7 @@ exit 0 %files daemon-driver-qemu %config(noreplace) %{_sysconfdir}/sysconfig/virtqemud %config(noreplace) %{_sysconfdir}/libvirt/virtqemud.conf +%config(noreplace) %{_prefix}/lib/sysctl.d/60-qemu-postcopy-migration.conf %{_datadir}/augeas/lenses/virtqemud.aug %{_datadir}/augeas/lenses/tests/test_virtqemud.aug %{_unitdir}/virtqemud.service diff --git a/src/qemu/meson.build b/src/qemu/meson.build index 3898d23877..66ef594006 100644 --- a/src/qemu/meson.build +++ b/src/qemu/meson.build @@ -170,6 +170,14 @@ if conf.has('WITH_QEMU') 'file': files('virtqemud.sysconf'), } + if conf.has('WITH_SYSCTL') + install_data( + 'postcopy-migration.sysctl', + install_dir: prefix / 'lib' / 'sysctl.d', + rename: [ '60-qemu-postcopy-migration.conf' ], + ) + endif + virt_install_dirs += [ localstatedir / 'lib' / 'libvirt' / 'qemu', runstatedir / 'libvirt' / 'qemu', diff --git a/src/qemu/postcopy-migration.sysctl b/src/qemu/postcopy-migration.sysctl new file mode 100644 index 0000000000..aa8f015ae0 --- /dev/null +++ b/src/qemu/postcopy-migration.sysctl @@ -0,0 +1,6 @@ +# This is needed to support post-copy migration for QEMU run by libvirt, +# i.e., unprivileged, as userfaultfd is by default only available to +# privileged processes. +# It can be safely overridden by a file in /etc/sysctl.d/ in case post-copy +# migration is not used on the host. +vm.unprivileged_userfaultfd = 1 -- 2.34.1

2 1

[libvirt PATCH] remote: Avoid crash in remoteSplitURIScheme()
by Andrea Bolognani 10 Dec '21

10 Dec '21

We need to make sure the URI scheme is present before passing it to strchr(), otherwise we're going to get $ virt-ssh-helper foo Segmentation fault (core dumped) Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- src/remote/remote_sockets.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/src/remote/remote_sockets.c b/src/remote/remote_sockets.c index 2979576680..c315b24d30 100644 --- a/src/remote/remote_sockets.c +++ b/src/remote/remote_sockets.c @@ -69,7 +69,15 @@ remoteSplitURIScheme(virURI *uri, char **driver, remoteDriverTransport *transport) { - char *p = strchr(uri->scheme, '+'); + char *p = NULL; + + if (!uri->scheme) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("missing scheme for URI")); + return -1; + } + + p = strchr(uri->scheme, '+'); if (p) *driver = g_strndup(uri->scheme, p - uri->scheme); -- 2.31.1

3 5

[PATCH] Revert "lxc: controller: Fix container launch on cgroup v1"
by Michal Privoznik 10 Dec '21

10 Dec '21

Unfortunately, this fix breakes machinectl in a very nasty way, for instance 'machinectl shell' drops into the host shell. It's worse than being unable to start a container with CGroupsV1. Revert until a proper fix is figured out. This reverts commit 1b9ce05ce241a581d4e80228c92ceb0266f21f94. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/lxc/lxc_controller.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/lxc/lxc_controller.c b/src/lxc/lxc_controller.c index 039efcd7c7..7c593ee3a9 100644 --- a/src/lxc/lxc_controller.c +++ b/src/lxc/lxc_controller.c @@ -865,12 +865,12 @@ static int virLXCControllerSetupCgroupLimits(virLXCController *ctrl) nodeset = virDomainNumatuneGetNodeset(ctrl->def->numa, auto_nodeset, -1); if (!(ctrl->cgroup = virLXCCgroupCreate(ctrl->def, - getpid(), + ctrl->initpid, ctrl->nnicindexes, ctrl->nicindexes))) goto cleanup; - if (virCgroupAddMachineProcess(ctrl->cgroup, ctrl->initpid) < 0) + if (virCgroupAddMachineProcess(ctrl->cgroup, getpid()) < 0) goto cleanup; /* Add all qemu-nbd tasks to the cgroup */ -- 2.32.0

2 1

Issue labe improvements
by Peter Krempa 10 Dec '21

10 Dec '21

Hi, from time to time when I try to go through upstream issues I always feel that the labels we have are suboptimal and don't always allow to track the current state of the issue. Recently I've had a look at the qemu issues and found what I was lacking. Specifically I'm lacking the 'workflow' class of labels they use. I propose we adopt the following changes: 1) Convert the existing 'bug', 'enhancement', 'support', and 'discussion' labels into a set of scoped labels (again inspiration taken from qemu): kind::bug kind::enhancement kind::support kind::discussion kind::documentation This is mostly as the above kinds are mutually exclusive. 2) Introduce the workflow label similarly to what qemu uses: workflow::Confirmed/Triaged (<- confirmed for bugs, triaged for enhancements) workflow::Needs Info (replaces "needinfo") workflow::In progress (replaces "Doing") I'd also potentially like to have a 'Unconfirmed' state for when the bug has enough info, but it's unknown why it's happening. I want to specifically avoid the ambiguous "Triaged" when used on it's own. 3) Convert host-* labels into a scoped label. Hosts are usually mutually exclusive 4) Convert driver-* into scoped labels. Usually issues are not exceeding these boundaries 5) Remove the following unused or ambiguous labels: - critical - incident - Doing - To Do - gsoc::20* (all seem to be unused) I'm willing to go ahead and re-triage open stuff if nobody objects to the above changes.

4 3

[PATCH 0/2] remote_daemon: Set shutdown callbacks only after init is done
by Michal Privoznik 10 Dec '21

10 Dec '21

*** BLURB HERE *** Michal Prívozník (2): remote_daemon: Set shutdown callbacks only after init is done Revert "qemu: Avoid crash in qemuStateShutdownPrepare() and qemuStateShutdownWait()" src/qemu/qemu_driver.c | 6 ------ src/remote/remote_daemon.c | 7 ++++--- 2 files changed, 4 insertions(+), 9 deletions(-) -- 2.32.0

2 4