June 2022 - Devel - libvirt List Archives

[PATCH v2 0/9] qemu: Enable SCHED_CORE for domains and helper processes
by Michal Privoznik 10 Aug '22

10 Aug '22

v2 of: https://listman.redhat.com/archives/libvir-list/2022-May/230902.html While technically, the original series was "just" and RFC, it got proper review and thus I'm marking this as v2. What's still missing? ===================== * Per Dario's suggestion, we should take vcpu pinning (and possibly emulator pinning as well) into account, so that the guest doesn't have to do SCHED_CORE inside it. * per-VM knob to enable/disable SCHED_CORE. But as discussed under my RFC patches, it's probably not needed, at least for now. It can always be added later, should somebody need it. Diff to v1: =========== * Turned qemu.conf knob into an enum rather than boolean, so that users can chose whether helper processes are placed into the emulator group too * Cleaned up virCommand* code; per Dan's suggestion use one variable to differentiate runAlone/runAmong states, instead of two. Michal Prívozník (9): qemu_dbus: Separate PID read code into qemuDBusGetPID qemu_vhost_user_gpu: Export qemuVhostUserGPUGetPid() qemu_tpm: Expose qemuTPMEmulatorGetPid() qemu_virtiofs: Separate PID read code into qemuVirtioFSGetPid virprocess: Core Scheduling support virCommand: Introduce APIs for core scheduling qemu_conf: Introduce a knob to set SCHED_CORE qemu: Enable SCHED_CORE for domains and helper processes qemu: Place helper processes into the same trusted group src/libvirt_private.syms | 6 ++ src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf.in | 14 ++++ src/qemu/qemu_conf.c | 42 ++++++++++ src/qemu/qemu_conf.h | 11 +++ src/qemu/qemu_dbus.c | 42 +++++++--- src/qemu/qemu_dbus.h | 4 + src/qemu/qemu_extdevice.c | 120 ++++++++++++++++++++++++++++ src/qemu/qemu_extdevice.h | 3 + src/qemu/qemu_process.c | 9 +++ src/qemu/qemu_security.c | 4 + src/qemu/qemu_tpm.c | 2 +- src/qemu/qemu_tpm.h | 7 ++ src/qemu/qemu_vhost_user_gpu.c | 2 +- src/qemu/qemu_vhost_user_gpu.h | 8 ++ src/qemu/qemu_virtiofs.c | 41 +++++++--- src/qemu/qemu_virtiofs.h | 5 ++ src/qemu/test_libvirtd_qemu.aug.in | 1 + src/util/vircommand.c | 63 +++++++++++++++ src/util/vircommand.h | 5 ++ src/util/virprocess.c | 124 +++++++++++++++++++++++++++++ src/util/virprocess.h | 8 ++ 22 files changed, 494 insertions(+), 28 deletions(-) -- 2.35.1

4 17

[libvirt] Semantic change in v4.3 slows down image creation and snapshots (on ZFS)
by Christian Ehrhardt 31 Jul '22

31 Jul '22

Hi, when discussing bug [1] triggered by [2] Daniel and I have found that the change [3] which is in since v4.3 was a semantic change to the behavior of libvirt. virt-manager used and still does issue a volume XML like: <volume> <name>disk.qcow2</name> <capacity>16106127360</capacity> <allocation>16106127360</allocation> <target> <format type="qcow2"/> <features> <lazy_refcounts/> </features> </target> </volume> Up until the commit above that was using preallocation=metadata, but now uses preallocation=falloc. In many cases (read: Fileystems) that won't be very user-visible, but if you are running on ZFS (or anything else that can't fallocate well) this is made worse by [4] slowing down these operations a lot. After the POV for virt-manager seems to have settled on the bug [1] referenced the remaining question is if libvirt should revert/modify the change [3] - and that we agreed required a wider discussion to be had here, hence this mail. [1]: https://bugzilla.redhat.com/show_bug.cgi?id=1759454 [2]: https://bugs.launchpad.net/virt-manager/+bug/1847105 [3]: https://libvirt.org/git/?p=libvirt.git;a=commit;h=c9ec7088c7a3f4cd26bb471f1… [4]: https://github.com/zfsonlinux/zfs/issues/326 -- Christian Ehrhardt Software Engineer, Ubuntu Server Canonical Ltd

4 3

[libvirt][PATCH v11 0/4] Support query and use SGX
by Lin Yang 25 Jul '22

25 Jul '22

This patch series provides support for enabling Intel's Software Guard Extensions (SGX) feature in guest VM. Giving the SGX support in QEMU had been merged. Intel SGX is a set of instructions that increases the security of application code and data, giving them more protection from disclosure or modification. Developers can partition sensitive information into enclaves, which are areas of execution in memory with more security protection. It depends on QEMU fixing[1], which will move cpu QOM object from /machine/unattached/device[nn] to /machine/cpu[nn]. It requires libvirt to change the default cpu QOM object location once QEMU patch gets accepted, but it is out of this SGX patch scope. The typical flow looks below at very high level: 1. Calls virConnectGetDomainCapabilities API to domain capabilities that includes the following SGX information. <feature> ... <sgx supported='yes'> <epc_size unit='KiB'>N</epc_size> </sgx> ... </feature> 2. User requests to start a guest calling virCreateXML() with SGX requirement. It does not support NUMA yet, since latest QEMU 6.2 release does not support NUMA. It should contain <devices> ... <memory model='sgx-epc'> <target> <size unit='KiB'>N</size> </target> </memory> ... </devices> Please note that SGX NUMA support will be implemented in future patches. [1] https://lists.nongnu.org/archive/html/qemu-devel/2022-01/msg03534.html Haibin Huang (2): qemu: provide support to query the SGX capability conf: expose SGX feature in domain capabilities Lin Yang (2): conf: Introduce SGX EPC element into device memory xml qemu: Add command-line to generate SGX EPC memory backend docs/formatdomain.rst | 9 +- docs/formatdomaincaps.rst | 26 ++++ src/conf/domain_capabilities.c | 29 ++++ src/conf/domain_capabilities.h | 13 ++ src/conf/domain_conf.c | 6 + src/conf/domain_conf.h | 1 + src/conf/domain_validate.c | 16 ++ src/conf/schemas/domaincaps.rng | 22 ++- src/conf/schemas/domaincommon.rng | 1 + src/libvirt_private.syms | 1 + src/qemu/qemu_alias.c | 6 +- src/qemu/qemu_capabilities.c | 143 ++++++++++++++++++ src/qemu/qemu_capabilities.h | 6 + src/qemu/qemu_capspriv.h | 4 + src/qemu/qemu_command.c | 54 ++++++- src/qemu/qemu_domain.c | 38 +++-- src/qemu/qemu_domain_address.c | 6 + src/qemu/qemu_driver.c | 1 + src/qemu/qemu_monitor.c | 10 ++ src/qemu/qemu_monitor.h | 3 + src/qemu/qemu_monitor_json.c | 104 ++++++++++++- src/qemu/qemu_monitor_json.h | 9 ++ src/qemu/qemu_process.c | 2 + src/qemu/qemu_validate.c | 8 + src/security/security_apparmor.c | 1 + src/security/security_dac.c | 2 + src/security/security_selinux.c | 2 + tests/domaincapsdata/bhyve_basic.x86_64.xml | 1 + tests/domaincapsdata/bhyve_fbuf.x86_64.xml | 1 + tests/domaincapsdata/bhyve_uefi.x86_64.xml | 1 + tests/domaincapsdata/empty.xml | 1 + tests/domaincapsdata/libxl-xenfv.xml | 1 + tests/domaincapsdata/libxl-xenpv.xml | 1 + .../domaincapsdata/qemu_2.11.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.11.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.11.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.11.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.12.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.12.0-tcg.x86_64.xml | 1 + .../qemu_2.12.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.12.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.12.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_2.12.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.12.0.x86_64.xml | 1 + .../domaincapsdata/qemu_3.0.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_3.0.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_3.0.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_3.0.0.s390x.xml | 1 + tests/domaincapsdata/qemu_3.0.0.x86_64.xml | 1 + .../domaincapsdata/qemu_3.1.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_3.1.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_3.1.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_3.1.0.x86_64.xml | 1 + .../domaincapsdata/qemu_4.0.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_4.0.0-tcg.x86_64.xml | 1 + .../qemu_4.0.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_4.0.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_4.0.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_4.0.0.s390x.xml | 1 + tests/domaincapsdata/qemu_4.0.0.x86_64.xml | 1 + .../domaincapsdata/qemu_4.1.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_4.1.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_4.1.0.x86_64.xml | 1 + .../domaincapsdata/qemu_4.2.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_4.2.0-tcg.x86_64.xml | 1 + .../qemu_4.2.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_4.2.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_4.2.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_4.2.0.s390x.xml | 1 + tests/domaincapsdata/qemu_4.2.0.x86_64.xml | 1 + .../domaincapsdata/qemu_5.0.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_5.0.0-tcg.x86_64.xml | 1 + .../qemu_5.0.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_5.0.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_5.0.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_5.0.0.x86_64.xml | 1 + .../domaincapsdata/qemu_5.1.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_5.1.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_5.1.0.sparc.xml | 1 + tests/domaincapsdata/qemu_5.1.0.x86_64.xml | 1 + .../domaincapsdata/qemu_5.2.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_5.2.0-tcg.x86_64.xml | 1 + .../qemu_5.2.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_5.2.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_5.2.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_5.2.0.s390x.xml | 1 + tests/domaincapsdata/qemu_5.2.0.x86_64.xml | 1 + .../domaincapsdata/qemu_6.0.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_6.0.0-tcg.x86_64.xml | 1 + .../qemu_6.0.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_6.0.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_6.0.0.s390x.xml | 1 + tests/domaincapsdata/qemu_6.0.0.x86_64.xml | 1 + .../domaincapsdata/qemu_6.1.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_6.1.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_6.1.0.x86_64.xml | 1 + .../domaincapsdata/qemu_6.2.0-q35.x86_64.xml | 4 + .../domaincapsdata/qemu_6.2.0-tcg.x86_64.xml | 4 + .../qemu_6.2.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_6.2.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_6.2.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_6.2.0.x86_64.xml | 4 + .../domaincapsdata/qemu_7.0.0-q35.x86_64.xml | 4 + .../domaincapsdata/qemu_7.0.0-tcg.x86_64.xml | 4 + .../qemu_7.0.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_7.0.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_7.0.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_7.0.0.x86_64.xml | 4 + .../caps_6.2.0.x86_64.replies | 22 ++- .../caps_6.2.0.x86_64.xml | 5 + .../caps_7.0.0.x86_64.replies | 22 ++- .../caps_7.0.0.x86_64.xml | 5 + .../sgx-epc.x86_64-6.2.0.args | 37 +++++ tests/qemuxml2argvdata/sgx-epc.xml | 36 +++++ tests/qemuxml2argvtest.c | 2 + .../sgx-epc.x86_64-latest.xml | 52 +++++++ tests/qemuxml2xmltest.c | 2 + 117 files changed, 769 insertions(+), 36 deletions(-) create mode 100644 tests/qemuxml2argvdata/sgx-epc.x86_64-6.2.0.args create mode 100644 tests/qemuxml2argvdata/sgx-epc.xml create mode 100644 tests/qemuxml2xmloutdata/sgx-epc.x86_64-latest.xml -- 2.25.1

7 17

[PATCH] cmdQemuMonitorCommandQMPWrap: Reset ignored errors from JSON parsing
by Peter Krempa 25 Jul '22

25 Jul '22

'cmdQemuMonitorCommandQMPWrap' is checking whether the user provided string is not valid JSON to avoid wrapping it. In cases where it's not JSON we ignore the error and add the wrapper. If the caller then reports a different non-libvirt error the error from the JSON parsing would be printed as well. Reset errors we ignore: # virsh qemu-monitor-command cd --pass-fds a asdf error: Unable to parse FD number 'a' error: internal error: cannot parse json asdf: lexical error: invalid char in json text. asdf (right here) ------^ In the above case 'asdf' is not valid JSON, but the code did wrap it into '{"execute":"asdf"}', the only problem is the argument for --pass-fds. Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- tools/virsh-domain.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 9b1b14cdc2..743660e794 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -9769,6 +9769,8 @@ cmdQemuMonitorCommandQMPWrap(vshControl *ctl, if (virJSONValueIsObject(fullcmdjson)) return g_steal_pointer(&fullcmd); + vshResetLibvirtError(); + /* we try to wrap the command and possible arguments into a JSON object, if * we as fall back we pass through what we've got from the user */ @@ -9787,6 +9789,8 @@ cmdQemuMonitorCommandQMPWrap(vshControl *ctl, * JSON object wrapper and try using that */ g_auto(virBuffer) buf = VIR_BUFFER_INITIALIZER; + vshResetLibvirtError(); + virBufferAddLit(&buf, "{"); /* opt points to the _ARGV option bit containing the command so we'll * iterate through the arguments now */ -- 2.35.1

2 2

[libvirt PATCH 0/9] [RFC] Dynamic CPU models
by Tim Wiederhake 18 Jul '22

18 Jul '22

libvirt and qemu cpu models are out of sync. libvirt cpu models are considered static and never changing, whereas qemu cpu models have changed over time. This leads to problems and confusion for users, as the cpu models they specify may have different features than what they expect. This series introduces two new APIs to enumerate and retrieve cpu models dynamically from the hypervisor at runtime, allowing the user to make use of cpu models from newer hypervisor versions even if libvirt is not aware of them. These two new functions are intended as building blocks onto which higher-level functionality can be build, e.g. the ability to specify a hypervisor cpu model directly in a domain xml. With these two functions alone this has to be done manually for now, i.e. enumerate the available cpu models, retrieve the cpu description and manually insert it into the domain xml accordingly: $ virsh hypervisor-cpu-models --arch x86_64 Name Alias ---------------------------------------------------- max host base qemu64-v1 qemu64 qemu64-v1 qemu32-v1 qemu32 qemu32-v1 phenom-v1 phenom phenom-v1 (...) $ virsh hypervisor-cpu-definition --machine pc --arch x86_64 qemu64 <cpu> <model>qemu64</model> <feature name='cmov'/> <feature name='mmx'/> <feature name='xd'/> <feature name='x-intel-pt-auto-level'/> <feature name='kvm_asyncpf'/> <feature name='kvm-asyncpf'/> <feature name='legacy-cache'/> <feature name='vmware-cpuid-freq'/> <feature name='mce'/> <feature name='mca'/> <feature name='msr'/> <feature name='fxsr'/> <feature name='cpuid-0xb'/> <feature name='kvm_pv_eoi'/> <feature name='pni'/> <feature name='x2apic'/> <feature name='i64'/> <feature name='pae'/> <feature name='pat'/> <feature name='sse'/> <feature name='kvm_nopiodelay'/> <feature name='kvm-nopiodelay'/> <feature name='kvmclock-stable-bit'/> <feature name='hypervisor'/> <feature name='syscall'/> <feature name='x-migrate-smi-count'/> <feature name='full-cpuid-auto-level'/> <feature name='sse3'/> <feature name='sse2'/> <feature name='kvm-pv-eoi'/> <feature name='cx8'/> <feature name='pge'/> <feature name='fill-mtrr-mask'/> <feature name='cx16'/> <feature name='de'/> <feature name='clflush'/> <feature name='tsc'/> <feature name='fpu'/> <feature name='check'/> <feature name='apic'/> <feature name='kvm-steal-time'/> <feature name='kvm_steal_time'/> <feature name='kvmclock'/> <feature name='l3-cache'/> <feature name='nx'/> <feature name='tcg-cpuid'/> <feature name='lm'/> <feature name='pse'/> <feature name='sep'/> <feature name='kvm'/> <feature name='lahf-lm'/> <feature name='lahf_lm'/> <feature name='mtrr'/> <feature name='pse36'/> </cpu> The returned cpu description is currently completely unfiltered, as can be seen by the duplicate entries differing only in "-" vs. "_" usage, inclusion of experimental feature flags and generally, flags that are not recognized by libvirt. One possibility to adress this would be to extend virQEMUCapsCPUFeatureTranslate. Before I continue with this, I would love to hear other people's thoughts, comments and potential use cases. Cheers, Tim Tim Wiederhake (9): qemuMonitorCPUDefInfo: Add alias libvirt: introduce virConnectGetHypervisorCPUModelNames public API remote: Add RPC support for the virConnectGetHypervisorCPUModelNames API qemu: implement virConnectGetHypervisorCPUModelNames API tools: Report hypervisor cpu model names libvirt: introduce virConnectGetHypervisorCPUModelDefinition public API remote: Add support for the virConnectGetHypervisorCPUModelDefinition API qemu: Implement virConnectGetHypervisorCPUModelDefinition API tools: Report hypervisor cpu model definitions docs/manpages/virsh.rst | 27 ++++++ include/libvirt/libvirt-host.h | 13 +++ src/driver-hypervisor.h | 17 ++++ src/libvirt-host.c | 109 +++++++++++++++++++++++ src/libvirt_public.syms | 2 + src/qemu/qemu_driver.c | 105 ++++++++++++++++++++++ src/qemu/qemu_monitor.c | 2 + src/qemu/qemu_monitor.h | 1 + src/qemu/qemu_monitor_json.c | 3 + src/remote/remote_daemon_dispatch.c | 76 ++++++++++++++++ src/remote/remote_driver.c | 100 +++++++++++++++++++++ src/remote/remote_protocol.x | 37 +++++++- src/remote_protocol-structs | 27 ++++++ tools/virsh-host.c | 132 ++++++++++++++++++++++++++++ 14 files changed, 650 insertions(+), 1 deletion(-) -- 2.31.1

2 22

[PATCH] qemu_monitor_json: Implement logic for setting iothread.thread-pool-{min, max}
by Michal Privoznik 07 Jul '22

07 Jul '22

When virDomainSetIOThreadParams() API is called, well its QEMU impl: qemuDomainSetIOThreadParams() then typed params are parsed by qemuDomainIOThreadParseParams() into this qemuMonitorIOThreadInfo struct. In the struct we have a <int, bool> pair for every IOThread attribute we can tune through monitor. The struct is then passed to qemuMonitorJSONSetIOThread() which looks at the bool and if set then the corresponding attribute is set to given value. Each attribute is thus changed in a separate call. While this works for attributes independent of each other ("poll-max-ns", "poll-grow", "poll-shrink"), it does not always work for the other attributes ("thread-pool-min" and "thread-pool-max"). The limitation here is that the lower boundary (minimum) has to be lower (or equal to) the upper boundary (maximum) at all times. This means, that in some cases we might need to set attributes in reversed order to meet the constraint. Resolves: https://gitlab.com/libvirt/libvirt/-/issues/339 Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/qemu/qemu_monitor_json.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c index 3aad2ab212..80696de731 100644 --- a/src/qemu/qemu_monitor_json.c +++ b/src/qemu/qemu_monitor_json.c @@ -7428,6 +7428,7 @@ qemuMonitorJSONSetIOThread(qemuMonitor *mon, { g_autofree char *path = NULL; qemuMonitorJSONObjectProperty prop; + bool setMaxFirst = false; path = g_strdup_printf("/objects/iothread%u", iothreadInfo->iothread_id); @@ -7443,8 +7444,32 @@ qemuMonitorJSONSetIOThread(qemuMonitor *mon, VIR_IOTHREAD_SET_PROP("poll-max-ns", poll_max_ns); VIR_IOTHREAD_SET_PROP("poll-grow", poll_grow); VIR_IOTHREAD_SET_PROP("poll-shrink", poll_shrink); - VIR_IOTHREAD_SET_PROP("thread-pool-min", thread_pool_min); - VIR_IOTHREAD_SET_PROP("thread-pool-max", thread_pool_max); + + if (iothreadInfo->set_thread_pool_min && + iothreadInfo->set_thread_pool_max) { + int curr_max = -1; + + /* By default, the minimum is set first, followed by the maximum. But + * if the current maximum is below the minimum we want to set we need + * to set the maximum first. Otherwise would get an error because we + * would be attempting to shift minimum above maximum. */ + prop.type = QEMU_MONITOR_OBJECT_PROPERTY_INT; + if (qemuMonitorJSONGetObjectProperty(mon, path, + "thread-pool-max", &prop) < 0) + return -1; + curr_max = prop.val.iv; + + if (curr_max < iothreadInfo->thread_pool_min) + setMaxFirst = true; + } + + if (setMaxFirst) { + VIR_IOTHREAD_SET_PROP("thread-pool-max", thread_pool_max); + VIR_IOTHREAD_SET_PROP("thread-pool-min", thread_pool_min); + } else { + VIR_IOTHREAD_SET_PROP("thread-pool-min", thread_pool_min); + VIR_IOTHREAD_SET_PROP("thread-pool-max", thread_pool_max); + } #undef VIR_IOTHREAD_SET_PROP -- 2.35.1

2 1

[libvirt PATCH v2] tools: add virt-qmp-proxy for proxying QMP clients to libvirt QEMU guests
by Daniel P. Berrangé 01 Jul '22

01 Jul '22

Libvirt provides QMP passthrough APIs for the QEMU driver and these are exposed in virsh. It is not especially pleasant, however, using the raw QMP JSON syntax. QEMU has a tool 'qmp-shell' which can speak QMP and exposes a human friendly interactive shell. It is not possible to use this with libvirt managed guest, however, since only one client can attach to he QMP socket at any point in time. The virt-qmp-proxy tool aims to solve this problem. It opens a UNIX socket and listens for incoming client connections, speaking QMP on the connected socket. It will forward any QMP commands received onto the running libvirt QEMU guest, and forward any replies back to the QMP client. $ virsh start demo $ virt-qmp-proxy demo demo.qmp & $ qmp-shell demo.qmp Welcome to the QMP low-level shell! Connected to QEMU 6.2.0 (QEMU) query-kvm { "return": { "enabled": true, "present": true } } Note this tool of course has the same risks as the raw libvirt QMP passthrough. It is safe to run query commands to fetch information but commands which change the QEMU state risk disrupting libvirt's management of QEMU, potentially resulting in data loss/corruption in the worst case. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- Changed in v2: - Rewrote to not be such a gross hack, specifically - Wired up usage of libvirt event loop for sock I/O - Register with libvirt for QMP events - Incrementally read from socket & try json parsing until we get a full command, instead of assuming a full command in one read - Forwarding of passed FDs in both directions (libvirt -> client untested, since AFAIK, no QMP cmd returns FDs currently) Other thought.... This patch is against libvirt.git but has a dependancy on the libvirt-python.git APIs. If we put this in libvirt-client RPM then we get a new dep on python. Perhaps better to have this live in libvirt-python.git/examples, though I would like it present as a standard tool ? Another option is to bundle with virt-install which is a python app commonly present on virt hosts ? docs/manpages/meson.build | 1 + docs/manpages/virt-qmp-proxy.rst | 120 +++++++++++ tools/meson.build | 5 + tools/virt-qmp-proxy | 360 +++++++++++++++++++++++++++++++ 4 files changed, 486 insertions(+) create mode 100644 docs/manpages/virt-qmp-proxy.rst create mode 100755 tools/virt-qmp-proxy diff --git a/docs/manpages/meson.build b/docs/manpages/meson.build index ba673cf472..4162a9969a 100644 --- a/docs/manpages/meson.build +++ b/docs/manpages/meson.build @@ -18,6 +18,7 @@ docs_man_files = [ { 'name': 'virt-pki-query-dn', 'section': '1', 'install': true }, { 'name': 'virt-pki-validate', 'section': '1', 'install': true }, { 'name': 'virt-qemu-run', 'section': '1', 'install': conf.has('WITH_QEMU') }, + { 'name': 'virt-qmp-proxy', 'section': '1', 'install': conf.has('WITH_QEMU') }, { 'name': 'virt-xml-validate', 'section': '1', 'install': true }, { 'name': 'libvirt-guests', 'section': '8', 'install': conf.has('WITH_LIBVIRTD') }, diff --git a/docs/manpages/virt-qmp-proxy.rst b/docs/manpages/virt-qmp-proxy.rst new file mode 100644 index 0000000000..30b9f6699d --- /dev/null +++ b/docs/manpages/virt-qmp-proxy.rst @@ -0,0 +1,120 @@ +============== +virt-qmp-proxy +============== + +-------------------------------------------------- +Expose a QMP proxy server for a libvirt QEMU guest +-------------------------------------------------- + +:Manual section: 1 +:Manual group: Virtualization Support + +.. contents:: + + +SYNOPSIS +======== + +``virt-qmp-proxy`` [*OPTION*]... *DOMAIN* *QMP-SOCKET-PATH* + + +DESCRIPTION +=========== + +This tool provides a way to expose a QMP proxy server that communicates +with a QEMU guest managed by libvirt. This enables standard QMP client +tools to interact with libvirt managed guests. + +**NOTE: use of this tool will result in the running QEMU guest being +marked as tainted.** It is strongly recommended that this tool *only be +used to send commands which query information* about the running guest. +If this tool is used to make changes to the state of the guest, this +may have negative interactions with the QEMU driver, resulting in an +inability to manage the guest operation thereafter, and in the worst +case **potentially lead to data loss or corruption**. + +The ``virt-qmp-proxy`` program will listen on a UNIX socket for incoming +client connections, and run the QMP protocol over the connection. Any +commands received will be sent to the running libvirt guest, and replies +sent back. + +The ``virt-qemu-proxy`` program may be interrupted (eg Ctrl-C) when it +is no longer required. The libvirt QEMU guest will continue running. + + +OPTIONS +======= + +*DOMAIN* + +The ID or UUID or Name of the libvirt QEMU guest. + +*QMP-SOCKET-PATH* + +The filesystem path at which to run the QMP server, listening for +incoming connections. + +``-c`` *CONNECTION-URI* +``--connect``\ =\ *CONNECTION-URI* + +The URI for the connection to the libvirt QEMU driver. If omitted, +a URI will be auto-detected. + +``-v``, ``--verbose`` + +Run in verbose mode, printing all QMP commands and replies that +are handled. + +``-h``, ``--help`` + +Display the command line help. + + +EXIT STATUS +=========== + +Upon successful shutdown, an exit status of 0 will be set. Upon +failure a non-zero status will be set. + + +AUTHOR +====== + +Daniel P. Berrangé + + +BUGS +==== + +Please report all bugs you discover. This should be done via either: + +#. the mailing list + + `https://libvirt.org/contact.html <https://libvirt.org/contact.html>`_ + +#. the bug tracker + + `https://libvirt.org/bugs.html <https://libvirt.org/bugs.html>`_ + +Alternatively, you may report bugs to your software distributor / vendor. + +COPYRIGHT +========= + +Copyright (C) 2022 by Red Hat, Inc. + + +LICENSE +======= + +``virt-qemu-proxy`` is distributed under the terms of the GNU LGPL v2+. +This is free software; see the source for copying conditions. There +is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR +PURPOSE + + +SEE ALSO +======== + +virsh(1), `https://libvirt.org/ <https://libvirt.org/>`_, +`QMP reference <https://www.qemu.org/docs/master/interop/qemu-qmp-ref.html>`_ diff --git a/tools/meson.build b/tools/meson.build index bb28a904dc..4e959ecf0b 100644 --- a/tools/meson.build +++ b/tools/meson.build @@ -320,6 +320,11 @@ if conf.has('WITH_LIBVIRTD') endif endif +if conf.has('WITH_QEMU') + install_data('virt-qmp-proxy', + install_dir: bindir) +endif + if bash_completion_dep.found() subdir('bash-completion') endif diff --git a/tools/virt-qmp-proxy b/tools/virt-qmp-proxy new file mode 100755 index 0000000000..d85342bd2b --- /dev/null +++ b/tools/virt-qmp-proxy @@ -0,0 +1,360 @@ +#!/usr/bin/env python3 + +import argparse +import array +import libvirt +import libvirt_qemu +import os +import re +import socket +import sys +import traceback +import json +import fcntl + + +debug = False + +def get_domain(uri, domstr): + conn = libvirt.open(uri) + + dom = None + saveex = None + + def try_lookup(cb): + try: + return cb() + except libvirt.libvirtError as ex: + nonlocal saveex + if saveex is None: + saveex = ex + + if re.match(r'^\d+$', domstr): + dom = try_lookup(lambda: conn.lookupByID(int(domstr))) + + if dom is None and re.match(r'^[-a-f0-9]{36}|[a-f0-9]{32}$', domstr): + dom = try_lookup(lambda: conn.lookupByUUIDString(domstr)) + + if dom is None: + dom = try_lookup(lambda: conn.lookupByName(domstr)) + + if dom is None: + raise saveex + + if not dom.isActive(): + raise Exception("Domain must be running to use QMP") + + return conn, dom + + +class QMPProxy(object): + + def __init__(self, conn, dom, serversock, verbose): + self.conn = conn + self.dom = dom + self.verbose = verbose + + self.serversock = serversock + self.serverwatch = 0 + + self.clientsock = None + self.clientwatch = 0 + + self.api2sock = bytes([]) + self.api2sockfds = [] + + self.sock2api = bytes([]) + self.sock2apifds = [] + + self.serverwatch = libvirt.virEventAddHandle( + self.serversock.fileno(), libvirt.VIR_EVENT_HANDLE_READABLE, + self.handle_server_io, self) + + libvirt_qemu.qemuMonitorEventRegister(self.conn, self.dom, + None, + self.handle_qmp_event, + self) + + + @staticmethod + def handle_qmp_event(conn, dom, event, secs, usecs, details, self): + evdoc = { + "event": event, + "timestamp": {"seconds": secs, "microseconds": usecs } + } + if details is not None: + evdoc["data"] = details + + ev = json.dumps(evdoc) + if self.verbose: + print(ev) + ev += "\r\n" + self.api2sock += ev.encode("utf-8") + self.update_client_events() + + + def recv_with_fds(self): + # Match VIR_NET_MESSAGE_NUM_FDS_MAX in virnetprotocol.x + maxfds = 32 + fds = array.array('i') + cmsgdatalen = socket.CMSG_LEN(maxfds * fds.itemsize) + + data, cmsgdata, flags, addr = self.clientsock.recvmsg(1024, + cmsgdatalen) + for cmsg_level, cmsg_type, cmsg_data in cmsgdata: + if (cmsg_level == socket.SOL_SOCKET and + cmsg_type == socket.SCM_RIGHTS): + fds.frombytes(cmsg_data[:len(cmsg_data) - + (len(cmsg_data) % fds.itemsize)]) + else: + raise Exception("Unexpected CMSGDATA level %d type %d" % ( + cmsg_level, cmsg_type)) + + return data, [self.make_file(fd) for fd in fds] + + + def send_with_fds(self, data, fds): + cfds = [fd.fileno() for fd in fds] + + cmsgdata = [(socket.SOL_SOCKET, socket.SCM_RIGHTS, + array.array("i", cfds))] + + return self.clientsock.sendmsg([data], cmsgdata) + + + @staticmethod + def make_file(fd): + flags = fcntl.fcntl(fd, fcntl.F_GETFL) + + mask = os.O_RDONLY | os.O_WRONLY | os.O_RDWR | os.O_APPEND + flags = flags & mask + mode = "" + if flags == os.O_RDONLY: + mode = "rb" + elif flags == os.O_WRONLY: + mode = "wb" + elif flags == os.O_RDWR: + mode = "r+b" + elif flags == (os.O_WRONLY | os.O_APPEND): + mode = "ab" + elif flags == (os.O_RDWR | os.O_APPEND): + mode = "a+b" + + return os.fdopen(fd, mode) + + + def add_client(self, sock): + ver = self.conn.getVersion() + major = int(ver / 1000000) % 1000 + minor = int(ver / 1000) % 1000 + micro = ver % 1000 + + greetingobj = { + "QMP": { + "version": { + "qemu": { + "major": major, + "minor": minor, + "micro": micro, + }, + "package": f"qemu-{major}.{minor}.{micro}", + }, + "capabilities": [ + "oob" + ], + } + } + greeting = json.dumps(greetingobj) + if self.verbose: + print(greeting) + greeting += "\r\n" + + self.clientsock = sock + self.clientwatch = libvirt.virEventAddHandle( + self.clientsock.fileno(), libvirt.VIR_EVENT_HANDLE_WRITABLE, + self.handle_client_io, self) + self.api2sock += greeting.encode("utf-8") + self.update_server_events() + + + def remove_client(self): + libvirt.virEventRemoveHandle(self.clientwatch) + self.clientsock.close() + self.clientsock = None + self.clientwatch = 0 + self.update_server_events() + + self.api2sock = bytes([]) + self.api2sockfds = [] + + self.sock2api = bytes([]) + self.sock2apifds = [] + + + def update_client_events(self): + # For simplicity of tracking distinct QMP cmds and their passed FDs + # we don't try to support "pipelining", only a single cmd may be + # inflight + if len(self.api2sock) > 0: + events = libvirt.VIR_EVENT_HANDLE_WRITABLE + else: + events = libvirt.VIR_EVENT_HANDLE_READABLE + + libvirt.virEventUpdateHandle(self.clientwatch, events) + + + def update_server_events(self): + if self.clientsock is not None: + libvirt.virEventUpdateHandle(self.serverwatch, 0) + else: + libvirt.virEventUpdateHandle(self.serverwatch, + libvirt.VIR_EVENT_HANDLE_READABLE) + + + def try_command(self): + try: + cmdstr = self.sock2api.decode("utf-8") + cmd = json.loads(cmdstr) + + if self.verbose: + cmdstr = cmdstr.strip() + print(cmdstr) + except Exception as ex: + if debug: + print("Incomplete %s: %s" % ( self.sock2api, ex)) + return + + id = None + if "id" in cmd: + id = cmd["id"] + del cmd["id"] + + if cmd.get("execute", "") == "qmp_capabilities": + resobj = { + "return": {}, + } + resfds = [] + else: + if hasattr(libvirt_qemu, "qemuMonitorCommandWithFiles"): + res, resfds = libvirt_qemu.qemuMonitorCommandWithFiles( + self.dom, json.dumps(cmd), [f.fileno() for f in self.sock2apifds]) + resobj = json.loads(res) + else: + if len(self.sock2apifds) > 0: + raise Exception("FD passing not supported") + res = libvirt_qemu.qemuMonitorCommand( + self.dom, json.dumps(cmd)) + resfds = [] + resobj = json.loads(res) + + if "id" in resobj: + del resobj["id"] + if id is not None: + resobj["id"] = id + + res = json.dumps(resobj) + if self.verbose: + print(res) + res += "\r\n" + + self.sock2api = bytes([]) + self.sock2apifds = [] + self.api2sock += res.encode("utf-8") + self.api2sockfds = resfds + + + @staticmethod + def handle_client_io(watch, fd, events, self): + error = False + try: + if events & libvirt.VIR_EVENT_HANDLE_WRITABLE: + done = self.send_with_fds(self.api2sock, self.api2sockfds) + if done > 0: + self.api2sock = self.api2sock[done:] + self.api2sockfds = [] + elif events & libvirt.VIR_EVENT_HANDLE_READABLE: + data, fds = self.recv_with_fds() + if len(data) == 0: + error = True + else: + self.sock2api += data + if len(fds): + self.sock2apifds += fds + + self.try_command() + else: + error = True + except Exception as e: + global debug + if debug: + print("%s: %s" % (sys.argv[0], str(e))) + print(traceback.format_exc()) + error = True + + if error: + self.remove_client() + else: + self.update_client_events() + + + @staticmethod + def handle_server_io(watch, fd, events, self): + if self.clientsock is None: + sock, addr = self.serversock.accept() + self.add_client(sock) + else: + self.update_server_events() + + +def parse_commandline(): + parser = argparse.ArgumentParser(description="Libvirt QMP proxy") + parser.add_argument("--connect", "-c", + help="Libvirt QEMU driver connection URI") + parser.add_argument("--debug", "-d", action='store_true', + help="Display debugging information") + parser.add_argument("--verbose", "-v", action='store_true', + help="Display QMP traffic") + parser.add_argument("domain", metavar="DOMAIN", + help="Libvirt guest domain ID/UUID/Name") + parser.add_argument("sockpath", metavar="QMP-SOCK-PATH", + help="UNIX socket path for QMP server") + + return parser.parse_args() + + +def main(): + args = parse_commandline() + global debug + debug = args.debug + + if not debug: + libvirt.registerErrorHandler(lambda opaque, error: None, None) + + libvirt.virEventRegisterDefaultImpl() + + conn, dom = get_domain(args.connect, args.domain) + + if conn.getType() != "QEMU": + raise Exception("QMP proxy requires a QEMU driver connection not %s" % + conn.getType()) + + sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + if os.path.exists(args.sockpath): + os.unlink(args.sockpath) + sock.bind(args.sockpath) + sock.listen(1) + + proxy = QMPProxy(conn, dom, sock, args.verbose) + + while True: + libvirt.virEventRunDefaultImpl() + + +try: + main() + sys.exit(0) +except Exception as e: + print("%s: %s" % (sys.argv[0], str(e))) + if debug: + print(traceback.format_exc()) + sys.exit(1) -- 2.36.1

3 2

[PATCH] security_selinux.c: Relabel existing mode="bind" UNIX sockets
by David Michael 01 Jul '22

01 Jul '22

This supports sockets created by libvirt and passed by FD using the same method as in security_dac.c. Signed-off-by: David Michael <david(a)bigbadwolfsecurity.com> --- Hi, Custom SELinux labels are not applied to sockets when they have mode="bind", but other security models (DAC) allow changing these sockets. Can the same method be used to support SELinux? Thanks. David src/security/security_selinux.c | 6 ++++-- tests/securityselinuxlabeldata/chardev.txt | 2 +- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/src/security/security_selinux.c b/src/security/security_selinux.c index e2f34a27dc..8b258c9e36 100644 --- a/src/security/security_selinux.c +++ b/src/security/security_selinux.c @@ -2541,7 +2541,9 @@ virSecuritySELinuxSetChardevLabel(virSecurityManager *mgr, break; case VIR_DOMAIN_CHR_TYPE_UNIX: - if (!dev_source->data.nix.listen) { + if (!dev_source->data.nix.listen || + (dev_source->data.nix.path && + virFileExists(dev_source->data.nix.path))) { if (virSecuritySELinuxSetFilecon(mgr, dev_source->data.nix.path, imagelabel, @@ -2618,7 +2620,7 @@ virSecuritySELinuxRestoreChardevLabel(virSecurityManager *mgr, case VIR_DOMAIN_CHR_TYPE_UNIX: if (!dev_source->data.nix.listen) { if (virSecuritySELinuxRestoreFileLabel(mgr, - dev_source->data.file.path, + dev_source->data.nix.path, true) < 0) goto done; } diff --git a/tests/securityselinuxlabeldata/chardev.txt b/tests/securityselinuxlabeldata/chardev.txt index 3f4b6302b9..bdb367f7a5 100644 --- a/tests/securityselinuxlabeldata/chardev.txt +++ b/tests/securityselinuxlabeldata/chardev.txt @@ -2,6 +2,6 @@ /plain.dev;system_u:object_r:svirt_image_t:s0:c41,c264 /plain.fifo;system_u:object_r:svirt_image_t:s0:c41,c264 /nolabel.sock; -/plain.sock; +/plain.sock;system_u:object_r:svirt_image_t:s0:c41,c264 /yeslabel.sock;system_u:object_r:svirt_image_t:s0:c41,c264 /altlabel.sock;system_u:object_r:svirt_image_custom_t:s0:c41,c264 -- 2.36.1

4 7

[PATCH 0/3] qemu: add an API for "query-stats" QMP command
by Amneesh Singh 30 Jun '22

30 Jun '22

QEMU is gaining introspectable statistics which can be queried via the "query-stats" QMP command. This patchset aims to add an API for the same. The returned JSON for "query-stats" is an array of objects containing their own respective array of statistics. Patch 1 adds the API which returns the deserialized JSON in the form of a GPtrArray of GHashTables. Patch 2 adds the "query-stats" to QEMU capabilities. Patch 3 uses the API to query the halt poll success time and the halt poll failure time. Relevant QEMU patches can be found here: https://lore.kernel.org/all/20220523150509.349412-1-pbonzini@redhat.com/ This patchset is part of the 2022 GSOC contributor project. All reviews and comments are appreciated :). Amneesh Singh (3): qemu_monitor: add qemuMonitorQueryStats qemu_capabilities: add "query-stats" QMP command to the QEMU capabilities qemu_driver: use qemuMonitorQueryStats to extract halt poll time src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_driver.c | 48 +++++++++++-- src/qemu/qemu_monitor.c | 46 +++++++++++++ src/qemu/qemu_monitor.h | 36 ++++++++++ src/qemu/qemu_monitor_json.c | 128 +++++++++++++++++++++++++++++++++++ src/qemu/qemu_monitor_json.h | 6 ++ 7 files changed, 263 insertions(+), 4 deletions(-) -- 2.36.1

3 11

[libvirt PATCH 0/4] qemu_migration: Apply max-postcopy-bandwidth on post-copy resume
by Jiri Denemark 30 Jun '22

30 Jun '22

Resolves: https://gitlab.com/libvirt/libvirt/-/issues/333 Jiri Denemark (4): qemu: Pass migration flags to qemuMigrationParamsApply qemu_migration_params: Replace qemuMigrationParamTypes array qemu_migration: Pass migParams to qemuMigrationSrcResume qemu_migration: Apply max-postcopy-bandwidth on post-copy resume src/qemu/qemu_driver.c | 6 +- src/qemu/qemu_migration.c | 26 ++++-- src/qemu/qemu_migration_params.c | 131 ++++++++++++++++++--------- src/qemu/qemu_migration_params.h | 3 +- src/qemu/qemu_migration_paramspriv.h | 3 +- tests/qemumigparamstest.c | 2 +- tests/qemumigrationcookiexmltest.c | 2 +- 7 files changed, 116 insertions(+), 57 deletions(-) -- 2.35.1

2 5