5:44 a.m.
From: Marc-André Lureau <marcandre.lureau(a)redhat.com>
This series adds support for running virtio GPUs in seperate
processes, thanks to vhost-user backend.
The QEMU support landed for 4.1. There are several benefits of running
the GPU/virgl in an external process, since Mesa is rather heavy on
the qemu main loop, and may block for a while, or crash.
The external GPU process is started with one end of a socket pair, the
other end is given to a QEMU chardev attached to a device. The
external process is also added to the cgroup to limit resources usage.
vhost-user is a generic mechanism that allows implementing virtio device
dataplane handling in a separate userspace process. vhost-user-gpu here
notably moves the virgl 3d handling out of the main qemu process. The
external process will be /usr/libexec/vhost-user-gpu, which comes from
qemu.git contrib/vhost-user-gpu code, first released in qemu-4.1.
Part of this series deals with discovering the location on disk of the
vhost-user-gpu binary, and what capabilities it provides. This uses a
similar mechanism to firmware.json, described in qemu
docs/interop/vhost-user.json
https://github.com/qemu/qemu/blob/master/docs/interop/vhost-user.json
qemu 4.1 ships a 50-qemu-gpu.json to match. I believe virtio-fs
will use a similar mechanism when it lands in upstream qemu, as
virtiofsd is a separate process that communicates with qemu over
vhost-user.
For a bit more background on vhost-user-gpu process handling and
the json interop motivation, here's some of the qemu discussion:
https://lists.nongnu.org/archive/html/qemu-devel/2018-08/msg02610.html
https://lists.nongnu.org/archive/html/qemu-devel/2018-09/msg00807.html
For this series, the XML to enable this is:
<video model='virtio'>
<driver name='vhostuser'/>
<acceleration accel3d='yes' rendernode='/path/to/rendernode'/>
</video>
rendernode is optional
qemu_vhost_user.c handles vhost-user.json
qemu_vhost_user_gpu.c handles the process management for
vhost-user-gpu
v5: addressing v4 reviews
- replaced "util: ignore EACCESS in virDirOpenIfExists" with more
specific "qemu-interop: ignore non-readable directories"
- use virFileSanitizePath & virBufferEscapeString on rendernode path
- fix ->accel NULL crash, replace qemuSetupVideoAccelCgroup with
qemuSetupVideoCgroup
- fix src/vz virDomainDefAddImplicitVideo usage
- few indent changes, rebased, added r-b tags
v4:
- rebased
- simplify vhost-user-gpu pidfile checking
- fix check/syntax-check
v3:
- rename qemu_configs -> qemu_interop_config
- replace <model .. vhostuser='yes'/> with <driver
name='vhostuser'/>
- move vhost_user_binary to virDomainVideoDriverDef
- some VIR_AUTO usage improvements
- introduce qemuDomainVideoPrivate to store vhost-user fd
- improved selection of -device model
- use DO_TEST_CAPS_LATEST
- allocate a rendernode with virHostGetDRMRenderNode() - but no clear idea how to have a
test
- add a patch to ignore EPERM in virDirOpenIfExists()
- better domain checks/validate
- added Ján r-b
- no s-o-b from Cole, per request and commits taken from his git branch
- rebase, indentation/style fixes, update doc, version..
v2:
- rebase to master
- if test file build by dropping LDADDS usage
- syntax-check issues:
* use #pragma once
* if () bracket issues
* jump label indent issues
* error message %s usage
* size_t for loops
Marc-André Lureau (20):
qemu: generalize qemuFetchConfigs
qemu-interop: ignore non-readable directories
conf: format/parse/rng/docs for video <driver name='qemu|vhostuser'/>
domain: add rendernode attribute on <accel>
qemu-cgroup: allow accel rendernode access
qemu: add vhost-user-gpu capabilities checks
qemu: check that qemu is vhost-user-vga capable
qemu: validate virtio-gpu with vhost-user
qemu: restrict 'virgl=' option to non-vhostuser video type
qemu: add vhost-user helpers
qemu: add qemuSecurityStartVhostUserGPU helper
conf: add privateData to virDomainVideoDef
qemu: add qemuDomainVideoPrivate
qemu: add vhost-user-gpu helper unit
tests: mock execv/execve
tests: wrap vhost-user paths in qemuxml2argvtest
qemu: prepare domain for vhost-user GPU
qemu: start/stop the vhost-user-gpu external device
qemu: build vhost-user GPU devices
tests: add vhost-user-gpu xml2argv tests
docs/formatdomain.html.in | 18 +-
docs/schemas/domaincommon.rng | 13 +
src/conf/domain_conf.c | 85 +++-
src/conf/domain_conf.h | 22 +-
src/qemu/Makefile.inc.am | 6 +
src/qemu/qemu_capabilities.c | 6 +
src/qemu/qemu_capabilities.h | 4 +
src/qemu/qemu_cgroup.c | 28 ++
src/qemu/qemu_command.c | 65 ++-
src/qemu/qemu_domain.c | 52 ++-
src/qemu/qemu_domain.h | 12 +
src/qemu/qemu_extdevice.c | 74 ++-
src/qemu/qemu_extdevice.h | 5 +
src/qemu/qemu_firmware.c | 144 +-----
src/qemu/qemu_interop_config.c | 189 ++++++++
src/qemu/qemu_interop_config.h | 25 ++
src/qemu/qemu_process.c | 61 ++-
src/qemu/qemu_security.c | 40 ++
src/qemu/qemu_security.h | 6 +
src/qemu/qemu_vhost_user.c | 422 ++++++++++++++++++
src/qemu/qemu_vhost_user.h | 48 ++
src/qemu/qemu_vhost_user_gpu.c | 275 ++++++++++++
src/qemu/qemu_vhost_user_gpu.h | 49 ++
src/vz/vz_sdk.c | 2 +-
tests/Makefile.am | 9 +
.../caps_4.1.0.x86_64.xml | 2 +
.../etc/qemu/vhost-user/40-gpu.json | 1 +
.../etc/qemu/vhost-user/50-gpu.json | 0
.../qemu/vhost-user/test-vhost-user-gpu | 11 +
.../usr/share/qemu/vhost-user/30-gpu.json | 1 +
.../usr/share/qemu/vhost-user/50-gpu.json | 8 +
.../usr/share/qemu/vhost-user/60-gpu.json | 1 +
tests/qemuvhostusertest.c | 132 ++++++
...host-user-gpu-secondary.x86_64-latest.args | 43 ++
.../vhost-user-gpu-secondary.xml | 46 ++
.../vhost-user-vga.x86_64-latest.args | 40 ++
tests/qemuxml2argvdata/vhost-user-vga.xml | 42 ++
tests/qemuxml2argvdata/virtio-options.args | 5 +-
tests/qemuxml2argvdata/virtio-options.xml | 4 +-
tests/qemuxml2argvtest.c | 16 +-
tests/virfilewrapper.c | 22 +
41 files changed, 1828 insertions(+), 206 deletions(-)
create mode 100644 src/qemu/qemu_interop_config.c
create mode 100644 src/qemu/qemu_interop_config.h
create mode 100644 src/qemu/qemu_vhost_user.c
create mode 100644 src/qemu/qemu_vhost_user.h
create mode 100644 src/qemu/qemu_vhost_user_gpu.c
create mode 100644 src/qemu/qemu_vhost_user_gpu.h
create mode 120000 tests/qemuvhostuserdata/etc/qemu/vhost-user/40-gpu.json
create mode 100644 tests/qemuvhostuserdata/etc/qemu/vhost-user/50-gpu.json
create mode 100755
tests/qemuvhostuserdata/usr/libexec/qemu/vhost-user/test-vhost-user-gpu
create mode 120000 tests/qemuvhostuserdata/usr/share/qemu/vhost-user/30-gpu.json
create mode 100644 tests/qemuvhostuserdata/usr/share/qemu/vhost-user/50-gpu.json
create mode 120000 tests/qemuvhostuserdata/usr/share/qemu/vhost-user/60-gpu.json
create mode 100644 tests/qemuvhostusertest.c
create mode 100644 tests/qemuxml2argvdata/vhost-user-gpu-secondary.x86_64-latest.args
create mode 100644 tests/qemuxml2argvdata/vhost-user-gpu-secondary.xml
create mode 100644 tests/qemuxml2argvdata/vhost-user-vga.x86_64-latest.args
create mode 100644 tests/qemuxml2argvdata/vhost-user-vga.xml
--
2.23.0