1:40 p.m.
After a bit of a lengthy delay, this is the second version of this patch
series. See https://bugzilla.redhat.com/show_bug.cgi?id=2016527 for more
information about the goal, but the summary is that RHEL does not want to ship
the qemu storage plugins for curl and ssh. Handling them outside of the qemu
process provides several advantages such as reduced attack surface and
stability.
A quick summary of the code:
- at startup I query to see whether nbdkit exists on the host and if
so, I query which plugins/filters are installed. These capabilities
are cached and stored in the qemu driver
- When the driver prepares the domain, we go through each disk source
and determine whether the nbdkit capabilities allow us to support
this disk via nbdkit, and if so, we allocate a qemuNbdkitProcess
object and stash it in the private data of the virStorageSource.
- The presence or absence of this qemuNbdkitProcess data then indicates
whether this disk will be served to qemu indirectly via nbdkit or
directly
- When we launch the qemuProcess, as part of the "external device
start" step, I launch a ndkit process for each disk that is supported
by nbdkit.
- for devices which are served by an intermediate ndkit process, I
change the qemu commandline in the following ways:
- I no longer pass auth/cookie secrets to qemu (those are handled by
nbdkit)
- I replace the actual network URL of the remote disk source with the
path to the nbdkit unix socket
Open questions
- selinux: I need some help from people more familiar with selinux to figure
out what is needed here. When selinux is enforcing, I get a failure to
launch nbdkit to serve the disks. I suspect we need a new context and policy
for /usr/sbin/nbdkit that allows it to transition to the appropriate selinux
context. The current context (on fedora) is "system_u:object_r:bin_t:s0".
When I (temporarily) change the context to something like qemu_exec_t,
I am able to start nbdkit and the domain launches.
Known shortcomings
- creating disks (in ssh) still isn't supported. I wanted to send out the
patch series anyway since it's been delayed too long already.
Changes since v1:
- split into multiple patches
- added a build option for nbdkit_moddir
- don't instantiate any secret / cookie props for disks that are being served
by nbdkit since we don't send secrets to qemu anymore
- ensure that nbdkit processes are started/stopped for the entire backing
chain
- switch to virFileCache-based capabilities for nbdkit so that we don't need
to requery every time
- switch to using pipes for communicating sensitive data to nbdkit
- use pidfile support built into virCommand rather than nbdkit's --pidfile
argument
- added significantly more tests
Jonathon Jongsma (16):
schema: allow 'ssh' as a protocol for network disks
qemu: Add qemuNbdkitCaps to qemu driver
qemu: expand nbdkit capabilities
util: Allow virFileCache data to be any GObject
qemu: implement basic virFileCache for nbdkit caps
qemu: implement persistent file cache for nbdkit caps
qemu: use file cache for nbdkit caps
qemu: Add qemuNbdkitProcess
qemu: add functions to start and stop nbdkit
tests: add ability to test various nbdkit capabilities
qemu: split qemuDomainSecretStorageSourcePrepare
qemu: use nbdkit to serve network disks if available
qemu: include nbdkit state in private xml
tests: add tests for nbdkit invocation
qemu: pass sensitive data to nbdkit via pipe
qemu: add test for authenticating a https network disk
build-aux/syntax-check.mk | 4 +-
docs/formatdomain.rst | 2 +-
meson.build | 6 +
meson_options.txt | 1 +
po/POTFILES | 1 +
src/conf/schemas/domaincommon.rng | 1 +
src/qemu/meson.build | 1 +
src/qemu/qemu_block.c | 168 ++-
src/qemu/qemu_command.c | 4 +-
src/qemu/qemu_conf.c | 22 +
src/qemu/qemu_conf.h | 6 +
src/qemu/qemu_domain.c | 176 ++-
src/qemu/qemu_domain.h | 4 +
src/qemu/qemu_driver.c | 3 +
src/qemu/qemu_extdevice.c | 84 ++
src/qemu/qemu_nbdkit.c | 1051 +++++++++++++++++
src/qemu/qemu_nbdkit.h | 90 ++
src/qemu/qemu_nbdkitpriv.h | 46 +
src/util/virfilecache.c | 15 +-
src/util/virfilecache.h | 2 +-
src/util/virutil.h | 2 +-
tests/meson.build | 1 +
.../disk-cdrom-network.args.disk0 | 7 +
.../disk-cdrom-network.args.disk1 | 9 +
.../disk-cdrom-network.args.disk1.pipe.45 | 1 +
.../disk-cdrom-network.args.disk2 | 9 +
.../disk-cdrom-network.args.disk2.pipe.47 | 1 +
.../disk-network-http.args.disk0 | 7 +
.../disk-network-http.args.disk1 | 6 +
.../disk-network-http.args.disk2 | 7 +
.../disk-network-http.args.disk2.pipe.45 | 1 +
.../disk-network-http.args.disk3 | 8 +
.../disk-network-http.args.disk3.pipe.47 | 1 +
...work-source-curl-nbdkit-backing.args.disk0 | 8 +
...rce-curl-nbdkit-backing.args.disk0.pipe.45 | 1 +
.../disk-network-source-curl.args.1.pipe.1 | 1 +
.../disk-network-source-curl.args.disk0 | 8 +
...isk-network-source-curl.args.disk0.pipe.45 | 1 +
.../disk-network-source-curl.args.disk1 | 10 +
...isk-network-source-curl.args.disk1.pipe.47 | 1 +
...isk-network-source-curl.args.disk1.pipe.49 | 1 +
.../disk-network-source-curl.args.disk2 | 8 +
...isk-network-source-curl.args.disk2.pipe.49 | 1 +
...isk-network-source-curl.args.disk2.pipe.51 | 1 +
.../disk-network-source-curl.args.disk3 | 7 +
.../disk-network-source-curl.args.disk4 | 7 +
.../disk-network-ssh.args.disk0 | 7 +
tests/qemunbdkittest.c | 271 +++++
...sk-cdrom-network-nbdkit.x86_64-latest.args | 42 +
.../disk-cdrom-network-nbdkit.xml | 1 +
...isk-network-http-nbdkit.x86_64-latest.args | 45 +
.../disk-network-http-nbdkit.xml | 1 +
...rce-curl-nbdkit-backing.x86_64-latest.args | 38 +
...isk-network-source-curl-nbdkit-backing.xml | 45 +
...work-source-curl-nbdkit.x86_64-latest.args | 50 +
.../disk-network-source-curl-nbdkit.xml | 1 +
...isk-network-source-curl.x86_64-latest.args | 54 +
.../disk-network-source-curl.xml | 74 ++
...disk-network-ssh-nbdkit.x86_64-latest.args | 36 +
.../disk-network-ssh-nbdkit.xml | 1 +
.../disk-network-ssh.x86_64-latest.args | 36 +
tests/qemuxml2argvdata/disk-network-ssh.xml | 31 +
tests/qemuxml2argvtest.c | 18 +
tests/testutilsqemu.c | 27 +
tests/testutilsqemu.h | 5 +
65 files changed, 2474 insertions(+), 111 deletions(-)
create mode 100644 src/qemu/qemu_nbdkit.c
create mode 100644 src/qemu/qemu_nbdkit.h
create mode 100644 src/qemu/qemu_nbdkitpriv.h
create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk0
create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk1
create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk1.pipe.45
create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk2
create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk2.pipe.47
create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk0
create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk1
create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk2
create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk2.pipe.45
create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk3
create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk3.pipe.47
create mode 100644
tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.disk0
create mode 100644
tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.disk0.pipe.45
create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.1.pipe.1
create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk0
create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk0.pipe.45
create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1
create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.47
create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.49
create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2
create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.49
create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.51
create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk3
create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk4
create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk0
create mode 100644 tests/qemunbdkittest.c
create mode 100644 tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.x86_64-latest.args
create mode 120000 tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.xml
create mode 100644 tests/qemuxml2argvdata/disk-network-http-nbdkit.x86_64-latest.args
create mode 120000 tests/qemuxml2argvdata/disk-network-http-nbdkit.xml
create mode 100644
tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.x86_64-latest.args
create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.xml
create mode 100644
tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.x86_64-latest.args
create mode 120000 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.xml
create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args
create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl.xml
create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-nbdkit.x86_64-latest.args
create mode 120000 tests/qemuxml2argvdata/disk-network-ssh-nbdkit.xml
create mode 100644 tests/qemuxml2argvdata/disk-network-ssh.x86_64-latest.args
create mode 100644 tests/qemuxml2argvdata/disk-network-ssh.xml
--
2.37.1