[libvirt PATCH v8 00/37] Use nbdkit for http/ftp/ssh network drives in libvirt

This is the eighth version of this patch series. See https://bugzilla.redhat.com/show_bug.cgi?id=2016527 for more information. Note that testing this requires selinux policy changes which are not fully done, but there is a new policy in development that has allowed me to run with selinux in enforcing mode for the common cases. See https://bugzilla.redhat.com/show_bug.cgi?id=2182505 for more information. The following scenarios should work now with selinux enabled using the selinux policy from that bug: - http/https disks - ssh disks with password authentication - ssh disks with passwordless keyfile The one major thing that doesn't work and is difficult to get working with selinux enabled is the ssh-agent. This is because there doesn't seem to be any selinux policy for ssh-agent, so by default the ssh-agent socket is labeled unconfined_t. We cannot allow access from the libvirt/qemu to unconfined_t because that would open up access to just about anything on the host. So additional work will likely be necessary for ssh-agent/libvirt interaction in the future. Fortunately ssh-agent is something that never was really supported with the old qemu block driver either, so I think we could potentially merge this patchset either without the ssh-agent patches or with a note that ssh-agent won't work with selinux enabled. Changes in v8: - Hopefully addressed all of Peter's issues, in addition to: - updated documentation to say 9.8.0, since 9.7.0 is currently in freeze - used WITH_NBDKIT instead of WITH_DECL_SYS_PIDFD_OPEN to make the code a bit more concise and understandable - enabled ci by adding libnbd to the dependencies, which uncovered a couple additional minor issues with those platforms that don't support the pidfd_open syscall - don't run nbdkit tests when WITH_NBDKIT is not defined - avoid warnings with unused function arguments - note that the ubuntu containers are currently failing due to a LeakSanitizer error, but I haven't reproduced it locally and can't figure out how to get better information from the leak sanitizer. Pointers appreciated: https://gitlab.com/jjongsma/libvirt/-/jobs/4991631193 - One change of note is a new patch "qemu: improve error handling when restarting nbdkit". In order to provide better error reporting to the user and avoid VIR_WARN as suggested by Peter, some functions now return an error and this error is propagated up to qemuProcessReconnect(). This could potentially result in running domains being killed upon a libvirt restart, but only if they were in a state where they were was not a running nbdkit backend or libvirt couldn't monitor the process nbdkit. Jonathon Jongsma (37): schema: allow 'ssh' as a protocol for network disks qemu: Add functions for determining nbdkit availability qemu: expand nbdkit capabilities util: Allow virFileCache data to be any GObject qemu: implement basic virFileCache for nbdkit caps qemu: implement persistent file cache for nbdkit caps qemu: use file cache for nbdkit caps qemu: Add qemuNbdkitProcess qemu: query nbdkit module dir from binary qemu: add functions to start and stop nbdkit Generalize qemuDomainLogContextNew() qemu: Extract qemuDomainLogContext into a new file qemu: move qemuProcessReadLog() to qemuLogContext qemu: log error output from nbdkit tests: add ability to test various nbdkit capabilities qemu: split qemuDomainSecretStorageSourcePrepare qemu: include nbdkit state in private xml util: secure erase virCommand send buffers qemu: pass sensitive data to nbdkit via pipe qemu: use nbdkit to serve network disks if available util: make virCommandSetSendBuffer testable tests: add tests for nbdkit invocation qemu: add test for authenticating a https network disk qemu: Add Taint for nbdkit restart failure qemu: Monitor nbdkit process for exit qemu: improve error handling when restarting nbdkit qemu: try to connect to nbdkit early to detect errors schema: add password configuration for ssh disk qemu: implement password auth for ssh disks with nbdkit schema: add configuration for host verification of ssh disks qemu: implement knownHosts for ssh disks with nbdkit schema: add keyfile configuration for ssh disks qemu: implement keyfile auth for ssh disks with nbdkit schema: add ssh-agent configuration for ssh disks qemu: implement ssh-agent auth for ssh disks with nbdkit rpm: update spec file for for nbdkit support ci: add libnbd to build build-aux/syntax-check.mk | 2 +- ci/buildenv/almalinux-8.sh | 1 + ci/buildenv/centos-stream-8.sh | 1 + ci/buildenv/centos-stream-9.sh | 1 + ci/buildenv/debian-12-cross-aarch64.sh | 1 + ci/buildenv/debian-12-cross-armv6l.sh | 1 + ci/buildenv/debian-12-cross-armv7l.sh | 1 + ci/buildenv/debian-12-cross-i686.sh | 1 + ci/buildenv/debian-12-cross-mips64el.sh | 1 + ci/buildenv/debian-12-cross-mipsel.sh | 1 + ci/buildenv/debian-12-cross-ppc64le.sh | 1 + ci/buildenv/debian-12-cross-s390x.sh | 1 + ci/buildenv/debian-12.sh | 1 + ci/buildenv/debian-sid-cross-aarch64.sh | 1 + ci/buildenv/debian-sid-cross-armv6l.sh | 1 + ci/buildenv/debian-sid-cross-armv7l.sh | 1 + ci/buildenv/debian-sid-cross-i686.sh | 1 + ci/buildenv/debian-sid-cross-mips64el.sh | 1 + ci/buildenv/debian-sid-cross-mipsel.sh | 1 + ci/buildenv/debian-sid-cross-ppc64le.sh | 1 + ci/buildenv/debian-sid-cross-s390x.sh | 1 + ci/buildenv/debian-sid.sh | 1 + ci/buildenv/fedora-37.sh | 1 + ci/buildenv/fedora-38-cross-mingw32.sh | 1 + ci/buildenv/fedora-38-cross-mingw64.sh | 1 + ci/buildenv/fedora-38.sh | 1 + ci/buildenv/fedora-rawhide-cross-mingw32.sh | 1 + ci/buildenv/fedora-rawhide-cross-mingw64.sh | 1 + ci/buildenv/fedora-rawhide.sh | 1 + ci/buildenv/opensuse-leap-15.sh | 1 + ci/buildenv/opensuse-tumbleweed.sh | 1 + ci/buildenv/ubuntu-2204.sh | 1 + ci/containers/almalinux-8.Dockerfile | 1 + ci/containers/centos-stream-8.Dockerfile | 1 + ci/containers/centos-stream-9.Dockerfile | 1 + .../debian-12-cross-aarch64.Dockerfile | 1 + .../debian-12-cross-armv6l.Dockerfile | 1 + .../debian-12-cross-armv7l.Dockerfile | 1 + ci/containers/debian-12-cross-i686.Dockerfile | 1 + .../debian-12-cross-mips64el.Dockerfile | 1 + .../debian-12-cross-mipsel.Dockerfile | 1 + .../debian-12-cross-ppc64le.Dockerfile | 1 + .../debian-12-cross-s390x.Dockerfile | 1 + ci/containers/debian-12.Dockerfile | 1 + .../debian-sid-cross-aarch64.Dockerfile | 1 + .../debian-sid-cross-armv6l.Dockerfile | 1 + .../debian-sid-cross-armv7l.Dockerfile | 1 + .../debian-sid-cross-i686.Dockerfile | 1 + .../debian-sid-cross-mips64el.Dockerfile | 1 + .../debian-sid-cross-mipsel.Dockerfile | 1 + .../debian-sid-cross-ppc64le.Dockerfile | 1 + .../debian-sid-cross-s390x.Dockerfile | 1 + ci/containers/debian-sid.Dockerfile | 1 + ci/containers/fedora-37.Dockerfile | 1 + .../fedora-38-cross-mingw32.Dockerfile | 1 + .../fedora-38-cross-mingw64.Dockerfile | 1 + ci/containers/fedora-38.Dockerfile | 1 + .../fedora-rawhide-cross-mingw32.Dockerfile | 1 + .../fedora-rawhide-cross-mingw64.Dockerfile | 1 + ci/containers/fedora-rawhide.Dockerfile | 1 + ci/containers/opensuse-leap-15.Dockerfile | 1 + ci/containers/opensuse-tumbleweed.Dockerfile | 1 + ci/containers/ubuntu-2204.Dockerfile | 1 + ci/lcitool/projects/libvirt.yml | 1 + docs/formatdomain.rst | 45 +- libvirt.spec.in | 8 + meson.build | 18 + meson_options.txt | 1 + po/POTFILES | 2 + src/conf/domain_conf.c | 38 + src/conf/domain_conf.h | 1 + src/conf/schemas/domaincommon.rng | 55 + src/conf/storage_source_conf.c | 6 + src/conf/storage_source_conf.h | 6 +- src/libvirt_private.syms | 1 + src/qemu/meson.build | 3 + src/qemu/qemu_block.c | 162 ++- src/qemu/qemu_conf.c | 22 + src/qemu/qemu_conf.h | 6 + src/qemu/qemu_domain.c | 436 +++--- src/qemu/qemu_domain.h | 31 +- src/qemu/qemu_driver.c | 20 + src/qemu/qemu_extdevice.c | 62 + src/qemu/qemu_hotplug.c | 7 + src/qemu/qemu_logcontext.c | 329 +++++ src/qemu/qemu_logcontext.h | 41 + src/qemu/qemu_nbdkit.c | 1291 +++++++++++++++++ src/qemu/qemu_nbdkit.h | 119 ++ src/qemu/qemu_nbdkitpriv.h | 31 + src/qemu/qemu_process.c | 126 +- src/qemu/qemu_process.h | 3 + src/util/vircommand.c | 19 +- src/util/vircommand.h | 8 + src/util/vircommandpriv.h | 4 + src/util/virfilecache.c | 14 +- src/util/virfilecache.h | 2 +- tests/meson.build | 5 + tests/qemublocktest.c | 2 +- ...w2-invalid.json => network-ssh-qcow2.json} | 0 ...cow2-invalid.xml => network-ssh-qcow2.xml} | 0 .../disk-cdrom-network.args.disk0 | 6 + .../disk-cdrom-network.args.disk1 | 8 + .../disk-cdrom-network.args.disk1.pipe.778 | 1 + .../disk-cdrom-network.args.disk2 | 8 + .../disk-cdrom-network.args.disk2.pipe.780 | 1 + .../disk-network-http.args.disk0 | 6 + .../disk-network-http.args.disk1 | 5 + .../disk-network-http.args.disk2 | 6 + .../disk-network-http.args.disk2.pipe.778 | 1 + .../disk-network-http.args.disk3 | 7 + .../disk-network-http.args.disk3.pipe.780 | 1 + ...work-source-curl-nbdkit-backing.args.disk0 | 7 + ...ce-curl-nbdkit-backing.args.disk0.pipe.778 | 1 + .../disk-network-source-curl.args.disk0 | 7 + ...sk-network-source-curl.args.disk0.pipe.778 | 1 + .../disk-network-source-curl.args.disk1 | 9 + ...sk-network-source-curl.args.disk1.pipe.780 | 1 + ...sk-network-source-curl.args.disk1.pipe.782 | 1 + .../disk-network-source-curl.args.disk2 | 7 + ...sk-network-source-curl.args.disk2.pipe.782 | 1 + ...sk-network-source-curl.args.disk2.pipe.784 | 1 + .../disk-network-source-curl.args.disk3 | 6 + .../disk-network-source-curl.args.disk4 | 6 + .../disk-network-ssh-key.args.disk0 | 9 + .../disk-network-ssh-key.args.disk1 | 9 + .../disk-network-ssh-password.args.disk0 | 9 + ...k-network-ssh-password.args.disk0.pipe.778 | 1 + .../disk-network-ssh.args.disk0 | 7 + .../disk-network-ssh.args.disk1 | 8 + .../disk-network-ssh.args.disk1.pipe.778 | 1 + .../disk-network-ssh.args.disk2 | 9 + tests/qemunbdkittest.c | 310 ++++ tests/qemustatusxml2xmldata/modern-in.xml | 4 + ...sk-cdrom-network-nbdkit.x86_64-latest.args | 42 + .../disk-cdrom-network-nbdkit.xml | 1 + ...isk-network-http-nbdkit.x86_64-latest.args | 44 + .../disk-network-http-nbdkit.xml | 1 + ...rce-curl-nbdkit-backing.x86_64-latest.args | 37 + ...isk-network-source-curl-nbdkit-backing.xml | 45 + ...work-source-curl-nbdkit.x86_64-latest.args | 49 + .../disk-network-source-curl-nbdkit.xml | 1 + ...isk-network-source-curl.x86_64-latest.args | 53 + .../disk-network-source-curl.xml | 74 + .../qemuxml2argvdata/disk-network-ssh-key.xml | 44 + ...disk-network-ssh-nbdkit.x86_64-latest.args | 35 + .../disk-network-ssh-nbdkit.xml | 1 + ...sk-network-ssh-password.x86_64-latest.args | 35 + .../disk-network-ssh-password.xml | 35 + .../disk-network-ssh.x86_64-latest.args | 35 + tests/qemuxml2argvdata/disk-network-ssh.xml | 32 + tests/qemuxml2argvtest.c | 23 + tests/testutilsqemu.c | 26 + tests/testutilsqemu.h | 4 + 153 files changed, 3599 insertions(+), 472 deletions(-) create mode 100644 src/qemu/qemu_logcontext.c create mode 100644 src/qemu/qemu_logcontext.h create mode 100644 src/qemu/qemu_nbdkit.c create mode 100644 src/qemu/qemu_nbdkit.h create mode 100644 src/qemu/qemu_nbdkitpriv.h rename tests/qemublocktestdata/imagecreate/{network-ssh-qcow2-invalid.json => network-ssh-qcow2.json} (100%) rename tests/qemublocktestdata/imagecreate/{network-ssh-qcow2-invalid.xml => network-ssh-qcow2.xml} (100%) create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk1.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk2 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk2.pipe.780 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk2 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk2.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk3 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk3.pipe.780 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.disk0.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk0.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.780 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.782 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.782 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.784 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk3 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk4 create mode 100644 tests/qemunbdkitdata/disk-network-ssh-key.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-ssh-key.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-network-ssh-password.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-ssh-password.args.disk0.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk1.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk2 create mode 100644 tests/qemunbdkittest.c create mode 100644 tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-http-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-network-http-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl.xml create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-key.xml create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-network-ssh-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-password.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-password.xml create mode 100644 tests/qemuxml2argvdata/disk-network-ssh.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-ssh.xml -- 2.41.0

There was support in the code for parsing protocol='ssh' on network disk sources, but it was not present in the xml schema. Add this to the schema. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/conf/schemas/domaincommon.rng | 1 + tests/qemublocktest.c | 2 +- ...w2-invalid.json => network-ssh-qcow2.json} | 0 ...cow2-invalid.xml => network-ssh-qcow2.xml} | 0 .../disk-network-ssh.x86_64-latest.args | 35 +++++++++++++++++++ tests/qemuxml2argvdata/disk-network-ssh.xml | 31 ++++++++++++++++ tests/qemuxml2argvtest.c | 1 + 7 files changed, 69 insertions(+), 1 deletion(-) rename tests/qemublocktestdata/imagecreate/{network-ssh-qcow2-invalid.json => network-ssh-qcow2.json} (100%) rename tests/qemublocktestdata/imagecreate/{network-ssh-qcow2-invalid.xml => network-ssh-qcow2.xml} (100%) create mode 100644 tests/qemuxml2argvdata/disk-network-ssh.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-ssh.xml diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincommon.rng index de3bd1c35c..4a475f5c36 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -2179,6 +2179,7 @@ <choice> <value>sheepdog</value> <value>tftp</value> + <value>ssh</value> </choice> </attribute> <attribute name="name"/> diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c index 9a968477d7..8bad69e7ac 100644 --- a/tests/qemublocktest.c +++ b/tests/qemublocktest.c @@ -1213,7 +1213,7 @@ mymain(void) TEST_IMAGE_CREATE("network-gluster-qcow2", NULL); TEST_IMAGE_CREATE("network-rbd-qcow2", NULL); - TEST_IMAGE_CREATE("network-ssh-qcow2-invalid", NULL); + TEST_IMAGE_CREATE("network-ssh-qcow2", NULL); #define TEST_BITMAP_DETECT(testname) \ do { \ diff --git a/tests/qemublocktestdata/imagecreate/network-ssh-qcow2-invalid.json b/tests/qemublocktestdata/imagecreate/network-ssh-qcow2.json similarity index 100% rename from tests/qemublocktestdata/imagecreate/network-ssh-qcow2-invalid.json rename to tests/qemublocktestdata/imagecreate/network-ssh-qcow2.json diff --git a/tests/qemublocktestdata/imagecreate/network-ssh-qcow2-invalid.xml b/tests/qemublocktestdata/imagecreate/network-ssh-qcow2.xml similarity index 100% rename from tests/qemublocktestdata/imagecreate/network-ssh-qcow2-invalid.xml rename to tests/qemublocktestdata/imagecreate/network-ssh-qcow2.xml diff --git a/tests/qemuxml2argvdata/disk-network-ssh.x86_64-latest.args b/tests/qemuxml2argvdata/disk-network-ssh.x86_64-latest.args new file mode 100644 index 0000000000..b7fd30032b --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-ssh.x86_64-latest.args @@ -0,0 +1,35 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1 \ +USER=test \ +LOGNAME=test \ +XDG_DATA_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=QEMUGuest1,debug-threads=on \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=off,dump-guest-core=off,memory-backend=pc.ram,acpi=off \ +-accel kvm \ +-cpu qemu64 \ +-m size=219136k \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}' \ +-overcommit mem-lock=off \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,fd=1729,server=on,wait=off \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-boot strict=on \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0x2"}' \ +-blockdev '{"driver":"ssh","path":"test.img","server":{"host":"example.org","port":"2222"},"node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw","file":"libvirt-1-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x2","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ +-msg timestamp=on diff --git a/tests/qemuxml2argvdata/disk-network-ssh.xml b/tests/qemuxml2argvdata/disk-network-ssh.xml new file mode 100644 index 0000000000..355add4fea --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-ssh.xml @@ -0,0 +1,31 @@ +<domain type='kvm'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='x86_64' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <disk type='network' device='disk'> + <driver name='qemu' type='raw'/> + <source protocol='ssh' name='test.img'> + <host name='example.org' port='2222'/> + <timeout seconds='1234'/> + <readahead size='1024'/> + </source> + <target dev='vda' bus='virtio'/> + </disk> + <controller type='usb' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='none'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 6e63f8a2eb..1b76b32812 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1254,6 +1254,7 @@ mymain(void) VIR_FREE(driver.config->nbdTLSx509secretUUID); VIR_FREE(driver.config->vxhsTLSx509secretUUID); driver.config->vxhsTLS = 0; + DO_TEST_CAPS_LATEST("disk-network-ssh"); DO_TEST_CAPS_LATEST("disk-no-boot"); DO_TEST_CAPS_LATEST("disk-nvme"); DO_TEST_CAPS_VER("disk-vhostuser-numa", "4.2.0"); -- 2.41.0

In future commits, we will optionally use nbdkit to serve some remote disk sources. This patch queries to see whether nbdkit is installed on the host and queries it for capabilities. The data will be used in later commits. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/meson.build | 1 + src/qemu/qemu_conf.h | 1 + src/qemu/qemu_nbdkit.c | 200 +++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_nbdkit.h | 50 +++++++++++ 4 files changed, 252 insertions(+) create mode 100644 src/qemu/qemu_nbdkit.c create mode 100644 src/qemu/qemu_nbdkit.h diff --git a/src/qemu/meson.build b/src/qemu/meson.build index c8806bbc36..9be6996195 100644 --- a/src/qemu/meson.build +++ b/src/qemu/meson.build @@ -28,6 +28,7 @@ qemu_driver_sources = [ 'qemu_monitor_json.c', 'qemu_monitor_text.c', 'qemu_namespace.c', + 'qemu_nbdkit.c', 'qemu_passt.c', 'qemu_process.c', 'qemu_qapi.c', diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index 11c740d28f..4f610d86a1 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -36,6 +36,7 @@ #include "virthreadpool.h" #include "locking/lock_manager.h" #include "qemu_capabilities.h" +#include "qemu_nbdkit.h" #include "virclosecallbacks.h" #include "virhostdev.h" #include "virfile.h" diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c new file mode 100644 index 0000000000..e4e8fd568e --- /dev/null +++ b/src/qemu/qemu_nbdkit.c @@ -0,0 +1,200 @@ +/* + * qemu_nbdkit.c: helpers for using nbdkit + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + */ + +#include <config.h> +#include <glib.h> + +#include "vircommand.h" +#include "virerror.h" +#include "virlog.h" +#include "virpidfile.h" +#include "virutil.h" +#include "qemu_block.h" +#include "qemu_conf.h" +#include "qemu_domain.h" +#include "qemu_extdevice.h" +#include "qemu_nbdkit.h" +#include "qemu_security.h" + +#include <fcntl.h> + +#define VIR_FROM_THIS VIR_FROM_QEMU + +VIR_LOG_INIT("qemu.nbdkit"); + +VIR_ENUM_IMPL(qemuNbdkitCaps, + QEMU_NBDKIT_CAPS_LAST, + /* 0 */ + "plugin-curl", /* QEMU_NBDKIT_CAPS_PLUGIN_CURL */ + "plugin-ssh", /* QEMU_NBDKIT_CAPS_PLUGIN_SSH */ + "filter-readahead", /* QEMU_NBDKIT_CAPS_FILTER_READAHEAD */ +); + +struct _qemuNbdkitCaps { + GObject parent; + + char *path; + char *version; + + virBitmap *flags; +}; +G_DEFINE_TYPE(qemuNbdkitCaps, qemu_nbdkit_caps, G_TYPE_OBJECT); + + +static void +qemuNbdkitCheckCommandCap(qemuNbdkitCaps *nbdkit, + virCommand *cmd, + qemuNbdkitCapsFlags cap) +{ + if (virCommandRun(cmd, NULL) != 0) + return; + + VIR_DEBUG("Setting nbdkit capability %i", cap); + ignore_value(virBitmapSetBit(nbdkit->flags, cap)); +} + + +static void +qemuNbdkitQueryFilter(qemuNbdkitCaps *nbdkit, + const char *filter, + qemuNbdkitCapsFlags cap) +{ + g_autoptr(virCommand) cmd = virCommandNewArgList(nbdkit->path, + "--version", + NULL); + + virCommandAddArgPair(cmd, "--filter", filter); + + /* use null plugin to check for filter */ + virCommandAddArg(cmd, "null"); + + qemuNbdkitCheckCommandCap(nbdkit, cmd, cap); +} + + +static void +qemuNbdkitQueryPlugin(qemuNbdkitCaps *nbdkit, + const char *plugin, + qemuNbdkitCapsFlags cap) +{ + g_autoptr(virCommand) cmd = virCommandNewArgList(nbdkit->path, + plugin, + "--version", + NULL); + + qemuNbdkitCheckCommandCap(nbdkit, cmd, cap); +} + + +static void +qemuNbdkitCapsQueryPlugins(qemuNbdkitCaps *nbdkit) +{ + qemuNbdkitQueryPlugin(nbdkit, "curl", QEMU_NBDKIT_CAPS_PLUGIN_CURL); + qemuNbdkitQueryPlugin(nbdkit, "ssh", QEMU_NBDKIT_CAPS_PLUGIN_SSH); +} + + +static void +qemuNbdkitCapsQueryFilters(qemuNbdkitCaps *nbdkit) +{ + qemuNbdkitQueryFilter(nbdkit, "readahead", + QEMU_NBDKIT_CAPS_FILTER_READAHEAD); +} + + +static int +qemuNbdkitCapsQueryVersion(qemuNbdkitCaps *nbdkit) +{ + g_autoptr(virCommand) cmd = virCommandNewArgList(nbdkit->path, + "--version", + NULL); + + virCommandSetOutputBuffer(cmd, &nbdkit->version); + + if (virCommandRun(cmd, NULL) != 0) + return -1; + + VIR_DEBUG("Got nbdkit version %s", nbdkit->version); + return 0; +} + + +static void +qemuNbdkitCapsFinalize(GObject *object) +{ + qemuNbdkitCaps *nbdkit = QEMU_NBDKIT_CAPS(object); + + g_clear_pointer(&nbdkit->path, g_free); + g_clear_pointer(&nbdkit->version, g_free); + g_clear_pointer(&nbdkit->flags, virBitmapFree); + + G_OBJECT_CLASS(qemu_nbdkit_caps_parent_class)->finalize(object); +} + + +void +qemu_nbdkit_caps_init(qemuNbdkitCaps *caps) +{ + caps->flags = virBitmapNew(QEMU_NBDKIT_CAPS_LAST); + caps->version = NULL; +} + + +static void +qemu_nbdkit_caps_class_init(qemuNbdkitCapsClass *klass) +{ + GObjectClass *obj = G_OBJECT_CLASS(klass); + + obj->finalize = qemuNbdkitCapsFinalize; +} + + +qemuNbdkitCaps * +qemuNbdkitCapsNew(const char *path) +{ + qemuNbdkitCaps *caps = g_object_new(QEMU_TYPE_NBDKIT_CAPS, NULL); + caps->path = g_strdup(path); + + return caps; +} + + +G_GNUC_UNUSED static void +qemuNbdkitCapsQuery(qemuNbdkitCaps *caps) +{ + qemuNbdkitCapsQueryPlugins(caps); + qemuNbdkitCapsQueryFilters(caps); + qemuNbdkitCapsQueryVersion(caps); +} + + +bool +qemuNbdkitCapsGet(qemuNbdkitCaps *nbdkitCaps, + qemuNbdkitCapsFlags flag) +{ + return virBitmapIsBitSet(nbdkitCaps->flags, flag); +} + + +void +qemuNbdkitCapsSet(qemuNbdkitCaps *nbdkitCaps, + qemuNbdkitCapsFlags flag) +{ + ignore_value(virBitmapSetBit(nbdkitCaps->flags, flag)); +} diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h new file mode 100644 index 0000000000..e191e1fdb4 --- /dev/null +++ b/src/qemu/qemu_nbdkit.h @@ -0,0 +1,50 @@ +/* + * qemu_nbdkit.h: helpers for using nbdkit + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + */ + +#pragma once + +#include "internal.h" +#include "virenum.h" + +typedef struct _qemuNbdkitCaps qemuNbdkitCaps; + +typedef enum { + /* 0 */ + QEMU_NBDKIT_CAPS_PLUGIN_CURL, + QEMU_NBDKIT_CAPS_PLUGIN_SSH, + QEMU_NBDKIT_CAPS_FILTER_READAHEAD, + + QEMU_NBDKIT_CAPS_LAST, +} qemuNbdkitCapsFlags; + +VIR_ENUM_DECL(qemuNbdkitCaps); + +qemuNbdkitCaps * +qemuNbdkitCapsNew(const char *path); + +bool +qemuNbdkitCapsGet(qemuNbdkitCaps *nbdkitCaps, + qemuNbdkitCapsFlags flag); + +void +qemuNbdkitCapsSet(qemuNbdkitCaps *nbdkitCaps, + qemuNbdkitCapsFlags flag); + +#define QEMU_TYPE_NBDKIT_CAPS qemu_nbdkit_caps_get_type() +G_DECLARE_FINAL_TYPE(qemuNbdkitCaps, qemu_nbdkit_caps, QEMU, NBDKIT_CAPS, GObject); -- 2.41.0

In order to add caching of the nbdkit capabilities, we will need to compare against file modification times, etc. So look up this information when creating the nbdkit caps. Add a nbdkit_moddir build option to allow the builder to specify the location to look for nbdkit plugins and filters. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_nbdkit.c | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index e4e8fd568e..58828dd89a 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -20,6 +20,7 @@ #include <config.h> #include <glib.h> +#include "configmake.h" #include "vircommand.h" #include "virerror.h" #include "virlog.h" @@ -38,6 +39,10 @@ VIR_LOG_INIT("qemu.nbdkit"); +#define NBDKIT_MODDIR LIBDIR "/nbdkit" +#define NBDKIT_PLUGINDIR NBDKIT_MODDIR "/plugins" +#define NBDKIT_FILTERDIR NBDKIT_MODDIR "/filters" + VIR_ENUM_IMPL(qemuNbdkitCaps, QEMU_NBDKIT_CAPS_LAST, /* 0 */ @@ -51,6 +56,11 @@ struct _qemuNbdkitCaps { char *path; char *version; + time_t ctime; + time_t libvirtCtime; + time_t pluginDirMtime; + time_t filterDirMtime; + unsigned int libvirtVersion; virBitmap *flags; }; @@ -175,9 +185,41 @@ qemuNbdkitCapsNew(const char *path) } +static time_t +qemuNbdkitGetDirMtime(const char *moddir) +{ + struct stat st; + + if (stat(moddir, &st) < 0) { + VIR_DEBUG("Failed to stat nbdkit module directory '%s': %s", + moddir, + g_strerror(errno)); + return 0; + } + + return st.st_mtime; +} + + G_GNUC_UNUSED static void qemuNbdkitCapsQuery(qemuNbdkitCaps *caps) { + struct stat st; + + if (stat(caps->path, &st) < 0) { + VIR_DEBUG("Failed to stat nbdkit binary '%s': %s", + caps->path, + g_strerror(errno)); + caps->ctime = 0; + return; + } + + caps->ctime = st.st_ctime; + caps->filterDirMtime = qemuNbdkitGetDirMtime(NBDKIT_FILTERDIR); + caps->pluginDirMtime = qemuNbdkitGetDirMtime(NBDKIT_PLUGINDIR); + caps->libvirtCtime = virGetSelfLastChanged(); + caps->libvirtVersion = LIBVIR_VERSION_NUMBER; + qemuNbdkitCapsQueryPlugins(caps); qemuNbdkitCapsQueryFilters(caps); qemuNbdkitCapsQueryVersion(caps); -- 2.41.0

Since the libvirt documentation suggests to prefer GObject over virObject, and since virObject is a GObject, change virFileCache to allow GObjects as data. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/util/virfilecache.c | 14 ++++++++------ src/util/virfilecache.h | 2 +- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/src/util/virfilecache.c b/src/util/virfilecache.c index c730de066e..6f698016a1 100644 --- a/src/util/virfilecache.c +++ b/src/util/virfilecache.c @@ -170,7 +170,7 @@ virFileCacheLoad(virFileCache *cache, *data = g_steal_pointer(&loadData); cleanup: - virObjectUnref(loadData); + g_clear_pointer(&loadData, g_object_unref); return ret; } @@ -207,7 +207,7 @@ virFileCacheNewData(virFileCache *cache, return NULL; if (virFileCacheSave(cache, name, data) < 0) { - g_clear_pointer(&data, virObjectUnref); + g_clear_object(&data); } } @@ -239,7 +239,7 @@ virFileCacheNew(const char *dir, if (!(cache = virObjectNew(virFileCacheClass))) return NULL; - cache->table = virHashNew(virObjectUnref); + cache->table = virHashNew(g_object_unref); cache->dir = g_strdup(dir); @@ -270,7 +270,7 @@ virFileCacheValidate(virFileCache *cache, if (*data) { VIR_DEBUG("Caching data '%p' for '%s'", *data, name); if (virHashAddEntry(cache->table, name, *data) < 0) { - g_clear_pointer(data, virObjectUnref); + g_clear_pointer(data, g_object_unref); } } } @@ -300,7 +300,8 @@ virFileCacheLookup(virFileCache *cache, data = virHashLookup(cache->table, name); virFileCacheValidate(cache, name, &data); - virObjectRef(data); + if (data) + g_object_ref(data); virObjectUnlock(cache); return data; @@ -331,7 +332,8 @@ virFileCacheLookupByFunc(virFileCache *cache, data = virHashSearch(cache->table, iter, iterData, &name); virFileCacheValidate(cache, name, &data); - virObjectRef(data); + if (data) + g_object_ref(data); virObjectUnlock(cache); return data; diff --git a/src/util/virfilecache.h b/src/util/virfilecache.h index c3bc0f529c..944741c0a7 100644 --- a/src/util/virfilecache.h +++ b/src/util/virfilecache.h @@ -48,7 +48,7 @@ typedef bool * @priv: private data created together with cache * * Creates a new data based on the @name. The returned data must be - * an instance of virObject. + * an instance of GObject. * * Returns data object or NULL on error. */ -- 2.41.0

Preparatory step for caching nbdkit capabilities. This patch implements the newData and isValid virFileCacheHandlers callback functions. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_nbdkit.c | 89 +++++++++++++++++++++++++++++++++++++++++- src/qemu/qemu_nbdkit.h | 4 ++ 2 files changed, 92 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 58828dd89a..9828f562f7 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -201,7 +201,7 @@ qemuNbdkitGetDirMtime(const char *moddir) } -G_GNUC_UNUSED static void +static void qemuNbdkitCapsQuery(qemuNbdkitCaps *caps) { struct stat st; @@ -240,3 +240,90 @@ qemuNbdkitCapsSet(qemuNbdkitCaps *nbdkitCaps, { ignore_value(virBitmapSetBit(nbdkitCaps->flags, flag)); } + + +static bool +virNbkditCapsCheckModdir(const char *moddir, + time_t expectedMtime) +{ + time_t mtime = qemuNbdkitGetDirMtime(moddir); + + if (mtime != expectedMtime) { + VIR_DEBUG("Outdated capabilities for nbdkit: module directory '%s' changed (%lld vs %lld)", + moddir, (long long)mtime, (long long)expectedMtime); + return false; + } + return true; +} + + +static bool +virNbdkitCapsIsValid(void *data, + void *privData G_GNUC_UNUSED) +{ + qemuNbdkitCaps *nbdkitCaps = data; + struct stat st; + + if (!nbdkitCaps->path) + return true; + + if (!virNbkditCapsCheckModdir(NBDKIT_PLUGINDIR, nbdkitCaps->pluginDirMtime)) + return false; + if (!virNbkditCapsCheckModdir(NBDKIT_FILTERDIR, nbdkitCaps->filterDirMtime)) + return false; + + if (nbdkitCaps->libvirtCtime != virGetSelfLastChanged() || + nbdkitCaps->libvirtVersion != LIBVIR_VERSION_NUMBER) { + VIR_DEBUG("Outdated capabilities for '%s': libvirt changed (%lld vs %lld, %lu vs %lu)", + nbdkitCaps->path, + (long long)nbdkitCaps->libvirtCtime, + (long long)virGetSelfLastChanged(), + (unsigned long)nbdkitCaps->libvirtVersion, + (unsigned long)LIBVIR_VERSION_NUMBER); + return false; + } + + if (stat(nbdkitCaps->path, &st) < 0) { + VIR_DEBUG("Failed to stat nbdkit binary '%s': %s", + nbdkitCaps->path, + g_strerror(errno)); + return false; + } + + if (st.st_ctime != nbdkitCaps->ctime) { + VIR_DEBUG("Outdated capabilities for '%s': nbdkit binary changed (%lld vs %lld)", + nbdkitCaps->path, + (long long)st.st_ctime, (long long)nbdkitCaps->ctime); + return false; + } + + return true; +} + + +static void* +virNbdkitCapsNewData(const char *binary, + void *privData G_GNUC_UNUSED) +{ + qemuNbdkitCaps *caps = qemuNbdkitCapsNew(binary); + qemuNbdkitCapsQuery(caps); + + return caps; +} + + +virFileCacheHandlers nbdkitCapsCacheHandlers = { + .isValid = virNbdkitCapsIsValid, + .newData = virNbdkitCapsNewData, + .loadFile = NULL, + .saveFile = NULL, + .privFree = NULL, +}; + + +virFileCache* +qemuNbdkitCapsCacheNew(const char *cachedir) +{ + g_autofree char *dir = g_build_filename(cachedir, "nbdkitcapabilities", NULL); + return virFileCacheNew(dir, "xml", &nbdkitCapsCacheHandlers); +} diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index e191e1fdb4..4aba7c8455 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -21,6 +21,7 @@ #include "internal.h" #include "virenum.h" +#include "virfilecache.h" typedef struct _qemuNbdkitCaps qemuNbdkitCaps; @@ -38,6 +39,9 @@ VIR_ENUM_DECL(qemuNbdkitCaps); qemuNbdkitCaps * qemuNbdkitCapsNew(const char *path); +virFileCache * +qemuNbdkitCapsCacheNew(const char *cachedir); + bool qemuNbdkitCapsGet(qemuNbdkitCaps *nbdkitCaps, qemuNbdkitCapsFlags flag); -- 2.41.0

Implement the loadFile and saveFile virFileCacheHandlers callbacks so that nbdkit capabilities are cached perstistently across daemon restarts. The format and implementation is modeled on the qemu capabilities, but simplified slightly. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- po/POTFILES | 1 + src/qemu/qemu_nbdkit.c | 226 ++++++++++++++++++++++++++++++++++++++++- 2 files changed, 225 insertions(+), 2 deletions(-) diff --git a/po/POTFILES b/po/POTFILES index 5d6ec195b4..6167f98ac5 100644 --- a/po/POTFILES +++ b/po/POTFILES @@ -181,6 +181,7 @@ src/qemu/qemu_monitor.c src/qemu/qemu_monitor_json.c src/qemu/qemu_monitor_text.c src/qemu/qemu_namespace.c +src/qemu/qemu_nbdkit.c src/qemu/qemu_passt.c src/qemu/qemu_process.c src/qemu/qemu_qapi.c diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 9828f562f7..97612b637f 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -312,11 +312,233 @@ virNbdkitCapsNewData(const char *binary, } +static int +qemuNbdkitCapsValidateBinary(qemuNbdkitCaps *nbdkitCaps, + xmlXPathContextPtr ctxt) +{ + g_autofree char *str = NULL; + + if (!(str = virXPathString("string(./path)", ctxt))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("missing path in nbdkit capabilities cache")); + return -1; + } + + if (STRNEQ(str, nbdkitCaps->path)) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Expected caps for '%1$s' but saw '%2$s'"), + nbdkitCaps->path, str); + return -1; + } + + return 0; +} + + +static int +qemuNbdkitCapsParseFlags(qemuNbdkitCaps *nbdkitCaps, + xmlXPathContextPtr ctxt) +{ + g_autofree xmlNodePtr *nodes = NULL; + size_t i; + int n; + + if ((n = virXPathNodeSet("./flag", ctxt, &nodes)) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("failed to parse qemu capabilities flags")); + return -1; + } + + VIR_DEBUG("Got flags %d", n); + for (i = 0; i < n; i++) { + unsigned int flag; + + if (virXMLPropEnum(nodes[i], "name", qemuNbdkitCapsTypeFromString, + VIR_XML_PROP_REQUIRED, &flag) < 0) + return -1; + + qemuNbdkitCapsSet(nbdkitCaps, flag); + } + + return 0; +} + + +/* + * Parsing a doc that looks like + * + * <nbdkitCaps> + * <path>/some/path</path> + * <nbdkitctime>234235253</nbdkitctime> + * <plugindirmtime>234235253</plugindirmtime> + * <filterdirmtime>234235253</filterdirmtime> + * <selfctime>234235253</selfctime> + * <selfvers>1002016</selfvers> + * <flag name='foo'/> + * <flag name='bar'/> + * ... + * </nbdkitCaps> + * + * Returns 0 on success, 1 if outdated, -1 on error + */ +static int +qemuNbdkitCapsLoadCache(qemuNbdkitCaps *nbdkitCaps, + const char *filename) +{ + g_autoptr(xmlDoc) doc = NULL; + g_autoptr(xmlXPathContext) ctxt = NULL; + long long int l; + + if (!(doc = virXMLParse(filename, NULL, NULL, "nbdkitCaps", &ctxt, NULL, false))) + return -1; + + if (virXPathLongLong("string(./selfctime)", ctxt, &l) < 0) { + VIR_DEBUG("missing selfctime in nbdkit capabilities XML"); + return -1; + } + nbdkitCaps->libvirtCtime = (time_t)l; + + nbdkitCaps->libvirtVersion = 0; + virXPathUInt("string(./selfvers)", ctxt, &nbdkitCaps->libvirtVersion); + + if (nbdkitCaps->libvirtCtime != virGetSelfLastChanged() || + nbdkitCaps->libvirtVersion != LIBVIR_VERSION_NUMBER) { + VIR_DEBUG("Outdated capabilities in %s: libvirt changed (%lld vs %lld, %lu vs %lu), stopping load", + nbdkitCaps->path, + (long long)nbdkitCaps->libvirtCtime, + (long long)virGetSelfLastChanged(), + (unsigned long)nbdkitCaps->libvirtVersion, + (unsigned long)LIBVIR_VERSION_NUMBER); + return 1; + } + + if (qemuNbdkitCapsValidateBinary(nbdkitCaps, ctxt) < 0) + return -1; + + if (virXPathLongLong("string(./nbdkitctime)", ctxt, &l) < 0) { + VIR_DEBUG("missing nbdkitctime in nbdkit capabilities XML"); + return -1; + } + nbdkitCaps->ctime = (time_t)l; + + if (virXPathLongLong("string(./plugindirmtime)", ctxt, &l) < 0) { + VIR_DEBUG("missing plugindirmtime in nbdkit capabilities XML"); + return -1; + } + nbdkitCaps->pluginDirMtime = (time_t)l; + + if (virXPathLongLong("string(./filterdirmtime)", ctxt, &l) < 0) { + VIR_DEBUG("missing filterdirmtime in nbdkit capabilities XML"); + return -1; + } + nbdkitCaps->filterDirMtime = (time_t)l; + + if (qemuNbdkitCapsParseFlags(nbdkitCaps, ctxt) < 0) + return -1; + + if ((nbdkitCaps->version = virXPathString("string(./version)", ctxt)) == NULL) { + VIR_DEBUG("missing version in nbdkit capabilities cache"); + return -1; + } + + return 0; +} + + +static void* +virNbdkitCapsLoadFile(const char *filename, + const char *binary, + void *privData G_GNUC_UNUSED, + bool *outdated) +{ + g_autoptr(qemuNbdkitCaps) nbdkitCaps = qemuNbdkitCapsNew(binary); + int ret; + + if (!nbdkitCaps) + return NULL; + + ret = qemuNbdkitCapsLoadCache(nbdkitCaps, filename); + if (ret < 0) + return NULL; + if (ret == 1) { + *outdated = true; + return NULL; + } + + return g_steal_pointer(&nbdkitCaps); +} + + +static char* +qemuNbdkitCapsFormatCache(qemuNbdkitCaps *nbdkitCaps) +{ + g_auto(virBuffer) buf = VIR_BUFFER_INITIALIZER; + size_t i; + + virBufferAddLit(&buf, "<nbdkitCaps>\n"); + virBufferAdjustIndent(&buf, 2); + + virBufferEscapeString(&buf, "<path>%s</path>\n", + nbdkitCaps->path); + virBufferAsprintf(&buf, "<nbdkitctime>%lu</nbdkitctime>\n", + nbdkitCaps->ctime); + virBufferAsprintf(&buf, "<plugindirmtime>%lu</plugindirmtime>\n", + nbdkitCaps->pluginDirMtime); + virBufferAsprintf(&buf, "<filterdirmtime>%lu</filterdirmtime>\n", + nbdkitCaps->filterDirMtime); + virBufferAsprintf(&buf, "<selfctime>%lu</selfctime>\n", + nbdkitCaps->libvirtCtime); + virBufferAsprintf(&buf, "<selfvers>%u</selfvers>\n", + nbdkitCaps->libvirtVersion); + + for (i = 0; i < QEMU_NBDKIT_CAPS_LAST; i++) { + if (qemuNbdkitCapsGet(nbdkitCaps, i)) { + virBufferAsprintf(&buf, "<flag name='%s'/>\n", + qemuNbdkitCapsTypeToString(i)); + } + } + + virBufferAsprintf(&buf, "<version>%s</version>\n", + nbdkitCaps->version); + + virBufferAdjustIndent(&buf, -2); + virBufferAddLit(&buf, "</nbdkitCaps>\n"); + + return virBufferContentAndReset(&buf); +} + + +static int +virNbdkitCapsSaveFile(void *data, + const char *filename, + void *privData G_GNUC_UNUSED) +{ + qemuNbdkitCaps *nbdkitCaps = data; + g_autofree char *xml = NULL; + + xml = qemuNbdkitCapsFormatCache(nbdkitCaps); + + if (virFileWriteStr(filename, xml, 0600) < 0) { + virReportSystemError(errno, + _("Failed to save '%1$s' for '%2$s'"), + filename, nbdkitCaps->path); + return -1; + } + + VIR_DEBUG("Saved caps '%s' for '%s' with (%lu, %lu)", + filename, nbdkitCaps->path, + nbdkitCaps->ctime, + nbdkitCaps->libvirtCtime); + + return 0; +} + + virFileCacheHandlers nbdkitCapsCacheHandlers = { .isValid = virNbdkitCapsIsValid, .newData = virNbdkitCapsNewData, - .loadFile = NULL, - .saveFile = NULL, + .loadFile = virNbdkitCapsLoadFile, + .saveFile = virNbdkitCapsSaveFile, .privFree = NULL, }; -- 2.41.0

Add the virFileCache implementation for nbdkit capabilities to the qemu driver. This allows us to determine whether nbdkit is installed and which plugins are supported. it also has persistent caching and the capabilities are re-queried whenever something changes. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_conf.h | 3 +++ src/qemu/qemu_driver.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index 4f610d86a1..a44985fb8b 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -318,6 +318,9 @@ struct _virQEMUDriver { /* Immutable pointer, self-locking APIs */ virHashAtomic *migrationErrors; + + /* Immutable pointer, self-locking APIs */ + virFileCache *nbdkitCapsCache; }; virQEMUDriverConfig *virQEMUDriverConfigNew(bool privileged, diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 5db42f0753..ad8428948b 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -845,6 +845,8 @@ qemuStateInitialize(bool privileged, defsecmodel))) goto error; + qemu_driver->nbdkitCapsCache = qemuNbdkitCapsCacheNew(cfg->cacheDir); + /* If hugetlbfs is present, then we need to create a sub-directory within * it, since we can't assume the root mount point has permissions that * will let our spawned QEMU instances use it. */ @@ -1078,6 +1080,7 @@ qemuStateCleanup(void) ebtablesContextFree(qemu_driver->ebtables); VIR_FREE(qemu_driver->qemuImgBinary); virObjectUnref(qemu_driver->domains); + virObjectUnref(qemu_driver->nbdkitCapsCache); if (qemu_driver->lockFD != -1) virPidFileRelease(qemu_driver->config->stateDir, "driver", qemu_driver->lockFD); -- 2.41.0

An object for storing information about a nbdkit process that is serving a specific virStorageSource. At the moment, this information is just stored in the private data of virStorageSource and not used at all. Future commits will use this data to actually start a nbdkit process. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_conf.c | 22 ++++++++++++ src/qemu/qemu_conf.h | 2 ++ src/qemu/qemu_domain.c | 31 ++++++++++++++++ src/qemu/qemu_domain.h | 4 +++ src/qemu/qemu_nbdkit.c | 82 ++++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_nbdkit.h | 26 ++++++++++++++ 6 files changed, 167 insertions(+) diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index 3f811d064f..a0360e8d1b 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -1656,3 +1656,25 @@ qemuHugepageMakeBasedir(virQEMUDriver *driver, return 0; } + + +/* + * qemuGetNbdkitCaps: + * @driver: the qemu driver + * + * Gets the capabilities for Nbdkit for the specified driver. These can be used + * to determine whether a particular disk source can be served by nbdkit or + * not. + * + * Returns: a reference to qemuNbdkitCaps or NULL + */ +qemuNbdkitCaps* +qemuGetNbdkitCaps(virQEMUDriver *driver) +{ + char *nbdkitBinary = virFindFileInPath("nbdkit"); + + if (!nbdkitBinary) + return NULL; + + return virFileCacheLookup(driver->nbdkitCapsCache, nbdkitBinary); +} diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index a44985fb8b..1a3ba3a0fb 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -377,3 +377,5 @@ int qemuGetMemoryBackingPath(virQEMUDriver *driver, int qemuHugepageMakeBasedir(virQEMUDriver *driver, virHugeTLBFS *hugepage); + +qemuNbdkitCaps* qemuGetNbdkitCaps(virQEMUDriver *driver); diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index bfeddc7746..5a2eb4868a 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -882,6 +882,7 @@ qemuDomainStorageSourcePrivateDispose(void *obj) g_clear_pointer(&priv->httpcookie, qemuDomainSecretInfoFree); g_clear_pointer(&priv->tlsKeySecret, qemuDomainSecretInfoFree); g_clear_pointer(&priv->fdpass, qemuFDPassFree); + g_clear_pointer(&priv->nbdkitProcess, qemuNbdkitProcessFree); } @@ -10471,6 +10472,34 @@ qemuDomainPrepareStorageSourceNFS(virStorageSource *src) } +/* qemuPrepareStorageSourceNbdkit: + * @src: source for a disk + * + * If src is an network source that is managed by nbdkit, prepare data so that + * nbdkit can be launched before the domain is started + * + * Returns true if nbdkit will be used for this source, + */ +static bool +qemuDomainPrepareStorageSourceNbdkit(virStorageSource *src, + virQEMUDriverConfig *cfg, + const char *alias, + qemuDomainObjPrivate *priv) +{ + g_autoptr(qemuNbdkitCaps) nbdkit = NULL; + + if (virStorageSourceGetActualType(src) != VIR_STORAGE_TYPE_NETWORK) + return false; + + nbdkit = qemuGetNbdkitCaps(priv->driver); + if (!nbdkit) + return false; + + return qemuNbdkitInitStorageSource(nbdkit, src, priv->libDir, + alias, cfg->user, cfg->group); +} + + /* qemuProcessPrepareStorageSourceTLS: * @source: source for a disk * @cfg: driver configuration @@ -11300,6 +11329,8 @@ qemuDomainPrepareStorageSourceBlockdevNodename(virDomainDiskDef *disk, if (qemuDomainPrepareStorageSourceFDs(src, priv) < 0) return -1; + qemuDomainPrepareStorageSourceNbdkit(src, cfg, src->nodestorage, priv); + return 0; } diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 786f239495..89edc75fcf 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -33,6 +33,7 @@ #include "qemu_conf.h" #include "qemu_capabilities.h" #include "qemu_migration_params.h" +#include "qemu_nbdkit.h" #include "qemu_slirp.h" #include "qemu_fd.h" #include "virchrdev.h" @@ -308,6 +309,9 @@ struct _qemuDomainStorageSourcePrivate { /* file descriptors if user asks for FDs to be passed */ qemuFDPass *fdpass; + + /* an nbdkit process for serving network storage sources */ + qemuNbdkitProcess *nbdkitProcess; }; virObject *qemuDomainStorageSourcePrivateNew(void); diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 97612b637f..12c721f7f1 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -549,3 +549,85 @@ qemuNbdkitCapsCacheNew(const char *cachedir) g_autofree char *dir = g_build_filename(cachedir, "nbdkitcapabilities", NULL); return virFileCacheNew(dir, "xml", &nbdkitCapsCacheHandlers); } + + +static qemuNbdkitProcess * +qemuNbdkitProcessNew(virStorageSource *source, + const char *pidfile, + const char *socketfile) +{ + qemuNbdkitProcess *nbdkit = g_new0(qemuNbdkitProcess, 1); + /* weak reference -- source owns this object, so it will always outlive us */ + nbdkit->source = source; + nbdkit->user = -1; + nbdkit->group = -1; + nbdkit->pid = -1; + nbdkit->pidfile = g_strdup(pidfile); + nbdkit->socketfile = g_strdup(socketfile); + + return nbdkit; +} + + +bool +qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, + virStorageSource *source, + char *statedir, + const char *alias, + uid_t user, + gid_t group) +{ + qemuDomainStorageSourcePrivate *srcPriv = qemuDomainStorageSourcePrivateFetch(source); + g_autofree char *pidname = g_strdup_printf("nbdkit-%s.pid", alias); + g_autofree char *socketname = g_strdup_printf("nbdkit-%s.socket", alias); + g_autofree char *pidfile = g_build_filename(statedir, pidname, NULL); + g_autofree char *socketfile = g_build_filename(statedir, socketname, NULL); + qemuNbdkitProcess *proc; + + if (srcPriv->nbdkitProcess) + return false; + + switch (source->protocol) { + case VIR_STORAGE_NET_PROTOCOL_HTTP: + case VIR_STORAGE_NET_PROTOCOL_HTTPS: + case VIR_STORAGE_NET_PROTOCOL_FTP: + case VIR_STORAGE_NET_PROTOCOL_FTPS: + case VIR_STORAGE_NET_PROTOCOL_TFTP: + if (!virBitmapIsBitSet(caps->flags, QEMU_NBDKIT_CAPS_PLUGIN_CURL)) + return false; + break; + case VIR_STORAGE_NET_PROTOCOL_SSH: + if (!virBitmapIsBitSet(caps->flags, QEMU_NBDKIT_CAPS_PLUGIN_SSH)) + return false; + break; + case VIR_STORAGE_NET_PROTOCOL_NONE: + case VIR_STORAGE_NET_PROTOCOL_NBD: + case VIR_STORAGE_NET_PROTOCOL_RBD: + case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG: + case VIR_STORAGE_NET_PROTOCOL_GLUSTER: + case VIR_STORAGE_NET_PROTOCOL_ISCSI: + case VIR_STORAGE_NET_PROTOCOL_VXHS: + case VIR_STORAGE_NET_PROTOCOL_NFS: + case VIR_STORAGE_NET_PROTOCOL_LAST: + return false; + } + + proc = qemuNbdkitProcessNew(source, pidfile, socketfile); + proc->caps = g_object_ref(caps); + proc->user = user; + proc->group = group; + + srcPriv->nbdkitProcess = proc; + + return true; +} + + +void +qemuNbdkitProcessFree(qemuNbdkitProcess *proc) +{ + g_clear_pointer(&proc->pidfile, g_free); + g_clear_pointer(&proc->socketfile, g_free); + g_clear_object(&proc->caps); + g_free(proc); +} diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index 4aba7c8455..8844bba13c 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -20,10 +20,12 @@ #pragma once #include "internal.h" +#include "storage_source_conf.h" #include "virenum.h" #include "virfilecache.h" typedef struct _qemuNbdkitCaps qemuNbdkitCaps; +typedef struct _qemuNbdkitProcess qemuNbdkitProcess; typedef enum { /* 0 */ @@ -42,6 +44,14 @@ qemuNbdkitCapsNew(const char *path); virFileCache * qemuNbdkitCapsCacheNew(const char *cachedir); +bool +qemuNbdkitInitStorageSource(qemuNbdkitCaps *nbdkitCaps, + virStorageSource *source, + char *statedir, + const char *alias, + uid_t user, + gid_t group); + bool qemuNbdkitCapsGet(qemuNbdkitCaps *nbdkitCaps, qemuNbdkitCapsFlags flag); @@ -52,3 +62,19 @@ qemuNbdkitCapsSet(qemuNbdkitCaps *nbdkitCaps, #define QEMU_TYPE_NBDKIT_CAPS qemu_nbdkit_caps_get_type() G_DECLARE_FINAL_TYPE(qemuNbdkitCaps, qemu_nbdkit_caps, QEMU, NBDKIT_CAPS, GObject); + +struct _qemuNbdkitProcess { + qemuNbdkitCaps *caps; + virStorageSource *source; + + char *pidfile; + char *socketfile; + uid_t user; + gid_t group; + pid_t pid; +}; + +void +qemuNbdkitProcessFree(qemuNbdkitProcess *proc); + +G_DEFINE_AUTOPTR_CLEANUP_FUNC(qemuNbdkitProcess, qemuNbdkitProcessFree); -- 2.41.0

Rather than hard-coding the nbdkit module directory, query the nbdkit binary for the location to these directories. nbdkit provides a --dump-config optiont that outputs this information and can be easily parsed. We can also get the version from this output rather than executing `nbdkit --version` separately. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_nbdkit.c | 77 ++++++++++++++++++++++++++++++++---------- 1 file changed, 60 insertions(+), 17 deletions(-) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 12c721f7f1..9a2a89224d 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -20,7 +20,6 @@ #include <config.h> #include <glib.h> -#include "configmake.h" #include "vircommand.h" #include "virerror.h" #include "virlog.h" @@ -39,10 +38,6 @@ VIR_LOG_INIT("qemu.nbdkit"); -#define NBDKIT_MODDIR LIBDIR "/nbdkit" -#define NBDKIT_PLUGINDIR NBDKIT_MODDIR "/plugins" -#define NBDKIT_FILTERDIR NBDKIT_MODDIR "/filters" - VIR_ENUM_IMPL(qemuNbdkitCaps, QEMU_NBDKIT_CAPS_LAST, /* 0 */ @@ -56,6 +51,9 @@ struct _qemuNbdkitCaps { char *path; char *version; + char *filterDir; + char *pluginDir; + time_t ctime; time_t libvirtCtime; time_t pluginDirMtime; @@ -129,18 +127,47 @@ qemuNbdkitCapsQueryFilters(qemuNbdkitCaps *nbdkit) static int -qemuNbdkitCapsQueryVersion(qemuNbdkitCaps *nbdkit) +qemuNbdkitCapsQueryBuildConfig(qemuNbdkitCaps *nbdkit) { + size_t i; + g_autofree char *output = NULL; + g_auto(GStrv) lines = NULL; + const char *line; g_autoptr(virCommand) cmd = virCommandNewArgList(nbdkit->path, - "--version", + "--dump-config", NULL); - virCommandSetOutputBuffer(cmd, &nbdkit->version); + virCommandSetOutputBuffer(cmd, &output); if (virCommandRun(cmd, NULL) != 0) return -1; - VIR_DEBUG("Got nbdkit version %s", nbdkit->version); + lines = g_strsplit(output, "\n", 0); + if (!lines) + return -1; + + for (i = 0; (line = lines[i]); i++) { + const char *key; + const char *val; + char *p; + + p = strchr(line, '='); + if (!p) + continue; + + *p = '\0'; + key = line; + val = p + 1; + + VIR_DEBUG("Got nbdkit config value %s=%s", key, val); + + if (STREQ(key, "version")) + nbdkit->version = g_strdup(val); + else if (STREQ(key, "filterdir")) + nbdkit->filterDir = g_strdup(val); + else if (STREQ(key, "plugindir")) + nbdkit->pluginDir = g_strdup(val); + } return 0; } @@ -152,6 +179,8 @@ qemuNbdkitCapsFinalize(GObject *object) g_clear_pointer(&nbdkit->path, g_free); g_clear_pointer(&nbdkit->version, g_free); + g_clear_pointer(&nbdkit->filterDir, g_free); + g_clear_pointer(&nbdkit->pluginDir, g_free); g_clear_pointer(&nbdkit->flags, virBitmapFree); G_OBJECT_CLASS(qemu_nbdkit_caps_parent_class)->finalize(object); @@ -214,15 +243,15 @@ qemuNbdkitCapsQuery(qemuNbdkitCaps *caps) return; } + qemuNbdkitCapsQueryBuildConfig(caps); + qemuNbdkitCapsQueryPlugins(caps); + qemuNbdkitCapsQueryFilters(caps); + caps->ctime = st.st_ctime; - caps->filterDirMtime = qemuNbdkitGetDirMtime(NBDKIT_FILTERDIR); - caps->pluginDirMtime = qemuNbdkitGetDirMtime(NBDKIT_PLUGINDIR); + caps->filterDirMtime = qemuNbdkitGetDirMtime(caps->filterDir); + caps->pluginDirMtime = qemuNbdkitGetDirMtime(caps->pluginDir); caps->libvirtCtime = virGetSelfLastChanged(); caps->libvirtVersion = LIBVIR_VERSION_NUMBER; - - qemuNbdkitCapsQueryPlugins(caps); - qemuNbdkitCapsQueryFilters(caps); - qemuNbdkitCapsQueryVersion(caps); } @@ -267,9 +296,9 @@ virNbdkitCapsIsValid(void *data, if (!nbdkitCaps->path) return true; - if (!virNbkditCapsCheckModdir(NBDKIT_PLUGINDIR, nbdkitCaps->pluginDirMtime)) + if (!virNbkditCapsCheckModdir(nbdkitCaps->pluginDir, nbdkitCaps->pluginDirMtime)) return false; - if (!virNbkditCapsCheckModdir(NBDKIT_FILTERDIR, nbdkitCaps->filterDirMtime)) + if (!virNbkditCapsCheckModdir(nbdkitCaps->filterDir, nbdkitCaps->filterDirMtime)) return false; if (nbdkitCaps->libvirtCtime != virGetSelfLastChanged() || @@ -421,12 +450,22 @@ qemuNbdkitCapsLoadCache(qemuNbdkitCaps *nbdkitCaps, } nbdkitCaps->ctime = (time_t)l; + if ((nbdkitCaps->pluginDir = virXPathString("string(./plugindir)", ctxt)) == NULL) { + VIR_DEBUG("missing plugindir in nbdkit capabilities cache"); + return -1; + } + if (virXPathLongLong("string(./plugindirmtime)", ctxt, &l) < 0) { VIR_DEBUG("missing plugindirmtime in nbdkit capabilities XML"); return -1; } nbdkitCaps->pluginDirMtime = (time_t)l; + if ((nbdkitCaps->filterDir = virXPathString("string(./filterdir)", ctxt)) == NULL) { + VIR_DEBUG("missing filterdir in nbdkit capabilities cache"); + return -1; + } + if (virXPathLongLong("string(./filterdirmtime)", ctxt, &l) < 0) { VIR_DEBUG("missing filterdirmtime in nbdkit capabilities XML"); return -1; @@ -482,8 +521,12 @@ qemuNbdkitCapsFormatCache(qemuNbdkitCaps *nbdkitCaps) nbdkitCaps->path); virBufferAsprintf(&buf, "<nbdkitctime>%lu</nbdkitctime>\n", nbdkitCaps->ctime); + virBufferEscapeString(&buf, "<plugindir>%s</plugindir>\n", + nbdkitCaps->pluginDir); virBufferAsprintf(&buf, "<plugindirmtime>%lu</plugindirmtime>\n", nbdkitCaps->pluginDirMtime); + virBufferEscapeString(&buf, "<filterdir>%s</filterdir>\n", + nbdkitCaps->filterDir); virBufferAsprintf(&buf, "<filterdirmtime>%lu</filterdirmtime>\n", nbdkitCaps->filterDirMtime); virBufferAsprintf(&buf, "<selfctime>%lu</selfctime>\n", -- 2.41.0

Add some helper functions to build a virCommand object and run the nbdkit process for a given virStorageSource. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_nbdkit.c | 250 +++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_nbdkit.h | 10 ++ 2 files changed, 260 insertions(+) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 9a2a89224d..6bf962d0f1 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -24,6 +24,8 @@ #include "virerror.h" #include "virlog.h" #include "virpidfile.h" +#include "virsecureerase.h" +#include "virtime.h" #include "virutil.h" #include "qemu_block.h" #include "qemu_conf.h" @@ -666,6 +668,168 @@ qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, } +static int +qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *proc, + virCommand *cmd) +{ + g_autoptr(virURI) uri = qemuBlockStorageSourceGetURI(proc->source); + g_autofree char *uristring = virURIFormat(uri); + + /* nbdkit plugin name */ + virCommandAddArg(cmd, "curl"); + if (proc->source->protocol == VIR_STORAGE_NET_PROTOCOL_HTTP) { + /* allow http to be upgraded to https via e.g. redirect */ + virCommandAddArgPair(cmd, "protocols", "http,https"); + } else { + virCommandAddArgPair(cmd, "protocols", + virStorageNetProtocolTypeToString(proc->source->protocol)); + } + virCommandAddArgPair(cmd, "url", uristring); + + if (proc->source->auth) { + g_autoptr(virConnect) conn = virGetConnectSecret(); + g_autofree uint8_t *secret = NULL; + size_t secretlen = 0; + g_autofree char *password = NULL; + int secrettype; + virStorageAuthDef *authdef = proc->source->auth; + + virCommandAddArgPair(cmd, "user", + proc->source->auth->username); + + if ((secrettype = virSecretUsageTypeFromString(proc->source->auth->secrettype)) < 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("invalid secret type %1$s"), + proc->source->auth->secrettype); + return -1; + } + + if (virSecretGetSecretString(conn, + &authdef->seclookupdef, + secrettype, + &secret, + &secretlen) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("failed to get auth secret for storage")); + return -1; + } + + /* ensure that the secret is a NULL-terminated string */ + password = g_strndup((char*)secret, secretlen); + virSecureErase(secret, secretlen); + + /* for now, just report an error rather than passing the password in + * cleartext on the commandline */ + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Password not yet supported for nbdkit sources")); + + virSecureEraseString(password); + + return -1; + } + + if (proc->source->ncookies > 0) { + /* for now, just report an error rather than passing cookies in + * cleartext on the commandline */ + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Cookies not yet supported for nbdkit sources")); + return -1; + } + + if (proc->source->sslverify == VIR_TRISTATE_BOOL_NO) { + virCommandAddArgPair(cmd, "sslverify", "false"); + } + + if (proc->source->timeout > 0) { + g_autofree char *timeout = g_strdup_printf("%llu", proc->source->timeout); + virCommandAddArgPair(cmd, "timeout", timeout); + } + + return 0; +} + + +static int +qemuNbdkitProcessBuildCommandSSH(qemuNbdkitProcess *proc, + virCommand *cmd) +{ + const char *user = NULL; + virStorageNetHostDef *host = &proc->source->hosts[0]; + g_autofree char *portstr = g_strdup_printf("%u", host->port); + + /* nbdkit plugin name */ + virCommandAddArg(cmd, "ssh"); + + virCommandAddArgPair(cmd, "host", host->name); + virCommandAddArgPair(cmd, "port", portstr); + virCommandAddArgPair(cmd, "path", proc->source->path); + + if (proc->source->auth) + user = proc->source->auth->username; + else if (proc->source->ssh_user) + user = proc->source->ssh_user; + + if (user) + virCommandAddArgPair(cmd, "user", user); + + if (proc->source->ssh_host_key_check_disabled) + virCommandAddArgPair(cmd, "verify-remote-host", "false"); + + return 0; +} + + +static virCommand * +qemuNbdkitProcessBuildCommand(qemuNbdkitProcess *proc) +{ + g_autoptr(virCommand) cmd = virCommandNewArgList(proc->caps->path, + "--unix", + proc->socketfile, + "--foreground", + NULL); + + if (proc->source->readonly) + virCommandAddArg(cmd, "--readonly"); + + if (qemuNbdkitCapsGet(proc->caps, QEMU_NBDKIT_CAPS_FILTER_READAHEAD) && + proc->source->readahead > 0) + virCommandAddArgPair(cmd, "--filter", "readahead"); + + switch (proc->source->protocol) { + case VIR_STORAGE_NET_PROTOCOL_HTTP: + case VIR_STORAGE_NET_PROTOCOL_HTTPS: + case VIR_STORAGE_NET_PROTOCOL_FTP: + case VIR_STORAGE_NET_PROTOCOL_FTPS: + case VIR_STORAGE_NET_PROTOCOL_TFTP: + if (qemuNbdkitProcessBuildCommandCurl(proc, cmd) < 0) + return NULL; + break; + case VIR_STORAGE_NET_PROTOCOL_SSH: + if (qemuNbdkitProcessBuildCommandSSH(proc, cmd) < 0) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_NONE: + case VIR_STORAGE_NET_PROTOCOL_NBD: + case VIR_STORAGE_NET_PROTOCOL_RBD: + case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG: + case VIR_STORAGE_NET_PROTOCOL_GLUSTER: + case VIR_STORAGE_NET_PROTOCOL_ISCSI: + case VIR_STORAGE_NET_PROTOCOL_VXHS: + case VIR_STORAGE_NET_PROTOCOL_NFS: + case VIR_STORAGE_NET_PROTOCOL_LAST: + virReportError(VIR_ERR_NO_SUPPORT, + _("protocol '%1$s' is not supported by nbdkit"), + virStorageNetProtocolTypeToString(proc->source->protocol)); + return NULL; + } + + virCommandDaemonize(cmd); + + return g_steal_pointer(&cmd); +} + + void qemuNbdkitProcessFree(qemuNbdkitProcess *proc) { @@ -674,3 +838,89 @@ qemuNbdkitProcessFree(qemuNbdkitProcess *proc) g_clear_object(&proc->caps); g_free(proc); } + + +int +qemuNbdkitProcessStart(qemuNbdkitProcess *proc, + virDomainObj *vm, + virQEMUDriver *driver) +{ + g_autoptr(virCommand) cmd = NULL; + int rc; + int exitstatus = 0; + g_autofree char *errbuf = NULL; + virTimeBackOffVar timebackoff; + g_autoptr(virURI) uri = NULL; + g_autofree char *uristring = NULL; + + if (!(cmd = qemuNbdkitProcessBuildCommand(proc))) + return -1; + + VIR_DEBUG("starting nbdkit process for %s", proc->source->nodestorage); + virCommandSetErrorBuffer(cmd, &errbuf); + virCommandSetPidFile(cmd, proc->pidfile); + + if (qemuExtDeviceLogCommand(driver, vm, cmd, "nbdkit") < 0) + goto error; + + if (qemuSecurityCommandRun(driver, vm, cmd, proc->user, proc->group, true, &exitstatus) < 0) + goto error; + + if (exitstatus != 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Could not start 'nbdkit'. exitstatus: %1$d"), exitstatus); + goto error; + } + + if ((rc = virPidFileReadPath(proc->pidfile, &proc->pid)) < 0) { + virReportSystemError(-rc, + _("Failed to read pidfile %1$s"), + proc->pidfile); + goto error; + } + + if (virTimeBackOffStart(&timebackoff, 1, 1000) < 0) + goto error; + + while (virTimeBackOffWait(&timebackoff)) { + if (virFileExists(proc->socketfile)) + return 0; + + if (virProcessKill(proc->pid, 0) == 0) + continue; + + VIR_WARN("nbdkit died unexpectedly"); + goto errorlog; + } + + VIR_WARN("nbdkit socket did not show up"); + + errorlog: + if ((uri = qemuBlockStorageSourceGetURI(proc->source))) + uristring = virURIFormat(uri); + + virReportError(VIR_ERR_OPERATION_FAILED, + _("Failed to connect to nbdkit for '%1$s': %2$s"), + NULLSTR(uristring), NULLSTR(errbuf)); + + error: + qemuNbdkitProcessStop(proc); + return -1; +} + + +int +qemuNbdkitProcessStop(qemuNbdkitProcess *proc) +{ + if (proc->pid < 0) + return 0; + + VIR_DEBUG("Stopping nbdkit process %i", proc->pid); + virProcessKill(proc->pid, SIGTERM); + + unlink(proc->pidfile); + unlink(proc->socketfile); + proc->pid = -1; + + return 0; +} diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index 8844bba13c..ccd418b7d3 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -38,6 +38,8 @@ typedef enum { VIR_ENUM_DECL(qemuNbdkitCaps); +typedef struct _virQEMUDriver virQEMUDriver; + qemuNbdkitCaps * qemuNbdkitCapsNew(const char *path); @@ -74,6 +76,14 @@ struct _qemuNbdkitProcess { pid_t pid; }; +int +qemuNbdkitProcessStart(qemuNbdkitProcess *proc, + virDomainObj *vm, + virQEMUDriver *driver); + +int +qemuNbdkitProcessStop(qemuNbdkitProcess *proc); + void qemuNbdkitProcessFree(qemuNbdkitProcess *proc); -- 2.41.0

On Thu, Aug 31, 2023 at 04:39:50PM -0500, Jonathon Jongsma wrote:
Add some helper functions to build a virCommand object and run the nbdkit process for a given virStorageSource.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_nbdkit.c | 250 +++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_nbdkit.h | 10 ++ 2 files changed, 260 insertions(+)
diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 9a2a89224d..6bf962d0f1 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -24,6 +24,8 @@ #include "virerror.h" #include "virlog.h" #include "virpidfile.h" +#include "virsecureerase.h" +#include "virtime.h" #include "virutil.h" #include "qemu_block.h" #include "qemu_conf.h" @@ -666,6 +668,168 @@ qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, }
+static int +qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *proc, + virCommand *cmd) +{ + g_autoptr(virURI) uri = qemuBlockStorageSourceGetURI(proc->source); + g_autofree char *uristring = virURIFormat(uri); + + /* nbdkit plugin name */ + virCommandAddArg(cmd, "curl"); + if (proc->source->protocol == VIR_STORAGE_NET_PROTOCOL_HTTP) { + /* allow http to be upgraded to https via e.g. redirect */ + virCommandAddArgPair(cmd, "protocols", "http,https"); + } else { + virCommandAddArgPair(cmd, "protocols", + virStorageNetProtocolTypeToString(proc->source->protocol)); + } + virCommandAddArgPair(cmd, "url", uristring); + + if (proc->source->auth) { + g_autoptr(virConnect) conn = virGetConnectSecret(); + g_autofree uint8_t *secret = NULL; + size_t secretlen = 0; + g_autofree char *password = NULL; + int secrettype; + virStorageAuthDef *authdef = proc->source->auth; + + virCommandAddArgPair(cmd, "user", + proc->source->auth->username); + + if ((secrettype = virSecretUsageTypeFromString(proc->source->auth->secrettype)) < 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("invalid secret type %1$s"), + proc->source->auth->secrettype); + return -1; + } + + if (virSecretGetSecretString(conn, + &authdef->seclookupdef, + secrettype, + &secret, + &secretlen) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("failed to get auth secret for storage")); + return -1; + } + + /* ensure that the secret is a NULL-terminated string */ + password = g_strndup((char*)secret, secretlen); + virSecureErase(secret, secretlen); + + /* for now, just report an error rather than passing the password in + * cleartext on the commandline */ + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Password not yet supported for nbdkit sources")); + + virSecureEraseString(password); + + return -1; + } + + if (proc->source->ncookies > 0) { + /* for now, just report an error rather than passing cookies in + * cleartext on the commandline */ + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Cookies not yet supported for nbdkit sources")); + return -1; + } + + if (proc->source->sslverify == VIR_TRISTATE_BOOL_NO) { + virCommandAddArgPair(cmd, "sslverify", "false"); + } + + if (proc->source->timeout > 0) { + g_autofree char *timeout = g_strdup_printf("%llu", proc->source->timeout); + virCommandAddArgPair(cmd, "timeout", timeout); + } + + return 0; +} + + +static int +qemuNbdkitProcessBuildCommandSSH(qemuNbdkitProcess *proc, + virCommand *cmd) +{ + const char *user = NULL; + virStorageNetHostDef *host = &proc->source->hosts[0]; + g_autofree char *portstr = g_strdup_printf("%u", host->port); + + /* nbdkit plugin name */ + virCommandAddArg(cmd, "ssh"); + + virCommandAddArgPair(cmd, "host", host->name); + virCommandAddArgPair(cmd, "port", portstr); + virCommandAddArgPair(cmd, "path", proc->source->path); + + if (proc->source->auth) + user = proc->source->auth->username; + else if (proc->source->ssh_user) + user = proc->source->ssh_user; + + if (user) + virCommandAddArgPair(cmd, "user", user); + + if (proc->source->ssh_host_key_check_disabled) + virCommandAddArgPair(cmd, "verify-remote-host", "false"); + + return 0; +} + + +static virCommand * +qemuNbdkitProcessBuildCommand(qemuNbdkitProcess *proc) +{ + g_autoptr(virCommand) cmd = virCommandNewArgList(proc->caps->path, + "--unix", + proc->socketfile, + "--foreground", + NULL); + + if (proc->source->readonly) + virCommandAddArg(cmd, "--readonly"); + + if (qemuNbdkitCapsGet(proc->caps, QEMU_NBDKIT_CAPS_FILTER_READAHEAD) && + proc->source->readahead > 0) + virCommandAddArgPair(cmd, "--filter", "readahead"); + + switch (proc->source->protocol) { + case VIR_STORAGE_NET_PROTOCOL_HTTP: + case VIR_STORAGE_NET_PROTOCOL_HTTPS: + case VIR_STORAGE_NET_PROTOCOL_FTP: + case VIR_STORAGE_NET_PROTOCOL_FTPS: + case VIR_STORAGE_NET_PROTOCOL_TFTP: + if (qemuNbdkitProcessBuildCommandCurl(proc, cmd) < 0) + return NULL; + break; + case VIR_STORAGE_NET_PROTOCOL_SSH: + if (qemuNbdkitProcessBuildCommandSSH(proc, cmd) < 0) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_NONE: + case VIR_STORAGE_NET_PROTOCOL_NBD: + case VIR_STORAGE_NET_PROTOCOL_RBD: + case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG: + case VIR_STORAGE_NET_PROTOCOL_GLUSTER: + case VIR_STORAGE_NET_PROTOCOL_ISCSI: + case VIR_STORAGE_NET_PROTOCOL_VXHS: + case VIR_STORAGE_NET_PROTOCOL_NFS: + case VIR_STORAGE_NET_PROTOCOL_LAST: + virReportError(VIR_ERR_NO_SUPPORT, + _("protocol '%1$s' is not supported by nbdkit"), + virStorageNetProtocolTypeToString(proc->source->protocol)); + return NULL; + } + + virCommandDaemonize(cmd); + + return g_steal_pointer(&cmd); +} + + void qemuNbdkitProcessFree(qemuNbdkitProcess *proc) { @@ -674,3 +838,89 @@ qemuNbdkitProcessFree(qemuNbdkitProcess *proc) g_clear_object(&proc->caps); g_free(proc); } + + +int +qemuNbdkitProcessStart(qemuNbdkitProcess *proc, + virDomainObj *vm, + virQEMUDriver *driver) +{ + g_autoptr(virCommand) cmd = NULL; + int rc; + int exitstatus = 0; + g_autofree char *errbuf = NULL; + virTimeBackOffVar timebackoff; + g_autoptr(virURI) uri = NULL; + g_autofree char *uristring = NULL; + + if (!(cmd = qemuNbdkitProcessBuildCommand(proc))) + return -1; + + VIR_DEBUG("starting nbdkit process for %s", proc->source->nodestorage); + virCommandSetErrorBuffer(cmd, &errbuf); + virCommandSetPidFile(cmd, proc->pidfile); + + if (qemuExtDeviceLogCommand(driver, vm, cmd, "nbdkit") < 0) + goto error; + + if (qemuSecurityCommandRun(driver, vm, cmd, proc->user, proc->group, true, &exitstatus) < 0) + goto error; + + if (exitstatus != 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Could not start 'nbdkit'. exitstatus: %1$d"), exitstatus); + goto error; + } + + if ((rc = virPidFileReadPath(proc->pidfile, &proc->pid)) < 0) { + virReportSystemError(-rc, + _("Failed to read pidfile %1$s"), + proc->pidfile); + goto error; + } + + if (virTimeBackOffStart(&timebackoff, 1, 1000) < 0) + goto error; + + while (virTimeBackOffWait(&timebackoff)) { + if (virFileExists(proc->socketfile)) + return 0; + + if (virProcessKill(proc->pid, 0) == 0) + continue; + + VIR_WARN("nbdkit died unexpectedly"); + goto errorlog; + } + + VIR_WARN("nbdkit socket did not show up"); + + errorlog: + if ((uri = qemuBlockStorageSourceGetURI(proc->source))) + uristring = virURIFormat(uri); + + virReportError(VIR_ERR_OPERATION_FAILED, + _("Failed to connect to nbdkit for '%1$s': %2$s"), + NULLSTR(uristring), NULLSTR(errbuf)); + + error: + qemuNbdkitProcessStop(proc); + return -1; +} + + +int +qemuNbdkitProcessStop(qemuNbdkitProcess *proc) +{ + if (proc->pid < 0) + return 0; + + VIR_DEBUG("Stopping nbdkit process %i", proc->pid); + virProcessKill(proc->pid, SIGTERM);
Coverity complains here that the return value of virProcessKill() is not checked which leads me to a question if we should use virProcessKillPainfully() instead. With the code that is pushed the function qemuNbdkitProcessStop() is called only within the qemu_nbdkit.c for these cases: - in qemuNbdkitProcessRestart() before starting the process again where we do not check if the original process was killed correctly or not, - in qemuNbdkitStopStorageSource() where we check return value of qemuNbdkitProcessStop() but it will always be 0, - in qemuNbdkitProcessStart() as error path where we don't check any return value. To me it seems that the return value qemuNbdkitProcessStop can be changed to void as we always return 0 and use virProcessKillPainfully() or properly pass return value of virProcessKill() and check it for every use of qemuNbdkitProcessStop(). Pavel
+ + unlink(proc->pidfile); + unlink(proc->socketfile); + proc->pid = -1; + + return 0; +} diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index 8844bba13c..ccd418b7d3 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -38,6 +38,8 @@ typedef enum {
VIR_ENUM_DECL(qemuNbdkitCaps);
+typedef struct _virQEMUDriver virQEMUDriver; + qemuNbdkitCaps * qemuNbdkitCapsNew(const char *path);
@@ -74,6 +76,14 @@ struct _qemuNbdkitProcess { pid_t pid; };
+int +qemuNbdkitProcessStart(qemuNbdkitProcess *proc, + virDomainObj *vm, + virQEMUDriver *driver); + +int +qemuNbdkitProcessStop(qemuNbdkitProcess *proc); + void qemuNbdkitProcessFree(qemuNbdkitProcess *proc);
-- 2.41.0

On 9/20/23 7:24 AM, Pavel Hrdina wrote:
On Thu, Aug 31, 2023 at 04:39:50PM -0500, Jonathon Jongsma wrote:
Add some helper functions to build a virCommand object and run the nbdkit process for a given virStorageSource.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_nbdkit.c | 250 +++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_nbdkit.h | 10 ++ 2 files changed, 260 insertions(+)
diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 9a2a89224d..6bf962d0f1 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -24,6 +24,8 @@ #include "virerror.h" #include "virlog.h" #include "virpidfile.h" +#include "virsecureerase.h" +#include "virtime.h" #include "virutil.h" #include "qemu_block.h" #include "qemu_conf.h" @@ -666,6 +668,168 @@ qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, }
+static int +qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *proc, + virCommand *cmd) +{ + g_autoptr(virURI) uri = qemuBlockStorageSourceGetURI(proc->source); + g_autofree char *uristring = virURIFormat(uri); + + /* nbdkit plugin name */ + virCommandAddArg(cmd, "curl"); + if (proc->source->protocol == VIR_STORAGE_NET_PROTOCOL_HTTP) { + /* allow http to be upgraded to https via e.g. redirect */ + virCommandAddArgPair(cmd, "protocols", "http,https"); + } else { + virCommandAddArgPair(cmd, "protocols", + virStorageNetProtocolTypeToString(proc->source->protocol)); + } + virCommandAddArgPair(cmd, "url", uristring); + + if (proc->source->auth) { + g_autoptr(virConnect) conn = virGetConnectSecret(); + g_autofree uint8_t *secret = NULL; + size_t secretlen = 0; + g_autofree char *password = NULL; + int secrettype; + virStorageAuthDef *authdef = proc->source->auth; + + virCommandAddArgPair(cmd, "user", + proc->source->auth->username); + + if ((secrettype = virSecretUsageTypeFromString(proc->source->auth->secrettype)) < 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("invalid secret type %1$s"), + proc->source->auth->secrettype); + return -1; + } + + if (virSecretGetSecretString(conn, + &authdef->seclookupdef, + secrettype, + &secret, + &secretlen) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("failed to get auth secret for storage")); + return -1; + } + + /* ensure that the secret is a NULL-terminated string */ + password = g_strndup((char*)secret, secretlen); + virSecureErase(secret, secretlen); + + /* for now, just report an error rather than passing the password in + * cleartext on the commandline */ + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Password not yet supported for nbdkit sources")); + + virSecureEraseString(password); + + return -1; + } + + if (proc->source->ncookies > 0) { + /* for now, just report an error rather than passing cookies in + * cleartext on the commandline */ + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Cookies not yet supported for nbdkit sources")); + return -1; + } + + if (proc->source->sslverify == VIR_TRISTATE_BOOL_NO) { + virCommandAddArgPair(cmd, "sslverify", "false"); + } + + if (proc->source->timeout > 0) { + g_autofree char *timeout = g_strdup_printf("%llu", proc->source->timeout); + virCommandAddArgPair(cmd, "timeout", timeout); + } + + return 0; +} + + +static int +qemuNbdkitProcessBuildCommandSSH(qemuNbdkitProcess *proc, + virCommand *cmd) +{ + const char *user = NULL; + virStorageNetHostDef *host = &proc->source->hosts[0]; + g_autofree char *portstr = g_strdup_printf("%u", host->port); + + /* nbdkit plugin name */ + virCommandAddArg(cmd, "ssh"); + + virCommandAddArgPair(cmd, "host", host->name); + virCommandAddArgPair(cmd, "port", portstr); + virCommandAddArgPair(cmd, "path", proc->source->path); + + if (proc->source->auth) + user = proc->source->auth->username; + else if (proc->source->ssh_user) + user = proc->source->ssh_user; + + if (user) + virCommandAddArgPair(cmd, "user", user); + + if (proc->source->ssh_host_key_check_disabled) + virCommandAddArgPair(cmd, "verify-remote-host", "false"); + + return 0; +} + + +static virCommand * +qemuNbdkitProcessBuildCommand(qemuNbdkitProcess *proc) +{ + g_autoptr(virCommand) cmd = virCommandNewArgList(proc->caps->path, + "--unix", + proc->socketfile, + "--foreground", + NULL); + + if (proc->source->readonly) + virCommandAddArg(cmd, "--readonly"); + + if (qemuNbdkitCapsGet(proc->caps, QEMU_NBDKIT_CAPS_FILTER_READAHEAD) && + proc->source->readahead > 0) + virCommandAddArgPair(cmd, "--filter", "readahead"); + + switch (proc->source->protocol) { + case VIR_STORAGE_NET_PROTOCOL_HTTP: + case VIR_STORAGE_NET_PROTOCOL_HTTPS: + case VIR_STORAGE_NET_PROTOCOL_FTP: + case VIR_STORAGE_NET_PROTOCOL_FTPS: + case VIR_STORAGE_NET_PROTOCOL_TFTP: + if (qemuNbdkitProcessBuildCommandCurl(proc, cmd) < 0) + return NULL; + break; + case VIR_STORAGE_NET_PROTOCOL_SSH: + if (qemuNbdkitProcessBuildCommandSSH(proc, cmd) < 0) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_NONE: + case VIR_STORAGE_NET_PROTOCOL_NBD: + case VIR_STORAGE_NET_PROTOCOL_RBD: + case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG: + case VIR_STORAGE_NET_PROTOCOL_GLUSTER: + case VIR_STORAGE_NET_PROTOCOL_ISCSI: + case VIR_STORAGE_NET_PROTOCOL_VXHS: + case VIR_STORAGE_NET_PROTOCOL_NFS: + case VIR_STORAGE_NET_PROTOCOL_LAST: + virReportError(VIR_ERR_NO_SUPPORT, + _("protocol '%1$s' is not supported by nbdkit"), + virStorageNetProtocolTypeToString(proc->source->protocol)); + return NULL; + } + + virCommandDaemonize(cmd); + + return g_steal_pointer(&cmd); +} + + void qemuNbdkitProcessFree(qemuNbdkitProcess *proc) { @@ -674,3 +838,89 @@ qemuNbdkitProcessFree(qemuNbdkitProcess *proc) g_clear_object(&proc->caps); g_free(proc); } + + +int +qemuNbdkitProcessStart(qemuNbdkitProcess *proc, + virDomainObj *vm, + virQEMUDriver *driver) +{ + g_autoptr(virCommand) cmd = NULL; + int rc; + int exitstatus = 0; + g_autofree char *errbuf = NULL; + virTimeBackOffVar timebackoff; + g_autoptr(virURI) uri = NULL; + g_autofree char *uristring = NULL; + + if (!(cmd = qemuNbdkitProcessBuildCommand(proc))) + return -1; + + VIR_DEBUG("starting nbdkit process for %s", proc->source->nodestorage); + virCommandSetErrorBuffer(cmd, &errbuf); + virCommandSetPidFile(cmd, proc->pidfile); + + if (qemuExtDeviceLogCommand(driver, vm, cmd, "nbdkit") < 0) + goto error; + + if (qemuSecurityCommandRun(driver, vm, cmd, proc->user, proc->group, true, &exitstatus) < 0) + goto error; + + if (exitstatus != 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Could not start 'nbdkit'. exitstatus: %1$d"), exitstatus); + goto error; + } + + if ((rc = virPidFileReadPath(proc->pidfile, &proc->pid)) < 0) { + virReportSystemError(-rc, + _("Failed to read pidfile %1$s"), + proc->pidfile); + goto error; + } + + if (virTimeBackOffStart(&timebackoff, 1, 1000) < 0) + goto error; + + while (virTimeBackOffWait(&timebackoff)) { + if (virFileExists(proc->socketfile)) + return 0; + + if (virProcessKill(proc->pid, 0) == 0) + continue; + + VIR_WARN("nbdkit died unexpectedly"); + goto errorlog; + } + + VIR_WARN("nbdkit socket did not show up"); + + errorlog: + if ((uri = qemuBlockStorageSourceGetURI(proc->source))) + uristring = virURIFormat(uri); + + virReportError(VIR_ERR_OPERATION_FAILED, + _("Failed to connect to nbdkit for '%1$s': %2$s"), + NULLSTR(uristring), NULLSTR(errbuf)); + + error: + qemuNbdkitProcessStop(proc); + return -1; +} + + +int +qemuNbdkitProcessStop(qemuNbdkitProcess *proc) +{ + if (proc->pid < 0) + return 0; + + VIR_DEBUG("Stopping nbdkit process %i", proc->pid); + virProcessKill(proc->pid, SIGTERM);
Coverity complains here that the return value of virProcessKill() is not checked which leads me to a question if we should use virProcessKillPainfully() instead.
With the code that is pushed the function qemuNbdkitProcessStop() is called only within the qemu_nbdkit.c for these cases:
- in qemuNbdkitProcessRestart() before starting the process again where we do not check if the original process was killed correctly or not,
- in qemuNbdkitStopStorageSource() where we check return value of qemuNbdkitProcessStop() but it will always be 0,
- in qemuNbdkitProcessStart() as error path where we don't check any return value.
To me it seems that the return value qemuNbdkitProcessStop can be changed to void as we always return 0 and use virProcessKillPainfully() or properly pass return value of virProcessKill() and check it for every use of qemuNbdkitProcessStop().
Pavel
Good question. In one of my earlier series I had actually used virProcessKillPainfully(), but changed it based on a suggestion from Peter that it would be bad to kill it painfully if nbdkit was ever used in read-write mode. But apparently I forgot to handle a shutdown failure. An alternative would be to simply refuse to use nbdkit if the user requests read-write mode. Jonathon

On Thu, Sep 21, 2023 at 12:50:52 -0500, Jonathon Jongsma wrote:
On 9/20/23 7:24 AM, Pavel Hrdina wrote:
On Thu, Aug 31, 2023 at 04:39:50PM -0500, Jonathon Jongsma wrote:
Add some helper functions to build a virCommand object and run the nbdkit process for a given virStorageSource.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_nbdkit.c | 250 +++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_nbdkit.h | 10 ++ 2 files changed, 260 insertions(+)
[...]
+ VIR_DEBUG("Stopping nbdkit process %i", proc->pid); + virProcessKill(proc->pid, SIGTERM);
Coverity complains here that the return value of virProcessKill() is not checked which leads me to a question if we should use virProcessKillPainfully() instead.
With the code that is pushed the function qemuNbdkitProcessStop() is called only within the qemu_nbdkit.c for these cases:
- in qemuNbdkitProcessRestart() before starting the process again where we do not check if the original process was killed correctly or not,
- in qemuNbdkitStopStorageSource() where we check return value of qemuNbdkitProcessStop() but it will always be 0,
- in qemuNbdkitProcessStart() as error path where we don't check any return value.
To me it seems that the return value qemuNbdkitProcessStop can be changed to void as we always return 0 and use virProcessKillPainfully() or properly pass return value of virProcessKill() and check it for every use of qemuNbdkitProcessStop().
Pavel
Good question. In one of my earlier series I had actually used virProcessKillPainfully(), but changed it based on a suggestion from Peter that it would be bad to kill it painfully if nbdkit was ever used in read-write mode. But apparently I forgot to handle a shutdown failure. An alternative would be to simply refuse to use nbdkit if the user requests read-write mode.
We can use the same algorithm as with the qemu process where we first issue SIGTERM, thus if the process is responsive it can execute the shutdown actions. Otherwise it'll get SIGKILL right after.

On 9/21/23 1:10 PM, Peter Krempa wrote:
On Thu, Sep 21, 2023 at 12:50:52 -0500, Jonathon Jongsma wrote:
On 9/20/23 7:24 AM, Pavel Hrdina wrote:
On Thu, Aug 31, 2023 at 04:39:50PM -0500, Jonathon Jongsma wrote:
Add some helper functions to build a virCommand object and run the nbdkit process for a given virStorageSource.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_nbdkit.c | 250 +++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_nbdkit.h | 10 ++ 2 files changed, 260 insertions(+)
[...]
+ VIR_DEBUG("Stopping nbdkit process %i", proc->pid); + virProcessKill(proc->pid, SIGTERM);
Coverity complains here that the return value of virProcessKill() is not checked which leads me to a question if we should use virProcessKillPainfully() instead.
With the code that is pushed the function qemuNbdkitProcessStop() is called only within the qemu_nbdkit.c for these cases:
- in qemuNbdkitProcessRestart() before starting the process again where we do not check if the original process was killed correctly or not,
- in qemuNbdkitStopStorageSource() where we check return value of qemuNbdkitProcessStop() but it will always be 0,
- in qemuNbdkitProcessStart() as error path where we don't check any return value.
To me it seems that the return value qemuNbdkitProcessStop can be changed to void as we always return 0 and use virProcessKillPainfully() or properly pass return value of virProcessKill() and check it for every use of qemuNbdkitProcessStop().
Pavel
Good question. In one of my earlier series I had actually used virProcessKillPainfully(), but changed it based on a suggestion from Peter that it would be bad to kill it painfully if nbdkit was ever used in read-write mode. But apparently I forgot to handle a shutdown failure. An alternative would be to simply refuse to use nbdkit if the user requests read-write mode.
We can use the same algorithm as with the qemu process where we first issue SIGTERM, thus if the process is responsive it can execute the shutdown actions. Otherwise it'll get SIGKILL right after.
That sounds almost identical to what virProcessKillPainfully() does. Jonathon

Allow to specify a basename for the log file so that qemuDomainLogContextNew() can be used to create log contexts for secondary loggers. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_domain.c | 5 +++-- src/qemu/qemu_domain.h | 3 ++- src/qemu/qemu_process.c | 2 +- 3 files changed, 6 insertions(+), 4 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 5a2eb4868a..d79f9879df 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -7119,7 +7119,8 @@ void qemuDomainObjCheckNetTaint(virQEMUDriver *driver, qemuDomainLogContext *qemuDomainLogContextNew(virQEMUDriver *driver, - virDomainObj *vm) + virDomainObj *vm, + const char *basename) { g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); qemuDomainLogContext *ctxt = QEMU_DOMAIN_LOG_CONTEXT(g_object_new(QEMU_TYPE_DOMAIN_LOG_CONTEXT, NULL)); @@ -7128,7 +7129,7 @@ qemuDomainLogContext *qemuDomainLogContextNew(virQEMUDriver *driver, ctxt->writefd = -1; ctxt->readfd = -1; - ctxt->path = g_strdup_printf("%s/%s.log", cfg->logDir, vm->def->name); + ctxt->path = g_strdup_printf("%s/%s.log", cfg->logDir, basename); if (cfg->stdioLogD) { ctxt->manager = virLogManagerNew(driver->privileged); diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 89edc75fcf..a262555c8c 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -657,7 +657,8 @@ void qemuDomainObjCheckNetTaint(virQEMUDriver *driver, qemuDomainLogContext *logCtxt); qemuDomainLogContext *qemuDomainLogContextNew(virQEMUDriver *driver, - virDomainObj *vm); + virDomainObj *vm, + const char *basename); int qemuDomainLogContextWrite(qemuDomainLogContext *ctxt, const char *fmt, ...) G_GNUC_PRINTF(2, 3); ssize_t qemuDomainLogContextRead(qemuDomainLogContext *ctxt, diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index a6ed69cfe2..e0385d11be 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -7616,7 +7616,7 @@ qemuProcessLaunch(virConnectPtr conn, hookData.cfg = cfg; VIR_DEBUG("Creating domain log file"); - if (!(logCtxt = qemuDomainLogContextNew(driver, vm))) { + if (!(logCtxt = qemuDomainLogContextNew(driver, vm, vm->def->name))) { virLastErrorPrefixMessage("%s", _("can't connect to virtlogd")); goto cleanup; } -- 2.41.0

This will allow us to use it for nbdkit logging in upcoming commits. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- po/POTFILES | 1 + src/qemu/meson.build | 1 + src/qemu/qemu_domain.c | 247 ++-------------------------------- src/qemu/qemu_domain.h | 27 +--- src/qemu/qemu_logcontext.c | 264 +++++++++++++++++++++++++++++++++++++ src/qemu/qemu_logcontext.h | 38 ++++++ src/qemu/qemu_process.c | 44 +++---- 7 files changed, 346 insertions(+), 276 deletions(-) create mode 100644 src/qemu/qemu_logcontext.c create mode 100644 src/qemu/qemu_logcontext.h diff --git a/po/POTFILES b/po/POTFILES index 6167f98ac5..3a51aea5cb 100644 --- a/po/POTFILES +++ b/po/POTFILES @@ -174,6 +174,7 @@ src/qemu/qemu_hostdev.c src/qemu/qemu_hotplug.c src/qemu/qemu_interface.c src/qemu/qemu_interop_config.c +src/qemu/qemu_logcontext.c src/qemu/qemu_migration.c src/qemu/qemu_migration_cookie.c src/qemu/qemu_migration_params.c diff --git a/src/qemu/meson.build b/src/qemu/meson.build index 9be6996195..6d7a1bfbb0 100644 --- a/src/qemu/meson.build +++ b/src/qemu/meson.build @@ -21,6 +21,7 @@ qemu_driver_sources = [ 'qemu_hotplug.c', 'qemu_interface.c', 'qemu_interop_config.c', + 'qemu_logcontext.c', 'qemu_migration.c', 'qemu_migration_cookie.c', 'qemu_migration_params.c', diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index d79f9879df..23608f95bd 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -455,21 +455,8 @@ qemuDomainObjFromDomain(virDomainPtr domain) } -struct _qemuDomainLogContext { - GObject parent; - - int writefd; - int readfd; /* Only used if manager == NULL */ - off_t pos; - ino_t inode; /* Only used if manager != NULL */ - char *path; - virLogManager *manager; -}; - -G_DEFINE_TYPE(qemuDomainLogContext, qemu_domain_log_context, G_TYPE_OBJECT); static virClass *qemuDomainSaveCookieClass; -static void qemuDomainLogContextFinalize(GObject *obj); static void qemuDomainSaveCookieDispose(void *obj); @@ -482,32 +469,8 @@ qemuDomainOnceInit(void) return 0; } -static void qemu_domain_log_context_init(qemuDomainLogContext *logctxt G_GNUC_UNUSED) -{ -} - -static void qemu_domain_log_context_class_init(qemuDomainLogContextClass *klass) -{ - GObjectClass *obj = G_OBJECT_CLASS(klass); - - obj->finalize = qemuDomainLogContextFinalize; -} - VIR_ONCE_GLOBAL_INIT(qemuDomain); -static void -qemuDomainLogContextFinalize(GObject *object) -{ - qemuDomainLogContext *ctxt = QEMU_DOMAIN_LOG_CONTEXT(object); - VIR_DEBUG("ctxt=%p", ctxt); - - virLogManagerFree(ctxt->manager); - VIR_FREE(ctxt->path); - VIR_FORCE_CLOSE(ctxt->writefd); - VIR_FORCE_CLOSE(ctxt->readfd); - G_OBJECT_CLASS(qemu_domain_log_context_parent_class)->finalize(object); -} - /* qemuDomainGetMasterKeyFilePath: * @libDir: Directory path to domain lib files * @@ -6882,7 +6845,7 @@ static void G_GNUC_PRINTF(5, 6) qemuDomainObjTaintMsg(virQEMUDriver *driver, virDomainObj *obj, virDomainTaintFlags taint, - qemuDomainLogContext *logCtxt, + qemuLogContext *logCtxt, const char *fmt, ...) { virErrorPtr orig_err = NULL; @@ -6935,12 +6898,12 @@ qemuDomainObjTaintMsg(virQEMUDriver *driver, goto cleanup; if (logCtxt) { - rc = qemuDomainLogContextWrite(logCtxt, - "%s: Domain id=%d is tainted: %s%s%s%s\n", - timestamp, - obj->def->id, - virDomainTaintTypeToString(taint), - extraprefix, extramsg, extrasuffix); + rc = qemuLogContextWrite(logCtxt, + "%s: Domain id=%d is tainted: %s%s%s%s\n", + timestamp, + obj->def->id, + virDomainTaintTypeToString(taint), + extraprefix, extramsg, extrasuffix); } else { rc = qemuDomainLogAppendMessage(driver, obj, "%s: Domain id=%d is tainted: %s%s%s%s\n", @@ -6961,7 +6924,7 @@ qemuDomainObjTaintMsg(virQEMUDriver *driver, void qemuDomainObjTaint(virQEMUDriver *driver, virDomainObj *obj, virDomainTaintFlags taint, - qemuDomainLogContext *logCtxt) + qemuLogContext *logCtxt) { qemuDomainObjTaintMsg(driver, obj, taint, logCtxt, NULL); qemuDomainSaveStatus(obj); @@ -6970,7 +6933,7 @@ void qemuDomainObjTaint(virQEMUDriver *driver, static void qemuDomainObjCheckMachineTaint(virQEMUDriver *driver, virDomainObj *obj, - qemuDomainLogContext *logCtxt) + qemuLogContext *logCtxt) { qemuDomainObjPrivate *priv = obj->privateData; virQEMUCaps *qemuCaps = priv->qemuCaps; @@ -6988,7 +6951,7 @@ qemuDomainObjCheckMachineTaint(virQEMUDriver *driver, static void qemuDomainObjCheckCPUTaint(virQEMUDriver *driver, virDomainObj *obj, - qemuDomainLogContext *logCtxt, + qemuLogContext *logCtxt, bool incomingMigration) { qemuDomainObjPrivate *priv = obj->privateData; @@ -7020,7 +6983,7 @@ qemuDomainObjCheckCPUTaint(virQEMUDriver *driver, void qemuDomainObjCheckTaint(virQEMUDriver *driver, virDomainObj *obj, - qemuDomainLogContext *logCtxt, + qemuLogContext *logCtxt, bool incomingMigration) { size_t i; @@ -7076,7 +7039,7 @@ void qemuDomainObjCheckTaint(virQEMUDriver *driver, void qemuDomainObjCheckDiskTaint(virQEMUDriver *driver, virDomainObj *obj, virDomainDiskDef *disk, - qemuDomainLogContext *logCtxt) + qemuLogContext *logCtxt) { if (disk->rawio == VIR_TRISTATE_BOOL_YES) qemuDomainObjTaint(driver, obj, VIR_DOMAIN_TAINT_HIGH_PRIVILEGES, @@ -7093,7 +7056,7 @@ void qemuDomainObjCheckDiskTaint(virQEMUDriver *driver, void qemuDomainObjCheckHostdevTaint(virQEMUDriver *driver, virDomainObj *obj, virDomainHostdevDef *hostdev, - qemuDomainLogContext *logCtxt) + qemuLogContext *logCtxt) { if (!virHostdevIsSCSIDevice(hostdev)) return; @@ -7106,7 +7069,7 @@ void qemuDomainObjCheckHostdevTaint(virQEMUDriver *driver, void qemuDomainObjCheckNetTaint(virQEMUDriver *driver, virDomainObj *obj, virDomainNetDef *net, - qemuDomainLogContext *logCtxt) + qemuLogContext *logCtxt) { /* script is only useful for NET_TYPE_ETHERNET (qemu) and * NET_TYPE_BRIDGE (xen), but could be (incorrectly) specified for @@ -7118,163 +7081,6 @@ void qemuDomainObjCheckNetTaint(virQEMUDriver *driver, } -qemuDomainLogContext *qemuDomainLogContextNew(virQEMUDriver *driver, - virDomainObj *vm, - const char *basename) -{ - g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); - qemuDomainLogContext *ctxt = QEMU_DOMAIN_LOG_CONTEXT(g_object_new(QEMU_TYPE_DOMAIN_LOG_CONTEXT, NULL)); - - VIR_DEBUG("Context new %p stdioLogD=%d", ctxt, cfg->stdioLogD); - ctxt->writefd = -1; - ctxt->readfd = -1; - - ctxt->path = g_strdup_printf("%s/%s.log", cfg->logDir, basename); - - if (cfg->stdioLogD) { - ctxt->manager = virLogManagerNew(driver->privileged); - if (!ctxt->manager) - goto error; - - ctxt->writefd = virLogManagerDomainOpenLogFile(ctxt->manager, - "qemu", - vm->def->uuid, - vm->def->name, - ctxt->path, - 0, - &ctxt->inode, - &ctxt->pos); - if (ctxt->writefd < 0) - goto error; - } else { - if ((ctxt->writefd = open(ctxt->path, O_WRONLY | O_CREAT | O_APPEND, S_IRUSR | S_IWUSR)) < 0) { - virReportSystemError(errno, _("failed to create logfile %1$s"), - ctxt->path); - goto error; - } - if (virSetCloseExec(ctxt->writefd) < 0) { - virReportSystemError(errno, _("failed to set close-on-exec flag on %1$s"), - ctxt->path); - goto error; - } - - /* For unprivileged startup we must truncate the file since - * we can't rely on logrotate. We don't use O_TRUNC since - * it is better for SELinux policy if we truncate afterwards */ - if (!driver->privileged && - ftruncate(ctxt->writefd, 0) < 0) { - virReportSystemError(errno, _("failed to truncate %1$s"), - ctxt->path); - goto error; - } - - if ((ctxt->readfd = open(ctxt->path, O_RDONLY)) < 0) { - virReportSystemError(errno, _("failed to open logfile %1$s"), - ctxt->path); - goto error; - } - if (virSetCloseExec(ctxt->readfd) < 0) { - virReportSystemError(errno, _("failed to set close-on-exec flag on %1$s"), - ctxt->path); - goto error; - } - - if ((ctxt->pos = lseek(ctxt->writefd, 0, SEEK_END)) < 0) { - virReportSystemError(errno, _("failed to seek in log file %1$s"), - ctxt->path); - goto error; - } - } - - return ctxt; - - error: - g_clear_object(&ctxt); - return NULL; -} - - -int qemuDomainLogContextWrite(qemuDomainLogContext *ctxt, - const char *fmt, ...) -{ - va_list argptr; - g_autofree char *message = NULL; - int ret = -1; - - va_start(argptr, fmt); - - message = g_strdup_vprintf(fmt, argptr); - if (!ctxt->manager && - lseek(ctxt->writefd, 0, SEEK_END) < 0) { - virReportSystemError(errno, "%s", - _("Unable to seek to end of domain logfile")); - goto cleanup; - } - if (safewrite(ctxt->writefd, message, strlen(message)) < 0) { - virReportSystemError(errno, "%s", - _("Unable to write to domain logfile")); - goto cleanup; - } - - ret = 0; - - cleanup: - va_end(argptr); - return ret; -} - - -ssize_t qemuDomainLogContextRead(qemuDomainLogContext *ctxt, - char **msg) -{ - char *buf; - size_t buflen; - - VIR_DEBUG("Context read %p manager=%p inode=%llu pos=%llu", - ctxt, ctxt->manager, - (unsigned long long)ctxt->inode, - (unsigned long long)ctxt->pos); - - if (ctxt->manager) { - buf = virLogManagerDomainReadLogFile(ctxt->manager, - ctxt->path, - ctxt->inode, - ctxt->pos, - 1024 * 128, - 0); - if (!buf) - return -1; - buflen = strlen(buf); - } else { - ssize_t got; - - buflen = 1024 * 128; - - /* Best effort jump to start of messages */ - ignore_value(lseek(ctxt->readfd, ctxt->pos, SEEK_SET)); - - buf = g_new0(char, buflen); - - got = saferead(ctxt->readfd, buf, buflen - 1); - if (got < 0) { - VIR_FREE(buf); - virReportSystemError(errno, "%s", - _("Unable to read from log file")); - return -1; - } - - buf[got] = '\0'; - - buf = g_renew(char, buf, got + 1); - buflen = got; - } - - *msg = buf; - - return buflen; -} - - /** * qemuDomainLogAppendMessage: * @@ -7332,31 +7138,6 @@ qemuDomainLogAppendMessage(virQEMUDriver *driver, } -int qemuDomainLogContextGetWriteFD(qemuDomainLogContext *ctxt) -{ - return ctxt->writefd; -} - - -void qemuDomainLogContextMarkPosition(qemuDomainLogContext *ctxt) -{ - if (ctxt->manager) - virLogManagerDomainGetLogFilePosition(ctxt->manager, - ctxt->path, - 0, - &ctxt->inode, - &ctxt->pos); - else - ctxt->pos = lseek(ctxt->writefd, 0, SEEK_END); -} - - -virLogManager *qemuDomainLogContextGetManager(qemuDomainLogContext *ctxt) -{ - return ctxt->manager; -} - - /* Locate an appropriate 'qemu-img' binary. */ const char * qemuFindQemuImgBinary(virQEMUDriver *driver) diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index a262555c8c..ddd20e67b4 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -32,13 +32,13 @@ #include "qemu_domainjob.h" #include "qemu_conf.h" #include "qemu_capabilities.h" +#include "qemu_logcontext.h" #include "qemu_migration_params.h" #include "qemu_nbdkit.h" #include "qemu_slirp.h" #include "qemu_fd.h" #include "virchrdev.h" #include "virobject.h" -#include "logging/log_manager.h" #include "virdomainmomentobjlist.h" #include "virenum.h" #include "vireventthread.h" @@ -479,9 +479,6 @@ struct qemuProcessEvent { void qemuProcessEventFree(struct qemuProcessEvent *event); -#define QEMU_TYPE_DOMAIN_LOG_CONTEXT qemu_domain_log_context_get_type() -G_DECLARE_FINAL_TYPE(qemuDomainLogContext, qemu_domain_log_context, QEMU, DOMAIN_LOG_CONTEXT, GObject); - typedef struct _qemuDomainSaveCookie qemuDomainSaveCookie; struct _qemuDomainSaveCookie { virObject parent; @@ -634,39 +631,27 @@ char *qemuDomainDefFormatLive(virQEMUDriver *driver, void qemuDomainObjTaint(virQEMUDriver *driver, virDomainObj *obj, virDomainTaintFlags taint, - qemuDomainLogContext *logCtxt); + qemuLogContext *logCtxt); char **qemuDomainObjGetTainting(virQEMUDriver *driver, virDomainObj *obj); void qemuDomainObjCheckTaint(virQEMUDriver *driver, virDomainObj *obj, - qemuDomainLogContext *logCtxt, + qemuLogContext *logCtxt, bool incomingMigration); void qemuDomainObjCheckDiskTaint(virQEMUDriver *driver, virDomainObj *obj, virDomainDiskDef *disk, - qemuDomainLogContext *logCtxt); + qemuLogContext *logCtxt); void qemuDomainObjCheckHostdevTaint(virQEMUDriver *driver, virDomainObj *obj, virDomainHostdevDef *disk, - qemuDomainLogContext *logCtxt); + qemuLogContext *logCtxt); void qemuDomainObjCheckNetTaint(virQEMUDriver *driver, virDomainObj *obj, virDomainNetDef *net, - qemuDomainLogContext *logCtxt); - -qemuDomainLogContext *qemuDomainLogContextNew(virQEMUDriver *driver, - virDomainObj *vm, - const char *basename); -int qemuDomainLogContextWrite(qemuDomainLogContext *ctxt, - const char *fmt, ...) G_GNUC_PRINTF(2, 3); -ssize_t qemuDomainLogContextRead(qemuDomainLogContext *ctxt, - char **msg); -int qemuDomainLogContextGetWriteFD(qemuDomainLogContext *ctxt); -void qemuDomainLogContextMarkPosition(qemuDomainLogContext *ctxt); - -virLogManager *qemuDomainLogContextGetManager(qemuDomainLogContext *ctxt); + qemuLogContext *logCtxt); int qemuDomainLogAppendMessage(virQEMUDriver *driver, virDomainObj *vm, diff --git a/src/qemu/qemu_logcontext.c b/src/qemu/qemu_logcontext.c new file mode 100644 index 0000000000..0121ae5173 --- /dev/null +++ b/src/qemu/qemu_logcontext.c @@ -0,0 +1,264 @@ +/* + * qemu_logcontext.c: QEMU log context + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + */ + +#include <config.h> + +#include "qemu_logcontext.h" +#include "viralloc.h" +#include "virlog.h" +#include "virutil.h" + +#include <fcntl.h> + +#define VIR_FROM_THIS VIR_FROM_QEMU + +VIR_LOG_INIT("qemu.qemu_logcontext"); + + +struct _qemuLogContext { + GObject parent; + + int writefd; + int readfd; /* Only used if manager == NULL */ + off_t pos; + ino_t inode; /* Only used if manager != NULL */ + char *path; + virLogManager *manager; +}; + +G_DEFINE_TYPE(qemuLogContext, qemu_log_context, G_TYPE_OBJECT); + +static void +qemuLogContextFinalize(GObject *obj); + + +static void +qemu_log_context_init(qemuLogContext *logctxt G_GNUC_UNUSED) +{ +} + + +static void +qemu_log_context_class_init(qemuLogContextClass *klass) +{ + GObjectClass *obj = G_OBJECT_CLASS(klass); + + obj->finalize = qemuLogContextFinalize; +} + + +static void +qemuLogContextFinalize(GObject *object) +{ + qemuLogContext *ctxt = QEMU_LOG_CONTEXT(object); + VIR_DEBUG("ctxt=%p", ctxt); + + virLogManagerFree(ctxt->manager); + VIR_FREE(ctxt->path); + VIR_FORCE_CLOSE(ctxt->writefd); + VIR_FORCE_CLOSE(ctxt->readfd); + G_OBJECT_CLASS(qemu_log_context_parent_class)->finalize(object); +} + + +qemuLogContext * +qemuLogContextNew(virQEMUDriver *driver, + virDomainObj *vm, + const char *basename) +{ + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); + qemuLogContext *ctxt = QEMU_LOG_CONTEXT(g_object_new(QEMU_TYPE_LOG_CONTEXT, NULL)); + + VIR_DEBUG("Context new %p stdioLogD=%d", ctxt, cfg->stdioLogD); + ctxt->writefd = -1; + ctxt->readfd = -1; + + ctxt->path = g_strdup_printf("%s/%s.log", cfg->logDir, basename); + + if (cfg->stdioLogD) { + ctxt->manager = virLogManagerNew(driver->privileged); + if (!ctxt->manager) + goto error; + + ctxt->writefd = virLogManagerDomainOpenLogFile(ctxt->manager, + "qemu", + vm->def->uuid, + vm->def->name, + ctxt->path, + 0, + &ctxt->inode, + &ctxt->pos); + if (ctxt->writefd < 0) + goto error; + } else { + if ((ctxt->writefd = open(ctxt->path, O_WRONLY | O_CREAT | O_APPEND, S_IRUSR | S_IWUSR)) < 0) { + virReportSystemError(errno, _("failed to create logfile %1$s"), + ctxt->path); + goto error; + } + if (virSetCloseExec(ctxt->writefd) < 0) { + virReportSystemError(errno, _("failed to set close-on-exec flag on %1$s"), + ctxt->path); + goto error; + } + + /* For unprivileged startup we must truncate the file since + * we can't rely on logrotate. We don't use O_TRUNC since + * it is better for SELinux policy if we truncate afterwards */ + if (!driver->privileged && + ftruncate(ctxt->writefd, 0) < 0) { + virReportSystemError(errno, _("failed to truncate %1$s"), + ctxt->path); + goto error; + } + + if ((ctxt->readfd = open(ctxt->path, O_RDONLY)) < 0) { + virReportSystemError(errno, _("failed to open logfile %1$s"), + ctxt->path); + goto error; + } + if (virSetCloseExec(ctxt->readfd) < 0) { + virReportSystemError(errno, _("failed to set close-on-exec flag on %1$s"), + ctxt->path); + goto error; + } + + if ((ctxt->pos = lseek(ctxt->writefd, 0, SEEK_END)) < 0) { + virReportSystemError(errno, _("failed to seek in log file %1$s"), + ctxt->path); + goto error; + } + } + + return ctxt; + + error: + g_clear_object(&ctxt); + return NULL; +} + + +int +qemuLogContextWrite(qemuLogContext *ctxt, + const char *fmt, ...) +{ + va_list argptr; + g_autofree char *message = NULL; + int ret = -1; + + va_start(argptr, fmt); + + message = g_strdup_vprintf(fmt, argptr); + if (!ctxt->manager && + lseek(ctxt->writefd, 0, SEEK_END) < 0) { + virReportSystemError(errno, "%s", + _("Unable to seek to end of domain logfile")); + goto cleanup; + } + if (safewrite(ctxt->writefd, message, strlen(message)) < 0) { + virReportSystemError(errno, "%s", + _("Unable to write to domain logfile")); + goto cleanup; + } + + ret = 0; + + cleanup: + va_end(argptr); + return ret; +} + + +ssize_t +qemuLogContextRead(qemuLogContext *ctxt, + char **msg) +{ + char *buf; + size_t buflen; + + VIR_DEBUG("Context read %p manager=%p inode=%llu pos=%llu", + ctxt, ctxt->manager, + (unsigned long long)ctxt->inode, + (unsigned long long)ctxt->pos); + + if (ctxt->manager) { + buf = virLogManagerDomainReadLogFile(ctxt->manager, + ctxt->path, + ctxt->inode, + ctxt->pos, + 1024 * 128, + 0); + if (!buf) + return -1; + buflen = strlen(buf); + } else { + ssize_t got; + + buflen = 1024 * 128; + + /* Best effort jump to start of messages */ + ignore_value(lseek(ctxt->readfd, ctxt->pos, SEEK_SET)); + + buf = g_new0(char, buflen); + + got = saferead(ctxt->readfd, buf, buflen - 1); + if (got < 0) { + VIR_FREE(buf); + virReportSystemError(errno, "%s", + _("Unable to read from log file")); + return -1; + } + + buf[got] = '\0'; + + buf = g_renew(char, buf, got + 1); + buflen = got; + } + + *msg = buf; + + return buflen; +} + + +int +qemuLogContextGetWriteFD(qemuLogContext *ctxt) +{ + return ctxt->writefd; +} + + +void +qemuLogContextMarkPosition(qemuLogContext *ctxt) +{ + if (ctxt->manager) + virLogManagerDomainGetLogFilePosition(ctxt->manager, + ctxt->path, + 0, + &ctxt->inode, + &ctxt->pos); + else + ctxt->pos = lseek(ctxt->writefd, 0, SEEK_END); +} + + +virLogManager * +qemuLogContextGetManager(qemuLogContext *ctxt) +{ + return ctxt->manager; +} diff --git a/src/qemu/qemu_logcontext.h b/src/qemu/qemu_logcontext.h new file mode 100644 index 0000000000..6ad60b7b4a --- /dev/null +++ b/src/qemu/qemu_logcontext.h @@ -0,0 +1,38 @@ +/* + * qemu_logcontext.h: QEMU log context + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + */ + +#pragma once + +#include <glib-object.h> +#include "qemu_conf.h" +#include "logging/log_manager.h" + +#define QEMU_TYPE_LOG_CONTEXT qemu_log_context_get_type() +G_DECLARE_FINAL_TYPE(qemuLogContext, qemu_log_context, QEMU, LOG_CONTEXT, GObject); + +qemuLogContext *qemuLogContextNew(virQEMUDriver *driver, + virDomainObj *vm, + const char *basename); +int qemuLogContextWrite(qemuLogContext *ctxt, + const char *fmt, ...) G_GNUC_PRINTF(2, 3); +ssize_t qemuLogContextRead(qemuLogContext *ctxt, + char **msg); +int qemuLogContextGetWriteFD(qemuLogContext *ctxt); +void qemuLogContextMarkPosition(qemuLogContext *ctxt); + +virLogManager *qemuLogContextGetManager(qemuLogContext *ctxt); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index e0385d11be..e03d0d8b4d 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -1831,7 +1831,7 @@ qemuProcessMonitorReportLogError(qemuMonitor *mon, static void qemuProcessMonitorLogFree(void *opaque) { - qemuDomainLogContext *logCtxt = opaque; + qemuLogContext *logCtxt = opaque; g_clear_object(&logCtxt); } @@ -1857,7 +1857,7 @@ static int qemuConnectMonitor(virQEMUDriver *driver, virDomainObj *vm, int asyncJob, - qemuDomainLogContext *logCtxt, + qemuLogContext *logCtxt, bool reconnect) { qemuDomainObjPrivate *priv = vm->privateData; @@ -1921,7 +1921,7 @@ qemuConnectMonitor(virQEMUDriver *driver, * Returns 0 on success or -1 on error */ static int -qemuProcessReadLog(qemuDomainLogContext *logCtxt, +qemuProcessReadLog(qemuLogContext *logCtxt, char **msg, size_t max) { @@ -1931,7 +1931,7 @@ qemuProcessReadLog(qemuDomainLogContext *logCtxt, char *filter_next; size_t skip; - if ((got = qemuDomainLogContextRead(logCtxt, &buf)) < 0) + if ((got = qemuLogContextRead(logCtxt, &buf)) < 0) return -1; /* Filter out debug messages from intermediate libvirt process */ @@ -1974,7 +1974,7 @@ qemuProcessReadLog(qemuDomainLogContext *logCtxt, static int -qemuProcessReportLogError(qemuDomainLogContext *logCtxt, +qemuProcessReportLogError(qemuLogContext *logCtxt, const char *msgprefix) { g_autofree char *logmsg = NULL; @@ -1999,7 +1999,7 @@ qemuProcessMonitorReportLogError(qemuMonitor *mon G_GNUC_UNUSED, const char *msg, void *opaque) { - qemuDomainLogContext *logCtxt = opaque; + qemuLogContext *logCtxt = opaque; qemuProcessReportLogError(logCtxt, msg); } @@ -2300,7 +2300,7 @@ static int qemuProcessWaitForMonitor(virQEMUDriver *driver, virDomainObj *vm, int asyncJob, - qemuDomainLogContext *logCtxt) + qemuLogContext *logCtxt) { int ret = -1; g_autoptr(GHashTable) info = NULL; @@ -4664,7 +4664,7 @@ static void qemuLogOperation(virDomainObj *vm, const char *msg, virCommand *cmd, - qemuDomainLogContext *logCtxt) + qemuLogContext *logCtxt) { g_autofree char *timestamp = NULL; qemuDomainObjPrivate *priv = vm->privateData; @@ -4678,20 +4678,20 @@ qemuLogOperation(virDomainObj *vm, if ((timestamp = virTimeStringNow()) == NULL) return; - if (qemuDomainLogContextWrite(logCtxt, - "%s: %s %s, qemu version: %d.%d.%d%s, kernel: %s, hostname: %s\n", - timestamp, msg, VIR_LOG_VERSION_STRING, - (qemuVersion / 1000000) % 1000, - (qemuVersion / 1000) % 1000, - qemuVersion % 1000, - NULLSTR_EMPTY(package), - uts.release, - NULLSTR_EMPTY(hostname)) < 0) + if (qemuLogContextWrite(logCtxt, + "%s: %s %s, qemu version: %d.%d.%d%s, kernel: %s, hostname: %s\n", + timestamp, msg, VIR_LOG_VERSION_STRING, + (qemuVersion / 1000000) % 1000, + (qemuVersion / 1000) % 1000, + qemuVersion % 1000, + NULLSTR_EMPTY(package), + uts.release, + NULLSTR_EMPTY(hostname)) < 0) return; if (cmd) { g_autofree char *args = virCommandToString(cmd, true); - qemuDomainLogContextWrite(logCtxt, "%s\n", args); + qemuLogContextWrite(logCtxt, "%s\n", args); } } @@ -7566,7 +7566,7 @@ qemuProcessLaunch(virConnectPtr conn, int ret = -1; int rv; int logfile = -1; - g_autoptr(qemuDomainLogContext) logCtxt = NULL; + g_autoptr(qemuLogContext) logCtxt = NULL; qemuDomainObjPrivate *priv = vm->privateData; g_autoptr(virCommand) cmd = NULL; struct qemuProcessHookData hookData; @@ -7616,11 +7616,11 @@ qemuProcessLaunch(virConnectPtr conn, hookData.cfg = cfg; VIR_DEBUG("Creating domain log file"); - if (!(logCtxt = qemuDomainLogContextNew(driver, vm, vm->def->name))) { + if (!(logCtxt = qemuLogContextNew(driver, vm, vm->def->name))) { virLastErrorPrefixMessage("%s", _("can't connect to virtlogd")); goto cleanup; } - logfile = qemuDomainLogContextGetWriteFD(logCtxt); + logfile = qemuLogContextGetWriteFD(logCtxt); if (qemuProcessGenID(vm, flags) < 0) goto cleanup; @@ -7656,7 +7656,7 @@ qemuProcessLaunch(virConnectPtr conn, qemuDomainObjCheckTaint(driver, vm, logCtxt, incoming != NULL); - qemuDomainLogContextMarkPosition(logCtxt); + qemuLogContextMarkPosition(logCtxt); if (qemuProcessEnableDomainNamespaces(driver, vm) < 0) goto cleanup; -- 2.41.0

This code can be used by the nbdkit implementation for reading back filtered log data for error reporting. Move it to qemuLogContext so that it can be shared. Renamed to qemuLogContextReadFiltered(). Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_logcontext.c | 65 ++++++++++++++++++++++++++++++++++++ src/qemu/qemu_logcontext.h | 3 ++ src/qemu/qemu_process.c | 67 +------------------------------------- 3 files changed, 69 insertions(+), 66 deletions(-) diff --git a/src/qemu/qemu_logcontext.c b/src/qemu/qemu_logcontext.c index 0121ae5173..6e20f58bfa 100644 --- a/src/qemu/qemu_logcontext.c +++ b/src/qemu/qemu_logcontext.c @@ -21,6 +21,7 @@ #include "qemu_logcontext.h" #include "viralloc.h" #include "virlog.h" +#include "virstring.h" #include "virutil.h" #include <fcntl.h> @@ -236,6 +237,70 @@ qemuLogContextRead(qemuLogContext *ctxt, } +/** + * qemuLogContextFilter: Read and filter log for relevant messages + * @ctxt: the domain log context + * @msg: pointer to buffer to store the read messages in + * @max: maximum length of the message returned in @msg after filtering + * + * Reads log output from @ctxt and filters it. Skips messages not produced by + * the target executable or irrelevant messages. If @max is not zero, @buf will + * contain at most @max characters from the end of the log and @buf will start + * after a new line if possible. + */ +int +qemuLogContextReadFiltered(qemuLogContext *ctxt, + char **msg, + size_t max) +{ + char *buf; + char *eol; + char *filter_next; + size_t skip; + ssize_t got; + + if ((got = qemuLogContextRead(ctxt, &buf)) < 0) + return -1; + + /* Filter out debug messages from intermediate libvirt process */ + filter_next = buf; + while ((eol = strchr(filter_next, '\n'))) { + *eol = '\0'; + if (virLogProbablyLogMessage(filter_next) || + strstr(filter_next, "char device redirected to")) { + skip = (eol + 1) - filter_next; + memmove(filter_next, eol + 1, buf + got - eol); + got -= skip; + } else { + filter_next = eol + 1; + *eol = '\n'; + } + } + + if (got > 0 && + buf[got - 1] == '\n') { + buf[got - 1] = '\0'; + got--; + } + + if (max > 0 && got > max) { + skip = got - max; + + if (buf[skip - 1] != '\n' && + (eol = strchr(buf + skip, '\n')) && + !virStringIsEmpty(eol + 1)) + skip = eol + 1 - buf; + + memmove(buf, buf + skip, got - skip + 1); + got -= skip; + } + + buf = g_renew(char, buf, got + 1); + *msg = buf; + return 0; +} + + int qemuLogContextGetWriteFD(qemuLogContext *ctxt) { diff --git a/src/qemu/qemu_logcontext.h b/src/qemu/qemu_logcontext.h index 6ad60b7b4a..738e908bc3 100644 --- a/src/qemu/qemu_logcontext.h +++ b/src/qemu/qemu_logcontext.h @@ -32,6 +32,9 @@ int qemuLogContextWrite(qemuLogContext *ctxt, const char *fmt, ...) G_GNUC_PRINTF(2, 3); ssize_t qemuLogContextRead(qemuLogContext *ctxt, char **msg); +int qemuLogContextReadFiltered(qemuLogContext *ctxt, + char **msg, + size_t max); int qemuLogContextGetWriteFD(qemuLogContext *ctxt); void qemuLogContextMarkPosition(qemuLogContext *ctxt); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index e03d0d8b4d..a77d2ba7de 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -1908,71 +1908,6 @@ qemuConnectMonitor(virQEMUDriver *driver, } -/** - * qemuProcessReadLog: Read log file of a qemu VM - * @logCtxt: the domain log context - * @msg: pointer to buffer to store the read messages in - * @max: maximum length of the message returned in @msg - * - * Reads log of a qemu VM. Skips messages not produced by qemu or irrelevant - * messages. If @max is not zero, @msg will contain at most @max characters - * from the end of the log and @msg will start after a new line if possible. - * - * Returns 0 on success or -1 on error - */ -static int -qemuProcessReadLog(qemuLogContext *logCtxt, - char **msg, - size_t max) -{ - char *buf; - ssize_t got; - char *eol; - char *filter_next; - size_t skip; - - if ((got = qemuLogContextRead(logCtxt, &buf)) < 0) - return -1; - - /* Filter out debug messages from intermediate libvirt process */ - filter_next = buf; - while ((eol = strchr(filter_next, '\n'))) { - *eol = '\0'; - if (virLogProbablyLogMessage(filter_next) || - strstr(filter_next, "char device redirected to")) { - skip = (eol + 1) - filter_next; - memmove(filter_next, eol + 1, buf + got - eol); - got -= skip; - } else { - filter_next = eol + 1; - *eol = '\n'; - } - } - - if (got > 0 && - buf[got - 1] == '\n') { - buf[got - 1] = '\0'; - got--; - } - - if (max > 0 && got > max) { - skip = got - max; - - if (buf[skip - 1] != '\n' && - (eol = strchr(buf + skip, '\n')) && - !virStringIsEmpty(eol + 1)) - skip = eol + 1 - buf; - - memmove(buf, buf + skip, got - skip + 1); - got -= skip; - } - - buf = g_renew(char, buf, got + 1); - *msg = buf; - return 0; -} - - static int qemuProcessReportLogError(qemuLogContext *logCtxt, const char *msgprefix) @@ -1980,7 +1915,7 @@ qemuProcessReportLogError(qemuLogContext *logCtxt, g_autofree char *logmsg = NULL; /* assume that 1024 chars of qemu log is the right balance */ - if (qemuProcessReadLog(logCtxt, &logmsg, 1024) < 0) + if (qemuLogContextReadFiltered(logCtxt, &logmsg, 1024) < 0) return -1; virResetLastError(); -- 2.41.0

log stderr and stdout from nbdkit into its own log so that nbdkit-related issues can be debugged more easily. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_nbdkit.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 6bf962d0f1..2d70e72c42 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -852,12 +852,23 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, virTimeBackOffVar timebackoff; g_autoptr(virURI) uri = NULL; g_autofree char *uristring = NULL; + g_autofree char *basename = g_strdup_printf("%s-nbdkit-%i", vm->def->name, proc->source->id); + int logfd = -1; + g_autoptr(qemuLogContext) logContext = NULL; if (!(cmd = qemuNbdkitProcessBuildCommand(proc))) return -1; + if (!(logContext = qemuLogContextNew(driver, vm, basename))) { + virLastErrorPrefixMessage("%s", _("can't connect to virtlogd")); + return -1; + } + + logfd = qemuLogContextGetWriteFD(logContext); + VIR_DEBUG("starting nbdkit process for %s", proc->source->nodestorage); - virCommandSetErrorBuffer(cmd, &errbuf); + virCommandSetErrorFD(cmd, &logfd); + virCommandSetOutputFD(cmd, &logfd); virCommandSetPidFile(cmd, proc->pidfile); if (qemuExtDeviceLogCommand(driver, vm, cmd, "nbdkit") < 0) @@ -899,6 +910,9 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, if ((uri = qemuBlockStorageSourceGetURI(proc->source))) uristring = virURIFormat(uri); + if (qemuLogContextReadFiltered(logContext, &errbuf, 1024) < 0) + VIR_WARN("Unable to read from nbdkit log"); + virReportError(VIR_ERR_OPERATION_FAILED, _("Failed to connect to nbdkit for '%1$s': %2$s"), NULLSTR(uristring), NULLSTR(errbuf)); -- 2.41.0

Add new DO_TEST_CAPS_LATEST_NBDKIT macro to test xml2argv for various nbdkit capability scenarios. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_nbdkit.c | 20 +++++++++++++++++--- tests/qemuxml2argvtest.c | 11 +++++++++++ tests/testutilsqemu.c | 26 ++++++++++++++++++++++++++ tests/testutilsqemu.h | 4 ++++ 4 files changed, 58 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 2d70e72c42..81861bae4a 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -290,10 +290,16 @@ virNbkditCapsCheckModdir(const char *moddir, static bool virNbdkitCapsIsValid(void *data, - void *privData G_GNUC_UNUSED) + void *privData) { qemuNbdkitCaps *nbdkitCaps = data; struct stat st; + /* when run under test, we will use privData as a signal to indicate that + * we shouldn't touch the filesystem */ + bool skipValidation = (privData != NULL); + + if (skipValidation) + return true; if (!nbdkitCaps->path) return true; @@ -334,9 +340,17 @@ virNbdkitCapsIsValid(void *data, static void* virNbdkitCapsNewData(const char *binary, - void *privData G_GNUC_UNUSED) + void *privData) { - qemuNbdkitCaps *caps = qemuNbdkitCapsNew(binary); + /* when run under test, we will use privData as a signal to indicate that + * we shouldn't touch the filesystem */ + bool skipNewData = (privData != NULL); + qemuNbdkitCaps *caps = NULL; + + if (skipNewData) + return NULL; + + caps = qemuNbdkitCapsNew(binary); qemuNbdkitCapsQuery(caps); return caps; diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 1b76b32812..d64c21ae17 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -611,6 +611,14 @@ testCompareXMLToArgv(const void *data) if (qemuTestCapsCacheInsert(driver.qemuCapsCache, info->qemuCaps) < 0) goto cleanup; + if (info->nbdkitCaps) { + if (virFileCacheInsertData(driver.nbdkitCapsCache, TEST_NBDKIT_PATH, + g_object_ref(info->nbdkitCaps)) < 0) { + g_object_unref(info->nbdkitCaps); + goto cleanup; + } + } + if (info->migrateFrom && !(migrateURI = qemuMigrationDstGetURI(info->migrateFrom, info->migrateFd))) @@ -831,6 +839,9 @@ mymain(void) # define DO_TEST_CAPS_ARCH_VER(name, arch, ver) \ DO_TEST_CAPS_ARCH_VER_FULL(name, arch, ver, ARG_END) +# define DO_TEST_CAPS_LATEST_NBDKIT(name, ...) \ + DO_TEST_CAPS_ARCH_LATEST_FULL(name, "x86_64", ARG_NBDKIT_CAPS, __VA_ARGS__, QEMU_NBDKIT_CAPS_LAST, ARG_END) + # define DO_TEST_CAPS_LATEST(name) \ DO_TEST_CAPS_ARCH_LATEST(name, "x86_64") diff --git a/tests/testutilsqemu.c b/tests/testutilsqemu.c index fdbad16abe..9a607ab5a3 100644 --- a/tests/testutilsqemu.c +++ b/tests/testutilsqemu.c @@ -50,6 +50,10 @@ virFindFileInPath(const char *file) return g_strdup_printf("/usr/bin/%s", file); } + if (g_str_equal(file, "nbdkit")) { + return g_strdup(TEST_NBDKIT_PATH); + } + /* Nothing in tests should be relying on real files * in host OS, so we return NULL to try to force * an error in such a case @@ -288,6 +292,7 @@ void qemuTestDriverFree(virQEMUDriver *driver) virObjectUnref(driver->caps); virObjectUnref(driver->config); virObjectUnref(driver->securityManager); + g_clear_object(&driver->nbdkitCapsCache); virCPUDefFree(cpuDefault); virCPUDefFree(cpuHaswell); @@ -487,6 +492,12 @@ int qemuTestDriverInit(virQEMUDriver *driver) if (!driver->qemuCapsCache) goto error; + driver->nbdkitCapsCache = qemuNbdkitCapsCacheNew("/dev/null"); + /* the nbdkitCapsCache just interprets the presence of a non-null private + * data pointer as a signal to skip cache validation. This prevents the + * cache from trying to validate the plugindir mtime, etc during test */ + virFileCacheSetPriv(driver->nbdkitCapsCache, GUINT_TO_POINTER(1)); + driver->xmlopt = virQEMUDriverCreateXMLConf(driver, "none"); if (!driver->xmlopt) goto error; @@ -780,6 +791,14 @@ testQemuInfoSetArgs(struct testQemuInfo *info, ignore_value(virBitmapSetBit(info->args.fakeCapsDel, flag)); break; + case ARG_NBDKIT_CAPS: + if (!(info->args.fakeNbdkitCaps)) + info->args.fakeNbdkitCaps = virBitmapNew(QEMU_NBDKIT_CAPS_LAST); + + while ((flag = va_arg(argptr, int)) < QEMU_NBDKIT_CAPS_LAST) + ignore_value(virBitmapSetBit(info->args.fakeNbdkitCaps, flag)); + break; + case ARG_GIC: info->args.gic = va_arg(argptr, int); break; @@ -1054,6 +1073,11 @@ testQemuInfoInitArgs(struct testQemuInfo *info) for (cap = -1; (cap = virBitmapNextSetBit(info->args.fakeCapsDel, cap)) >= 0;) virQEMUCapsClear(info->qemuCaps, cap); + info->nbdkitCaps = qemuNbdkitCapsNew(TEST_NBDKIT_PATH); + + for (cap = -1; (cap = virBitmapNextSetBit(info->args.fakeNbdkitCaps, cap)) >= 0;) + qemuNbdkitCapsSet(info->nbdkitCaps, cap); + if (info->args.gic != GIC_NONE && testQemuCapsSetGIC(info->qemuCaps, info->args.gic) < 0) return -1; @@ -1072,6 +1096,8 @@ testQemuInfoClear(struct testQemuInfo *info) g_clear_pointer(&info->args.fakeCapsAdd, virBitmapFree); g_clear_pointer(&info->args.fakeCapsDel, virBitmapFree); g_clear_pointer(&info->args.fds, g_hash_table_unref); + g_clear_object(&info->nbdkitCaps); + g_clear_pointer(&info->args.fakeNbdkitCaps, virBitmapFree); } diff --git a/tests/testutilsqemu.h b/tests/testutilsqemu.h index 7845ac7cb6..ad6874fdc3 100644 --- a/tests/testutilsqemu.h +++ b/tests/testutilsqemu.h @@ -28,6 +28,7 @@ # define TEST_TPM_ENV_VAR "VIR_TEST_MOCK_FAKE_TPM_VERSION" # define TPM_VER_1_2 "1.2" # define TPM_VER_2_0 "2.0" +# define TEST_NBDKIT_PATH "/fakebindir/nbdkit" enum { GIC_NONE = 0, @@ -49,6 +50,7 @@ typedef enum { ARG_CAPS_VARIANT, ARG_CAPS_HOST_CPU_MODEL, ARG_FD_GROUP, /* name, nfds, fd[0], ... fd[n-1] */ + ARG_NBDKIT_CAPS, ARG_END, } testQemuInfoArgName; @@ -80,6 +82,7 @@ struct testQemuArgs { bool newargs; virBitmap *fakeCapsAdd; virBitmap *fakeCapsDel; + virBitmap *fakeNbdkitCaps; char *capsver; char *capsarch; const char *capsvariant; @@ -95,6 +98,7 @@ struct testQemuInfo { char *outfile; char *errfile; virQEMUCaps *qemuCaps; + qemuNbdkitCaps *nbdkitCaps; const char *migrateFrom; int migrateFd; unsigned int flags; -- 2.41.0

This prepares encryption secrets and authentication secrets. When we add nbdkit-backed network storage sources, we will not need to send authentication secrets to qemu, since they will be sent to nbdkit instead. So split this into two different functions. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_domain.c | 103 +++++++++++++++++++++++++---------------- 1 file changed, 62 insertions(+), 41 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 23608f95bd..951f3127d9 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -1398,38 +1398,70 @@ qemuDomainSecretStorageSourcePrepareCookies(qemuDomainObjPrivate *priv, /** - * qemuDomainSecretStorageSourcePrepare: + * qemuDomainSecretStorageSourcePrepareEncryption: * @priv: domain private object * @src: storage source struct to setup - * @authalias: prefix of the alias for secret holding authentication data - * @encalias: prefix of the alias for secret holding encryption password + * @alias: prefix of the alias for secret holding encryption password * - * Prepares data necessary for encryption and authentication of @src. The two - * alias prefixes are provided since in the backing chain authentication belongs - * to the storage protocol data whereas encryption is relevant to the format - * driver in qemu. The two will have different node names. + * Prepares data necessary for encryption of @src. * * Returns 0 on success; -1 on error while reporting an libvirt error. */ static int -qemuDomainSecretStorageSourcePrepare(qemuDomainObjPrivate *priv, - virStorageSource *src, - const char *aliasprotocol, - const char *aliasformat) +qemuDomainSecretStorageSourcePrepareEncryption(qemuDomainObjPrivate *priv, + virStorageSource *src, + const char *alias) { qemuDomainStorageSourcePrivate *srcPriv; - bool hasEnc = src->encryption && src->encryption->nsecrets > 0; + size_t nsecrets = 0; + size_t i; - if (virStorageSourceIsEmpty(src)) + if (!(src->encryption && src->encryption->nsecrets > 0)) return 0; - if (!src->auth && !hasEnc && src->ncookies == 0) + if (virStorageSourceIsEmpty(src)) return 0; - if (!(src->privateData = qemuDomainStorageSourcePrivateNew())) - return -1; + nsecrets = src->encryption->nsecrets; + + srcPriv = qemuDomainStorageSourcePrivateFetch(src); + + srcPriv->enccount = nsecrets; + srcPriv->encinfo = g_new0(qemuDomainSecretInfo *, nsecrets); + for (i = 0; i < nsecrets; ++i) { + if (!(srcPriv->encinfo[i] = qemuDomainSecretInfoSetupFromSecret(priv, alias, + "encryption", i, + VIR_SECRET_USAGE_TYPE_VOLUME, + NULL, + &src->encryption->secrets[i]->seclookupdef))) + return -1; + } + + return 0; +} + + +/** + * qemuDomainSecretStorageSourcePrepareAuth: + * @priv: domain private object + * @src: storage source struct to setup + * @alias: prefix of the alias for secret holding authentication data + * + * Prepares data necessary for authentication of @src. + * + * Returns 0 on success; -1 on error while reporting an libvirt error. + */ +static int +qemuDomainSecretStorageSourcePrepareAuth(qemuDomainObjPrivate *priv, + virStorageSource *src, + const char *alias) +{ + qemuDomainStorageSourcePrivate *srcPriv; + + if (virStorageSourceIsEmpty(src)) + return 0; - srcPriv = QEMU_DOMAIN_STORAGE_SOURCE_PRIVATE(src); + srcPriv = qemuDomainStorageSourcePrivateFetch(src); if (src->auth) { virSecretUsageType usageType = VIR_SECRET_USAGE_TYPE_ISCSI; @@ -1437,7 +1469,7 @@ qemuDomainSecretStorageSourcePrepare(qemuDomainObjPrivate *priv, if (src->protocol == VIR_STORAGE_NET_PROTOCOL_RBD) usageType = VIR_SECRET_USAGE_TYPE_CEPH; - if (!(srcPriv->secinfo = qemuDomainSecretInfoSetupFromSecret(priv, aliasprotocol, + if (!(srcPriv->secinfo = qemuDomainSecretInfoSetupFromSecret(priv, alias, "auth", 0, usageType, src->auth->username, @@ -1445,26 +1477,10 @@ qemuDomainSecretStorageSourcePrepare(qemuDomainObjPrivate *priv, return -1; } - if (hasEnc) { - size_t nsecrets = src->encryption->nsecrets; - size_t i; - - srcPriv->enccount = nsecrets; - srcPriv->encinfo = g_new0(qemuDomainSecretInfo *, nsecrets); - for (i = 0; i < nsecrets; ++i) { - if (!(srcPriv->encinfo[i] = qemuDomainSecretInfoSetupFromSecret(priv, aliasformat, - "encryption", i, - VIR_SECRET_USAGE_TYPE_VOLUME, - NULL, - &src->encryption->secrets[i]->seclookupdef))) - return -1; - } - } - if (src->ncookies && !(srcPriv->httpcookie = qemuDomainSecretStorageSourcePrepareCookies(priv, src, - aliasprotocol))) + alias))) return -1; return 0; @@ -11000,9 +11016,12 @@ qemuDomainPrepareDiskSourceLegacy(virDomainDiskDef *disk, qemuDomainPrepareStorageSourceConfig(disk->src, cfg); qemuDomainPrepareDiskSourceData(disk, disk->src); - if (qemuDomainSecretStorageSourcePrepare(priv, disk->src, - disk->info.alias, - disk->info.alias) < 0) + if (qemuDomainSecretStorageSourcePrepareEncryption(priv, disk->src, + disk->info.alias) < 0) + return -1; + + if (qemuDomainSecretStorageSourcePrepareAuth(priv, disk->src, + disk->info.alias) < 0) return -1; if (qemuDomainPrepareStorageSourcePR(disk->src, priv, disk->info.alias) < 0) @@ -11093,9 +11112,11 @@ qemuDomainPrepareStorageSourceBlockdevNodename(virDomainDiskDef *disk, qemuDomainPrepareStorageSourceConfig(src, cfg); qemuDomainPrepareDiskSourceData(disk, src); - if (qemuDomainSecretStorageSourcePrepare(priv, src, - src->nodestorage, - src->nodeformat) < 0) + if (qemuDomainSecretStorageSourcePrepareEncryption(priv, src, + src->nodeformat) < 0) + return -1; + if (qemuDomainSecretStorageSourcePrepareAuth(priv, src, + src->nodestorage) < 0) return -1; if (qemuDomainPrepareStorageSourcePR(src, priv, src->nodestorage) < 0) -- 2.41.0

Add xml to the private data for a disk source to represent the nbdkit process so that the state can be re-created if the libvirt daemon is restarted. Format: <nbdkit> <pidfile>/path/to/nbdkit.pid</pidfile> <socketfile>/path/to/nbdkit.socket</socketfile> </nbdkit> Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_domain.c | 52 +++++++++++++++++ src/qemu/qemu_nbdkit.c | 71 +++++++++++++++++++++++ src/qemu/qemu_nbdkit.h | 8 +++ src/qemu/qemu_process.c | 6 ++ tests/qemustatusxml2xmldata/modern-in.xml | 4 ++ 5 files changed, 141 insertions(+) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 951f3127d9..8429ce1028 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -1958,6 +1958,33 @@ qemuStorageSourcePrivateDataAssignSecinfo(qemuDomainSecretInfo **secinfo, } +static int +qemuStorageSourcePrivateDataParseNbdkit(xmlNodePtr node, + xmlXPathContextPtr ctxt, + virStorageSource *src) +{ + g_autofree char *pidfile = NULL; + g_autofree char *socketfile = NULL; + VIR_XPATH_NODE_AUTORESTORE(ctxt); + + ctxt->node = node; + + if (!(pidfile = virXPathString("string(./pidfile)", ctxt))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("missing nbdkit pidfile")); + return -1; + } + + if (!(socketfile = virXPathString("string(./socketfile)", ctxt))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("missing nbdkit socketfile")); + return -1; + } + + qemuNbdkitReconnectStorageSource(src, pidfile, socketfile); + + return 0; +} + + static int qemuStorageSourcePrivateDataParse(xmlXPathContextPtr ctxt, virStorageSource *src) @@ -1971,6 +1998,7 @@ qemuStorageSourcePrivateDataParse(xmlXPathContextPtr ctxt, bool fdsetPresent = false; unsigned int fdSetID; int enccount; + xmlNodePtr nbdkitnode = NULL; src->nodestorage = virXPathString("string(./nodenames/nodename[@type='storage']/@name)", ctxt); src->nodeformat = virXPathString("string(./nodenames/nodename[@type='format']/@name)", ctxt); @@ -2036,6 +2064,10 @@ qemuStorageSourcePrivateDataParse(xmlXPathContextPtr ctxt, virTristateBoolTypeFromString(thresholdEventWithIndex) == VIR_TRISTATE_BOOL_YES) src->thresholdEventWithIndex = true; + if ((nbdkitnode = virXPathNode("nbdkit", ctxt))) { + if (qemuStorageSourcePrivateDataParseNbdkit(nbdkitnode, ctxt, src) < 0) + return -1; + } return 0; } @@ -2053,6 +2085,23 @@ qemuStorageSourcePrivateDataFormatSecinfo(virBuffer *buf, } +static void +qemuStorageSourcePrivateDataFormatNbdkit(qemuNbdkitProcess *nbdkit, + virBuffer *buf) +{ + g_auto(virBuffer) childBuf = VIR_BUFFER_INIT_CHILD(buf); + + if (!nbdkit) + return; + + virBufferEscapeString(&childBuf, "<pidfile>%s</pidfile>\n", + nbdkit->pidfile); + virBufferEscapeString(&childBuf, "<socketfile>%s</socketfile>\n", + nbdkit->socketfile); + virXMLFormatElement(buf, "nbdkit", NULL, &childBuf); +} + + static int qemuStorageSourcePrivateDataFormat(virStorageSource *src, virBuffer *buf) @@ -2102,6 +2151,9 @@ qemuStorageSourcePrivateDataFormat(virStorageSource *src, if (src->thresholdEventWithIndex) virBufferAddLit(buf, "<thresholdEvent indexUsed='yes'/>\n"); + if (srcPriv) + qemuStorageSourcePrivateDataFormatNbdkit(srcPriv->nbdkitProcess, buf); + return 0; } diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 81861bae4a..e3923ab4f2 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -627,6 +627,77 @@ qemuNbdkitProcessNew(virStorageSource *source, return nbdkit; } +/** + * qemuNbdkitReconnectStorageSource: + * @source: a storage source + * @pidfile: a pidfile for an nbdkit process + * @socketfile: the socket file associated with the nbdkit process + * + * This function constructs a new qemuNbdkitProcess object with the given values for @pidfile and + * @socketfile and stores it in @source. This is intended to be called when the libvirt daemon is + * restarted and tries to reconnect to all currently-running domains. Since this function is called + * from the code that parses the current daemon state, it should not perform any filesystem + * operations, or anything else that might fail. Additional initialization will be done later by + * calling qemuNbdkitStorageSourceManageProcess(). + */ +void +qemuNbdkitReconnectStorageSource(virStorageSource *source, + const char *pidfile, + const char *socketfile) +{ + qemuDomainStorageSourcePrivate *srcpriv = qemuDomainStorageSourcePrivateFetch(source); + + if (srcpriv->nbdkitProcess) { + VIR_WARN("source already has an nbdkit process"); + return; + } + + srcpriv->nbdkitProcess = qemuNbdkitProcessNew(source, pidfile, socketfile); +} + + +static void +qemuNbdkitStorageSourceManageProcessOne(virStorageSource *source) +{ + qemuDomainStorageSourcePrivate *srcpriv = QEMU_DOMAIN_STORAGE_SOURCE_PRIVATE(source); + qemuNbdkitProcess *proc; + + if (!srcpriv) + return; + + proc = srcpriv->nbdkitProcess; + + if (!proc) + return; + + if (proc->pid <= 0) { + if (virPidFileReadPath(proc->pidfile, &proc->pid) < 0) { + VIR_WARN("Unable to read pidfile '%s'", proc->pidfile); + return; + } + } + + if (virProcessKill(proc->pid, 0) < 0) + VIR_WARN("nbdkit process %i is not alive", proc->pid); +} + +/** + * qemuNbdkitStorageSourceManageProcess: + * @source: a storage source + * @vm: the vm that owns this storage source + * + * This function re-enables monitoring of any nbdkit processes associated with the backing chain of + * @source. It is intended to be called after libvirt restarts and has loaded its current state from + * disk and is attempting to re-connect to active domains. + */ +void +qemuNbdkitStorageSourceManageProcess(virStorageSource *source) +{ + virStorageSource *backing; + for (backing = source; backing != NULL; backing = backing->backingStore) + qemuNbdkitStorageSourceManageProcessOne(backing); +} + bool qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index ccd418b7d3..7e2aeed4eb 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -54,6 +54,14 @@ qemuNbdkitInitStorageSource(qemuNbdkitCaps *nbdkitCaps, uid_t user, gid_t group); +void +qemuNbdkitReconnectStorageSource(virStorageSource *source, + const char *pidfile, + const char *socketfile); + +void +qemuNbdkitStorageSourceManageProcess(virStorageSource *src); + bool qemuNbdkitCapsGet(qemuNbdkitCaps *nbdkitCaps, qemuNbdkitCapsFlags flag); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index a77d2ba7de..d90990d8a5 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -8992,6 +8992,12 @@ qemuProcessReconnect(void *opaque) } } + for (i = 0; i < obj->def->ndisks; i++) + qemuNbdkitStorageSourceManageProcess(obj->def->disks[i]->src); + + if (obj->def->os.loader && obj->def->os.loader->nvram) + qemuNbdkitStorageSourceManageProcess(obj->def->os.loader->nvram); + /* update domain state XML with possibly updated state in virDomainObj */ if (virDomainObjSave(obj, driver->xmlopt, cfg->stateDir) < 0) goto error; diff --git a/tests/qemustatusxml2xmldata/modern-in.xml b/tests/qemustatusxml2xmldata/modern-in.xml index 95fc569029..e139c8d38c 100644 --- a/tests/qemustatusxml2xmldata/modern-in.xml +++ b/tests/qemustatusxml2xmldata/modern-in.xml @@ -345,6 +345,10 @@ <fdset type='storage' id='1337'/> </fdsets> <thresholdEvent indexUsed='yes'/> + <nbdkit> + <pidfile>/path/to/nbdkit.pid</pidfile> + <socketfile>/path/to/nbdkit.socket</socketfile> + </nbdkit> </privateData> </source> <backingStore/> -- 2.41.0

All users of virCommandSetSendBuffer() are using it to send sensitive data to a child process. So, since these buffers contain sensitive information, clear it with virSecureErase(). Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/util/vircommand.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/util/vircommand.c b/src/util/vircommand.c index 5f094c625a..899d413dd2 100644 --- a/src/util/vircommand.c +++ b/src/util/vircommand.c @@ -54,6 +54,7 @@ #include "virpidfile.h" #include "virprocess.h" #include "virbuffer.h" +#include "virsecureerase.h" #include "virthread.h" #include "virstring.h" @@ -1697,6 +1698,7 @@ virCommandFreeSendBuffers(virCommand *cmd) for (i = 0; i < virCommandGetNumSendBuffers(cmd); i++) { VIR_FORCE_CLOSE(cmd->sendBuffers[i].fd); + virSecureErase(cmd->sendBuffers[i].buffer, cmd->sendBuffers[i].buflen); VIR_FREE(cmd->sendBuffers[i].buffer); } VIR_FREE(cmd->sendBuffers); -- 2.41.0

Rather than passing passwords and cookies (which could contain passwords) to nbdkit via commandline arguments, use the alternate format that nbdkit supports where we can specify a file descriptor which nbdkit will read to get the password or cookies. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_nbdkit.c | 54 ++++++++++++++++++++++++++---------------- 1 file changed, 34 insertions(+), 20 deletions(-) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index e3923ab4f2..22a67b0748 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -24,7 +24,6 @@ #include "virerror.h" #include "virlog.h" #include "virpidfile.h" -#include "virsecureerase.h" #include "virtime.h" #include "virutil.h" #include "qemu_block.h" @@ -753,6 +752,29 @@ qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, } +static int +qemuNbdkitCommandPassDataByPipe(virCommand *cmd, + const char *argName, + unsigned char **buf, + size_t buflen) +{ + g_autofree char *fdfmt = NULL; + int fd = virCommandSetSendBuffer(cmd, buf, buflen); + + if (fd < 0) + return -1; + + /* some nbdkit arguments accept a variation where nbdkit will read the data + * from a file descriptor, e.g. password=-FD */ + fdfmt = g_strdup_printf("-%i", fd); + virCommandAddArgPair(cmd, argName, fdfmt); + + virCommandDoAsyncIO(cmd); + + return 0; +} + + static int qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *proc, virCommand *cmd) @@ -775,7 +797,6 @@ qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *proc, g_autoptr(virConnect) conn = virGetConnectSecret(); g_autofree uint8_t *secret = NULL; size_t secretlen = 0; - g_autofree char *password = NULL; int secrettype; virStorageAuthDef *authdef = proc->source->auth; @@ -799,26 +820,19 @@ qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *proc, return -1; } - /* ensure that the secret is a NULL-terminated string */ - password = g_strndup((char*)secret, secretlen); - virSecureErase(secret, secretlen); - - /* for now, just report an error rather than passing the password in - * cleartext on the commandline */ - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("Password not yet supported for nbdkit sources")); - - virSecureEraseString(password); - - return -1; + if (qemuNbdkitCommandPassDataByPipe(cmd, "password", + &secret, secretlen) < 0) + return -1; } - if (proc->source->ncookies > 0) { - /* for now, just report an error rather than passing cookies in - * cleartext on the commandline */ - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("Cookies not yet supported for nbdkit sources")); - return -1; + /* Create a pipe to send the cookies to the nbdkit process. */ + if (proc->source->ncookies) { + g_autofree char *cookies = qemuBlockStorageSourceGetCookieString(proc->source); + + if (qemuNbdkitCommandPassDataByPipe(cmd, "cookie", + (unsigned char**)&cookies, + strlen(cookies)) < 0) + return -1; } if (proc->source->sslverify == VIR_TRISTATE_BOOL_NO) { -- 2.41.0

For virStorageSource objects that contain an nbdkitProcess, start that nbdkit process to serve that network drive and then pass the nbdkit socket to qemu rather than sending the network url to qemu directly. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_block.c | 162 +++++++++++------- src/qemu/qemu_domain.c | 13 +- src/qemu/qemu_extdevice.c | 62 +++++++ src/qemu/qemu_hotplug.c | 7 + src/qemu/qemu_nbdkit.c | 42 +++++ src/qemu/qemu_nbdkit.h | 13 ++ ...sk-cdrom-network-nbdkit.x86_64-latest.args | 42 +++++ .../disk-cdrom-network-nbdkit.xml | 1 + ...isk-network-http-nbdkit.x86_64-latest.args | 44 +++++ .../disk-network-http-nbdkit.xml | 1 + ...rce-curl-nbdkit-backing.x86_64-latest.args | 37 ++++ ...isk-network-source-curl-nbdkit-backing.xml | 45 +++++ ...work-source-curl-nbdkit.x86_64-latest.args | 49 ++++++ .../disk-network-source-curl-nbdkit.xml | 1 + ...isk-network-source-curl.x86_64-latest.args | 52 ++++++ .../disk-network-source-curl.xml | 71 ++++++++ ...disk-network-ssh-nbdkit.x86_64-latest.args | 35 ++++ .../disk-network-ssh-nbdkit.xml | 1 + tests/qemuxml2argvtest.c | 6 + 19 files changed, 618 insertions(+), 66 deletions(-) create mode 100644 tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-http-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-network-http-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl.xml create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-network-ssh-nbdkit.xml diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c index dcdf883926..1a2dc8ffb4 100644 --- a/src/qemu/qemu_block.c +++ b/src/qemu/qemu_block.c @@ -438,6 +438,32 @@ qemuBlockStorageSourceGetCURLProps(virStorageSource *src, } +static virJSONValue * +qemuBlockStorageSourceGetNbdkitProps(virStorageSource *src) +{ + qemuDomainStorageSourcePrivate *srcPriv = QEMU_DOMAIN_STORAGE_SOURCE_PRIVATE(src); + virJSONValue *ret = NULL; + g_autoptr(virJSONValue) serverprops = NULL; + virStorageNetHostDef host = { .transport = VIR_STORAGE_NET_HOST_TRANS_UNIX }; + + /* srcPriv->nbdkitProcess will already be initialized if we can use nbdkit + * to proxy this storage source */ + if (!(srcPriv && srcPriv->nbdkitProcess)) + return NULL; + + host.socket = srcPriv->nbdkitProcess->socketfile; + serverprops = qemuBlockStorageSourceBuildJSONSocketAddress(&host); + + if (!serverprops) + return NULL; + + if (virJSONValueObjectAdd(&ret, "a:server", &serverprops, NULL) < 0) + return NULL; + + return ret; +} + + static virJSONValue * qemuBlockStorageSourceGetISCSIProps(virStorageSource *src, bool onlytarget) @@ -890,69 +916,75 @@ qemuBlockStorageSourceGetBackendProps(virStorageSource *src, return NULL; case VIR_STORAGE_TYPE_NETWORK: - switch ((virStorageNetProtocol) src->protocol) { - case VIR_STORAGE_NET_PROTOCOL_GLUSTER: - driver = "gluster"; - if (!(fileprops = qemuBlockStorageSourceGetGlusterProps(src, onlytarget))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_VXHS: - driver = "vxhs"; - if (!(fileprops = qemuBlockStorageSourceGetVxHSProps(src, onlytarget))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_HTTP: - case VIR_STORAGE_NET_PROTOCOL_HTTPS: - case VIR_STORAGE_NET_PROTOCOL_FTP: - case VIR_STORAGE_NET_PROTOCOL_FTPS: - case VIR_STORAGE_NET_PROTOCOL_TFTP: - driver = virStorageNetProtocolTypeToString(src->protocol); - if (!(fileprops = qemuBlockStorageSourceGetCURLProps(src, onlytarget))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_ISCSI: - driver = "iscsi"; - if (!(fileprops = qemuBlockStorageSourceGetISCSIProps(src, onlytarget))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_NBD: + /* prefer using nbdkit for sources that are supported */ + if ((fileprops = qemuBlockStorageSourceGetNbdkitProps(src))) { driver = "nbd"; - if (!(fileprops = qemuBlockStorageSourceGetNBDProps(src, onlytarget))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_RBD: - driver = "rbd"; - if (!(fileprops = qemuBlockStorageSourceGetRBDProps(src, onlytarget))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG: - driver = "sheepdog"; - if (!(fileprops = qemuBlockStorageSourceGetSheepdogProps(src))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_SSH: - driver = "ssh"; - if (!(fileprops = qemuBlockStorageSourceGetSshProps(src))) - return NULL; break; - - case VIR_STORAGE_NET_PROTOCOL_NFS: - driver = "nfs"; - if (!(fileprops = qemuBlockStorageSourceGetNFSProps(src))) - return NULL; - break; - - case VIR_STORAGE_NET_PROTOCOL_NONE: - case VIR_STORAGE_NET_PROTOCOL_LAST: - virReportEnumRangeError(virStorageNetProtocol, src->protocol); - return NULL; + } else { + switch ((virStorageNetProtocol) src->protocol) { + case VIR_STORAGE_NET_PROTOCOL_GLUSTER: + driver = "gluster"; + if (!(fileprops = qemuBlockStorageSourceGetGlusterProps(src, onlytarget))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_VXHS: + driver = "vxhs"; + if (!(fileprops = qemuBlockStorageSourceGetVxHSProps(src, onlytarget))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_HTTP: + case VIR_STORAGE_NET_PROTOCOL_HTTPS: + case VIR_STORAGE_NET_PROTOCOL_FTP: + case VIR_STORAGE_NET_PROTOCOL_FTPS: + case VIR_STORAGE_NET_PROTOCOL_TFTP: + driver = virStorageNetProtocolTypeToString(src->protocol); + if (!(fileprops = qemuBlockStorageSourceGetCURLProps(src, onlytarget))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_ISCSI: + driver = "iscsi"; + if (!(fileprops = qemuBlockStorageSourceGetISCSIProps(src, onlytarget))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_NBD: + driver = "nbd"; + if (!(fileprops = qemuBlockStorageSourceGetNBDProps(src, onlytarget))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_RBD: + driver = "rbd"; + if (!(fileprops = qemuBlockStorageSourceGetRBDProps(src, onlytarget))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_SHEEPDOG: + driver = "sheepdog"; + if (!(fileprops = qemuBlockStorageSourceGetSheepdogProps(src))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_SSH: + driver = "ssh"; + if (!(fileprops = qemuBlockStorageSourceGetSshProps(src))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_NFS: + driver = "nfs"; + if (!(fileprops = qemuBlockStorageSourceGetNFSProps(src))) + return NULL; + break; + + case VIR_STORAGE_NET_PROTOCOL_NONE: + case VIR_STORAGE_NET_PROTOCOL_LAST: + virReportEnumRangeError(virStorageNetProtocol, src->protocol); + return NULL; + } } break; } @@ -2261,6 +2293,7 @@ qemuBlockStorageSourceCreateGetStorageProps(virStorageSource *src, g_autoptr(virJSONValue) location = NULL; const char *driver = NULL; const char *filename = NULL; + qemuDomainStorageSourcePrivate *srcPriv = QEMU_DOMAIN_STORAGE_SOURCE_PRIVATE(src); switch (actualType) { case VIR_STORAGE_TYPE_FILE: @@ -2289,6 +2322,13 @@ qemuBlockStorageSourceCreateGetStorageProps(virStorageSource *src, break; case VIR_STORAGE_NET_PROTOCOL_SSH: + if (srcPriv->nbdkitProcess) { + /* disk creation not yet supported with nbdkit, and even if it + * was supported, it would not be done with blockdev-create + * props */ + return 0; + } + driver = "ssh"; if (!(location = qemuBlockStorageSourceGetSshProps(src))) return -1; diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 8429ce1028..46fe5a1cf4 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -11167,9 +11167,14 @@ qemuDomainPrepareStorageSourceBlockdevNodename(virDomainDiskDef *disk, if (qemuDomainSecretStorageSourcePrepareEncryption(priv, src, src->nodeformat) < 0) return -1; - if (qemuDomainSecretStorageSourcePrepareAuth(priv, src, - src->nodestorage) < 0) - return -1; + + if (!qemuDomainPrepareStorageSourceNbdkit(src, cfg, src->nodestorage, priv)) { + /* If we're using nbdkit to serve the storage source, we don't pass + * authentication secrets to qemu, but will pass them to nbdkit instead */ + if (qemuDomainSecretStorageSourcePrepareAuth(priv, src, + src->nodestorage) < 0) + return -1; + } if (qemuDomainPrepareStorageSourcePR(src, priv, src->nodestorage) < 0) return -1; @@ -11184,8 +11189,6 @@ qemuDomainPrepareStorageSourceBlockdevNodename(virDomainDiskDef *disk, if (qemuDomainPrepareStorageSourceFDs(src, priv) < 0) return -1; - qemuDomainPrepareStorageSourceNbdkit(src, cfg, src->nodestorage, priv); - return 0; } diff --git a/src/qemu/qemu_extdevice.c b/src/qemu/qemu_extdevice.c index bf59967eb7..42ecdf13d5 100644 --- a/src/qemu/qemu_extdevice.c +++ b/src/qemu/qemu_extdevice.c @@ -232,6 +232,17 @@ qemuExtDevicesStart(virQEMUDriver *driver, return -1; } + for (i = 0; i < def->ndisks; i++) { + virDomainDiskDef *disk = def->disks[i]; + if (qemuNbdkitStartStorageSource(driver, vm, disk->src) < 0) + return -1; + } + + if (def->os.loader && def->os.loader->nvram) { + if (qemuNbdkitStartStorageSource(driver, vm, def->os.loader->nvram) < 0) + return -1; + } + return 0; } @@ -283,6 +294,14 @@ qemuExtDevicesStop(virQEMUDriver *driver, fs->fsdriver == VIR_DOMAIN_FS_DRIVER_TYPE_VIRTIOFS) qemuVirtioFSStop(driver, vm, fs); } + + for (i = 0; i < def->ndisks; i++) { + virDomainDiskDef *disk = def->disks[i]; + qemuNbdkitStopStorageSource(disk->src); + } + + if (def->os.loader && def->os.loader->nvram) + qemuNbdkitStopStorageSource(def->os.loader->nvram); } @@ -319,10 +338,42 @@ qemuExtDevicesHasDevice(virDomainDef *def) return true; } + for (i = 0; i < def->ndisks; i++) { + virDomainDiskDef *disk = def->disks[i]; + virStorageSource *backing; + + for (backing = disk->src; backing; backing = backing->backingStore) { + qemuDomainStorageSourcePrivate* priv = + QEMU_DOMAIN_STORAGE_SOURCE_PRIVATE(backing); + if (priv && priv->nbdkitProcess) + return true; + } + } + + return false; } +/* recursively setup nbdkit cgroups for backing chain of src */ +static int +qemuExtDevicesSetupCgroupNbdkit(virStorageSource *src, + virCgroup *cgroup) +{ + virStorageSource *backing; + + for (backing = src; backing; backing = backing->backingStore) { + qemuDomainStorageSourcePrivate *priv = QEMU_DOMAIN_STORAGE_SOURCE_PRIVATE(src); + + if (priv && priv->nbdkitProcess && + qemuNbdkitProcessSetupCgroup(priv->nbdkitProcess, cgroup) < 0) + return -1; + } + + return 0; +} + + int qemuExtDevicesSetupCgroup(virQEMUDriver *driver, virDomainObj *vm, @@ -365,6 +416,17 @@ qemuExtDevicesSetupCgroup(virQEMUDriver *driver, return -1; } + for (i = 0; i < def->ndisks; i++) { + virDomainDiskDef *disk = def->disks[i]; + if (qemuExtDevicesSetupCgroupNbdkit(disk->src, cgroup) < 0) + return -1; + } + + if (def->os.loader && def->os.loader->nvram) { + if (qemuExtDevicesSetupCgroupNbdkit(def->os.loader->nvram, cgroup) < 0) + return -1; + } + for (i = 0; i < def->nfss; i++) { virDomainFSDef *fs = def->fss[i]; diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index d7d1db8ecc..dc06486922 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -1010,6 +1010,9 @@ qemuDomainAttachDeviceDiskLiveInternal(virQEMUDriver *driver, if (qemuHotplugAttachManagedPR(vm, disk->src, VIR_ASYNC_JOB_NONE) < 0) goto cleanup; + if (qemuNbdkitStartStorageSource(driver, vm, disk->src) < 0) + goto cleanup; + ret = qemuDomainAttachDiskGeneric(vm, disk, VIR_ASYNC_JOB_NONE); virDomainAuditDisk(vm, NULL, disk->src, "attach", ret == 0); @@ -1032,6 +1035,8 @@ qemuDomainAttachDeviceDiskLiveInternal(virQEMUDriver *driver, if (virStorageSourceChainHasManagedPR(disk->src)) ignore_value(qemuHotplugRemoveManagedPR(vm, VIR_ASYNC_JOB_NONE)); + + qemuNbdkitStopStorageSource(disk->src); } qemuDomainSecretDiskDestroy(disk); qemuDomainCleanupStorageSourceFD(disk->src); @@ -4491,6 +4496,8 @@ qemuDomainRemoveDiskDevice(virQEMUDriver *driver, qemuHotplugRemoveManagedPR(vm, VIR_ASYNC_JOB_NONE) < 0) goto cleanup; + qemuNbdkitStopStorageSource(disk->src); + if (disk->transient) { VIR_DEBUG("Removing transient overlay '%s' of disk '%s'", disk->src->path, disk->dst); diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 22a67b0748..299d8824f2 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -752,6 +752,40 @@ qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, } +int +qemuNbdkitStartStorageSource(virQEMUDriver *driver, + virDomainObj *vm, + virStorageSource *src) +{ + virStorageSource *backing; + + for (backing = src; backing != NULL; backing = backing->backingStore) { + qemuDomainStorageSourcePrivate *priv = QEMU_DOMAIN_STORAGE_SOURCE_PRIVATE(src); + + if (priv && priv->nbdkitProcess && + qemuNbdkitProcessStart(priv->nbdkitProcess, vm, driver) < 0) + return -1; + } + + return 0; +} + + +void +qemuNbdkitStopStorageSource(virStorageSource *src) +{ + virStorageSource *backing; + + for (backing = src; backing != NULL; backing = backing->backingStore) { + qemuDomainStorageSourcePrivate *priv = QEMU_DOMAIN_STORAGE_SOURCE_PRIVATE(src); + + if (priv && priv->nbdkitProcess && + qemuNbdkitProcessStop(priv->nbdkitProcess) < 0) + VIR_WARN("Unable to stop nbdkit for storage source '%s'", src->nodestorage); + } +} + + static int qemuNbdkitCommandPassDataByPipe(virCommand *cmd, const char *argName, @@ -939,6 +973,14 @@ qemuNbdkitProcessFree(qemuNbdkitProcess *proc) } +int +qemuNbdkitProcessSetupCgroup(qemuNbdkitProcess *proc, + virCgroup *cgroup) +{ + return virCgroupAddProcess(cgroup, proc->pid); +} + + int qemuNbdkitProcessStart(qemuNbdkitProcess *proc, virDomainObj *vm, diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index 7e2aeed4eb..36a2219d82 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -21,6 +21,7 @@ #include "internal.h" #include "storage_source_conf.h" +#include "vircgroup.h" #include "virenum.h" #include "virfilecache.h" @@ -59,6 +60,14 @@ qemuNbdkitReconnectStorageSource(virStorageSource *source, const char *pidfile, const char *socketfile); +int +qemuNbdkitStartStorageSource(virQEMUDriver *driver, + virDomainObj *vm, + virStorageSource *src); + +void +qemuNbdkitStopStorageSource(virStorageSource *src); + void qemuNbdkitStorageSourceManageProcess(virStorageSource *src); @@ -95,4 +104,8 @@ qemuNbdkitProcessStop(qemuNbdkitProcess *proc); void qemuNbdkitProcessFree(qemuNbdkitProcess *proc); +int +qemuNbdkitProcessSetupCgroup(qemuNbdkitProcess *proc, + virCgroup *cgroup); + G_DEFINE_AUTOPTR_CLEANUP_FUNC(qemuNbdkitProcess, qemuNbdkitProcessFree); diff --git a/tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.x86_64-latest.args b/tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.x86_64-latest.args new file mode 100644 index 0000000000..637f47bf2b --- /dev/null +++ b/tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.x86_64-latest.args @@ -0,0 +1,42 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1 \ +USER=test \ +LOGNAME=test \ +XDG_DATA_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=QEMUGuest1,debug-threads=on \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=off,dump-guest-core=off,memory-backend=pc.ram,acpi=on \ +-accel kvm \ +-cpu qemu64 \ +-m size=1048576k \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":1073741824}' \ +-overcommit mem-lock=off \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,fd=1729,server=on,wait=off \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-boot strict=on \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0x2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/nbdkit-libvirt-3-storage.socket"},"node-name":"libvirt-3-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-3-format","read-only":true,"driver":"raw","file":"libvirt-3-storage"}' \ +-device '{"driver":"ide-cd","bus":"ide.0","unit":0,"drive":"libvirt-3-format","id":"ide0-0-0","bootindex":1}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/nbdkit-libvirt-2-storage.socket"},"node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \ +-device '{"driver":"ide-cd","bus":"ide.0","unit":1,"drive":"libvirt-2-format","id":"ide0-0-1"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/nbdkit-libvirt-1-storage.socket"},"node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \ +-device '{"driver":"ide-cd","bus":"ide.1","unit":0,"drive":"libvirt-1-format","id":"ide0-1-0"}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.0","addr":"0x2"}' \ +-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ +-msg timestamp=on diff --git a/tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.xml b/tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.xml new file mode 120000 index 0000000000..55f677546f --- /dev/null +++ b/tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.xml @@ -0,0 +1 @@ +disk-cdrom-network.xml \ No newline at end of file diff --git a/tests/qemuxml2argvdata/disk-network-http-nbdkit.x86_64-latest.args b/tests/qemuxml2argvdata/disk-network-http-nbdkit.x86_64-latest.args new file mode 100644 index 0000000000..37e46dfc20 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-http-nbdkit.x86_64-latest.args @@ -0,0 +1,44 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1 \ +USER=test \ +LOGNAME=test \ +XDG_DATA_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=QEMUGuest1,debug-threads=on \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=off,dump-guest-core=off,memory-backend=pc.ram,acpi=off \ +-accel kvm \ +-cpu qemu64 \ +-m size=219136k \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}' \ +-overcommit mem-lock=off \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,fd=1729,server=on,wait=off \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-boot strict=on \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0x2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/nbdkit-libvirt-4-storage.socket"},"node-name":"libvirt-4-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-4-format","read-only":false,"driver":"raw","file":"libvirt-4-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x2","drive":"libvirt-4-format","id":"virtio-disk0","bootindex":1}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/nbdkit-libvirt-3-storage.socket"},"node-name":"libvirt-3-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-3-format","read-only":false,"driver":"raw","file":"libvirt-3-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x3","drive":"libvirt-3-format","id":"virtio-disk1"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/nbdkit-libvirt-2-storage.socket"},"node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-2-format","read-only":false,"driver":"raw","file":"libvirt-2-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x4","drive":"libvirt-2-format","id":"virtio-disk2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/nbdkit-libvirt-1-storage.socket"},"node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw","file":"libvirt-1-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x5","drive":"libvirt-1-format","id":"virtio-disk3"}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ +-msg timestamp=on diff --git a/tests/qemuxml2argvdata/disk-network-http-nbdkit.xml b/tests/qemuxml2argvdata/disk-network-http-nbdkit.xml new file mode 120000 index 0000000000..6a05204e8a --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-http-nbdkit.xml @@ -0,0 +1 @@ +disk-network-http.xml \ No newline at end of file diff --git a/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.x86_64-latest.args b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.x86_64-latest.args new file mode 100644 index 0000000000..25ec2513c2 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.x86_64-latest.args @@ -0,0 +1,37 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1 \ +USER=test \ +LOGNAME=test \ +XDG_DATA_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=QEMUGuest1,debug-threads=on \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=off,dump-guest-core=off,memory-backend=pc.ram,acpi=off \ +-accel tcg \ +-cpu qemu64 \ +-m size=219136k \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}' \ +-overcommit mem-lock=off \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,fd=1729,server=on,wait=off \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-boot strict=on \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0x2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/nbdkit-libvirt-2-storage.socket"},"node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"qcow2","file":"libvirt-2-storage"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/nbdkit-libvirt-1-storage.socket"},"node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"qcow2","file":"libvirt-1-storage","backing":"libvirt-2-format"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x2","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ +-msg timestamp=on diff --git a/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.xml b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.xml new file mode 100644 index 0000000000..37a30fcbd6 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.xml @@ -0,0 +1,45 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='x86_64' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-x86_64</emulator> + <disk type='network' device='disk'> + <driver name='qemu' type='qcow2'/> + <source protocol='https' name='path/to/disk1.qcow2'> + <host name='https.example.org' port='8443'/> + <cookies> + <cookie name='cookie1'>cookievalue1</cookie> + <cookie name='cookie2'>cookievalue2</cookie> + </cookies> + </source> + <backingStore type='network'> + <format type='qcow2'/> + <source protocol='https' name='path/to/backing.qcow2'> + <host name='https.example2.org' port='8444'/> + <cookies> + <cookie name='cookie3'>cookievalue3</cookie> + <cookie name='cookie4'>cookievalue4</cookie> + </cookies> + </source> + </backingStore> + <target dev='vda' bus='virtio'/> + <readonly/> + </disk> + <controller type='usb' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='none'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.x86_64-latest.args b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.x86_64-latest.args new file mode 100644 index 0000000000..c88661f369 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.x86_64-latest.args @@ -0,0 +1,49 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1 \ +USER=test \ +LOGNAME=test \ +XDG_DATA_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=QEMUGuest1,debug-threads=on \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=off,dump-guest-core=off,memory-backend=pc.ram,acpi=off \ +-accel tcg \ +-cpu qemu64 \ +-m size=219136k \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}' \ +-overcommit mem-lock=off \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,fd=1729,server=on,wait=off \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-boot strict=on \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0x2"}' \ +-device '{"driver":"ahci","id":"sata0","bus":"pci.0","addr":"0x2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/nbdkit-libvirt-5-storage.socket"},"node-name":"libvirt-5-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-5-format","read-only":true,"driver":"raw","file":"libvirt-5-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x3","drive":"libvirt-5-format","id":"virtio-disk0","bootindex":1}' \ +-object '{"qom-type":"secret","id":"libvirt-4-format-encryption-secret0","data":"9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1","keyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw==","format":"base64"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/nbdkit-libvirt-4-storage.socket"},"node-name":"libvirt-4-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-4-format","read-only":false,"driver":"luks","key-secret":"libvirt-4-format-encryption-secret0","file":"libvirt-4-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x4","drive":"libvirt-4-format","id":"virtio-disk4"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/nbdkit-libvirt-3-storage.socket"},"node-name":"libvirt-3-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-3-format","read-only":true,"driver":"raw","file":"libvirt-3-storage"}' \ +-device '{"driver":"ide-cd","bus":"sata0.1","drive":"libvirt-3-format","id":"sata0-0-1"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/nbdkit-libvirt-2-storage.socket"},"node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \ +-device '{"driver":"ide-cd","bus":"sata0.2","drive":"libvirt-2-format","id":"sata0-0-2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/nbdkit-libvirt-1-storage.socket"},"node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \ +-device '{"driver":"ide-cd","bus":"sata0.3","drive":"libvirt-1-format","id":"sata0-0-3"}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ +-msg timestamp=on diff --git a/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.xml b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.xml new file mode 120000 index 0000000000..4a1e40bd70 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.xml @@ -0,0 +1 @@ +disk-network-source-curl.xml \ No newline at end of file diff --git a/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args b/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args new file mode 100644 index 0000000000..cb0e5a92ea --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args @@ -0,0 +1,52 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1 \ +USER=test \ +LOGNAME=test \ +XDG_DATA_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=QEMUGuest1,debug-threads=on \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=off,dump-guest-core=off,memory-backend=pc.ram,acpi=off \ +-accel tcg \ +-cpu qemu64 \ +-m size=219136k \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}' \ +-overcommit mem-lock=off \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,fd=1729,server=on,wait=off \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-boot strict=on \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0x2"}' \ +-device '{"driver":"ahci","id":"sata0","bus":"pci.0","addr":"0x2"}' \ +-object '{"qom-type":"secret","id":"libvirt-5-storage-httpcookie-secret0","data":"BUU0KmnWfonHdjzhYhwVQZ5iTI1KweTJ22q8XWUVoBCVu1z70reDuczPBIabZtC3","keyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw==","format":"base64"}' \ +-blockdev '{"driver":"https","url":"https://https.example.org:8443/path/to/disk1.iso","cookie-secret":"libvirt-5-storage-httpcookie-secret0","node-name":"libvirt-5-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-5-format","read-only":true,"driver":"raw","file":"libvirt-5-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x3","drive":"libvirt-5-format","id":"virtio-disk0","bootindex":1}' \ +-object '{"qom-type":"secret","id":"libvirt-4-storage-httpcookie-secret0","data":"BUU0KmnWfonHdjzhYhwVQZ5iTI1KweTJ22q8XWUVoBCVu1z70reDuczPBIabZtC3","keyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw==","format":"base64"}' \ +-object '{"qom-type":"secret","id":"libvirt-4-format-encryption-secret0","data":"9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1","keyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw==","format":"base64"}' \ +-blockdev '{"driver":"https","url":"https://https.example.org:8443/path/to/disk5.iso?foo=bar","sslverify":false,"cookie-secret":"libvirt-4-storage-httpcookie-secret0","node-name":"libvirt-4-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-4-format","read-only":false,"driver":"luks","key-secret":"libvirt-4-format-encryption-secret0","file":"libvirt-4-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x4","drive":"libvirt-4-format","id":"virtio-disk4"}' \ +-object '{"qom-type":"secret","id":"libvirt-3-storage-httpcookie-secret0","data":"BUU0KmnWfonHdjzhYhwVQZ5iTI1KweTJ22q8XWUVoBBv7TuTgTkyAyOPpC2P5qLbOIypLoHpppjz+u5O+X8oT+jA1m7q/OJQ8dk2EFD5c0A=","keyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw==","format":"base64"}' \ +-blockdev '{"driver":"http","url":"http://http.example.org:8080/path/to/disk2.iso","cookie-secret":"libvirt-3-storage-httpcookie-secret0","node-name":"libvirt-3-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-3-format","read-only":true,"driver":"raw","file":"libvirt-3-storage"}' \ +-device '{"driver":"ide-cd","bus":"sata0.1","drive":"libvirt-3-format","id":"sata0-0-1"}' \ +-blockdev '{"driver":"ftp","url":"ftp://ftp.example.org:20/path/to/disk3.iso","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \ +-device '{"driver":"ide-cd","bus":"sata0.2","drive":"libvirt-2-format","id":"sata0-0-2"}' \ +-blockdev '{"driver":"ftps","url":"ftps://ftps.example.org:22/path/to/disk4.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \ +-device '{"driver":"ide-cd","bus":"sata0.3","drive":"libvirt-1-format","id":"sata0-0-3"}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ +-msg timestamp=on diff --git a/tests/qemuxml2argvdata/disk-network-source-curl.xml b/tests/qemuxml2argvdata/disk-network-source-curl.xml new file mode 100644 index 0000000000..1e50314abe --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-source-curl.xml @@ -0,0 +1,71 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='x86_64' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-x86_64</emulator> + <disk type='network' device='disk'> + <source protocol='https' name='path/to/disk1.iso'> + <host name='https.example.org' port='8443'/> + <cookies> + <cookie name='cookie1'>cookievalue1</cookie> + <cookie name='cookie2'>cookievalue2</cookie> + </cookies> + </source> + <target dev='vda' bus='virtio'/> + <readonly/> + </disk> + <disk type='network' device='cdrom'> + <source protocol='http' name='path/to/disk2.iso'> + <host name='http.example.org' port='8080'/> + <cookies> + <cookie name='cookie1'>cookievalue1</cookie> + <cookie name='cookie2'>cookievalue2</cookie> + <cookie name='cookie3'>cookievalue3</cookie> + </cookies> + </source> + <target dev='hdb' bus='sata'/> + </disk> + <disk type='network' device='cdrom'> + <source protocol='ftp' name='path/to/disk3.iso'> + <host name='ftp.example.org' port='20'/> + </source> + <target dev='hdc' bus='sata'/> + </disk> + <disk type='network' device='cdrom'> + <source protocol='ftps' name='path/to/disk4.iso'> + <host name='ftps.example.org' port='22'/> + </source> + <target dev='hdd' bus='sata'/> + </disk> + <disk type='network' device='disk'> + <source protocol='https' name='path/to/disk5.iso' query='foo=bar'> + <host name='https.example.org' port='8443'/> + <ssl verify='no'/> + <cookies> + <cookie name='cookie1'>cookievalue1</cookie> + <cookie name='cookie2'>cookievalue2</cookie> + </cookies> + <encryption format='luks'> + <secret type='passphrase' uuid='1148b693-0843-4cef-9f97-8feb4e1ae365'/> + </encryption> + </source> + <target dev='vde' bus='virtio'/> + </disk> + <controller type='usb' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='none'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvdata/disk-network-ssh-nbdkit.x86_64-latest.args b/tests/qemuxml2argvdata/disk-network-ssh-nbdkit.x86_64-latest.args new file mode 100644 index 0000000000..fd24e51570 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-ssh-nbdkit.x86_64-latest.args @@ -0,0 +1,35 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1 \ +USER=test \ +LOGNAME=test \ +XDG_DATA_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=QEMUGuest1,debug-threads=on \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=off,dump-guest-core=off,memory-backend=pc.ram,acpi=off \ +-accel kvm \ +-cpu qemu64 \ +-m size=219136k \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}' \ +-overcommit mem-lock=off \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,fd=1729,server=on,wait=off \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-boot strict=on \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0x2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/nbdkit-libvirt-1-storage.socket"},"node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw","file":"libvirt-1-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x2","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ +-msg timestamp=on diff --git a/tests/qemuxml2argvdata/disk-network-ssh-nbdkit.xml b/tests/qemuxml2argvdata/disk-network-ssh-nbdkit.xml new file mode 120000 index 0000000000..b0589bdfb5 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-ssh-nbdkit.xml @@ -0,0 +1 @@ +disk-network-ssh.xml \ No newline at end of file diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index d64c21ae17..0304f66f1d 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1208,6 +1208,7 @@ mymain(void) DO_TEST_CAPS_LATEST("disk-cdrom-empty-network-invalid"); DO_TEST_CAPS_LATEST("disk-cdrom-bus-other"); DO_TEST_CAPS_LATEST("disk-cdrom-network"); + DO_TEST_CAPS_LATEST_NBDKIT("disk-cdrom-network-nbdkit", QEMU_NBDKIT_CAPS_PLUGIN_CURL); DO_TEST_CAPS_LATEST("disk-cdrom-tray"); DO_TEST_CAPS_LATEST("disk-floppy"); DO_TEST_CAPS_LATEST("disk-floppy-q35"); @@ -1252,6 +1253,9 @@ mymain(void) /* qemu-6.0 is the last qemu version supporting sheepdog */ DO_TEST_CAPS_VER("disk-network-sheepdog", "6.0.0"); DO_TEST_CAPS_LATEST("disk-network-source-auth"); + DO_TEST_CAPS_LATEST("disk-network-source-curl"); + DO_TEST_CAPS_LATEST_NBDKIT("disk-network-source-curl-nbdkit", QEMU_NBDKIT_CAPS_PLUGIN_CURL); + DO_TEST_CAPS_LATEST_NBDKIT("disk-network-source-curl-nbdkit-backing", QEMU_NBDKIT_CAPS_PLUGIN_CURL); DO_TEST_CAPS_LATEST("disk-network-nfs"); driver.config->vxhsTLS = 1; driver.config->nbdTLSx509secretUUID = g_strdup("6fd3f62d-9fe7-4a4e-a869-7acd6376d8ea"); @@ -1262,10 +1266,12 @@ mymain(void) DO_TEST_CAPS_LATEST("disk-network-tlsx509-nbd-hostname"); DO_TEST_CAPS_VER("disk-network-tlsx509-vxhs", "5.0.0"); DO_TEST_CAPS_LATEST("disk-network-http"); + DO_TEST_CAPS_LATEST_NBDKIT("disk-network-http-nbdkit", QEMU_NBDKIT_CAPS_PLUGIN_CURL); VIR_FREE(driver.config->nbdTLSx509secretUUID); VIR_FREE(driver.config->vxhsTLSx509secretUUID); driver.config->vxhsTLS = 0; DO_TEST_CAPS_LATEST("disk-network-ssh"); + DO_TEST_CAPS_LATEST_NBDKIT("disk-network-ssh-nbdkit", QEMU_NBDKIT_CAPS_PLUGIN_SSH); DO_TEST_CAPS_LATEST("disk-no-boot"); DO_TEST_CAPS_LATEST("disk-nvme"); DO_TEST_CAPS_VER("disk-vhostuser-numa", "4.2.0"); -- 2.41.0

Add a private function to peek at the list of send buffers in virCommand so that it is testable Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/libvirt_private.syms | 1 + src/util/vircommand.c | 17 +++++++++-------- src/util/vircommand.h | 8 ++++++++ src/util/vircommandpriv.h | 4 ++++ 4 files changed, 22 insertions(+), 8 deletions(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 1e3e407097..e4da5388e7 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2117,6 +2117,7 @@ virCommandNewArgs; virCommandNewVAList; virCommandNonblockingFDs; virCommandPassFD; +virCommandPeekSendBuffers; virCommandRawStatus; virCommandRequireHandshake; virCommandRun; diff --git a/src/util/vircommand.c b/src/util/vircommand.c index 899d413dd2..7d7ce4297e 100644 --- a/src/util/vircommand.c +++ b/src/util/vircommand.c @@ -78,14 +78,6 @@ struct _virCommandFD { unsigned int flags; }; -typedef struct _virCommandSendBuffer virCommandSendBuffer; -struct _virCommandSendBuffer { - int fd; - unsigned char *buffer; - size_t buflen; - off_t offset; -}; - struct _virCommand { int has_error; /* 0 on success, -1 on error */ @@ -3515,3 +3507,12 @@ virCommandSetRunAmong(virCommand *cmd, cmd->schedCore = pid; } + +void +virCommandPeekSendBuffers(virCommand *cmd, + virCommandSendBuffer **buffers, + int *nbuffers) +{ + *buffers = cmd->sendBuffers; + *nbuffers = cmd->numSendBuffers; +} diff --git a/src/util/vircommand.h b/src/util/vircommand.h index d51449ac90..9bcdce35b9 100644 --- a/src/util/vircommand.h +++ b/src/util/vircommand.h @@ -24,6 +24,14 @@ #include "internal.h" #include "virbuffer.h" +typedef struct _virCommandSendBuffer virCommandSendBuffer; +struct _virCommandSendBuffer { + int fd; + unsigned char *buffer; + size_t buflen; + off_t offset; +}; + typedef struct _virCommand virCommand; /* This will execute in the context of the first child diff --git a/src/util/vircommandpriv.h b/src/util/vircommandpriv.h index ff17fa5ded..d579810bb5 100644 --- a/src/util/vircommandpriv.h +++ b/src/util/vircommandpriv.h @@ -47,3 +47,7 @@ void virCommandSetDryRun(virCommandDryRunToken *tok, bool bufCommandStripPath, virCommandDryRunCallback cb, void *opaque); + +void virCommandPeekSendBuffers(virCommand *cmd, + virCommandSendBuffer **buffers, + int *nbuffers); -- 2.41.0

We were testing the arguments that were being passed to qemu when a disk was being served by nbdkit, but the arguments used to start nbdkit itself were not testable. This adds a test to ensure that we're invoking nbdkit correctly for various disk source definitions. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- build-aux/syntax-check.mk | 2 +- src/qemu/qemu_nbdkit.c | 4 +- src/qemu/qemu_nbdkitpriv.h | 31 ++ tests/meson.build | 1 + .../disk-cdrom-network.args.disk0 | 6 + .../disk-cdrom-network.args.disk1 | 8 + .../disk-cdrom-network.args.disk1.pipe.778 | 1 + .../disk-cdrom-network.args.disk2 | 8 + .../disk-cdrom-network.args.disk2.pipe.780 | 1 + .../disk-network-http.args.disk0 | 6 + .../disk-network-http.args.disk1 | 5 + .../disk-network-http.args.disk2 | 6 + .../disk-network-http.args.disk2.pipe.778 | 1 + .../disk-network-http.args.disk3 | 7 + .../disk-network-http.args.disk3.pipe.780 | 1 + ...work-source-curl-nbdkit-backing.args.disk0 | 7 + ...ce-curl-nbdkit-backing.args.disk0.pipe.778 | 1 + .../disk-network-source-curl.args.disk0 | 7 + ...sk-network-source-curl.args.disk0.pipe.778 | 1 + .../disk-network-source-curl.args.disk1 | 7 + ...sk-network-source-curl.args.disk1.pipe.780 | 1 + .../disk-network-source-curl.args.disk2 | 7 + ...sk-network-source-curl.args.disk2.pipe.782 | 1 + .../disk-network-source-curl.args.disk3 | 6 + .../disk-network-source-curl.args.disk4 | 6 + .../disk-network-ssh.args.disk0 | 6 + tests/qemunbdkittest.c | 308 ++++++++++++++++++ 27 files changed, 444 insertions(+), 2 deletions(-) create mode 100644 src/qemu/qemu_nbdkitpriv.h create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk1.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk2 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk2.pipe.780 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk2 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk2.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk3 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk3.pipe.780 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.disk0.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk0.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.780 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.782 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk3 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk4 create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk0 create mode 100644 tests/qemunbdkittest.c diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk index 64c1e2773e..ec04402133 100644 --- a/build-aux/syntax-check.mk +++ b/build-aux/syntax-check.mk @@ -1370,7 +1370,7 @@ exclude_file_name_regexp--sc_prohibit_close = \ (\.p[yl]$$|\.spec\.in$$|^docs/|^(src/util/vir(file|event)\.c|src/libvirt-stream\.c|tests/(vir.+mock\.c|commandhelper\.c|qemusecuritymock\.c)|tools/nss/libvirt_nss_(leases|macs)\.c)|tools/virt-qemu-qmp-proxy$$) exclude_file_name_regexp--sc_prohibit_empty_lines_at_EOF = \ - (^tests/(nodedevmdevctl|viracpi|virhostcpu|virpcitest|virstoragetest)data/|docs/js/.*\.js|docs/fonts/.*\.woff|\.diff|tests/virconfdata/no-newline\.conf$$) + (^tests/(nodedevmdevctl|viracpi|virhostcpu|virpcitest|virstoragetest|qemunbdkit)data/|docs/js/.*\.js|docs/fonts/.*\.woff|\.diff|tests/virconfdata/no-newline\.conf$$) exclude_file_name_regexp--sc_prohibit_fork_wrappers = \ (^(src/(util/(vircommand|virdaemon)|lxc/lxc_controller)|tests/testutils)\.c$$) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 299d8824f2..df638e99c0 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -31,6 +31,8 @@ #include "qemu_domain.h" #include "qemu_extdevice.h" #include "qemu_nbdkit.h" +#define LIBVIRT_QEMU_NBDKITPRIV_H_ALLOW +#include "qemu_nbdkitpriv.h" #include "qemu_security.h" #include <fcntl.h> @@ -912,7 +914,7 @@ qemuNbdkitProcessBuildCommandSSH(qemuNbdkitProcess *proc, } -static virCommand * +virCommand * qemuNbdkitProcessBuildCommand(qemuNbdkitProcess *proc) { g_autoptr(virCommand) cmd = virCommandNewArgList(proc->caps->path, diff --git a/src/qemu/qemu_nbdkitpriv.h b/src/qemu/qemu_nbdkitpriv.h new file mode 100644 index 0000000000..64f9bb99d8 --- /dev/null +++ b/src/qemu/qemu_nbdkitpriv.h @@ -0,0 +1,31 @@ +/* + * qemu_nbdkitpriv.h: exposing some functions for testing + * + * Copyright (C) 2021 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + */ + +#ifndef LIBVIRT_QEMU_NBDKITPRIV_H_ALLOW +# error "qemu_nbdkitpriv.h may only be included by qemu_nbdkit.c or test suites" +#endif /* LIBVIRT_QEMU_NBDKITPRIV_H_ALLOW */ + +#pragma once + +#include "qemu_nbdkit.h" + +virCommand * +qemuNbdkitProcessBuildCommand(qemuNbdkitProcess *proc); diff --git a/tests/meson.build b/tests/meson.build index e76289da62..f05774263c 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -456,6 +456,7 @@ if conf.has('WITH_QEMU') { 'name': 'qemuvhostusertest', 'link_with': [ test_qemu_driver_lib ], 'link_whole': [ test_file_wrapper_lib ] }, { 'name': 'qemuxml2argvtest', 'link_with': [ test_qemu_driver_lib, test_utils_qemu_monitor_lib ], 'link_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, { 'name': 'qemuxml2xmltest', 'link_with': [ test_qemu_driver_lib ], 'link_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, + { 'name': 'qemunbdkittest', 'link_with': [ test_qemu_driver_lib ], 'link_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, ] endif diff --git a/tests/qemunbdkitdata/disk-cdrom-network.args.disk0 b/tests/qemunbdkitdata/disk-cdrom-network.args.disk0 new file mode 100644 index 0000000000..b2f3be4cba --- /dev/null +++ b/tests/qemunbdkitdata/disk-cdrom-network.args.disk0 @@ -0,0 +1,6 @@ +nbdkit \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground \ +--readonly curl \ +protocols=ftp \ +url=ftp://host.name:21/url/path/file.iso diff --git a/tests/qemunbdkitdata/disk-cdrom-network.args.disk1 b/tests/qemunbdkitdata/disk-cdrom-network.args.disk1 new file mode 100644 index 0000000000..a23f6573d6 --- /dev/null +++ b/tests/qemunbdkitdata/disk-cdrom-network.args.disk1 @@ -0,0 +1,8 @@ +nbdkit \ +--unix /tmp/statedir-1/nbdkit-test-disk-1.socket \ +--foreground \ +--readonly curl \ +protocols=ftps \ +url=ftps://host.name:990/url/path/file.iso \ +user=testuser \ +password=-777 diff --git a/tests/qemunbdkitdata/disk-cdrom-network.args.disk1.pipe.778 b/tests/qemunbdkitdata/disk-cdrom-network.args.disk1.pipe.778 new file mode 100644 index 0000000000..ccdd4033fc --- /dev/null +++ b/tests/qemunbdkitdata/disk-cdrom-network.args.disk1.pipe.778 @@ -0,0 +1 @@ +iscsi-mycluster_myname-secret \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-cdrom-network.args.disk2 b/tests/qemunbdkitdata/disk-cdrom-network.args.disk2 new file mode 100644 index 0000000000..04e918609a --- /dev/null +++ b/tests/qemunbdkitdata/disk-cdrom-network.args.disk2 @@ -0,0 +1,8 @@ +nbdkit \ +--unix /tmp/statedir-2/nbdkit-test-disk-2.socket \ +--foreground \ +--readonly curl \ +protocols=https \ +'url=https://host.name:443/url/path/file.iso?test=val' \ +user=testuser \ +password=-779 diff --git a/tests/qemunbdkitdata/disk-cdrom-network.args.disk2.pipe.780 b/tests/qemunbdkitdata/disk-cdrom-network.args.disk2.pipe.780 new file mode 100644 index 0000000000..ccdd4033fc --- /dev/null +++ b/tests/qemunbdkitdata/disk-cdrom-network.args.disk2.pipe.780 @@ -0,0 +1 @@ +iscsi-mycluster_myname-secret \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-http.args.disk0 b/tests/qemunbdkitdata/disk-network-http.args.disk0 new file mode 100644 index 0000000000..8316f353cb --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-http.args.disk0 @@ -0,0 +1,6 @@ +nbdkit \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground curl \ +protocols=http,https \ +url=http://example.org:80/test.img \ +timeout=1234 diff --git a/tests/qemunbdkitdata/disk-network-http.args.disk1 b/tests/qemunbdkitdata/disk-network-http.args.disk1 new file mode 100644 index 0000000000..a546a68b27 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-http.args.disk1 @@ -0,0 +1,5 @@ +nbdkit \ +--unix /tmp/statedir-1/nbdkit-test-disk-1.socket \ +--foreground curl \ +protocols=https \ +url=https://example.org:443/test2.img diff --git a/tests/qemunbdkitdata/disk-network-http.args.disk2 b/tests/qemunbdkitdata/disk-network-http.args.disk2 new file mode 100644 index 0000000000..1004547b3a --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-http.args.disk2 @@ -0,0 +1,6 @@ +nbdkit \ +--unix /tmp/statedir-2/nbdkit-test-disk-2.socket \ +--foreground curl \ +protocols=http,https \ +url=http://example.org:1234/test3.img \ +cookie=-777 diff --git a/tests/qemunbdkitdata/disk-network-http.args.disk2.pipe.778 b/tests/qemunbdkitdata/disk-network-http.args.disk2.pipe.778 new file mode 100644 index 0000000000..2c42c95930 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-http.args.disk2.pipe.778 @@ -0,0 +1 @@ +test=testcookievalue; test2="blurb" \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-http.args.disk3 b/tests/qemunbdkitdata/disk-network-http.args.disk3 new file mode 100644 index 0000000000..e3c357b89a --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-http.args.disk3 @@ -0,0 +1,7 @@ +nbdkit \ +--unix /tmp/statedir-3/nbdkit-test-disk-3.socket \ +--foreground curl \ +protocols=https \ +'url=https://example.org:1234/test4.img?par=val&other=ble' \ +cookie=-779 \ +sslverify=false diff --git a/tests/qemunbdkitdata/disk-network-http.args.disk3.pipe.780 b/tests/qemunbdkitdata/disk-network-http.args.disk3.pipe.780 new file mode 100644 index 0000000000..2c42c95930 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-http.args.disk3.pipe.780 @@ -0,0 +1 @@ +test=testcookievalue; test2="blurb" \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.disk0 b/tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.disk0 new file mode 100644 index 0000000000..605354433b --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.disk0 @@ -0,0 +1,7 @@ +nbdkit \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground \ +--readonly curl \ +protocols=https \ +url=https://https.example.org:8443/path/to/disk1.qcow2 \ +cookie=-777 diff --git a/tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.disk0.pipe.778 b/tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.disk0.pipe.778 new file mode 100644 index 0000000000..20af4ae383 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.disk0.pipe.778 @@ -0,0 +1 @@ +cookie1=cookievalue1; cookie2=cookievalue2 \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk0 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk0 new file mode 100644 index 0000000000..948dbfbe5a --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk0 @@ -0,0 +1,7 @@ +nbdkit \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground \ +--readonly curl \ +protocols=https \ +url=https://https.example.org:8443/path/to/disk1.iso \ +cookie=-777 diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk0.pipe.778 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk0.pipe.778 new file mode 100644 index 0000000000..20af4ae383 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk0.pipe.778 @@ -0,0 +1 @@ +cookie1=cookievalue1; cookie2=cookievalue2 \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1 new file mode 100644 index 0000000000..fde6a4f533 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1 @@ -0,0 +1,7 @@ +nbdkit \ +--unix /tmp/statedir-1/nbdkit-test-disk-1.socket \ +--foreground curl \ +protocols=https \ +'url=https://https.example.org:8443/path/to/disk5.iso?foo=bar' \ +cookie=-779 \ +sslverify=false diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.780 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.780 new file mode 100644 index 0000000000..20af4ae383 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.780 @@ -0,0 +1 @@ +cookie1=cookievalue1; cookie2=cookievalue2 \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk2 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2 new file mode 100644 index 0000000000..88c9fa35a1 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2 @@ -0,0 +1,7 @@ +nbdkit \ +--unix /tmp/statedir-2/nbdkit-test-disk-2.socket \ +--foreground \ +--readonly curl \ +protocols=http,https \ +url=http://http.example.org:8080/path/to/disk2.iso \ +cookie=-781 diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.782 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.782 new file mode 100644 index 0000000000..5c035e84c5 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.782 @@ -0,0 +1 @@ +cookie1=cookievalue1; cookie2=cookievalue2; cookie3=cookievalue3 \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk3 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk3 new file mode 100644 index 0000000000..f517baa948 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk3 @@ -0,0 +1,6 @@ +nbdkit \ +--unix /tmp/statedir-3/nbdkit-test-disk-3.socket \ +--foreground \ +--readonly curl \ +protocols=ftp \ +url=ftp://ftp.example.org:20/path/to/disk3.iso diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk4 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk4 new file mode 100644 index 0000000000..1df47a9d54 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk4 @@ -0,0 +1,6 @@ +nbdkit \ +--unix /tmp/statedir-4/nbdkit-test-disk-4.socket \ +--foreground \ +--readonly curl \ +protocols=ftps \ +url=ftps://ftps.example.org:22/path/to/disk4.iso diff --git a/tests/qemunbdkitdata/disk-network-ssh.args.disk0 b/tests/qemunbdkitdata/disk-network-ssh.args.disk0 new file mode 100644 index 0000000000..c04dc8bb03 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-ssh.args.disk0 @@ -0,0 +1,6 @@ +nbdkit \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground ssh \ +host=example.org \ +port=2222 \ +path=test.img diff --git a/tests/qemunbdkittest.c b/tests/qemunbdkittest.c new file mode 100644 index 0000000000..2a74f27a5a --- /dev/null +++ b/tests/qemunbdkittest.c @@ -0,0 +1,308 @@ +#include <config.h> + +#include <fcntl.h> +#include "internal.h" +#include "testutils.h" +#include "testutilsqemu.h" +#include "qemu/qemu_domain.h" +#include "qemu/qemu_nbdkit.h" +#define LIBVIRT_QEMU_NBDKITPRIV_H_ALLOW +#include "qemu/qemu_nbdkitpriv.h" +#include "vircommand.h" +#define LIBVIRT_VIRCOMMANDPRIV_H_ALLOW +#include "vircommandpriv.h" +#include "virutil.h" +#include "virsecret.h" +#include "datatypes.h" +#include "virmock.h" + +#define VIR_FROM_THIS VIR_FROM_QEMU + +static virQEMUDriver driver; + + +/* Some mock implementations for testing */ +#define PIPE_FD_START 777 +static int mockpipefd = PIPE_FD_START; + +static int (*real_virPipeQuiet)(int fds[2]); +static void +init_syms(void) +{ + VIR_MOCK_REAL_INIT(virPipeQuiet); +} + +static int +moveToStableFd(int fd) +{ + int newfd; + + /* don't overwrite an existing fd */ + if (fcntl(mockpipefd, F_GETFD) != -1) + abort(); + + newfd = dup2(fd, mockpipefd++); + + VIR_FORCE_CLOSE(fd); + + return newfd; +} + + +int +virPipeQuiet(int fds[2]) +{ + int tempfds[2]; + + init_syms(); + + if (real_virPipeQuiet(tempfds) < 0) + return -1; + + if ((fds[0] = moveToStableFd(tempfds[0])) < 0 || + (fds[1] = moveToStableFd(tempfds[1])) < 0) + return -1; + + return 0; +} + + +int +virSecretGetSecretString(virConnectPtr conn G_GNUC_UNUSED, + virSecretLookupTypeDef *seclookupdef, + virSecretUsageType secretUsageType, + uint8_t **secret, + size_t *secret_size) +{ + char uuidstr[VIR_UUID_BUFLEN]; + const char *secretname = NULL; + char *tmp = NULL; + + switch (seclookupdef->type) { + case VIR_SECRET_LOOKUP_TYPE_UUID: + virUUIDFormat(seclookupdef->u.uuid, uuidstr); + secretname = uuidstr; + break; + case VIR_SECRET_LOOKUP_TYPE_USAGE: + secretname = seclookupdef->u.usage; + break; + case VIR_SECRET_LOOKUP_TYPE_NONE: + case VIR_SECRET_LOOKUP_TYPE_LAST: + default: + virReportEnumRangeError(virSecretLookupType, seclookupdef->type); + return -1; + }; + + /* For testing, just generate a value for the secret that includes the type + * and the id of the secret */ + tmp = g_strdup_printf("%s-%s-secret", virSecretUsageTypeToString(secretUsageType), secretname); + *secret = (uint8_t*)tmp; + *secret_size = strlen(tmp) + 1; + + return 0; +} + +virConnectPtr virGetConnectSecret(void) +{ + return virGetConnect(); +} + +/* end of mock implementations */ + + +typedef struct { + const char *name; + char* infile; + char* outtemplate; + qemuNbdkitCaps *nbdkitcaps; + bool expectFail; +} TestInfo; + + +typedef enum { + NBDKIT_ARG_CAPS, + NBDKIT_ARG_EXPECT_FAIL, + NBDKIT_ARG_END +} NbdkitArgName; + + +static void +testInfoSetPaths(TestInfo *info) +{ + info->infile = g_strdup_printf("%s/qemuxml2argvdata/%s.xml", + abs_srcdir, info->name); + info->outtemplate = g_strdup_printf("%s/qemunbdkitdata/%s", + abs_srcdir, info->name); +} + +static void +testInfoClear(TestInfo *info) +{ + g_free(info->infile); + g_free(info->outtemplate); + g_clear_object(&info->nbdkitcaps); +} + +static void +testInfoSetArgs(TestInfo *info, ...) +{ + va_list argptr; + NbdkitArgName argname; + unsigned int cap; + + va_start(argptr, info); + while ((argname = va_arg(argptr, NbdkitArgName)) != NBDKIT_ARG_END) { + switch (argname) { + case NBDKIT_ARG_CAPS: + while ((cap = va_arg(argptr, unsigned int)) < QEMU_NBDKIT_CAPS_LAST) + qemuNbdkitCapsSet(info->nbdkitcaps, cap); + break; + case NBDKIT_ARG_EXPECT_FAIL: + info->expectFail = va_arg(argptr, unsigned int); + break; + case NBDKIT_ARG_END: + default: + break; + } + } +} + + +static int +testNbdkit(const void *data) +{ + const TestInfo *info = data; + g_autoptr(virDomainDef) def = NULL; + size_t i; + int ret = 0; + + /* restart mock pipe fds so tests are consistent */ + mockpipefd = PIPE_FD_START; + + if (!virFileExists(info->infile)) { + virReportError(VIR_ERR_INTERNAL_ERROR, + "Test input file '%s' is missing", info->infile); + return -1; + } + + if (!(def = virDomainDefParseFile(info->infile, driver.xmlopt, NULL, + VIR_DOMAIN_DEF_PARSE_SKIP_VALIDATE))) + return -1; + + for (i = 0; i < def->ndisks; i++) { + virDomainDiskDef *disk = def->disks[i]; + g_autofree char *statedir = g_strdup_printf("/tmp/statedir-%zi", i); + g_autofree char *alias = g_strdup_printf("test-disk-%zi", i); + g_autofree char *cmdfile = g_strdup_printf("%s.args.disk%zi", + info->outtemplate, i); + + if (qemuNbdkitInitStorageSource(info->nbdkitcaps, disk->src, statedir, + alias, 101, 101)) { + qemuDomainStorageSourcePrivate *srcPriv = + qemuDomainStorageSourcePrivateFetch(disk->src); + g_autoptr(virCommand) cmd = NULL; + g_autoptr(virCommandDryRunToken) dryRunToken = virCommandDryRunTokenNew(); + g_auto(virBuffer) buf = VIR_BUFFER_INITIALIZER; + g_autofree char *actualCmdline = NULL; + virCommandSendBuffer *sendbuffers; + int nsendbuffers; + size_t j; + + virCommandSetDryRun(dryRunToken, &buf, true, true, NULL, NULL); + cmd = qemuNbdkitProcessBuildCommand(srcPriv->nbdkitProcess); + + if (virCommandRun(cmd, NULL) < 0) { + ret = -1; + continue; + } + virCommandPeekSendBuffers(cmd, &sendbuffers, &nsendbuffers); + + if (!(actualCmdline = virBufferContentAndReset(&buf))) { + ret = -1; + continue; + } + + if (virTestCompareToFileFull(actualCmdline, cmdfile, false) < 0) + ret = -1; + + for (j = 0; j < nsendbuffers; j++) { + virCommandSendBuffer *buffer = &sendbuffers[j]; + g_autofree char *pipefile = g_strdup_printf("%s.pipe.%i", + cmdfile, + buffer->fd); + + if (virTestCompareToFile((const char*)buffer->buffer, pipefile) < 0) + ret = -1; + } + } else { + if (virFileExists(cmdfile)) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + "qemuNbdkitInitStorageSource() was not expected to fail"); + ret = -1; + } + } + } + + if (info->expectFail) { + if (ret == 0) { + ret = -1; + VIR_TEST_DEBUG("Error expected but there wasn't any."); + } else { + ret = 0; + } + } + return ret; +} + +static int +mymain(void) +{ + g_autoptr(GHashTable) capslatest = testQemuGetLatestCaps(); + g_autoptr(GHashTable) capscache = virHashNew(virObjectUnref); + int ret = 0; + + if (qemuTestDriverInit(&driver) < 0) + return EXIT_FAILURE; + + if (testQemuInsertRealCaps(driver.qemuCapsCache, "x86_64", "latest", "", + capslatest, capscache, NULL, NULL) < 0) { + ret = -1; + goto cleanup; + } + +#define DO_TEST_FULL(_name, ...) \ + do { \ + TestInfo info = { \ + .name = _name, \ + .nbdkitcaps = qemuNbdkitCapsNew(TEST_NBDKIT_PATH), \ + }; \ + testInfoSetPaths(&info); \ + testInfoSetArgs(&info, __VA_ARGS__); \ + virTestRunLog(&ret, "nbdkit " _name, testNbdkit, &info); \ + testInfoClear(&info); \ + } while (0) + +#define DO_TEST(_name, ...) \ + DO_TEST_FULL(_name, NBDKIT_ARG_CAPS, __VA_ARGS__, QEMU_NBDKIT_CAPS_LAST, NBDKIT_ARG_END) + +#define DO_TEST_FAILURE(_name, ...) \ + DO_TEST_FULL(_name, \ + NBDKIT_ARG_EXPECT_FAIL, 1, \ + NBDKIT_ARG_CAPS, __VA_ARGS__, QEMU_NBDKIT_CAPS_LAST, NBDKIT_ARG_END) + +#define DO_TEST_NOCAPS(_name) \ + DO_TEST_FULL(_name, NBDKIT_ARG_END) + + DO_TEST("disk-cdrom-network", QEMU_NBDKIT_CAPS_PLUGIN_CURL); + DO_TEST("disk-network-http", QEMU_NBDKIT_CAPS_PLUGIN_CURL); + DO_TEST("disk-network-source-curl-nbdkit-backing", QEMU_NBDKIT_CAPS_PLUGIN_CURL); + DO_TEST("disk-network-source-curl", QEMU_NBDKIT_CAPS_PLUGIN_CURL); + DO_TEST("disk-network-ssh", QEMU_NBDKIT_CAPS_PLUGIN_SSH); + + cleanup: + qemuTestDriverFree(&driver); + + return ret == 0 ? EXIT_SUCCESS : EXIT_FAILURE; +} + +VIR_TEST_MAIN(mymain) -- 2.41.0

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- tests/qemunbdkitdata/disk-network-source-curl.args.disk1 | 4 +++- .../disk-network-source-curl.args.disk1.pipe.780 | 2 +- .../disk-network-source-curl.args.disk1.pipe.782 | 1 + tests/qemunbdkitdata/disk-network-source-curl.args.disk2 | 2 +- .../disk-network-source-curl.args.disk2.pipe.784 | 1 + .../disk-network-source-curl.x86_64-latest.args | 3 ++- tests/qemuxml2argvdata/disk-network-source-curl.xml | 3 +++ 7 files changed, 12 insertions(+), 4 deletions(-) create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.782 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.784 diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1 index fde6a4f533..d1288dd242 100644 --- a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1 +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1 @@ -3,5 +3,7 @@ nbdkit \ --foreground curl \ protocols=https \ 'url=https://https.example.org:8443/path/to/disk5.iso?foo=bar' \ -cookie=-779 \ +user=myname \ +password=-779 \ +cookie=-781 \ sslverify=false diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.780 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.780 index 20af4ae383..ccdd4033fc 100644 --- a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.780 +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.780 @@ -1 +1 @@ -cookie1=cookievalue1; cookie2=cookievalue2 \ No newline at end of file +iscsi-mycluster_myname-secret \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.782 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.782 new file mode 100644 index 0000000000..20af4ae383 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.782 @@ -0,0 +1 @@ +cookie1=cookievalue1; cookie2=cookievalue2 \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk2 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2 index 88c9fa35a1..f1d0e1929e 100644 --- a/tests/qemunbdkitdata/disk-network-source-curl.args.disk2 +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2 @@ -4,4 +4,4 @@ nbdkit \ --readonly curl \ protocols=http,https \ url=http://http.example.org:8080/path/to/disk2.iso \ -cookie=-781 +cookie=-783 diff --git a/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.784 b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.784 new file mode 100644 index 0000000000..5c035e84c5 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.784 @@ -0,0 +1 @@ +cookie1=cookievalue1; cookie2=cookievalue2; cookie3=cookievalue3 \ No newline at end of file diff --git a/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args b/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args index cb0e5a92ea..f6ab5532cc 100644 --- a/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args +++ b/tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args @@ -32,9 +32,10 @@ XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.config \ -blockdev '{"driver":"https","url":"https://https.example.org:8443/path/to/disk1.iso","cookie-secret":"libvirt-5-storage-httpcookie-secret0","node-name":"libvirt-5-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-5-format","read-only":true,"driver":"raw","file":"libvirt-5-storage"}' \ -device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x3","drive":"libvirt-5-format","id":"virtio-disk0","bootindex":1}' \ +-object '{"qom-type":"secret","id":"libvirt-4-storage-auth-secret0","data":"9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1","keyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw==","format":"base64"}' \ -object '{"qom-type":"secret","id":"libvirt-4-storage-httpcookie-secret0","data":"BUU0KmnWfonHdjzhYhwVQZ5iTI1KweTJ22q8XWUVoBCVu1z70reDuczPBIabZtC3","keyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw==","format":"base64"}' \ -object '{"qom-type":"secret","id":"libvirt-4-format-encryption-secret0","data":"9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1","keyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw==","format":"base64"}' \ --blockdev '{"driver":"https","url":"https://https.example.org:8443/path/to/disk5.iso?foo=bar","sslverify":false,"cookie-secret":"libvirt-4-storage-httpcookie-secret0","node-name":"libvirt-4-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"driver":"https","url":"https://https.example.org:8443/path/to/disk5.iso?foo=bar","username":"myname","password-secret":"libvirt-4-storage-auth-secret0","sslverify":false,"cookie-secret":"libvirt-4-storage-httpcookie-secret0","node-name":"libvirt-4-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-4-format","read-only":false,"driver":"luks","key-secret":"libvirt-4-format-encryption-secret0","file":"libvirt-4-storage"}' \ -device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x4","drive":"libvirt-4-format","id":"virtio-disk4"}' \ -object '{"qom-type":"secret","id":"libvirt-3-storage-httpcookie-secret0","data":"BUU0KmnWfonHdjzhYhwVQZ5iTI1KweTJ22q8XWUVoBBv7TuTgTkyAyOPpC2P5qLbOIypLoHpppjz+u5O+X8oT+jA1m7q/OJQ8dk2EFD5c0A=","keyid":"masterKey0","iv":"AAECAwQFBgcICQoLDA0ODw==","format":"base64"}' \ diff --git a/tests/qemuxml2argvdata/disk-network-source-curl.xml b/tests/qemuxml2argvdata/disk-network-source-curl.xml index 1e50314abe..8c3982cd73 100644 --- a/tests/qemuxml2argvdata/disk-network-source-curl.xml +++ b/tests/qemuxml2argvdata/disk-network-source-curl.xml @@ -59,6 +59,9 @@ <encryption format='luks'> <secret type='passphrase' uuid='1148b693-0843-4cef-9f97-8feb4e1ae365'/> </encryption> + <auth username='myname'> + <secret type='iscsi' usage='mycluster_myname'/> + </auth> </source> <target dev='vde' bus='virtio'/> </disk> -- 2.41.0

Since the restart handler will trigger at an arbitrary time (when the nbdkit process crashes, for instance), it's difficult to provide feedback to the user if the restart is unsuccessful. Rather than just relying on a warning in the log, taint the domain so that there will be a slightly more user-visible notification. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> --- src/conf/domain_conf.c | 2 ++ src/conf/domain_conf.h | 1 + 2 files changed, 3 insertions(+) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index bb4f1fdb94..8feaf5d055 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -87,6 +87,7 @@ VIR_ENUM_IMPL(virDomainTaint, "custom-hypervisor-feature", "deprecated-config", "custom-device", + "nbdkit-restart", ); VIR_ENUM_IMPL(virDomainTaintMessage, @@ -105,6 +106,7 @@ VIR_ENUM_IMPL(virDomainTaintMessage, N_("hypervisor feature autodetection override"), N_("use of deprecated configuration settings"), N_("custom device configuration"), + N_("nbdkit restart failed"), ); VIR_ENUM_IMPL(virDomainVirt, diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index ca195a52d2..c0729905a8 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -3194,6 +3194,7 @@ typedef enum { VIR_DOMAIN_TAINT_CUSTOM_HYPERVISOR_FEATURE, /* custom hypervisor feature control */ VIR_DOMAIN_TAINT_DEPRECATED_CONFIG, /* Configuration that is marked deprecated */ VIR_DOMAIN_TAINT_CUSTOM_DEVICE, /* hypervisor device config customized */ + VIR_DOMAIN_TAINT_NBDKIT_RESTART, /* nbdkit could not be restarted */ VIR_DOMAIN_TAINT_LAST } virDomainTaintFlags; -- 2.41.0

On Thu, Aug 31, 2023 at 16:40:04 -0500, Jonathon Jongsma wrote:
Since the restart handler will trigger at an arbitrary time (when the nbdkit process crashes, for instance), it's difficult to provide feedback to the user if the restart is unsuccessful. Rather than just relying on a warning in the log, taint the domain so that there will be a slightly more user-visible notification.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> --- src/conf/domain_conf.c | 2 ++ src/conf/domain_conf.h | 1 + 2 files changed, 3 insertions(+)
Reviewed-by: Peter Krempa <pkrempa@redhat.com>

Adds the ability to monitor the nbdkit process so that we can take action in case the child exits unexpectedly. When the nbdkit process exits, we pause the vm, restart nbdkit, and then resume the vm. This allows the vm to continue working in the event of a nbdkit failure. Eventually we may want to generalize this functionality since we may need something similar for e.g. qemu-storage-daemon, etc. The process is monitored with the pidfd_open() syscall if it exists (since linux 5.3). Otherwise it resorts to checking whether the process is alive once a second. The one-second time period was chosen somewhat arbitrarily. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> --- meson.build | 11 +++ src/qemu/qemu_domain.c | 1 + src/qemu/qemu_domain.h | 1 + src/qemu/qemu_driver.c | 16 ++++ src/qemu/qemu_nbdkit.c | 169 ++++++++++++++++++++++++++++++++++++--- src/qemu/qemu_nbdkit.h | 8 +- src/qemu/qemu_process.c | 15 +++- src/qemu/qemu_process.h | 3 + tests/meson.build | 6 +- tests/qemuxml2argvtest.c | 6 +- 10 files changed, 219 insertions(+), 17 deletions(-) diff --git a/meson.build b/meson.build index 965ada483b..18ec312ee6 100644 --- a/meson.build +++ b/meson.build @@ -682,6 +682,13 @@ symbols = [ [ 'sched.h', 'cpu_set_t' ], ] +if host_machine.system() == 'linux' + symbols += [ + # process management + [ 'sys/syscall.h', 'SYS_pidfd_open' ], + ] +endif + foreach symbol : symbols if cc.has_header_symbol(symbol[0], symbol[1], args: '-D_GNU_SOURCE', prefix: symbol.get(2, '')) conf.set('WITH_DECL_@0@'.format(symbol[1].to_upper()), 1) @@ -2002,6 +2009,9 @@ endif conf.set_quoted('TLS_PRIORITY', get_option('tls_priority')) +if conf.has('WITH_DECL_SYS_PIDFD_OPEN') + conf.set('WITH_NBDKIT', 1) +endif # Various definitions @@ -2259,6 +2269,7 @@ misc_summary = { 'firewalld-zone': conf.has('WITH_FIREWALLD_ZONE'), 'nss': conf.has('WITH_NSS'), 'numad': conf.has('WITH_NUMAD'), + 'nbdkit': conf.has('WITH_NBDKIT'), 'Init script': init_script, 'Char device locks': chrdev_lock_files, 'Loader/NVRAM': loader_res, diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 46fe5a1cf4..2b8ece8f19 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -11511,6 +11511,7 @@ qemuProcessEventFree(struct qemuProcessEvent *event) case QEMU_PROCESS_EVENT_PR_DISCONNECT: case QEMU_PROCESS_EVENT_UNATTENDED_MIGRATION: case QEMU_PROCESS_EVENT_RESET: + case QEMU_PROCESS_EVENT_NBDKIT_EXITED: case QEMU_PROCESS_EVENT_LAST: break; } diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index ddd20e67b4..f018b45eb6 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -465,6 +465,7 @@ typedef enum { QEMU_PROCESS_EVENT_MEMORY_DEVICE_SIZE_CHANGE, QEMU_PROCESS_EVENT_UNATTENDED_MIGRATION, QEMU_PROCESS_EVENT_RESET, + QEMU_PROCESS_EVENT_NBDKIT_EXITED, QEMU_PROCESS_EVENT_LAST } qemuProcessEventType; diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index ad8428948b..c4236b872f 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -4040,6 +4040,19 @@ processResetEvent(virQEMUDriver *driver, } +static void +processNbdkitExitedEvent(virDomainObj *vm, + qemuNbdkitProcess *nbdkit) +{ + if (virDomainObjBeginJob(vm, VIR_JOB_MODIFY) < 0) + return; + + qemuNbdkitProcessRestart(nbdkit, vm); + + virDomainObjEndJob(vm); +} + + static void qemuProcessEventHandler(void *data, void *opaque) { struct qemuProcessEvent *processEvent = data; @@ -4097,6 +4110,9 @@ static void qemuProcessEventHandler(void *data, void *opaque) case QEMU_PROCESS_EVENT_RESET: processResetEvent(driver, vm); break; + case QEMU_PROCESS_EVENT_NBDKIT_EXITED: + processNbdkitExitedEvent(vm, processEvent->data); + break; case QEMU_PROCESS_EVENT_LAST: break; } diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index df638e99c0..b9ced1e5cf 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -19,6 +19,7 @@ #include <config.h> #include <glib.h> +#include <sys/syscall.h> #include "vircommand.h" #include "virerror.h" @@ -33,6 +34,7 @@ #include "qemu_nbdkit.h" #define LIBVIRT_QEMU_NBDKITPRIV_H_ALLOW #include "qemu_nbdkitpriv.h" +#include "qemu_process.h" #include "qemu_security.h" #include <fcntl.h> @@ -41,6 +43,12 @@ VIR_LOG_INIT("qemu.nbdkit"); +#if WITH_NBDKIT +# define WITHOUT_NBDKIT_UNUSED +#else +# define WITHOUT_NBDKIT_UNUSED G_GNUC_UNUSED +#endif + VIR_ENUM_IMPL(qemuNbdkitCaps, QEMU_NBDKIT_CAPS_LAST, /* 0 */ @@ -611,6 +619,116 @@ qemuNbdkitCapsCacheNew(const char *cachedir) } +void +qemuNbdkitProcessRestart(qemuNbdkitProcess *proc, + virDomainObj *vm) +{ + qemuDomainObjPrivate *vmpriv = vm->privateData; + virQEMUDriver *driver = vmpriv->driver; + + /* clean up resources associated with process */ + qemuNbdkitProcessStop(proc); + + if (qemuNbdkitProcessStart(proc, vm, driver) < 0) { + VIR_DEBUG("Unable to restart nbkdit process"); + virDomainObjTaint(vm, VIR_DOMAIN_TAINT_NBDKIT_RESTART); + } +} + + +#if WITH_NBDKIT +typedef struct { + qemuNbdkitProcess *proc; + virDomainObj *vm; +} qemuNbdkitProcessEventData; + + +static qemuNbdkitProcessEventData* +qemuNbdkitProcessEventDataNew(qemuNbdkitProcess *proc, + virDomainObj *vm) +{ + qemuNbdkitProcessEventData *d = g_new(qemuNbdkitProcessEventData, 1); + d->proc = proc; + d->vm = virObjectRef(vm); + return d; +} + + +static void +qemuNbdkitProcessEventDataFree(qemuNbdkitProcessEventData *d) +{ + virObjectUnref(d->vm); + g_free(d); +} + + +static void +qemuNbdkitProcessPidfdCb(int watch G_GNUC_UNUSED, + int fd, + int events G_GNUC_UNUSED, + void *opaque) +{ + qemuNbdkitProcessEventData *d = opaque; + + VIR_FORCE_CLOSE(fd); + /* submit an event so that it is handled in the per-vm event thread */ + qemuProcessHandleNbdkitExit(d->proc, d->vm); +} +#endif /* WITH_NBDKIT */ + + +static int +qemuNbdkitProcessStartMonitor(qemuNbdkitProcess *proc WITHOUT_NBDKIT_UNUSED, + virDomainObj *vm WITHOUT_NBDKIT_UNUSED) +{ +#if WITH_NBDKIT + int pidfd; + qemuNbdkitProcessEventData *data; + + pidfd = syscall(SYS_pidfd_open, proc->pid, 0); + if (pidfd < 0) { + virReportSystemError(errno, _("pidfd_open failed for %1$i"), proc->pid); + return -1; + } + + data = qemuNbdkitProcessEventDataNew(proc, vm); + if ((proc->eventwatch = virEventAddHandle(pidfd, + VIR_EVENT_HANDLE_READABLE, + qemuNbdkitProcessPidfdCb, + data, + (virFreeCallback)qemuNbdkitProcessEventDataFree)) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("failed to monitor nbdkit process %1$i"), + proc->pid); + VIR_FORCE_CLOSE(pidfd); + qemuNbdkitProcessEventDataFree(data); + return -1; + } + + VIR_DEBUG("Monitoring nbdkit process %i for exit", proc->pid); + + return 0; +#else + /* This should not be reachable */ + virReportError(VIR_ERR_NO_SUPPORT, "%s", + _("nbdkit support is not enabled")); + return -1; +#endif /* WITH_NBDKIT */ +} + + +static void +qemuNbdkitProcessStopMonitor(qemuNbdkitProcess *proc WITHOUT_NBDKIT_UNUSED) +{ +#if WITH_NBDKIT + if (proc->eventwatch > 0) { + virEventRemoveHandle(proc->eventwatch); + proc->eventwatch = 0; + } +#endif /* WITH_NBDKIT */ +} + + static qemuNbdkitProcess * qemuNbdkitProcessNew(virStorageSource *source, const char *pidfile, @@ -658,9 +776,11 @@ qemuNbdkitReconnectStorageSource(virStorageSource *source, static void -qemuNbdkitStorageSourceManageProcessOne(virStorageSource *source) +qemuNbdkitStorageSourceManageProcessOne(virStorageSource *source, + virDomainObj *vm) { qemuDomainStorageSourcePrivate *srcpriv = QEMU_DOMAIN_STORAGE_SOURCE_PRIVATE(source); + qemuDomainObjPrivate *vmpriv = vm->privateData; qemuNbdkitProcess *proc; if (!srcpriv) @@ -671,6 +791,9 @@ qemuNbdkitStorageSourceManageProcessOne(virStorageSource *source) if (!proc) return; + if (!proc->caps) + proc->caps = qemuGetNbdkitCaps(vmpriv->driver); + if (proc->pid <= 0) { if (virPidFileReadPath(proc->pidfile, &proc->pid) < 0) { VIR_WARN("Unable to read pidfile '%s'", proc->pidfile); @@ -678,8 +801,16 @@ qemuNbdkitStorageSourceManageProcessOne(virStorageSource *source) } } - if (virProcessKill(proc->pid, 0) < 0) - VIR_WARN("nbdkit process %i is not alive", proc->pid); + if (virProcessKill(proc->pid, 0) < 0) { + VIR_DEBUG("nbdkit process %i is not alive", proc->pid); + qemuNbdkitProcessRestart(proc, vm); + return; + } + + if (qemuNbdkitProcessStartMonitor(proc, vm) < 0) { + VIR_DEBUG("unable monitor nbdkit process"); + virDomainObjTaint(vm, VIR_DOMAIN_TAINT_NBDKIT_RESTART); + } } /** @@ -692,22 +823,28 @@ qemuNbdkitStorageSourceManageProcessOne(virStorageSource *source) * disk and is attempting to re-connect to active domains. */ void -qemuNbdkitStorageSourceManageProcess(virStorageSource *source) +qemuNbdkitStorageSourceManageProcess(virStorageSource *source, + virDomainObj *vm) { virStorageSource *backing; for (backing = source; backing != NULL; backing = backing->backingStore) - qemuNbdkitStorageSourceManageProcessOne(backing); + qemuNbdkitStorageSourceManageProcessOne(backing, vm); } bool -qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, - virStorageSource *source, - char *statedir, - const char *alias, - uid_t user, - gid_t group) +qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps WITHOUT_NBDKIT_UNUSED, + virStorageSource *source WITHOUT_NBDKIT_UNUSED, + char *statedir WITHOUT_NBDKIT_UNUSED, + const char *alias WITHOUT_NBDKIT_UNUSED, + uid_t user WITHOUT_NBDKIT_UNUSED, + gid_t group WITHOUT_NBDKIT_UNUSED) { +#if !WITH_NBDKIT + /* if nbdkit support is not enabled, don't construct the object so the + * calling function will fall back to qemu storage drivers */ + return false; +#else qemuDomainStorageSourcePrivate *srcPriv = qemuDomainStorageSourcePrivateFetch(source); g_autofree char *pidname = g_strdup_printf("nbdkit-%s.pid", alias); g_autofree char *socketname = g_strdup_printf("nbdkit-%s.socket", alias); @@ -751,6 +888,7 @@ qemuNbdkitInitStorageSource(qemuNbdkitCaps *caps, srcPriv->nbdkitProcess = proc; return true; +#endif /* WITH_NBDKIT */ } @@ -968,6 +1106,8 @@ qemuNbdkitProcessBuildCommand(qemuNbdkitProcess *proc) void qemuNbdkitProcessFree(qemuNbdkitProcess *proc) { + qemuNbdkitProcessStopMonitor(proc); + g_clear_pointer(&proc->pidfile, g_free); g_clear_pointer(&proc->socketfile, g_free); g_clear_object(&proc->caps); @@ -1037,8 +1177,11 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, goto error; while (virTimeBackOffWait(&timebackoff)) { - if (virFileExists(proc->socketfile)) + if (virFileExists(proc->socketfile)) { + if (qemuNbdkitProcessStartMonitor(proc, vm) < 0) + goto error; return 0; + } if (virProcessKill(proc->pid, 0) == 0) continue; @@ -1069,6 +1212,8 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, int qemuNbdkitProcessStop(qemuNbdkitProcess *proc) { + qemuNbdkitProcessStopMonitor(proc); + if (proc->pid < 0) return 0; diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index 36a2219d82..f33b049d38 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -69,7 +69,8 @@ void qemuNbdkitStopStorageSource(virStorageSource *src); void -qemuNbdkitStorageSourceManageProcess(virStorageSource *src); +qemuNbdkitStorageSourceManageProcess(virStorageSource *src, + virDomainObj *vm); bool qemuNbdkitCapsGet(qemuNbdkitCaps *nbdkitCaps, @@ -91,6 +92,7 @@ struct _qemuNbdkitProcess { uid_t user; gid_t group; pid_t pid; + int eventwatch; }; int @@ -98,6 +100,10 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, virDomainObj *vm, virQEMUDriver *driver); +void +qemuNbdkitProcessRestart(qemuNbdkitProcess *proc, + virDomainObj *vm); + int qemuNbdkitProcessStop(qemuNbdkitProcess *proc); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index d90990d8a5..a1c6cfd91e 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -8993,10 +8993,10 @@ qemuProcessReconnect(void *opaque) } for (i = 0; i < obj->def->ndisks; i++) - qemuNbdkitStorageSourceManageProcess(obj->def->disks[i]->src); + qemuNbdkitStorageSourceManageProcess(obj->def->disks[i]->src, obj); if (obj->def->os.loader && obj->def->os.loader->nvram) - qemuNbdkitStorageSourceManageProcess(obj->def->os.loader->nvram); + qemuNbdkitStorageSourceManageProcess(obj->def->os.loader->nvram, obj); /* update domain state XML with possibly updated state in virDomainObj */ if (virDomainObjSave(obj, driver->xmlopt, cfg->stateDir) < 0) @@ -9451,3 +9451,14 @@ qemuProcessQMPStart(qemuProcessQMP *proc) return 0; } + + +void +qemuProcessHandleNbdkitExit(qemuNbdkitProcess *nbdkit, + virDomainObj *vm) +{ + virObjectLock(vm); + VIR_DEBUG("nbdkit process %i died", nbdkit->pid); + qemuProcessEventSubmit(vm, QEMU_PROCESS_EVENT_NBDKIT_EXITED, 0, 0, nbdkit); + virObjectUnlock(vm); +} diff --git a/src/qemu/qemu_process.h b/src/qemu/qemu_process.h index cae1b49756..86a60d29c9 100644 --- a/src/qemu/qemu_process.h +++ b/src/qemu/qemu_process.h @@ -237,3 +237,6 @@ void qemuProcessRefreshDiskProps(virDomainDiskDef *disk, struct qemuDomainDiskInfo *info); int qemuProcessSetupEmulator(virDomainObj *vm); + +void qemuProcessHandleNbdkitExit(qemuNbdkitProcess *nbdkit, + virDomainObj *vm); diff --git a/tests/meson.build b/tests/meson.build index f05774263c..b235c5f4dd 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -456,8 +456,12 @@ if conf.has('WITH_QEMU') { 'name': 'qemuvhostusertest', 'link_with': [ test_qemu_driver_lib ], 'link_whole': [ test_file_wrapper_lib ] }, { 'name': 'qemuxml2argvtest', 'link_with': [ test_qemu_driver_lib, test_utils_qemu_monitor_lib ], 'link_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, { 'name': 'qemuxml2xmltest', 'link_with': [ test_qemu_driver_lib ], 'link_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, - { 'name': 'qemunbdkittest', 'link_with': [ test_qemu_driver_lib ], 'link_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, ] + if conf.has('WITH_NBDKIT') + tests += [ + { 'name': 'qemunbdkittest', 'link_with': [ test_qemu_driver_lib ], 'link_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, + ] + endif endif if conf.has('WITH_REMOTE') diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 0304f66f1d..216fd3a841 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -839,8 +839,12 @@ mymain(void) # define DO_TEST_CAPS_ARCH_VER(name, arch, ver) \ DO_TEST_CAPS_ARCH_VER_FULL(name, arch, ver, ARG_END) -# define DO_TEST_CAPS_LATEST_NBDKIT(name, ...) \ +# if WITH_NBDKIT +# define DO_TEST_CAPS_LATEST_NBDKIT(name, ...) \ DO_TEST_CAPS_ARCH_LATEST_FULL(name, "x86_64", ARG_NBDKIT_CAPS, __VA_ARGS__, QEMU_NBDKIT_CAPS_LAST, ARG_END) +# else +# define DO_TEST_CAPS_LATEST_NBDKIT(name, ...) +# endif /* WITH_NBDKIT */ # define DO_TEST_CAPS_LATEST(name) \ DO_TEST_CAPS_ARCH_LATEST(name, "x86_64") -- 2.41.0

On Thu, Aug 31, 2023 at 16:40:05 -0500, Jonathon Jongsma wrote:
Adds the ability to monitor the nbdkit process so that we can take action in case the child exits unexpectedly.
When the nbdkit process exits, we pause the vm, restart nbdkit, and then resume the vm. This allows the vm to continue working in the event of a nbdkit failure.
Eventually we may want to generalize this functionality since we may need something similar for e.g. qemu-storage-daemon, etc.
The process is monitored with the pidfd_open() syscall if it exists (since linux 5.3). Otherwise it resorts to checking whether the process is alive once a second. The one-second time period was chosen somewhat arbitrarily.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> --- meson.build | 11 +++ src/qemu/qemu_domain.c | 1 + src/qemu/qemu_domain.h | 1 + src/qemu/qemu_driver.c | 16 ++++ src/qemu/qemu_nbdkit.c | 169 ++++++++++++++++++++++++++++++++++++--- src/qemu/qemu_nbdkit.h | 8 +- src/qemu/qemu_process.c | 15 +++- src/qemu/qemu_process.h | 3 + tests/meson.build | 6 +- tests/qemuxml2argvtest.c | 6 +- 10 files changed, 219 insertions(+), 17 deletions(-)
[...]
diff --git a/tests/meson.build b/tests/meson.build index f05774263c..b235c5f4dd 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -456,8 +456,12 @@ if conf.has('WITH_QEMU') { 'name': 'qemuvhostusertest', 'link_with': [ test_qemu_driver_lib ], 'link_whole': [ test_file_wrapper_lib ] }, { 'name': 'qemuxml2argvtest', 'link_with': [ test_qemu_driver_lib, test_utils_qemu_monitor_lib ], 'link_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, { 'name': 'qemuxml2xmltest', 'link_with': [ test_qemu_driver_lib ], 'link_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, - { 'name': 'qemunbdkittest', 'link_with': [ test_qemu_driver_lib ], 'link_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, ] + if conf.has('WITH_NBDKIT') + tests += [ + { 'name': 'qemunbdkittest', 'link_with': [ test_qemu_driver_lib ], 'link_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, + ] + endif endif
if conf.has('WITH_REMOTE') diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 0304f66f1d..216fd3a841 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -839,8 +839,12 @@ mymain(void) # define DO_TEST_CAPS_ARCH_VER(name, arch, ver) \ DO_TEST_CAPS_ARCH_VER_FULL(name, arch, ver, ARG_END)
-# define DO_TEST_CAPS_LATEST_NBDKIT(name, ...) \ +# if WITH_NBDKIT +# define DO_TEST_CAPS_LATEST_NBDKIT(name, ...) \ DO_TEST_CAPS_ARCH_LATEST_FULL(name, "x86_64", ARG_NBDKIT_CAPS, __VA_ARGS__, QEMU_NBDKIT_CAPS_LAST, ARG_END) +# else +# define DO_TEST_CAPS_LATEST_NBDKIT(name, ...) +# endif /* WITH_NBDKIT */
# define DO_TEST_CAPS_LATEST(name) \ DO_TEST_CAPS_ARCH_LATEST(name, "x86_64")
These two hunks seem misplaced and don't semantically align with the rest of the patch. Reviewed-by: Peter Krempa <pkrempa@redhat.com>

On 9/5/23 2:54 AM, Peter Krempa wrote:
On Thu, Aug 31, 2023 at 16:40:05 -0500, Jonathon Jongsma wrote:
Adds the ability to monitor the nbdkit process so that we can take action in case the child exits unexpectedly.
When the nbdkit process exits, we pause the vm, restart nbdkit, and then resume the vm. This allows the vm to continue working in the event of a nbdkit failure.
Eventually we may want to generalize this functionality since we may need something similar for e.g. qemu-storage-daemon, etc.
The process is monitored with the pidfd_open() syscall if it exists (since linux 5.3). Otherwise it resorts to checking whether the process is alive once a second. The one-second time period was chosen somewhat arbitrarily.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> --- meson.build | 11 +++ src/qemu/qemu_domain.c | 1 + src/qemu/qemu_domain.h | 1 + src/qemu/qemu_driver.c | 16 ++++ src/qemu/qemu_nbdkit.c | 169 ++++++++++++++++++++++++++++++++++++--- src/qemu/qemu_nbdkit.h | 8 +- src/qemu/qemu_process.c | 15 +++- src/qemu/qemu_process.h | 3 + tests/meson.build | 6 +- tests/qemuxml2argvtest.c | 6 +- 10 files changed, 219 insertions(+), 17 deletions(-)
[...]
diff --git a/tests/meson.build b/tests/meson.build index f05774263c..b235c5f4dd 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -456,8 +456,12 @@ if conf.has('WITH_QEMU') { 'name': 'qemuvhostusertest', 'link_with': [ test_qemu_driver_lib ], 'link_whole': [ test_file_wrapper_lib ] }, { 'name': 'qemuxml2argvtest', 'link_with': [ test_qemu_driver_lib, test_utils_qemu_monitor_lib ], 'link_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, { 'name': 'qemuxml2xmltest', 'link_with': [ test_qemu_driver_lib ], 'link_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, - { 'name': 'qemunbdkittest', 'link_with': [ test_qemu_driver_lib ], 'link_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, ] + if conf.has('WITH_NBDKIT') + tests += [ + { 'name': 'qemunbdkittest', 'link_with': [ test_qemu_driver_lib ], 'link_whole': [ test_utils_qemu_lib, test_file_wrapper_lib ] }, + ] + endif endif
if conf.has('WITH_REMOTE') diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 0304f66f1d..216fd3a841 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -839,8 +839,12 @@ mymain(void) # define DO_TEST_CAPS_ARCH_VER(name, arch, ver) \ DO_TEST_CAPS_ARCH_VER_FULL(name, arch, ver, ARG_END)
-# define DO_TEST_CAPS_LATEST_NBDKIT(name, ...) \ +# if WITH_NBDKIT +# define DO_TEST_CAPS_LATEST_NBDKIT(name, ...) \ DO_TEST_CAPS_ARCH_LATEST_FULL(name, "x86_64", ARG_NBDKIT_CAPS, __VA_ARGS__, QEMU_NBDKIT_CAPS_LAST, ARG_END) +# else +# define DO_TEST_CAPS_LATEST_NBDKIT(name, ...) +# endif /* WITH_NBDKIT */
# define DO_TEST_CAPS_LATEST(name) \ DO_TEST_CAPS_ARCH_LATEST(name, "x86_64")
These two hunks seem misplaced and don't semantically align with the rest of the patch.
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Yes, they don't appear very related on first glance. But this is the first commit that added a dependency on the pidfd_open syscall (and thus added the "WITH_NBDKIT" preprocessor symbol). Before this patch, nbdkit tests would run fine and generate an nbdkit commandline on any platform as long as the nbdkit capabilities were set. But after this patch, any platform that did not provide this syscall would fall back to using the qemu block driver and generate a different commandline, causing tests to fail. Jonathon

Change the return value for qemuNbdkitProcessRestart() and qemuNbdkitStorageSourceManageProcess() to return an error status. The main effect of this change is that when libvirt starts up and reconnects to an already-running domain with an nbdkit-backed disk, it will return an error if it fails to restart any nbdkit processes that are not found to be running or if it fails to monitor any running nbdkit processes. These failures will result in the domain being killed. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> --- src/qemu/qemu_driver.c | 3 ++- src/qemu/qemu_nbdkit.c | 34 ++++++++++++++++------------------ src/qemu/qemu_nbdkit.h | 4 ++-- src/qemu/qemu_process.c | 6 ++++-- 4 files changed, 24 insertions(+), 23 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index c4236b872f..a469b1d04a 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -4047,7 +4047,8 @@ processNbdkitExitedEvent(virDomainObj *vm, if (virDomainObjBeginJob(vm, VIR_JOB_MODIFY) < 0) return; - qemuNbdkitProcessRestart(nbdkit, vm); + if (qemuNbdkitProcessRestart(nbdkit, vm) < 0) + virDomainObjTaint(vm, VIR_DOMAIN_TAINT_NBDKIT_RESTART); virDomainObjEndJob(vm); } diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index b9ced1e5cf..2ad34d6484 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -619,7 +619,7 @@ qemuNbdkitCapsCacheNew(const char *cachedir) } -void +int qemuNbdkitProcessRestart(qemuNbdkitProcess *proc, virDomainObj *vm) { @@ -629,10 +629,7 @@ qemuNbdkitProcessRestart(qemuNbdkitProcess *proc, /* clean up resources associated with process */ qemuNbdkitProcessStop(proc); - if (qemuNbdkitProcessStart(proc, vm, driver) < 0) { - VIR_DEBUG("Unable to restart nbkdit process"); - virDomainObjTaint(vm, VIR_DOMAIN_TAINT_NBDKIT_RESTART); - } + return qemuNbdkitProcessStart(proc, vm, driver); } @@ -775,7 +772,7 @@ qemuNbdkitReconnectStorageSource(virStorageSource *source, } -static void +static int qemuNbdkitStorageSourceManageProcessOne(virStorageSource *source, virDomainObj *vm) { @@ -784,33 +781,31 @@ qemuNbdkitStorageSourceManageProcessOne(virStorageSource *source, qemuNbdkitProcess *proc; if (!srcpriv) - return; + return 0; proc = srcpriv->nbdkitProcess; if (!proc) - return; + return 0; if (!proc->caps) proc->caps = qemuGetNbdkitCaps(vmpriv->driver); if (proc->pid <= 0) { if (virPidFileReadPath(proc->pidfile, &proc->pid) < 0) { - VIR_WARN("Unable to read pidfile '%s'", proc->pidfile); - return; + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Unable to read pidfile '%1$s'"), + proc->pidfile); + return -1; } } if (virProcessKill(proc->pid, 0) < 0) { VIR_DEBUG("nbdkit process %i is not alive", proc->pid); - qemuNbdkitProcessRestart(proc, vm); - return; + return qemuNbdkitProcessRestart(proc, vm); } - if (qemuNbdkitProcessStartMonitor(proc, vm) < 0) { - VIR_DEBUG("unable monitor nbdkit process"); - virDomainObjTaint(vm, VIR_DOMAIN_TAINT_NBDKIT_RESTART); - } + return qemuNbdkitProcessStartMonitor(proc, vm); } /** @@ -822,13 +817,16 @@ qemuNbdkitStorageSourceManageProcessOne(virStorageSource *source, * @source. It is intended to be called after libvirt restarts and has loaded its current state from * disk and is attempting to re-connect to active domains. */ -void +int qemuNbdkitStorageSourceManageProcess(virStorageSource *source, virDomainObj *vm) { virStorageSource *backing; for (backing = source; backing != NULL; backing = backing->backingStore) - qemuNbdkitStorageSourceManageProcessOne(backing, vm); + if (qemuNbdkitStorageSourceManageProcessOne(backing, vm) < 0) + return -1; + + return 0; } diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index f33b049d38..a818ff65e1 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -68,7 +68,7 @@ qemuNbdkitStartStorageSource(virQEMUDriver *driver, void qemuNbdkitStopStorageSource(virStorageSource *src); -void +int qemuNbdkitStorageSourceManageProcess(virStorageSource *src, virDomainObj *vm); @@ -100,7 +100,7 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, virDomainObj *vm, virQEMUDriver *driver); -void +int qemuNbdkitProcessRestart(qemuNbdkitProcess *proc, virDomainObj *vm); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index a1c6cfd91e..ddf95a064f 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -8993,10 +8993,12 @@ qemuProcessReconnect(void *opaque) } for (i = 0; i < obj->def->ndisks; i++) - qemuNbdkitStorageSourceManageProcess(obj->def->disks[i]->src, obj); + if (qemuNbdkitStorageSourceManageProcess(obj->def->disks[i]->src, obj) < 0) + goto error; if (obj->def->os.loader && obj->def->os.loader->nvram) - qemuNbdkitStorageSourceManageProcess(obj->def->os.loader->nvram, obj); + if (qemuNbdkitStorageSourceManageProcess(obj->def->os.loader->nvram, obj) < 0) + goto error; /* update domain state XML with possibly updated state in virDomainObj */ if (virDomainObjSave(obj, driver->xmlopt, cfg->stateDir) < 0) -- 2.41.0

On Thu, Aug 31, 2023 at 16:40:06 -0500, Jonathon Jongsma wrote:
Change the return value for qemuNbdkitProcessRestart() and qemuNbdkitStorageSourceManageProcess() to return an error status. The main effect of this change is that when libvirt starts up and reconnects to an already-running domain with an nbdkit-backed disk, it will return an error if it fails to restart any nbdkit processes that are not found to be running or if it fails to monitor any running nbdkit processes. These failures will result in the domain being killed.
I don't quite understand why this is a separate patch, but ...
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> --- src/qemu/qemu_driver.c | 3 ++- src/qemu/qemu_nbdkit.c | 34 ++++++++++++++++------------------ src/qemu/qemu_nbdkit.h | 4 ++-- src/qemu/qemu_process.c | 6 ++++-- 4 files changed, 24 insertions(+), 23 deletions(-)
Reviewed-by: Peter Krempa <pkrempa@redhat.com>

On 9/5/23 2:55 AM, Peter Krempa wrote:
On Thu, Aug 31, 2023 at 16:40:06 -0500, Jonathon Jongsma wrote:
Change the return value for qemuNbdkitProcessRestart() and qemuNbdkitStorageSourceManageProcess() to return an error status. The main effect of this change is that when libvirt starts up and reconnects to an already-running domain with an nbdkit-backed disk, it will return an error if it fails to restart any nbdkit processes that are not found to be running or if it fails to monitor any running nbdkit processes. These failures will result in the domain being killed.
I don't quite understand why this is a separate patch, but ...
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> --- src/qemu/qemu_driver.c | 3 ++- src/qemu/qemu_nbdkit.c | 34 ++++++++++++++++------------------ src/qemu/qemu_nbdkit.h | 4 ++-- src/qemu/qemu_process.c | 6 ++++-- 4 files changed, 24 insertions(+), 23 deletions(-)
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Yes, I probably should have explained why it was separate. I initially made it separate in case you felt that the behavior change was not acceptable and wanted me to drop it. I will squash it into the previous patch. Jonathon

When using nbdkit to serve a network disk source, the nbdkit process will start and wait for an nbd connection before actually attempting to connect to the (remote) disk location. Because of this, nbdkit will not report an error until after qemu is launched and tries to read from the disk. This results in a fairly user-unfriendly error saying that qemu was unable to start because "Requested export not available". Ideally we'd like to be able to tell the user *why* the export is not available, but this sort of information is only available to nbdkit, not qemu. It could be because the url was incorrect, or because of an authentication failure, or one of many other possibilities. To make this friendlier for users and easier to detect misconfigurations, try to connect to nbdkit immediately after starting nbdkit and before we try to start qemu. This requires adding a dependency on libnbd. If an error occurs when connecting to nbdkit, read back from the nbdkit error log and provide that information in the error report from qemuNbdkitProcessStart(). User-visible change demonstrated below: Previous error: $ virsh start nbdkit-test 2023-01-18 19:47:45.778+0000: 30895: error : virNetClientProgramDispatchError:172 : internal error: process exited while connecting to monitor: 2023-01-18T19:47:45.704658Z qemu-system-x86_64: -blockdev {"driver":"nbd","server":{"type":"unix", "path":"/var/lib/libvirt/qemu/domain-1-nbdkit-test/nbdkit-libvirt-1-storage.socket"}, "node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}: Requested export not available error: Failed to start domain 'nbdkit-test' error: internal error: process exited while connecting to monitor: 2023-01-18T19:47:45.704658Z qemu-system-x86_64: -blockdev {"driver":"nbd","server":{"type":"unix", "path":"/var/lib/libvirt/qemu/domain-1-nbdkit-test/nbdkit-libvirt-1-storage.socket"}, "node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}: Requested export not available After this change: $ virsh start nbdkit-test 2023-01-18 19:44:36.242+0000: 30895: error : virNetClientProgramDispatchError:172 : internal error: Failed to connect to nbdkit for 'http://localhost:8888/nonexistent.iso': nbdkit: curl[1]: error: problem doing HEAD request to fetch size of URL [http://localhost:8888/nonexistent.iso]: HTTP response code said error: The requested URL returned error: 404 error: Failed to start domain 'nbdkit-test' error: internal error: Failed to connect to nbdkit for 'http://localhost:8888/nonexistent.iso]: error: problem doing HEAD request to fetch size of URL [http://localhost:8888/nonexistent.iso]: HTTP response code said error: The requested URL returned error: 404 Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- meson.build | 7 +++++++ meson_options.txt | 1 + src/qemu/meson.build | 1 + src/qemu/qemu_nbdkit.c | 23 +++++++++++++++++++++++ 4 files changed, 32 insertions(+) diff --git a/meson.build b/meson.build index 18ec312ee6..132269addc 100644 --- a/meson.build +++ b/meson.build @@ -1002,6 +1002,12 @@ endif libiscsi_version = '1.18.0' libiscsi_dep = dependency('libiscsi', version: '>=' + libiscsi_version, required: get_option('libiscsi')) +libnbd_version = '1.0' +libnbd_dep = dependency('libnbd', version: '>=' + libnbd_version, required: get_option('libnbd')) +if libnbd_dep.found() + conf.set('WITH_LIBNBD', 1) +endif + libnl_version = '3.0' if not get_option('libnl').disabled() and host_machine.system() == 'linux' libnl_dep = dependency('libnl-3.0', version: '>=' + libnl_version, required: get_option('libnl')) @@ -2219,6 +2225,7 @@ libs_summary = { 'glusterfs': glusterfs_dep.found(), 'libiscsi': libiscsi_dep.found(), 'libkvm': libkvm_dep.found(), + 'libnbd': libnbd_dep.found(), 'libnl': libnl_dep.found(), 'libparted': libparted_dep.found(), 'libpcap': libpcap_dep.found(), diff --git a/meson_options.txt b/meson_options.txt index 9174c4021c..ba6e49afc5 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -25,6 +25,7 @@ option('curl', type: 'feature', value: 'auto', description: 'curl support') option('fuse', type: 'feature', value: 'auto', description: 'fuse support') option('glusterfs', type: 'feature', value: 'auto', description: 'glusterfs support') option('libiscsi', type: 'feature', value: 'auto', description: 'libiscsi support') +option('libnbd', type: 'feature', value: 'auto', description: 'libnbd support') option('libnl', type: 'feature', value: 'auto', description: 'libnl support') option('libpcap', type: 'feature', value: 'auto', description: 'libpcap support') option('libssh', type: 'feature', value: 'auto', description: 'libssh support') diff --git a/src/qemu/meson.build b/src/qemu/meson.build index 6d7a1bfbb0..607b597c8c 100644 --- a/src/qemu/meson.build +++ b/src/qemu/meson.build @@ -99,6 +99,7 @@ if conf.has('WITH_QEMU') access_dep, capng_dep, gnutls_dep, + libnbd_dep, libnl_dep, log_dep, selinux_dep, diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 2ad34d6484..db86a18321 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -19,6 +19,9 @@ #include <config.h> #include <glib.h> +#if WITH_LIBNBD +# include <libnbd.h> +#endif #include <sys/syscall.h> #include "vircommand.h" @@ -1136,6 +1139,9 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, g_autofree char *basename = g_strdup_printf("%s-nbdkit-%i", vm->def->name, proc->source->id); int logfd = -1; g_autoptr(qemuLogContext) logContext = NULL; +#if WITH_LIBNBD + struct nbd_handle *nbd = NULL; +#endif if (!(cmd = qemuNbdkitProcessBuildCommand(proc))) return -1; @@ -1176,6 +1182,23 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, while (virTimeBackOffWait(&timebackoff)) { if (virFileExists(proc->socketfile)) { +#if WITH_LIBNBD + /* if the disk source was misconfigured, nbdkit will not produce an error + * until somebody connects to the socket and tries to access the nbd + * export. This results in poor user experience because the only error we + * would get from qemu is something like "Requested export not available". + * So let's try to access it ourselves so that we can error out early and + * provide a useful message to the user. + */ + nbd = nbd_create(); + if (nbd_connect_unix(nbd, proc->socketfile) < 0) { + VIR_WARN("nbd_connect_unix failed: %s", nbd_get_error()); + nbd_close(nbd); + goto errorlog; + } + nbd_close(nbd); + +#endif if (qemuNbdkitProcessStartMonitor(proc, vm) < 0) goto error; return 0; -- 2.41.0

Right now, ssh network disks are not usable. There is some basic support in libvirt that is meant to support disk chains that have backing disks located at ssh urls, but there is no real way for a user to configure a ssh-based disk. This commit allows users to configure an ssh disk with password authentication. Implementation will follow. <disk type='network'> <source protocol='ssh' ...> <auth username='myusername'> <secret type='iscsi' usage='secretname'/> </auth> </disk> Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- docs/formatdomain.rst | 27 ++++++++++++++------------- src/conf/schemas/domaincommon.rng | 23 ++++++++++++++++++++++- 2 files changed, 36 insertions(+), 14 deletions(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index 68f54ab3ed..39d4230ec0 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -2778,7 +2778,7 @@ paravirtualized driver is specified via the ``disk`` element. ``network`` The ``protocol`` attribute specifies the protocol to access to the requested image. Possible values are "nbd", "iscsi", "rbd", "sheepdog", - "gluster", "vxhs", "nfs", "http", "https", "ftp", ftps", or "tftp". + "gluster", "vxhs", "nfs", "http", "https", "ftp", ftps", "tftp", or "ssh". For any ``protocol`` other than ``nbd`` an additional attribute ``name`` is mandatory to specify which volume/image will be used. @@ -2930,18 +2930,19 @@ paravirtualized driver is specified via the ``disk`` element. ``auth`` :since:`Since libvirt 3.9.0` , the ``auth`` element is supported for a disk ``type`` "network" that is using a ``source`` element with the - ``protocol`` attributes "rbd" or "iscsi". If present, the ``auth`` element - provides the authentication credentials needed to access the source. It - includes a mandatory attribute ``username``, which identifies the username - to use during authentication, as well as a sub-element ``secret`` with - mandatory attribute ``type``, to tie back to a `libvirt secret - object <formatsecret.html>`__ that holds the actual password or other - credentials (the domain XML intentionally does not expose the password, - only the reference to the object that does manage the password). Known - secret types are "ceph" for Ceph RBD network sources and "iscsi" for CHAP - authentication of iSCSI targets. Both will require either a ``uuid`` - attribute with the UUID of the secret object or a ``usage`` attribute - matching the key that was specified in the secret object. + ``protocol`` attributes "rbd", "iscsi", or "ssh". If present, the + ``auth`` element provides the authentication credentials needed to access + the source. It includes a mandatory attribute ``username``, which + identifies the username to use during authentication, as well as a + sub-element ``secret`` with mandatory attribute ``type``, to tie back to + a `libvirt secret object <formatsecret.html>`__ that holds the actual + password or other credentials (the domain XML intentionally does not + expose the password, only the reference to the object that does manage + the password). Known secret types are "ceph" for Ceph RBD network sources + and "iscsi" for CHAP authentication of iSCSI targets. Both will require + either a ``uuid`` attribute with the UUID of the secret object or a + ``usage`` attribute matching the key that was specified in the secret + object. ``encryption`` :since:`Since libvirt 3.9.0` , the ``encryption`` can be a sub-element of the ``source`` element for encrypted storage sources. If present, diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincommon.rng index 4a475f5c36..cd838a475c 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -2172,6 +2172,27 @@ </element> </define> + <define name="diskSourceNetworkProtocolSSH"> + <element name="source"> + <interleave> + <attribute name="protocol"> + <choice> + <value>ssh</value> + </choice> + </attribute> + <attribute name="name"/> + <ref name="diskSourceCommon"/> + <ref name="diskSourceNetworkHost"/> + <optional> + <ref name="encryption"/> + </optional> + <ref name="diskSourceNetworkProtocolPropsCommon"/> + <optional> + <ref name="diskAuth"/> + </optional> + </interleave> + </element> + </define> <define name="diskSourceNetworkProtocolSimple"> <element name="source"> <interleave> @@ -2179,7 +2200,6 @@ <choice> <value>sheepdog</value> <value>tftp</value> - <value>ssh</value> </choice> </attribute> <attribute name="name"/> @@ -2289,6 +2309,7 @@ <ref name="diskSourceNetworkProtocolHTTPS"/> <ref name="diskSourceNetworkProtocolFTPS"/> <ref name="diskSourceNetworkProtocolFTP"/> + <ref name="diskSourceNetworkProtocolSSH"/> <ref name="diskSourceNetworkProtocolSimple"/> <ref name="diskSourceNetworkProtocolVxHS"/> <ref name="diskSourceNetworkProtocolNFS"/> -- 2.41.0

For ssh disks that are served by nbdkit, lookup the password from the configured secret and securely pass it to the nbdkit process using fd passing. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_nbdkit.c | 84 ++++++++++--------- .../disk-network-ssh-password.args.disk0 | 8 ++ ...k-network-ssh-password.args.disk0.pipe.778 | 1 + .../disk-network-ssh.args.disk1 | 8 ++ .../disk-network-ssh.args.disk1.pipe.778 | 1 + tests/qemunbdkittest.c | 1 + ...sk-network-ssh-password.x86_64-latest.args | 35 ++++++++ .../disk-network-ssh-password.xml | 34 ++++++++ tests/qemuxml2argvtest.c | 1 + 9 files changed, 134 insertions(+), 39 deletions(-) create mode 100644 tests/qemunbdkitdata/disk-network-ssh-password.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-ssh-password.args.disk0.pipe.778 create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk1.pipe.778 create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-password.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-password.xml diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index db86a18321..4cd91e282b 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -950,6 +950,43 @@ qemuNbdkitCommandPassDataByPipe(virCommand *cmd, } +static int +qemuNbdkitProcessBuildCommandAuth(virStorageAuthDef *authdef, + virCommand *cmd) +{ + g_autoptr(virConnect) conn = NULL; + g_autofree uint8_t *secret = NULL; + size_t secretlen = 0; + int secrettype; + + if (!authdef) + return 0; + + if ((secrettype = virSecretUsageTypeFromString(authdef->secrettype)) < 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("invalid secret type %1$s"), + authdef->secrettype); + return -1; + } + + conn = virGetConnectSecret(); + if (virSecretGetSecretString(conn, + &authdef->seclookupdef, + secrettype, + &secret, + &secretlen) < 0) + return -1; + + virCommandAddArgPair(cmd, "user", authdef->username); + + if (qemuNbdkitCommandPassDataByPipe(cmd, "password", + &secret, secretlen) < 0) + return -1; + + return 0; +} + + static int qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *proc, virCommand *cmd) @@ -968,37 +1005,8 @@ qemuNbdkitProcessBuildCommandCurl(qemuNbdkitProcess *proc, } virCommandAddArgPair(cmd, "url", uristring); - if (proc->source->auth) { - g_autoptr(virConnect) conn = virGetConnectSecret(); - g_autofree uint8_t *secret = NULL; - size_t secretlen = 0; - int secrettype; - virStorageAuthDef *authdef = proc->source->auth; - - virCommandAddArgPair(cmd, "user", - proc->source->auth->username); - - if ((secrettype = virSecretUsageTypeFromString(proc->source->auth->secrettype)) < 0) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, - _("invalid secret type %1$s"), - proc->source->auth->secrettype); - return -1; - } - - if (virSecretGetSecretString(conn, - &authdef->seclookupdef, - secrettype, - &secret, - &secretlen) < 0) { - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("failed to get auth secret for storage")); - return -1; - } - - if (qemuNbdkitCommandPassDataByPipe(cmd, "password", - &secret, secretlen) < 0) - return -1; - } + if (proc->source->auth && qemuNbdkitProcessBuildCommandAuth(proc->source->auth, cmd) < 0) + return -1; /* Create a pipe to send the cookies to the nbdkit process. */ if (proc->source->ncookies) { @@ -1027,7 +1035,6 @@ static int qemuNbdkitProcessBuildCommandSSH(qemuNbdkitProcess *proc, virCommand *cmd) { - const char *user = NULL; virStorageNetHostDef *host = &proc->source->hosts[0]; g_autofree char *portstr = g_strdup_printf("%u", host->port); @@ -1038,13 +1045,12 @@ qemuNbdkitProcessBuildCommandSSH(qemuNbdkitProcess *proc, virCommandAddArgPair(cmd, "port", portstr); virCommandAddArgPair(cmd, "path", proc->source->path); - if (proc->source->auth) - user = proc->source->auth->username; - else if (proc->source->ssh_user) - user = proc->source->ssh_user; - - if (user) - virCommandAddArgPair(cmd, "user", user); + if (proc->source->auth) { + if (qemuNbdkitProcessBuildCommandAuth(proc->source->auth, cmd) < 0) + return -1; + } else if (proc->source->ssh_user) { + virCommandAddArgPair(cmd, "user", proc->source->ssh_user); + } if (proc->source->ssh_host_key_check_disabled) virCommandAddArgPair(cmd, "verify-remote-host", "false"); diff --git a/tests/qemunbdkitdata/disk-network-ssh-password.args.disk0 b/tests/qemunbdkitdata/disk-network-ssh-password.args.disk0 new file mode 100644 index 0000000000..30711f7f07 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-ssh-password.args.disk0 @@ -0,0 +1,8 @@ +nbdkit \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground ssh \ +host=example.org \ +port=2222 \ +path=test2.img \ +user=testuser \ +password=-777 diff --git a/tests/qemunbdkitdata/disk-network-ssh-password.args.disk0.pipe.778 b/tests/qemunbdkitdata/disk-network-ssh-password.args.disk0.pipe.778 new file mode 100644 index 0000000000..ccdd4033fc --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-ssh-password.args.disk0.pipe.778 @@ -0,0 +1 @@ +iscsi-mycluster_myname-secret \ No newline at end of file diff --git a/tests/qemunbdkitdata/disk-network-ssh.args.disk1 b/tests/qemunbdkitdata/disk-network-ssh.args.disk1 new file mode 100644 index 0000000000..9a8a16c8d5 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-ssh.args.disk1 @@ -0,0 +1,8 @@ +nbdkit \ +--unix /tmp/statedir-1/nbdkit-test-disk-1.socket \ +--foreground ssh \ +host=example.org \ +port=2222 \ +path=test2.img \ +user=testuser \ +password=-777 diff --git a/tests/qemunbdkitdata/disk-network-ssh.args.disk1.pipe.778 b/tests/qemunbdkitdata/disk-network-ssh.args.disk1.pipe.778 new file mode 100644 index 0000000000..ccdd4033fc --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-ssh.args.disk1.pipe.778 @@ -0,0 +1 @@ +iscsi-mycluster_myname-secret \ No newline at end of file diff --git a/tests/qemunbdkittest.c b/tests/qemunbdkittest.c index 2a74f27a5a..a51b287f34 100644 --- a/tests/qemunbdkittest.c +++ b/tests/qemunbdkittest.c @@ -298,6 +298,7 @@ mymain(void) DO_TEST("disk-network-source-curl-nbdkit-backing", QEMU_NBDKIT_CAPS_PLUGIN_CURL); DO_TEST("disk-network-source-curl", QEMU_NBDKIT_CAPS_PLUGIN_CURL); DO_TEST("disk-network-ssh", QEMU_NBDKIT_CAPS_PLUGIN_SSH); + DO_TEST("disk-network-ssh-password", QEMU_NBDKIT_CAPS_PLUGIN_SSH); cleanup: qemuTestDriverFree(&driver); diff --git a/tests/qemuxml2argvdata/disk-network-ssh-password.x86_64-latest.args b/tests/qemuxml2argvdata/disk-network-ssh-password.x86_64-latest.args new file mode 100644 index 0000000000..fd24e51570 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-ssh-password.x86_64-latest.args @@ -0,0 +1,35 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1 \ +USER=test \ +LOGNAME=test \ +XDG_DATA_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=QEMUGuest1,debug-threads=on \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=off,dump-guest-core=off,memory-backend=pc.ram,acpi=off \ +-accel kvm \ +-cpu qemu64 \ +-m size=219136k \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":224395264}' \ +-overcommit mem-lock=off \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,fd=1729,server=on,wait=off \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-boot strict=on \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0x2"}' \ +-blockdev '{"driver":"nbd","server":{"type":"unix","path":"/var/lib/libvirt/qemu/domain--1-QEMUGuest1/nbdkit-libvirt-1-storage.socket"},"node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw","file":"libvirt-1-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x2","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ +-msg timestamp=on diff --git a/tests/qemuxml2argvdata/disk-network-ssh-password.xml b/tests/qemuxml2argvdata/disk-network-ssh-password.xml new file mode 100644 index 0000000000..266acb761f --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-ssh-password.xml @@ -0,0 +1,34 @@ +<domain type='kvm'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='x86_64' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <disk type='network' device='disk'> + <driver name='qemu' type='raw'/> + <source protocol='ssh' name='test2.img'> + <host name='example.org' port='2222'/> + <timeout seconds='1234'/> + <readahead size='1024'/> + <auth username='testuser'> + <secret type='iscsi' usage='mycluster_myname'/> + </auth> + </source> + <target dev='vda' bus='virtio'/> + </disk> + <controller type='usb' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='none'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 216fd3a841..4541172672 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1276,6 +1276,7 @@ mymain(void) driver.config->vxhsTLS = 0; DO_TEST_CAPS_LATEST("disk-network-ssh"); DO_TEST_CAPS_LATEST_NBDKIT("disk-network-ssh-nbdkit", QEMU_NBDKIT_CAPS_PLUGIN_SSH); + DO_TEST_CAPS_LATEST_NBDKIT("disk-network-ssh-password", QEMU_NBDKIT_CAPS_PLUGIN_SSH); DO_TEST_CAPS_LATEST("disk-no-boot"); DO_TEST_CAPS_LATEST("disk-nvme"); DO_TEST_CAPS_VER("disk-vhostuser-numa", "4.2.0"); -- 2.41.0

In order to make ssh disks usable, we need to be able to validate a remote host. To do this, add a <knownHosts> xml element for ssh disks to allow the user to specify a location for a file that contains known host keys. Implementation to follow. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- docs/formatdomain.rst | 8 ++++++++ src/conf/schemas/domaincommon.rng | 11 +++++++++++ 2 files changed, 19 insertions(+) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index 39d4230ec0..496a8ebfbe 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -3021,6 +3021,14 @@ paravirtualized driver is specified via the ``disk`` element. paused and will be rerun after a successful reconnect. After that time, any delayed requests and all future requests before a successful reconnect will immediately fail. If not set the default QEMU value is 0. + ``knownHosts`` + For storage accessed via the ``ssh`` protocol, this element configures a + path to a file that will be used to verify the remote host. This file + must contain the expected host key for the remote host or the connection + will fail. The location of the file is specified via the ``path`` + attribute. + :since:`Since 9.8.0` + For a "file" or "volume" disk type which represents a cdrom or floppy (the ``device`` attribute), it is possible to define policy what to do with the diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincommon.rng index cd838a475c..ca43586323 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -2172,6 +2172,14 @@ </element> </define> + <define name="diskSourceNetworkProtocolSSHHostVerify"> + <element name="knownHosts"> + <attribute name="path"> + <ref name="absFilePath"/> + </attribute> + </element> + </define> + <define name="diskSourceNetworkProtocolSSH"> <element name="source"> <interleave> @@ -2187,6 +2195,9 @@ <ref name="encryption"/> </optional> <ref name="diskSourceNetworkProtocolPropsCommon"/> + <optional> + <ref name="diskSourceNetworkProtocolSSHHostVerify"/> + </optional> <optional> <ref name="diskAuth"/> </optional> -- 2.41.0

For ssh disks that are served by nbdkit, use the configured value for knownHosts and pass it to the nbdkit process. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/conf/domain_conf.c | 8 ++++++ src/conf/storage_source_conf.c | 2 ++ src/conf/storage_source_conf.h | 2 ++ src/qemu/qemu_extdevice.c | 4 +-- src/qemu/qemu_hotplug.c | 4 +-- src/qemu/qemu_nbdkit.c | 25 +++++++++++++++---- src/qemu/qemu_nbdkit.h | 6 +++-- .../disk-network-ssh-password.args.disk0 | 3 ++- .../disk-network-ssh.args.disk0 | 3 ++- .../disk-network-ssh-password.xml | 1 + tests/qemuxml2argvdata/disk-network-ssh.xml | 1 + 11 files changed, 46 insertions(+), 13 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 8feaf5d055..842b6404b5 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -7268,6 +7268,11 @@ virDomainDiskSourceNetworkParse(xmlNodePtr node, return -1; } } + if (src->protocol == VIR_STORAGE_NET_PROTOCOL_SSH && + (tmpnode = virXPathNode("./knownHosts", ctxt))) { + if (!(src->ssh_known_hosts_file = virXMLPropStringRequired(tmpnode, "path"))) + return -1; + } return 0; } @@ -22274,6 +22279,9 @@ virDomainDiskSourceFormatNetwork(virBuffer *attrBuf, if (src->timeout) virBufferAsprintf(childBuf, "<timeout seconds='%llu'/>\n", src->timeout); + + if (src->protocol == VIR_STORAGE_NET_PROTOCOL_SSH && src->ssh_known_hosts_file) + virBufferEscapeString(childBuf, "<knownHosts path='%s'/>\n", src->ssh_known_hosts_file); } diff --git a/src/conf/storage_source_conf.c b/src/conf/storage_source_conf.c index dcac3a8ff6..906bc36a9b 100644 --- a/src/conf/storage_source_conf.c +++ b/src/conf/storage_source_conf.c @@ -895,6 +895,7 @@ virStorageSourceCopy(const virStorageSource *src, /* ssh config passthrough for libguestfs */ def->ssh_host_key_check_disabled = src->ssh_host_key_check_disabled; def->ssh_user = g_strdup(src->ssh_user); + def->ssh_known_hosts_file = g_strdup(src->ssh_known_hosts_file); def->nfs_user = g_strdup(src->nfs_user); def->nfs_group = g_strdup(src->nfs_group); @@ -1170,6 +1171,7 @@ virStorageSourceClear(virStorageSource *def) VIR_FREE(def->tlsHostname); VIR_FREE(def->ssh_user); + VIR_FREE(def->ssh_known_hosts_file); VIR_FREE(def->nfs_user); VIR_FREE(def->nfs_group); diff --git a/src/conf/storage_source_conf.h b/src/conf/storage_source_conf.h index f13e7c756a..8a9c7d07e2 100644 --- a/src/conf/storage_source_conf.h +++ b/src/conf/storage_source_conf.h @@ -410,6 +410,8 @@ struct _virStorageSource { /* these must not be used apart from formatting the output JSON in the qemu driver */ char *ssh_user; bool ssh_host_key_check_disabled; + /* additional ssh variables */ + char *ssh_known_hosts_file; /* nfs_user and nfs_group store the strings passed in by the user for NFS params. * nfs_uid and nfs_gid represent the converted/looked up ID numbers which are used diff --git a/src/qemu/qemu_extdevice.c b/src/qemu/qemu_extdevice.c index 42ecdf13d5..3cf3867056 100644 --- a/src/qemu/qemu_extdevice.c +++ b/src/qemu/qemu_extdevice.c @@ -297,11 +297,11 @@ qemuExtDevicesStop(virQEMUDriver *driver, for (i = 0; i < def->ndisks; i++) { virDomainDiskDef *disk = def->disks[i]; - qemuNbdkitStopStorageSource(disk->src); + qemuNbdkitStopStorageSource(disk->src, vm); } if (def->os.loader && def->os.loader->nvram) - qemuNbdkitStopStorageSource(def->os.loader->nvram); + qemuNbdkitStopStorageSource(def->os.loader->nvram, vm); } diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index dc06486922..bbc5177206 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -1036,7 +1036,7 @@ qemuDomainAttachDeviceDiskLiveInternal(virQEMUDriver *driver, if (virStorageSourceChainHasManagedPR(disk->src)) ignore_value(qemuHotplugRemoveManagedPR(vm, VIR_ASYNC_JOB_NONE)); - qemuNbdkitStopStorageSource(disk->src); + qemuNbdkitStopStorageSource(disk->src, vm); } qemuDomainSecretDiskDestroy(disk); qemuDomainCleanupStorageSourceFD(disk->src); @@ -4496,7 +4496,7 @@ qemuDomainRemoveDiskDevice(virQEMUDriver *driver, qemuHotplugRemoveManagedPR(vm, VIR_ASYNC_JOB_NONE) < 0) goto cleanup; - qemuNbdkitStopStorageSource(disk->src); + qemuNbdkitStopStorageSource(disk->src, vm); if (disk->transient) { VIR_DEBUG("Removing transient overlay '%s' of disk '%s'", diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 4cd91e282b..afac71e21a 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -630,7 +630,7 @@ qemuNbdkitProcessRestart(qemuNbdkitProcess *proc, virQEMUDriver *driver = vmpriv->driver; /* clean up resources associated with process */ - qemuNbdkitProcessStop(proc); + qemuNbdkitProcessStop(proc, vm); return qemuNbdkitProcessStart(proc, vm, driver); } @@ -913,7 +913,8 @@ qemuNbdkitStartStorageSource(virQEMUDriver *driver, void -qemuNbdkitStopStorageSource(virStorageSource *src) +qemuNbdkitStopStorageSource(virStorageSource *src, + virDomainObj *vm) { virStorageSource *backing; @@ -921,7 +922,7 @@ qemuNbdkitStopStorageSource(virStorageSource *src) qemuDomainStorageSourcePrivate *priv = QEMU_DOMAIN_STORAGE_SOURCE_PRIVATE(src); if (priv && priv->nbdkitProcess && - qemuNbdkitProcessStop(priv->nbdkitProcess) < 0) + qemuNbdkitProcessStop(priv->nbdkitProcess, vm) < 0) VIR_WARN("Unable to stop nbdkit for storage source '%s'", src->nodestorage); } } @@ -1055,6 +1056,9 @@ qemuNbdkitProcessBuildCommandSSH(qemuNbdkitProcess *proc, if (proc->source->ssh_host_key_check_disabled) virCommandAddArgPair(cmd, "verify-remote-host", "false"); + if (proc->source->ssh_known_hosts_file) + virCommandAddArgPair(cmd, "known-hosts", proc->source->ssh_known_hosts_file); + return 0; } @@ -1167,6 +1171,10 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, if (qemuExtDeviceLogCommand(driver, vm, cmd, "nbdkit") < 0) goto error; + if (proc->source->ssh_known_hosts_file && + qemuSecurityDomainSetPathLabel(driver, vm, proc->source->ssh_known_hosts_file, false) < 0) + goto error; + if (qemuSecurityCommandRun(driver, vm, cmd, proc->user, proc->group, true, &exitstatus) < 0) goto error; @@ -1231,16 +1239,23 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, NULLSTR(uristring), NULLSTR(errbuf)); error: - qemuNbdkitProcessStop(proc); + qemuNbdkitProcessStop(proc, vm); return -1; } int -qemuNbdkitProcessStop(qemuNbdkitProcess *proc) +qemuNbdkitProcessStop(qemuNbdkitProcess *proc, + virDomainObj *vm) { + qemuDomainObjPrivate *vmpriv = vm->privateData; + virQEMUDriver *driver = vmpriv->driver; + qemuNbdkitProcessStopMonitor(proc); + if (proc->source->ssh_known_hosts_file) + qemuSecurityDomainRestorePathLabel(driver, vm, proc->source->ssh_known_hosts_file); + if (proc->pid < 0) return 0; diff --git a/src/qemu/qemu_nbdkit.h b/src/qemu/qemu_nbdkit.h index a818ff65e1..853b2cca6f 100644 --- a/src/qemu/qemu_nbdkit.h +++ b/src/qemu/qemu_nbdkit.h @@ -66,7 +66,8 @@ qemuNbdkitStartStorageSource(virQEMUDriver *driver, virStorageSource *src); void -qemuNbdkitStopStorageSource(virStorageSource *src); +qemuNbdkitStopStorageSource(virStorageSource *src, + virDomainObj *vm); int qemuNbdkitStorageSourceManageProcess(virStorageSource *src, @@ -105,7 +106,8 @@ qemuNbdkitProcessRestart(qemuNbdkitProcess *proc, virDomainObj *vm); int -qemuNbdkitProcessStop(qemuNbdkitProcess *proc); +qemuNbdkitProcessStop(qemuNbdkitProcess *proc, + virDomainObj *vm); void qemuNbdkitProcessFree(qemuNbdkitProcess *proc); diff --git a/tests/qemunbdkitdata/disk-network-ssh-password.args.disk0 b/tests/qemunbdkitdata/disk-network-ssh-password.args.disk0 index 30711f7f07..ee2d7c3343 100644 --- a/tests/qemunbdkitdata/disk-network-ssh-password.args.disk0 +++ b/tests/qemunbdkitdata/disk-network-ssh-password.args.disk0 @@ -5,4 +5,5 @@ host=example.org \ port=2222 \ path=test2.img \ user=testuser \ -password=-777 +password=-777 \ +known-hosts=/path/to/knownhosts diff --git a/tests/qemunbdkitdata/disk-network-ssh.args.disk0 b/tests/qemunbdkitdata/disk-network-ssh.args.disk0 index c04dc8bb03..481b218936 100644 --- a/tests/qemunbdkitdata/disk-network-ssh.args.disk0 +++ b/tests/qemunbdkitdata/disk-network-ssh.args.disk0 @@ -3,4 +3,5 @@ nbdkit \ --foreground ssh \ host=example.org \ port=2222 \ -path=test.img +path=test.img \ +known-hosts=/path/to/ssh_known_hosts diff --git a/tests/qemuxml2argvdata/disk-network-ssh-password.xml b/tests/qemuxml2argvdata/disk-network-ssh-password.xml index 266acb761f..bdb4cf6e35 100644 --- a/tests/qemuxml2argvdata/disk-network-ssh-password.xml +++ b/tests/qemuxml2argvdata/disk-network-ssh-password.xml @@ -22,6 +22,7 @@ <auth username='testuser'> <secret type='iscsi' usage='mycluster_myname'/> </auth> + <knownHosts path='/path/to/knownhosts'/> </source> <target dev='vda' bus='virtio'/> </disk> diff --git a/tests/qemuxml2argvdata/disk-network-ssh.xml b/tests/qemuxml2argvdata/disk-network-ssh.xml index 355add4fea..a3aeca0c99 100644 --- a/tests/qemuxml2argvdata/disk-network-ssh.xml +++ b/tests/qemuxml2argvdata/disk-network-ssh.xml @@ -19,6 +19,7 @@ <host name='example.org' port='2222'/> <timeout seconds='1234'/> <readahead size='1024'/> + <knownHosts path="/path/to/ssh_known_hosts"/> </source> <target dev='vda' bus='virtio'/> </disk> -- 2.41.0

Authenticating via key file to an ssh server is often preferable to logging in via password. In order to support this functionality add a new <identity> xml element for ssh disks that allows the user to specify a keyfile and username. Example configuration: <disk type='network'> <source protocol='ssh' ...> <identity keyfile='/path/to/id_rsa' username='myusername'/> ... </source> ... </disk> Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- docs/formatdomain.rst | 7 +++++++ src/conf/schemas/domaincommon.rng | 19 ++++++++++++++++++- 2 files changed, 25 insertions(+), 1 deletion(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index 496a8ebfbe..baa2fdce7d 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -3005,6 +3005,13 @@ paravirtualized driver is specified via the ``disk`` element. of these attributes is omitted, then that field is assumed to be the default value for the current system. If both ``user`` and ``group`` are intended to be default, then the entire element may be omitted. + + When using an ``ssh`` protocol, this element is used to enable + authentication via ssh keys. In this configuration, the element has two + attributes. The ``username`` attribute specifies the name of the user on + the remote server and the ``keyfile`` attribute specifies the path to the + keyfile. Note that this only works for ssh keys that are not + password-protected. ``reconnect`` For disk type ``vhostuser`` configures reconnect timeout if the connection is lost. This is set with the two mandatory attributes ``enabled`` and diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincommon.rng index ca43586323..47c5ee2a31 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -2180,6 +2180,19 @@ </element> </define> + <define name="diskSourceNetworkProtocolSSHKeyDef"> + <element name="identity"> + <interleave> + <attribute name="username"> + <ref name="genericName"/> + </attribute> + <attribute name="keyfile"> + <ref name="absFilePath"/> + </attribute> + </interleave> + </element> + </define> + <define name="diskSourceNetworkProtocolSSH"> <element name="source"> <interleave> @@ -2199,11 +2212,15 @@ <ref name="diskSourceNetworkProtocolSSHHostVerify"/> </optional> <optional> - <ref name="diskAuth"/> + <choice> + <ref name="diskSourceNetworkProtocolSSHKeyDef"/> + <ref name="diskAuth"/> + </choice> </optional> </interleave> </element> </define> + <define name="diskSourceNetworkProtocolSimple"> <element name="source"> <interleave> -- 2.41.0

For ssh disks that are served by nbdkit, we can support logging in with an ssh key file. Pass the path to the configured key file and the username to the nbdkit process. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/conf/domain_conf.c | 30 +++++++++++++---- src/conf/storage_source_conf.c | 2 ++ src/conf/storage_source_conf.h | 5 ++- src/qemu/qemu_nbdkit.c | 15 +++++++-- .../disk-network-ssh-key.args.disk0 | 9 +++++ .../disk-network-ssh.args.disk2 | 9 +++++ tests/qemunbdkittest.c | 1 + .../qemuxml2argvdata/disk-network-ssh-key.xml | 33 +++++++++++++++++++ 8 files changed, 93 insertions(+), 11 deletions(-) create mode 100644 tests/qemunbdkitdata/disk-network-ssh-key.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk2 create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-key.xml diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 842b6404b5..929e115bce 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -7268,10 +7268,18 @@ virDomainDiskSourceNetworkParse(xmlNodePtr node, return -1; } } - if (src->protocol == VIR_STORAGE_NET_PROTOCOL_SSH && - (tmpnode = virXPathNode("./knownHosts", ctxt))) { - if (!(src->ssh_known_hosts_file = virXMLPropStringRequired(tmpnode, "path"))) - return -1; + if (src->protocol == VIR_STORAGE_NET_PROTOCOL_SSH) { + if ((tmpnode = virXPathNode("./knownHosts", ctxt))) { + if (!(src->ssh_known_hosts_file = virXMLPropStringRequired(tmpnode, "path"))) + return -1; + } + if ((tmpnode = virXPathNode("./identity", ctxt))) { + if (!(src->ssh_user = virXMLPropStringRequired(tmpnode, "username"))) + return -1; + + if (!(src->ssh_keyfile = virXMLPropStringRequired(tmpnode, "keyfile"))) + return -1; + } } return 0; @@ -22280,8 +22288,18 @@ virDomainDiskSourceFormatNetwork(virBuffer *attrBuf, if (src->timeout) virBufferAsprintf(childBuf, "<timeout seconds='%llu'/>\n", src->timeout); - if (src->protocol == VIR_STORAGE_NET_PROTOCOL_SSH && src->ssh_known_hosts_file) - virBufferEscapeString(childBuf, "<knownHosts path='%s'/>\n", src->ssh_known_hosts_file); + if (src->protocol == VIR_STORAGE_NET_PROTOCOL_SSH) { + if (src->ssh_known_hosts_file) + virBufferEscapeString(childBuf, "<knownHosts path='%s'/>\n", src->ssh_known_hosts_file); + if (src->ssh_keyfile) { + virBufferAddLit(childBuf, "<identity"); + + virBufferEscapeString(childBuf, " username='%s'", src->ssh_user); + virBufferEscapeString(childBuf, " keyfile='%s'", src->ssh_keyfile); + + virBufferAddLit(childBuf, "/>\n"); + } + } } diff --git a/src/conf/storage_source_conf.c b/src/conf/storage_source_conf.c index 906bc36a9b..ce9c1f66c2 100644 --- a/src/conf/storage_source_conf.c +++ b/src/conf/storage_source_conf.c @@ -896,6 +896,7 @@ virStorageSourceCopy(const virStorageSource *src, def->ssh_host_key_check_disabled = src->ssh_host_key_check_disabled; def->ssh_user = g_strdup(src->ssh_user); def->ssh_known_hosts_file = g_strdup(src->ssh_known_hosts_file); + def->ssh_keyfile = g_strdup(src->ssh_keyfile); def->nfs_user = g_strdup(src->nfs_user); def->nfs_group = g_strdup(src->nfs_group); @@ -1172,6 +1173,7 @@ virStorageSourceClear(virStorageSource *def) VIR_FREE(def->ssh_user); VIR_FREE(def->ssh_known_hosts_file); + VIR_FREE(def->ssh_keyfile); VIR_FREE(def->nfs_user); VIR_FREE(def->nfs_group); diff --git a/src/conf/storage_source_conf.h b/src/conf/storage_source_conf.h index 8a9c7d07e2..8c805664af 100644 --- a/src/conf/storage_source_conf.h +++ b/src/conf/storage_source_conf.h @@ -406,12 +406,11 @@ struct _virStorageSource { bool hostcdrom; /* backing device is a cdrom */ - /* passthrough variables for the ssh driver which we don't handle properly */ - /* these must not be used apart from formatting the output JSON in the qemu driver */ + /* ssh variables */ char *ssh_user; bool ssh_host_key_check_disabled; - /* additional ssh variables */ char *ssh_known_hosts_file; + char *ssh_keyfile; /* nfs_user and nfs_group store the strings passed in by the user for NFS params. * nfs_uid and nfs_gid represent the converted/looked up ID numbers which are used diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index afac71e21a..0a6c7962b0 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -1049,8 +1049,12 @@ qemuNbdkitProcessBuildCommandSSH(qemuNbdkitProcess *proc, if (proc->source->auth) { if (qemuNbdkitProcessBuildCommandAuth(proc->source->auth, cmd) < 0) return -1; - } else if (proc->source->ssh_user) { - virCommandAddArgPair(cmd, "user", proc->source->ssh_user); + } else { + if (proc->source->ssh_keyfile) + virCommandAddArgPair(cmd, "identity", proc->source->ssh_keyfile); + + if (proc->source->ssh_user) + virCommandAddArgPair(cmd, "user", proc->source->ssh_user); } if (proc->source->ssh_host_key_check_disabled) @@ -1171,6 +1175,10 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, if (qemuExtDeviceLogCommand(driver, vm, cmd, "nbdkit") < 0) goto error; + if (proc->source->ssh_keyfile && + qemuSecurityDomainSetPathLabel(driver, vm, proc->source->ssh_keyfile, false) < 0) + goto error; + if (proc->source->ssh_known_hosts_file && qemuSecurityDomainSetPathLabel(driver, vm, proc->source->ssh_known_hosts_file, false) < 0) goto error; @@ -1256,6 +1264,9 @@ qemuNbdkitProcessStop(qemuNbdkitProcess *proc, if (proc->source->ssh_known_hosts_file) qemuSecurityDomainRestorePathLabel(driver, vm, proc->source->ssh_known_hosts_file); + if (proc->source->ssh_keyfile) + qemuSecurityDomainRestorePathLabel(driver, vm, proc->source->ssh_keyfile); + if (proc->pid < 0) return 0; diff --git a/tests/qemunbdkitdata/disk-network-ssh-key.args.disk0 b/tests/qemunbdkitdata/disk-network-ssh-key.args.disk0 new file mode 100644 index 0000000000..0b52bfe0fb --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-ssh-key.args.disk0 @@ -0,0 +1,9 @@ +nbdkit \ +--unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ +--foreground ssh \ +host=example.org \ +port=2222 \ +path=test.img \ +identity=/path/to/id_rsa \ +user=myuser \ +known-hosts=/path/to/ssh_known_hosts diff --git a/tests/qemunbdkitdata/disk-network-ssh.args.disk2 b/tests/qemunbdkitdata/disk-network-ssh.args.disk2 new file mode 100644 index 0000000000..e269a34351 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-ssh.args.disk2 @@ -0,0 +1,9 @@ +nbdkit \ +--unix /tmp/statedir-2/nbdkit-test-disk-2.socket \ +--foreground ssh \ +host=example.org \ +port=2222 \ +path=test2.img \ +identity=/path/to/id_rsa \ +user=myuser \ +known-hosts=/path/to/ssh_known_hosts diff --git a/tests/qemunbdkittest.c b/tests/qemunbdkittest.c index a51b287f34..559196a1cd 100644 --- a/tests/qemunbdkittest.c +++ b/tests/qemunbdkittest.c @@ -299,6 +299,7 @@ mymain(void) DO_TEST("disk-network-source-curl", QEMU_NBDKIT_CAPS_PLUGIN_CURL); DO_TEST("disk-network-ssh", QEMU_NBDKIT_CAPS_PLUGIN_SSH); DO_TEST("disk-network-ssh-password", QEMU_NBDKIT_CAPS_PLUGIN_SSH); + DO_TEST("disk-network-ssh-key", QEMU_NBDKIT_CAPS_PLUGIN_SSH); cleanup: qemuTestDriverFree(&driver); diff --git a/tests/qemuxml2argvdata/disk-network-ssh-key.xml b/tests/qemuxml2argvdata/disk-network-ssh-key.xml new file mode 100644 index 0000000000..81b92231fa --- /dev/null +++ b/tests/qemuxml2argvdata/disk-network-ssh-key.xml @@ -0,0 +1,33 @@ +<domain type='kvm'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='x86_64' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <disk type='network' device='disk'> + <driver name='qemu' type='raw'/> + <source protocol='ssh' name='test.img'> + <host name='example.org' port='2222'/> + <timeout seconds='1234'/> + <readahead size='1024'/> + <identity username='myuser' keyfile='/path/to/id_rsa'/> + <knownHosts path="/path/to/ssh_known_hosts"/> + </source> + <target dev='vda' bus='virtio'/> + </disk> + <controller type='usb' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='none'/> + </devices> +</domain> -- 2.41.0

Add the ability to specify a path to a ssh-agent socket in order to use the ssh-agent to authenticate to remote ssh disks. Example configuration: <disk type='network'> </source protocol='ssh' ...> <identity username='myusername' agentsock='/path/to/socket'/> ... </source> ... </disk> Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- docs/formatdomain.rst | 13 ++++++++----- src/conf/schemas/domaincommon.rng | 11 ++++++++--- 2 files changed, 16 insertions(+), 8 deletions(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index baa2fdce7d..714fee4fbf 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -3007,11 +3007,14 @@ paravirtualized driver is specified via the ``disk`` element. are intended to be default, then the entire element may be omitted. When using an ``ssh`` protocol, this element is used to enable - authentication via ssh keys. In this configuration, the element has two - attributes. The ``username`` attribute specifies the name of the user on - the remote server and the ``keyfile`` attribute specifies the path to the - keyfile. Note that this only works for ssh keys that are not - password-protected. + authentication via ssh keys. In this configuration, the element has three + possible attributes. The ``username`` attribute is required and specifies + the name of the user on the remote server. ssh keys can be specified in + one of two ways. The first way is by adding them to an ssh-agent and + providing the path to the ssh-agent socket in the ``agentsock`` + attribute. This method works for ssh keys with or without password + protection. Alternatively, for ssh keys without a password, the ssh key + can be specified directly by setting the ``keyfile`` attribute. ``reconnect`` For disk type ``vhostuser`` configures reconnect timeout if the connection is lost. This is set with the two mandatory attributes ``enabled`` and diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincommon.rng index 47c5ee2a31..d8dd1b8c69 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -2186,9 +2186,14 @@ <attribute name="username"> <ref name="genericName"/> </attribute> - <attribute name="keyfile"> - <ref name="absFilePath"/> - </attribute> + <choice> + <attribute name="keyfile"> + <ref name="absFilePath"/> + </attribute> + <attribute name="agentsock"> + <ref name="absFilePath"/> + </attribute> + </choice> </interleave> </element> </define> -- 2.41.0

It's not possible to use password-protected ssh keys directly with libvirt because libvirt doesn't have any way to prompt a user for the password. To accomodate password-protected key files, an administrator can add these keys to an ssh agent and then configure the domain with the path to the ssh-agent socket. Note that this requires an administrator or management app to configure the ssh-agent with an appropriate socket path and add the necessary keys to it. In addition, it does not currently work with selinux enabled. The ssh-agent socket would need a label that libvirt would be allowed to access rather than unconfined_t. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- src/conf/domain_conf.c | 14 ++++++++++++-- src/conf/storage_source_conf.c | 2 ++ src/conf/storage_source_conf.h | 1 + src/qemu/qemu_nbdkit.c | 10 ++++++++++ .../disk-network-ssh-key.args.disk0 | 6 +++--- .../disk-network-ssh-key.args.disk1 | 9 +++++++++ tests/qemuxml2argvdata/disk-network-ssh-key.xml | 17 ++++++++++++++--- 7 files changed, 51 insertions(+), 8 deletions(-) create mode 100644 tests/qemunbdkitdata/disk-network-ssh-key.args.disk1 diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 929e115bce..398f40d2be 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -7277,8 +7277,17 @@ virDomainDiskSourceNetworkParse(xmlNodePtr node, if (!(src->ssh_user = virXMLPropStringRequired(tmpnode, "username"))) return -1; - if (!(src->ssh_keyfile = virXMLPropStringRequired(tmpnode, "keyfile"))) + /* optional path to an ssh key file */ + src->ssh_keyfile = virXMLPropString(tmpnode, "keyfile"); + + /* optional ssh-agent socket location */ + src->ssh_agent = virXMLPropString(tmpnode, "agentsock"); + if (!src->ssh_keyfile && !src->ssh_agent) { + virReportError(VIR_ERR_XML_ERROR, + _("element '%1$s' requires either 'keyfile' or 'agentsock' attribute"), + tmpnode->name); return -1; + } } } @@ -22291,11 +22300,12 @@ virDomainDiskSourceFormatNetwork(virBuffer *attrBuf, if (src->protocol == VIR_STORAGE_NET_PROTOCOL_SSH) { if (src->ssh_known_hosts_file) virBufferEscapeString(childBuf, "<knownHosts path='%s'/>\n", src->ssh_known_hosts_file); - if (src->ssh_keyfile) { + if (src->ssh_keyfile || src->ssh_agent) { virBufferAddLit(childBuf, "<identity"); virBufferEscapeString(childBuf, " username='%s'", src->ssh_user); virBufferEscapeString(childBuf, " keyfile='%s'", src->ssh_keyfile); + virBufferEscapeString(childBuf, " agentsock='%s'", src->ssh_agent); virBufferAddLit(childBuf, "/>\n"); } diff --git a/src/conf/storage_source_conf.c b/src/conf/storage_source_conf.c index ce9c1f66c2..cafa031dfe 100644 --- a/src/conf/storage_source_conf.c +++ b/src/conf/storage_source_conf.c @@ -897,6 +897,7 @@ virStorageSourceCopy(const virStorageSource *src, def->ssh_user = g_strdup(src->ssh_user); def->ssh_known_hosts_file = g_strdup(src->ssh_known_hosts_file); def->ssh_keyfile = g_strdup(src->ssh_keyfile); + def->ssh_agent = g_strdup(src->ssh_agent); def->nfs_user = g_strdup(src->nfs_user); def->nfs_group = g_strdup(src->nfs_group); @@ -1174,6 +1175,7 @@ virStorageSourceClear(virStorageSource *def) VIR_FREE(def->ssh_user); VIR_FREE(def->ssh_known_hosts_file); VIR_FREE(def->ssh_keyfile); + VIR_FREE(def->ssh_agent); VIR_FREE(def->nfs_user); VIR_FREE(def->nfs_group); diff --git a/src/conf/storage_source_conf.h b/src/conf/storage_source_conf.h index 8c805664af..061faa66cb 100644 --- a/src/conf/storage_source_conf.h +++ b/src/conf/storage_source_conf.h @@ -411,6 +411,7 @@ struct _virStorageSource { bool ssh_host_key_check_disabled; char *ssh_known_hosts_file; char *ssh_keyfile; + char *ssh_agent; /* nfs_user and nfs_group store the strings passed in by the user for NFS params. * nfs_uid and nfs_gid represent the converted/looked up ID numbers which are used diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 0a6c7962b0..66b09cd240 100644 --- a/src/qemu/qemu_nbdkit.c +++ b/src/qemu/qemu_nbdkit.c @@ -1057,6 +1057,9 @@ qemuNbdkitProcessBuildCommandSSH(qemuNbdkitProcess *proc, virCommandAddArgPair(cmd, "user", proc->source->ssh_user); } + if (proc->source->ssh_agent) + virCommandAddEnvPair(cmd, "SSH_AUTH_SOCK", proc->source->ssh_agent); + if (proc->source->ssh_host_key_check_disabled) virCommandAddArgPair(cmd, "verify-remote-host", "false"); @@ -1179,6 +1182,10 @@ qemuNbdkitProcessStart(qemuNbdkitProcess *proc, qemuSecurityDomainSetPathLabel(driver, vm, proc->source->ssh_keyfile, false) < 0) goto error; + if (proc->source->ssh_agent && + qemuSecurityDomainSetPathLabel(driver, vm, proc->source->ssh_agent, false) < 0) + goto error; + if (proc->source->ssh_known_hosts_file && qemuSecurityDomainSetPathLabel(driver, vm, proc->source->ssh_known_hosts_file, false) < 0) goto error; @@ -1267,6 +1274,9 @@ qemuNbdkitProcessStop(qemuNbdkitProcess *proc, if (proc->source->ssh_keyfile) qemuSecurityDomainRestorePathLabel(driver, vm, proc->source->ssh_keyfile); + if (proc->source->ssh_agent) + qemuSecurityDomainRestorePathLabel(driver, vm, proc->source->ssh_agent); + if (proc->pid < 0) return 0; diff --git a/tests/qemunbdkitdata/disk-network-ssh-key.args.disk0 b/tests/qemunbdkitdata/disk-network-ssh-key.args.disk0 index 0b52bfe0fb..f627700490 100644 --- a/tests/qemunbdkitdata/disk-network-ssh-key.args.disk0 +++ b/tests/qemunbdkitdata/disk-network-ssh-key.args.disk0 @@ -1,9 +1,9 @@ +SSH_AUTH_SOCK=/path/to/agent/socket \ nbdkit \ --unix /tmp/statedir-0/nbdkit-test-disk-0.socket \ --foreground ssh \ host=example.org \ port=2222 \ -path=test.img \ -identity=/path/to/id_rsa \ +path=test1.img \ user=myuser \ -known-hosts=/path/to/ssh_known_hosts +known-hosts=/path/to/ssh_known_hosts1 diff --git a/tests/qemunbdkitdata/disk-network-ssh-key.args.disk1 b/tests/qemunbdkitdata/disk-network-ssh-key.args.disk1 new file mode 100644 index 0000000000..80df9c30c6 --- /dev/null +++ b/tests/qemunbdkitdata/disk-network-ssh-key.args.disk1 @@ -0,0 +1,9 @@ +nbdkit \ +--unix /tmp/statedir-1/nbdkit-test-disk-1.socket \ +--foreground ssh \ +host=example.org \ +port=2222 \ +path=test2.img \ +identity=/path/to/id_rsa \ +user=myuser2 \ +known-hosts=/path/to/ssh_known_hosts2 diff --git a/tests/qemuxml2argvdata/disk-network-ssh-key.xml b/tests/qemuxml2argvdata/disk-network-ssh-key.xml index 81b92231fa..fda01e7e68 100644 --- a/tests/qemuxml2argvdata/disk-network-ssh-key.xml +++ b/tests/qemuxml2argvdata/disk-network-ssh-key.xml @@ -15,12 +15,23 @@ <devices> <disk type='network' device='disk'> <driver name='qemu' type='raw'/> - <source protocol='ssh' name='test.img'> + <source protocol='ssh' name='test1.img'> <host name='example.org' port='2222'/> <timeout seconds='1234'/> <readahead size='1024'/> - <identity username='myuser' keyfile='/path/to/id_rsa'/> - <knownHosts path="/path/to/ssh_known_hosts"/> + <identity username='myuser' agentsock='/path/to/agent/socket'/> + <knownHosts path="/path/to/ssh_known_hosts1"/> + </source> + <target dev='vda' bus='virtio'/> + </disk> + <disk type='network' device='disk'> + <driver name='qemu' type='raw'/> + <source protocol='ssh' name='test2.img'> + <host name='example.org' port='2222'/> + <timeout seconds='1234'/> + <readahead size='1024'/> + <identity username='myuser2' keyfile='/path/to/id_rsa'/> + <knownHosts path="/path/to/ssh_known_hosts2"/> </source> <target dev='vda' bus='virtio'/> </disk> -- 2.41.0

Require libnbd-devel when building the qemu driver, recommend nbdkit packages. Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> --- libvirt.spec.in | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/libvirt.spec.in b/libvirt.spec.in index b471afebb1..744fa5c88d 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -312,6 +312,7 @@ BuildRequires: util-linux BuildRequires: libacl-devel # From QEMU RPMs, used by virstoragetest BuildRequires: /usr/bin/qemu-img +BuildRequires: libnbd-devel %endif # For LVM drivers BuildRequires: lvm2 @@ -768,6 +769,9 @@ Requires: numad Recommends: passt Recommends: passt-selinux %endif +Recommends: nbdkit +Recommends: nbdkit-curl-plugin +Recommends: nbdkit-ssh-plugin %description daemon-driver-qemu The qemu driver plugin for the libvirtd daemon, providing @@ -1074,8 +1078,10 @@ exit 1 %if %{with_qemu} %define arg_qemu -Ddriver_qemu=enabled + %define arg_libnbd -Dlibnbd=enabled %else %define arg_qemu -Ddriver_qemu=disabled + %define arg_libnbd -Dlibnbd=disabled %endif %if %{with_openvz} @@ -1264,6 +1270,7 @@ export SOURCE_DATE_EPOCH=$(stat --printf='%Y' %{_specdir}/libvirt.spec) -Dyajl=enabled \ %{?arg_sanlock} \ -Dlibpcap=enabled \ + %{?arg_libnbd} \ -Dlibnl=enabled \ -Daudit=enabled \ -Ddtrace=enabled \ @@ -1327,6 +1334,7 @@ export SOURCE_DATE_EPOCH=$(stat --printf='%Y' %{_specdir}/libvirt.spec) -Dglusterfs=disabled \ -Dhost_validate=disabled \ -Dlibiscsi=disabled \ + -Dlibnbd=disabled \ -Dlibnl=disabled \ -Dlibpcap=disabled \ -Dlibssh2=disabled \ -- 2.41.0

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> --- ci/buildenv/almalinux-8.sh | 1 + ci/buildenv/centos-stream-8.sh | 1 + ci/buildenv/centos-stream-9.sh | 1 + ci/buildenv/debian-12-cross-aarch64.sh | 1 + ci/buildenv/debian-12-cross-armv6l.sh | 1 + ci/buildenv/debian-12-cross-armv7l.sh | 1 + ci/buildenv/debian-12-cross-i686.sh | 1 + ci/buildenv/debian-12-cross-mips64el.sh | 1 + ci/buildenv/debian-12-cross-mipsel.sh | 1 + ci/buildenv/debian-12-cross-ppc64le.sh | 1 + ci/buildenv/debian-12-cross-s390x.sh | 1 + ci/buildenv/debian-12.sh | 1 + ci/buildenv/debian-sid-cross-aarch64.sh | 1 + ci/buildenv/debian-sid-cross-armv6l.sh | 1 + ci/buildenv/debian-sid-cross-armv7l.sh | 1 + ci/buildenv/debian-sid-cross-i686.sh | 1 + ci/buildenv/debian-sid-cross-mips64el.sh | 1 + ci/buildenv/debian-sid-cross-mipsel.sh | 1 + ci/buildenv/debian-sid-cross-ppc64le.sh | 1 + ci/buildenv/debian-sid-cross-s390x.sh | 1 + ci/buildenv/debian-sid.sh | 1 + ci/buildenv/fedora-37.sh | 1 + ci/buildenv/fedora-38-cross-mingw32.sh | 1 + ci/buildenv/fedora-38-cross-mingw64.sh | 1 + ci/buildenv/fedora-38.sh | 1 + ci/buildenv/fedora-rawhide-cross-mingw32.sh | 1 + ci/buildenv/fedora-rawhide-cross-mingw64.sh | 1 + ci/buildenv/fedora-rawhide.sh | 1 + ci/buildenv/opensuse-leap-15.sh | 1 + ci/buildenv/opensuse-tumbleweed.sh | 1 + ci/buildenv/ubuntu-2204.sh | 1 + ci/containers/almalinux-8.Dockerfile | 1 + ci/containers/centos-stream-8.Dockerfile | 1 + ci/containers/centos-stream-9.Dockerfile | 1 + ci/containers/debian-12-cross-aarch64.Dockerfile | 1 + ci/containers/debian-12-cross-armv6l.Dockerfile | 1 + ci/containers/debian-12-cross-armv7l.Dockerfile | 1 + ci/containers/debian-12-cross-i686.Dockerfile | 1 + ci/containers/debian-12-cross-mips64el.Dockerfile | 1 + ci/containers/debian-12-cross-mipsel.Dockerfile | 1 + ci/containers/debian-12-cross-ppc64le.Dockerfile | 1 + ci/containers/debian-12-cross-s390x.Dockerfile | 1 + ci/containers/debian-12.Dockerfile | 1 + ci/containers/debian-sid-cross-aarch64.Dockerfile | 1 + ci/containers/debian-sid-cross-armv6l.Dockerfile | 1 + ci/containers/debian-sid-cross-armv7l.Dockerfile | 1 + ci/containers/debian-sid-cross-i686.Dockerfile | 1 + ci/containers/debian-sid-cross-mips64el.Dockerfile | 1 + ci/containers/debian-sid-cross-mipsel.Dockerfile | 1 + ci/containers/debian-sid-cross-ppc64le.Dockerfile | 1 + ci/containers/debian-sid-cross-s390x.Dockerfile | 1 + ci/containers/debian-sid.Dockerfile | 1 + ci/containers/fedora-37.Dockerfile | 1 + ci/containers/fedora-38-cross-mingw32.Dockerfile | 1 + ci/containers/fedora-38-cross-mingw64.Dockerfile | 1 + ci/containers/fedora-38.Dockerfile | 1 + ci/containers/fedora-rawhide-cross-mingw32.Dockerfile | 1 + ci/containers/fedora-rawhide-cross-mingw64.Dockerfile | 1 + ci/containers/fedora-rawhide.Dockerfile | 1 + ci/containers/opensuse-leap-15.Dockerfile | 1 + ci/containers/opensuse-tumbleweed.Dockerfile | 1 + ci/containers/ubuntu-2204.Dockerfile | 1 + ci/lcitool/projects/libvirt.yml | 1 + 63 files changed, 63 insertions(+) diff --git a/ci/buildenv/almalinux-8.sh b/ci/buildenv/almalinux-8.sh index 086b4d946b..e3c0909557 100644 --- a/ci/buildenv/almalinux-8.sh +++ b/ci/buildenv/almalinux-8.sh @@ -45,6 +45,7 @@ function install_buildenv() { libcap-ng-devel \ libcurl-devel \ libiscsi-devel \ + libnbd-devel \ libnl3-devel \ libpcap-devel \ libpciaccess-devel \ diff --git a/ci/buildenv/centos-stream-8.sh b/ci/buildenv/centos-stream-8.sh index 6b3de502df..f6de8ee7f7 100644 --- a/ci/buildenv/centos-stream-8.sh +++ b/ci/buildenv/centos-stream-8.sh @@ -46,6 +46,7 @@ function install_buildenv() { libcap-ng-devel \ libcurl-devel \ libiscsi-devel \ + libnbd-devel \ libnl3-devel \ libpcap-devel \ libpciaccess-devel \ diff --git a/ci/buildenv/centos-stream-9.sh b/ci/buildenv/centos-stream-9.sh index 454e1f6322..d6657425a2 100644 --- a/ci/buildenv/centos-stream-9.sh +++ b/ci/buildenv/centos-stream-9.sh @@ -43,6 +43,7 @@ function install_buildenv() { libblkid-devel \ libcap-ng-devel \ libcurl-devel \ + libnbd-devel \ libnl3-devel \ libpcap-devel \ libpciaccess-devel \ diff --git a/ci/buildenv/debian-12-cross-aarch64.sh b/ci/buildenv/debian-12-cross-aarch64.sh index 6b03b2ea59..7d762ac668 100644 --- a/ci/buildenv/debian-12-cross-aarch64.sh +++ b/ci/buildenv/debian-12-cross-aarch64.sh @@ -27,6 +27,7 @@ function install_buildenv() { iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/buildenv/debian-12-cross-armv6l.sh b/ci/buildenv/debian-12-cross-armv6l.sh index 51dc4ce88a..e3d0619808 100644 --- a/ci/buildenv/debian-12-cross-armv6l.sh +++ b/ci/buildenv/debian-12-cross-armv6l.sh @@ -27,6 +27,7 @@ function install_buildenv() { iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/buildenv/debian-12-cross-armv7l.sh b/ci/buildenv/debian-12-cross-armv7l.sh index ecd1453bff..fe7d55f194 100644 --- a/ci/buildenv/debian-12-cross-armv7l.sh +++ b/ci/buildenv/debian-12-cross-armv7l.sh @@ -27,6 +27,7 @@ function install_buildenv() { iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/buildenv/debian-12-cross-i686.sh b/ci/buildenv/debian-12-cross-i686.sh index 3922591cd8..362ed4d450 100644 --- a/ci/buildenv/debian-12-cross-i686.sh +++ b/ci/buildenv/debian-12-cross-i686.sh @@ -27,6 +27,7 @@ function install_buildenv() { iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/buildenv/debian-12-cross-mips64el.sh b/ci/buildenv/debian-12-cross-mips64el.sh index 6a6250ac03..e38a68106e 100644 --- a/ci/buildenv/debian-12-cross-mips64el.sh +++ b/ci/buildenv/debian-12-cross-mips64el.sh @@ -27,6 +27,7 @@ function install_buildenv() { iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/buildenv/debian-12-cross-mipsel.sh b/ci/buildenv/debian-12-cross-mipsel.sh index 4b60a97704..90dac8c22a 100644 --- a/ci/buildenv/debian-12-cross-mipsel.sh +++ b/ci/buildenv/debian-12-cross-mipsel.sh @@ -27,6 +27,7 @@ function install_buildenv() { iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/buildenv/debian-12-cross-ppc64le.sh b/ci/buildenv/debian-12-cross-ppc64le.sh index c1fd55b55d..b314d12f1a 100644 --- a/ci/buildenv/debian-12-cross-ppc64le.sh +++ b/ci/buildenv/debian-12-cross-ppc64le.sh @@ -27,6 +27,7 @@ function install_buildenv() { iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/buildenv/debian-12-cross-s390x.sh b/ci/buildenv/debian-12-cross-s390x.sh index 563eb98ef7..03b6cdadf2 100644 --- a/ci/buildenv/debian-12-cross-s390x.sh +++ b/ci/buildenv/debian-12-cross-s390x.sh @@ -27,6 +27,7 @@ function install_buildenv() { iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/buildenv/debian-12.sh b/ci/buildenv/debian-12.sh index a40e327db4..5976fc0815 100644 --- a/ci/buildenv/debian-12.sh +++ b/ci/buildenv/debian-12.sh @@ -43,6 +43,7 @@ function install_buildenv() { libglusterfs-dev \ libgnutls28-dev \ libiscsi-dev \ + libnbd-dev \ libnl-3-dev \ libnl-route-3-dev \ libnuma-dev \ diff --git a/ci/buildenv/debian-sid-cross-aarch64.sh b/ci/buildenv/debian-sid-cross-aarch64.sh index 6b03b2ea59..7d762ac668 100644 --- a/ci/buildenv/debian-sid-cross-aarch64.sh +++ b/ci/buildenv/debian-sid-cross-aarch64.sh @@ -27,6 +27,7 @@ function install_buildenv() { iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/buildenv/debian-sid-cross-armv6l.sh b/ci/buildenv/debian-sid-cross-armv6l.sh index 51dc4ce88a..e3d0619808 100644 --- a/ci/buildenv/debian-sid-cross-armv6l.sh +++ b/ci/buildenv/debian-sid-cross-armv6l.sh @@ -27,6 +27,7 @@ function install_buildenv() { iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/buildenv/debian-sid-cross-armv7l.sh b/ci/buildenv/debian-sid-cross-armv7l.sh index ecd1453bff..fe7d55f194 100644 --- a/ci/buildenv/debian-sid-cross-armv7l.sh +++ b/ci/buildenv/debian-sid-cross-armv7l.sh @@ -27,6 +27,7 @@ function install_buildenv() { iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/buildenv/debian-sid-cross-i686.sh b/ci/buildenv/debian-sid-cross-i686.sh index 3922591cd8..362ed4d450 100644 --- a/ci/buildenv/debian-sid-cross-i686.sh +++ b/ci/buildenv/debian-sid-cross-i686.sh @@ -27,6 +27,7 @@ function install_buildenv() { iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/buildenv/debian-sid-cross-mips64el.sh b/ci/buildenv/debian-sid-cross-mips64el.sh index 6a6250ac03..e38a68106e 100644 --- a/ci/buildenv/debian-sid-cross-mips64el.sh +++ b/ci/buildenv/debian-sid-cross-mips64el.sh @@ -27,6 +27,7 @@ function install_buildenv() { iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/buildenv/debian-sid-cross-mipsel.sh b/ci/buildenv/debian-sid-cross-mipsel.sh index 4b60a97704..90dac8c22a 100644 --- a/ci/buildenv/debian-sid-cross-mipsel.sh +++ b/ci/buildenv/debian-sid-cross-mipsel.sh @@ -27,6 +27,7 @@ function install_buildenv() { iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/buildenv/debian-sid-cross-ppc64le.sh b/ci/buildenv/debian-sid-cross-ppc64le.sh index c1fd55b55d..b314d12f1a 100644 --- a/ci/buildenv/debian-sid-cross-ppc64le.sh +++ b/ci/buildenv/debian-sid-cross-ppc64le.sh @@ -27,6 +27,7 @@ function install_buildenv() { iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/buildenv/debian-sid-cross-s390x.sh b/ci/buildenv/debian-sid-cross-s390x.sh index 563eb98ef7..03b6cdadf2 100644 --- a/ci/buildenv/debian-sid-cross-s390x.sh +++ b/ci/buildenv/debian-sid-cross-s390x.sh @@ -27,6 +27,7 @@ function install_buildenv() { iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/buildenv/debian-sid.sh b/ci/buildenv/debian-sid.sh index a40e327db4..5976fc0815 100644 --- a/ci/buildenv/debian-sid.sh +++ b/ci/buildenv/debian-sid.sh @@ -43,6 +43,7 @@ function install_buildenv() { libglusterfs-dev \ libgnutls28-dev \ libiscsi-dev \ + libnbd-dev \ libnl-3-dev \ libnl-route-3-dev \ libnuma-dev \ diff --git a/ci/buildenv/fedora-37.sh b/ci/buildenv/fedora-37.sh index 23886ae77c..5308bc685d 100644 --- a/ci/buildenv/fedora-37.sh +++ b/ci/buildenv/fedora-37.sh @@ -43,6 +43,7 @@ function install_buildenv() { libcap-ng-devel \ libcurl-devel \ libiscsi-devel \ + libnbd-devel \ libnl3-devel \ libpcap-devel \ libpciaccess-devel \ diff --git a/ci/buildenv/fedora-38-cross-mingw32.sh b/ci/buildenv/fedora-38-cross-mingw32.sh index a45c84802f..babfa6450a 100644 --- a/ci/buildenv/fedora-38-cross-mingw32.sh +++ b/ci/buildenv/fedora-38-cross-mingw32.sh @@ -26,6 +26,7 @@ function install_buildenv() { iptables \ iscsi-initiator-utils \ kmod \ + libnbd-devel \ libxml2 \ libxslt \ lvm2 \ diff --git a/ci/buildenv/fedora-38-cross-mingw64.sh b/ci/buildenv/fedora-38-cross-mingw64.sh index 1e2b2bace2..18ae4c3f3d 100644 --- a/ci/buildenv/fedora-38-cross-mingw64.sh +++ b/ci/buildenv/fedora-38-cross-mingw64.sh @@ -26,6 +26,7 @@ function install_buildenv() { iptables \ iscsi-initiator-utils \ kmod \ + libnbd-devel \ libxml2 \ libxslt \ lvm2 \ diff --git a/ci/buildenv/fedora-38.sh b/ci/buildenv/fedora-38.sh index 23886ae77c..5308bc685d 100644 --- a/ci/buildenv/fedora-38.sh +++ b/ci/buildenv/fedora-38.sh @@ -43,6 +43,7 @@ function install_buildenv() { libcap-ng-devel \ libcurl-devel \ libiscsi-devel \ + libnbd-devel \ libnl3-devel \ libpcap-devel \ libpciaccess-devel \ diff --git a/ci/buildenv/fedora-rawhide-cross-mingw32.sh b/ci/buildenv/fedora-rawhide-cross-mingw32.sh index ed87040be6..218159f17d 100644 --- a/ci/buildenv/fedora-rawhide-cross-mingw32.sh +++ b/ci/buildenv/fedora-rawhide-cross-mingw32.sh @@ -27,6 +27,7 @@ function install_buildenv() { iptables \ iscsi-initiator-utils \ kmod \ + libnbd-devel \ libxml2 \ libxslt \ lvm2 \ diff --git a/ci/buildenv/fedora-rawhide-cross-mingw64.sh b/ci/buildenv/fedora-rawhide-cross-mingw64.sh index c57e81dea3..42a3df41f5 100644 --- a/ci/buildenv/fedora-rawhide-cross-mingw64.sh +++ b/ci/buildenv/fedora-rawhide-cross-mingw64.sh @@ -27,6 +27,7 @@ function install_buildenv() { iptables \ iscsi-initiator-utils \ kmod \ + libnbd-devel \ libxml2 \ libxslt \ lvm2 \ diff --git a/ci/buildenv/fedora-rawhide.sh b/ci/buildenv/fedora-rawhide.sh index 7a774c898c..ed69d9cdf9 100644 --- a/ci/buildenv/fedora-rawhide.sh +++ b/ci/buildenv/fedora-rawhide.sh @@ -44,6 +44,7 @@ function install_buildenv() { libcap-ng-devel \ libcurl-devel \ libiscsi-devel \ + libnbd-devel \ libnl3-devel \ libpcap-devel \ libpciaccess-devel \ diff --git a/ci/buildenv/opensuse-leap-15.sh b/ci/buildenv/opensuse-leap-15.sh index 93cb99c83b..e649f8c093 100644 --- a/ci/buildenv/opensuse-leap-15.sh +++ b/ci/buildenv/opensuse-leap-15.sh @@ -42,6 +42,7 @@ function install_buildenv() { libcurl-devel \ libgnutls-devel \ libiscsi-devel \ + libnbd-devel \ libnl3-devel \ libnuma-devel \ libpcap-devel \ diff --git a/ci/buildenv/opensuse-tumbleweed.sh b/ci/buildenv/opensuse-tumbleweed.sh index 0d064cc6ac..a67aae46c1 100644 --- a/ci/buildenv/opensuse-tumbleweed.sh +++ b/ci/buildenv/opensuse-tumbleweed.sh @@ -42,6 +42,7 @@ function install_buildenv() { libcurl-devel \ libgnutls-devel \ libiscsi-devel \ + libnbd-devel \ libnl3-devel \ libnuma-devel \ libpcap-devel \ diff --git a/ci/buildenv/ubuntu-2204.sh b/ci/buildenv/ubuntu-2204.sh index bb041e8eca..8d70f2fa5f 100644 --- a/ci/buildenv/ubuntu-2204.sh +++ b/ci/buildenv/ubuntu-2204.sh @@ -43,6 +43,7 @@ function install_buildenv() { libglusterfs-dev \ libgnutls28-dev \ libiscsi-dev \ + libnbd-dev \ libnl-3-dev \ libnl-route-3-dev \ libnuma-dev \ diff --git a/ci/containers/almalinux-8.Dockerfile b/ci/containers/almalinux-8.Dockerfile index d27fae106c..f3e2086e42 100644 --- a/ci/containers/almalinux-8.Dockerfile +++ b/ci/containers/almalinux-8.Dockerfile @@ -46,6 +46,7 @@ RUN dnf update -y && \ libcap-ng-devel \ libcurl-devel \ libiscsi-devel \ + libnbd-devel \ libnl3-devel \ libpcap-devel \ libpciaccess-devel \ diff --git a/ci/containers/centos-stream-8.Dockerfile b/ci/containers/centos-stream-8.Dockerfile index bc4439b506..984ccf3a81 100644 --- a/ci/containers/centos-stream-8.Dockerfile +++ b/ci/containers/centos-stream-8.Dockerfile @@ -47,6 +47,7 @@ RUN dnf distro-sync -y && \ libcap-ng-devel \ libcurl-devel \ libiscsi-devel \ + libnbd-devel \ libnl3-devel \ libpcap-devel \ libpciaccess-devel \ diff --git a/ci/containers/centos-stream-9.Dockerfile b/ci/containers/centos-stream-9.Dockerfile index 47e9008b72..10940b27df 100644 --- a/ci/containers/centos-stream-9.Dockerfile +++ b/ci/containers/centos-stream-9.Dockerfile @@ -44,6 +44,7 @@ RUN dnf distro-sync -y && \ libblkid-devel \ libcap-ng-devel \ libcurl-devel \ + libnbd-devel \ libnl3-devel \ libpcap-devel \ libpciaccess-devel \ diff --git a/ci/containers/debian-12-cross-aarch64.Dockerfile b/ci/containers/debian-12-cross-aarch64.Dockerfile index a4824aadc1..fa71e6ce1a 100644 --- a/ci/containers/debian-12-cross-aarch64.Dockerfile +++ b/ci/containers/debian-12-cross-aarch64.Dockerfile @@ -29,6 +29,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/containers/debian-12-cross-armv6l.Dockerfile b/ci/containers/debian-12-cross-armv6l.Dockerfile index 38e40dbda0..0a1927e449 100644 --- a/ci/containers/debian-12-cross-armv6l.Dockerfile +++ b/ci/containers/debian-12-cross-armv6l.Dockerfile @@ -29,6 +29,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/containers/debian-12-cross-armv7l.Dockerfile b/ci/containers/debian-12-cross-armv7l.Dockerfile index 630418f497..eced26b6fc 100644 --- a/ci/containers/debian-12-cross-armv7l.Dockerfile +++ b/ci/containers/debian-12-cross-armv7l.Dockerfile @@ -29,6 +29,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/containers/debian-12-cross-i686.Dockerfile b/ci/containers/debian-12-cross-i686.Dockerfile index 547f7a4b9d..0adac546c5 100644 --- a/ci/containers/debian-12-cross-i686.Dockerfile +++ b/ci/containers/debian-12-cross-i686.Dockerfile @@ -29,6 +29,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/containers/debian-12-cross-mips64el.Dockerfile b/ci/containers/debian-12-cross-mips64el.Dockerfile index bd4cc1ab6c..a10720d46f 100644 --- a/ci/containers/debian-12-cross-mips64el.Dockerfile +++ b/ci/containers/debian-12-cross-mips64el.Dockerfile @@ -29,6 +29,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/containers/debian-12-cross-mipsel.Dockerfile b/ci/containers/debian-12-cross-mipsel.Dockerfile index 4eb3eff835..21cb174825 100644 --- a/ci/containers/debian-12-cross-mipsel.Dockerfile +++ b/ci/containers/debian-12-cross-mipsel.Dockerfile @@ -29,6 +29,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/containers/debian-12-cross-ppc64le.Dockerfile b/ci/containers/debian-12-cross-ppc64le.Dockerfile index 890db100f7..db3ea27999 100644 --- a/ci/containers/debian-12-cross-ppc64le.Dockerfile +++ b/ci/containers/debian-12-cross-ppc64le.Dockerfile @@ -29,6 +29,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/containers/debian-12-cross-s390x.Dockerfile b/ci/containers/debian-12-cross-s390x.Dockerfile index d3fed3a47c..4a753d7859 100644 --- a/ci/containers/debian-12-cross-s390x.Dockerfile +++ b/ci/containers/debian-12-cross-s390x.Dockerfile @@ -29,6 +29,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/containers/debian-12.Dockerfile b/ci/containers/debian-12.Dockerfile index 9867e56955..8e7fddfed9 100644 --- a/ci/containers/debian-12.Dockerfile +++ b/ci/containers/debian-12.Dockerfile @@ -45,6 +45,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ libglusterfs-dev \ libgnutls28-dev \ libiscsi-dev \ + libnbd-dev \ libnl-3-dev \ libnl-route-3-dev \ libnuma-dev \ diff --git a/ci/containers/debian-sid-cross-aarch64.Dockerfile b/ci/containers/debian-sid-cross-aarch64.Dockerfile index c98ba2f751..52de106555 100644 --- a/ci/containers/debian-sid-cross-aarch64.Dockerfile +++ b/ci/containers/debian-sid-cross-aarch64.Dockerfile @@ -29,6 +29,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/containers/debian-sid-cross-armv6l.Dockerfile b/ci/containers/debian-sid-cross-armv6l.Dockerfile index de3ad751d6..3c5f7f5e09 100644 --- a/ci/containers/debian-sid-cross-armv6l.Dockerfile +++ b/ci/containers/debian-sid-cross-armv6l.Dockerfile @@ -29,6 +29,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/containers/debian-sid-cross-armv7l.Dockerfile b/ci/containers/debian-sid-cross-armv7l.Dockerfile index b271c97950..62a26cfeaa 100644 --- a/ci/containers/debian-sid-cross-armv7l.Dockerfile +++ b/ci/containers/debian-sid-cross-armv7l.Dockerfile @@ -29,6 +29,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/containers/debian-sid-cross-i686.Dockerfile b/ci/containers/debian-sid-cross-i686.Dockerfile index a12507feca..36d38f0b77 100644 --- a/ci/containers/debian-sid-cross-i686.Dockerfile +++ b/ci/containers/debian-sid-cross-i686.Dockerfile @@ -29,6 +29,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/containers/debian-sid-cross-mips64el.Dockerfile b/ci/containers/debian-sid-cross-mips64el.Dockerfile index 9a4c8387ce..260c3cd5f4 100644 --- a/ci/containers/debian-sid-cross-mips64el.Dockerfile +++ b/ci/containers/debian-sid-cross-mips64el.Dockerfile @@ -29,6 +29,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/containers/debian-sid-cross-mipsel.Dockerfile b/ci/containers/debian-sid-cross-mipsel.Dockerfile index 2d18455474..81895b23dd 100644 --- a/ci/containers/debian-sid-cross-mipsel.Dockerfile +++ b/ci/containers/debian-sid-cross-mipsel.Dockerfile @@ -29,6 +29,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/containers/debian-sid-cross-ppc64le.Dockerfile b/ci/containers/debian-sid-cross-ppc64le.Dockerfile index fa646e3d54..f2f11cb68c 100644 --- a/ci/containers/debian-sid-cross-ppc64le.Dockerfile +++ b/ci/containers/debian-sid-cross-ppc64le.Dockerfile @@ -29,6 +29,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/containers/debian-sid-cross-s390x.Dockerfile b/ci/containers/debian-sid-cross-s390x.Dockerfile index e1d94bd642..4b38e597d6 100644 --- a/ci/containers/debian-sid-cross-s390x.Dockerfile +++ b/ci/containers/debian-sid-cross-s390x.Dockerfile @@ -29,6 +29,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ iptables \ kmod \ libc-dev-bin \ + libnbd-dev \ libxml2-utils \ locales \ lvm2 \ diff --git a/ci/containers/debian-sid.Dockerfile b/ci/containers/debian-sid.Dockerfile index 9c08afdb3d..956f6254d9 100644 --- a/ci/containers/debian-sid.Dockerfile +++ b/ci/containers/debian-sid.Dockerfile @@ -45,6 +45,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ libglusterfs-dev \ libgnutls28-dev \ libiscsi-dev \ + libnbd-dev \ libnl-3-dev \ libnl-route-3-dev \ libnuma-dev \ diff --git a/ci/containers/fedora-37.Dockerfile b/ci/containers/fedora-37.Dockerfile index d8e87bd310..967e210d53 100644 --- a/ci/containers/fedora-37.Dockerfile +++ b/ci/containers/fedora-37.Dockerfile @@ -54,6 +54,7 @@ exec "$@"\n' > /usr/bin/nosync && \ libcap-ng-devel \ libcurl-devel \ libiscsi-devel \ + libnbd-devel \ libnl3-devel \ libpcap-devel \ libpciaccess-devel \ diff --git a/ci/containers/fedora-38-cross-mingw32.Dockerfile b/ci/containers/fedora-38-cross-mingw32.Dockerfile index 1292f9874e..7d3267b0f8 100644 --- a/ci/containers/fedora-38-cross-mingw32.Dockerfile +++ b/ci/containers/fedora-38-cross-mingw32.Dockerfile @@ -37,6 +37,7 @@ exec "$@"\n' > /usr/bin/nosync && \ iptables \ iscsi-initiator-utils \ kmod \ + libnbd-devel \ libxml2 \ libxslt \ lvm2 \ diff --git a/ci/containers/fedora-38-cross-mingw64.Dockerfile b/ci/containers/fedora-38-cross-mingw64.Dockerfile index b45aab5ceb..286b22c434 100644 --- a/ci/containers/fedora-38-cross-mingw64.Dockerfile +++ b/ci/containers/fedora-38-cross-mingw64.Dockerfile @@ -37,6 +37,7 @@ exec "$@"\n' > /usr/bin/nosync && \ iptables \ iscsi-initiator-utils \ kmod \ + libnbd-devel \ libxml2 \ libxslt \ lvm2 \ diff --git a/ci/containers/fedora-38.Dockerfile b/ci/containers/fedora-38.Dockerfile index 0b67a56b80..95d4d0780d 100644 --- a/ci/containers/fedora-38.Dockerfile +++ b/ci/containers/fedora-38.Dockerfile @@ -54,6 +54,7 @@ exec "$@"\n' > /usr/bin/nosync && \ libcap-ng-devel \ libcurl-devel \ libiscsi-devel \ + libnbd-devel \ libnl3-devel \ libpcap-devel \ libpciaccess-devel \ diff --git a/ci/containers/fedora-rawhide-cross-mingw32.Dockerfile b/ci/containers/fedora-rawhide-cross-mingw32.Dockerfile index 41221256f8..113e18159b 100644 --- a/ci/containers/fedora-rawhide-cross-mingw32.Dockerfile +++ b/ci/containers/fedora-rawhide-cross-mingw32.Dockerfile @@ -38,6 +38,7 @@ exec "$@"\n' > /usr/bin/nosync && \ iptables \ iscsi-initiator-utils \ kmod \ + libnbd-devel \ libxml2 \ libxslt \ lvm2 \ diff --git a/ci/containers/fedora-rawhide-cross-mingw64.Dockerfile b/ci/containers/fedora-rawhide-cross-mingw64.Dockerfile index 8737203d83..19408fd177 100644 --- a/ci/containers/fedora-rawhide-cross-mingw64.Dockerfile +++ b/ci/containers/fedora-rawhide-cross-mingw64.Dockerfile @@ -38,6 +38,7 @@ exec "$@"\n' > /usr/bin/nosync && \ iptables \ iscsi-initiator-utils \ kmod \ + libnbd-devel \ libxml2 \ libxslt \ lvm2 \ diff --git a/ci/containers/fedora-rawhide.Dockerfile b/ci/containers/fedora-rawhide.Dockerfile index a2ca2e9c61..2cba6a3a38 100644 --- a/ci/containers/fedora-rawhide.Dockerfile +++ b/ci/containers/fedora-rawhide.Dockerfile @@ -55,6 +55,7 @@ exec "$@"\n' > /usr/bin/nosync && \ libcap-ng-devel \ libcurl-devel \ libiscsi-devel \ + libnbd-devel \ libnl3-devel \ libpcap-devel \ libpciaccess-devel \ diff --git a/ci/containers/opensuse-leap-15.Dockerfile b/ci/containers/opensuse-leap-15.Dockerfile index dfb5b27bbb..110692173e 100644 --- a/ci/containers/opensuse-leap-15.Dockerfile +++ b/ci/containers/opensuse-leap-15.Dockerfile @@ -43,6 +43,7 @@ RUN zypper update -y && \ libcurl-devel \ libgnutls-devel \ libiscsi-devel \ + libnbd-devel \ libnl3-devel \ libnuma-devel \ libpcap-devel \ diff --git a/ci/containers/opensuse-tumbleweed.Dockerfile b/ci/containers/opensuse-tumbleweed.Dockerfile index 33a3bf559a..d4b68e25c4 100644 --- a/ci/containers/opensuse-tumbleweed.Dockerfile +++ b/ci/containers/opensuse-tumbleweed.Dockerfile @@ -43,6 +43,7 @@ RUN zypper dist-upgrade -y && \ libcurl-devel \ libgnutls-devel \ libiscsi-devel \ + libnbd-devel \ libnl3-devel \ libnuma-devel \ libpcap-devel \ diff --git a/ci/containers/ubuntu-2204.Dockerfile b/ci/containers/ubuntu-2204.Dockerfile index 5b7f6e96af..7d7b549a36 100644 --- a/ci/containers/ubuntu-2204.Dockerfile +++ b/ci/containers/ubuntu-2204.Dockerfile @@ -45,6 +45,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ libglusterfs-dev \ libgnutls28-dev \ libiscsi-dev \ + libnbd-dev \ libnl-3-dev \ libnl-route-3-dev \ libnuma-dev \ diff --git a/ci/lcitool/projects/libvirt.yml b/ci/lcitool/projects/libvirt.yml index 0234a1cae3..6df1710d2a 100644 --- a/ci/lcitool/projects/libvirt.yml +++ b/ci/lcitool/projects/libvirt.yml @@ -36,6 +36,7 @@ packages: - libcap-ng - libcurl - libiscsi + - libnbd - libnl3 - libnlroute3 - libnuma -- 2.41.0

On Thu, Aug 31, 2023 at 16:39:40 -0500, Jonathon Jongsma wrote: [...]
Jonathon Jongsma (37): schema: allow 'ssh' as a protocol for network disks qemu: Add functions for determining nbdkit availability qemu: expand nbdkit capabilities util: Allow virFileCache data to be any GObject qemu: implement basic virFileCache for nbdkit caps qemu: implement persistent file cache for nbdkit caps qemu: use file cache for nbdkit caps qemu: Add qemuNbdkitProcess qemu: query nbdkit module dir from binary qemu: add functions to start and stop nbdkit Generalize qemuDomainLogContextNew() qemu: Extract qemuDomainLogContext into a new file qemu: move qemuProcessReadLog() to qemuLogContext qemu: log error output from nbdkit tests: add ability to test various nbdkit capabilities qemu: split qemuDomainSecretStorageSourcePrepare qemu: include nbdkit state in private xml util: secure erase virCommand send buffers qemu: pass sensitive data to nbdkit via pipe qemu: use nbdkit to serve network disks if available util: make virCommandSetSendBuffer testable tests: add tests for nbdkit invocation qemu: add test for authenticating a https network disk qemu: Add Taint for nbdkit restart failure qemu: Monitor nbdkit process for exit qemu: improve error handling when restarting nbdkit qemu: try to connect to nbdkit early to detect errors schema: add password configuration for ssh disk qemu: implement password auth for ssh disks with nbdkit schema: add configuration for host verification of ssh disks qemu: implement knownHosts for ssh disks with nbdkit schema: add keyfile configuration for ssh disks qemu: implement keyfile auth for ssh disks with nbdkit schema: add ssh-agent configuration for ssh disks qemu: implement ssh-agent auth for ssh disks with nbdkit
Please push these patches. I plan working on qemu storage daemon integration which has the potential to conflict with this.
rpm: update spec file for for nbdkit support
This one can be omitted if you're still waiting for the selinux policy.
ci: add libnbd to build
This one probably should work even now.

On 9/19/23 2:44 AM, Peter Krempa wrote:
On Thu, Aug 31, 2023 at 16:39:40 -0500, Jonathon Jongsma wrote:
[...]
Jonathon Jongsma (37): schema: allow 'ssh' as a protocol for network disks qemu: Add functions for determining nbdkit availability qemu: expand nbdkit capabilities util: Allow virFileCache data to be any GObject qemu: implement basic virFileCache for nbdkit caps qemu: implement persistent file cache for nbdkit caps qemu: use file cache for nbdkit caps qemu: Add qemuNbdkitProcess qemu: query nbdkit module dir from binary qemu: add functions to start and stop nbdkit Generalize qemuDomainLogContextNew() qemu: Extract qemuDomainLogContext into a new file qemu: move qemuProcessReadLog() to qemuLogContext qemu: log error output from nbdkit tests: add ability to test various nbdkit capabilities qemu: split qemuDomainSecretStorageSourcePrepare qemu: include nbdkit state in private xml util: secure erase virCommand send buffers qemu: pass sensitive data to nbdkit via pipe qemu: use nbdkit to serve network disks if available util: make virCommandSetSendBuffer testable tests: add tests for nbdkit invocation qemu: add test for authenticating a https network disk qemu: Add Taint for nbdkit restart failure qemu: Monitor nbdkit process for exit qemu: improve error handling when restarting nbdkit qemu: try to connect to nbdkit early to detect errors schema: add password configuration for ssh disk qemu: implement password auth for ssh disks with nbdkit schema: add configuration for host verification of ssh disks qemu: implement knownHosts for ssh disks with nbdkit schema: add keyfile configuration for ssh disks qemu: implement keyfile auth for ssh disks with nbdkit schema: add ssh-agent configuration for ssh disks qemu: implement ssh-agent auth for ssh disks with nbdkit
Please push these patches. I plan working on qemu storage daemon integration which has the potential to conflict with this.
rpm: update spec file for for nbdkit support
This one can be omitted if you're still waiting for the selinux policy.
ci: add libnbd to build
This one probably should work even now.
Sorry for the delay. It should be pushed now. Jonathon
participants (3)
-
Jonathon Jongsma
-
Pavel Hrdina
-
Peter Krempa