5:41 a.m.
In this section (under %package libvirt-daemon-driver-qemu):
%if %{with_nbdkit}
Recommends: nbdkit
Recommends: nbdkit-curl-plugin
Recommends: nbdkit-ssh-plugin
%endif
You could save a bit of space by removing "Recommends: nbdkit" and
replacing it with:
Recommends: nbdkit-server
Recommends: nbdkit-readahead-filter
In Fedora packaging "nbdkit" is a meta-package which pulls in
nbdkit-server, nbdkit-basic-plugins and nbdkit-basic-filters. This
meta-package is intended to get a useful, functional nbdkit, but it's
not very minimal. The basic plugins/filters pull in about an extra
1.3MB.
nbdkit-basic-{plugins,filters} also "Provides" the names of the
individual plugins and filters, so if (for example) you wanted the
file plugin then you could just "Recommends: nbdkit-file-plugin" which
would pull in nbdkit-basic-plugins (or something else if we ever moved
that plugin to another package).
nbdkit-curl-plugin / nbdkit-ssh-plugin are packaged separately because
they have non-trivial dependencies on external libraries, and using
Recommends for those is absolutely correct. They depend on
nbdkit-server so it's not strictly needed to also "Recommends:
nbdkit-server", but it is better for clarity.
nbdkit-server includes nbdkit-null-plugin so you can still use that
for probing filters.
(I asked another question on a Jira ticket about if you are using
--readonly since that improves performance with nbdkit-curl-plugin.
Checking the source code I see you are, so that's great.)
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
6:01 a.m.
New subject: Another comment on nbdkit support
Few more things ...
(1) Looking at the parameters to nbdkit in
qemuNbdkitProcessBuildCommand, it might make sense to consider
--exit-with-parent. What this does on Linux is it causes nbdkit to
call:
prctl(PR_SET_PDEATHSIG, SIGTERM);
which causes the kernel to send SIGTERM to nbdkit if the parent
(virtqemud, presumably?) ever goes away.
This is just a belt and braces approach to ensuring that nbdkit is
always cleaned up, even if libvirt dies without managing to clean up.
So a couple of potential problems:
(a) --exit-with-parent is only supported on a subset of platforms
(currently Linux, BSD and macOS). You have to check:
$ nbdkit --dump-config | grep exit
exit_with_parent=yes
if you might use nbdkit outside those platforms. (Using
--exit-with-parent on a non-supported platform will give an error.)
(b) Is virtqemud actually persistent? I'm not totally sure what
happens for long-running libvirt domains if you restart libvirtd. Of
course if the parent process did go away in this case then it would
kill nbdkit which would be bad presumably ...
(2) I'm fairly sure you'll find you need to use --selinux-label at
some point. This does some SELinux/sVirt voodoo on the socket. We
found that this was necessary:
nbdkit -U /tmp/sock --selinux-label=system_u:object_r:svirt_socket_t:s0 ...
chcon system_u:object_r:svirt_image_t:s0 /tmp/sock
to allow qemu clients to connect to nbdkit when SELinux is enabled,
but only some of the time (like, it works fine without this on either
Fedora or RHEL, but not the other one, I forget which way round now).
(3) I'm wondering if, when libvirt's own debugging is enabled, we
should also add -v (verbose) and maybe even these -D flags:
-v -D curl.verbose=1 # for nbdkit-curl-plugin
-v -D ssh.log=2 # for nbdkit-ssh-plugin
nbdkit will send debug messages to stderr or syslog (see
--log=stderr|syslog option), which may or may not be useful.
(4) case VIR_STORAGE_NET_PROTOCOL_NBD gives an error. Technically
nbdkit does support NBD forwarding (nbdkit-nbd-plugin), but I think
you're correct not to use this. It's better to have qemu connect
directly using its own NBD support.
(5) Now we have the infrastructure, I think a massively valuable
feature would be VDDK support (nbdkit-vddk-plugin). Having libvirt
being able to boot qemu VMs backed directly by VMware servers would be
huge.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
6:11 a.m.
New subject: Another comment on nbdkit support
On Fri, Jan 19, 2024 at 12:01:55 +0000, Richard W.M. Jones wrote:
Few more things ...
(1) Looking at the parameters to nbdkit in
qemuNbdkitProcessBuildCommand, it might make sense to consider
--exit-with-parent. What this does on Linux is it causes nbdkit to
call:
prctl(PR_SET_PDEATHSIG, SIGTERM);
which causes the kernel to send SIGTERM to nbdkit if the parent
(virtqemud, presumably?) ever goes away.
We definitely do not want this. The VM will continue running if
virtqemud is restarted, thus nbdkit must continue running too.
This is just a belt and braces approach to ensuring that nbdkit is
always cleaned up, even if libvirt dies without managing to clean up.
So a couple of potential problems:
(a) --exit-with-parent is only supported on a subset of platforms
(currently Linux, BSD and macOS). You have to check:
$ nbdkit --dump-config | grep exit
exit_with_parent=yes
if you might use nbdkit outside those platforms. (Using
--exit-with-parent on a non-supported platform will give an error.)
(b) Is virtqemud actually persistent? I'm not totally sure what
happens for long-running libvirt domains if you restart libvirtd. Of
course if the parent process did go away in this case then it would
kill nbdkit which would be bad presumably ...
(2) I'm fairly sure you'll find you need to use --selinux-label at
some point. This does some SELinux/sVirt voodoo on the socket. We
found that this was necessary:
nbdkit -U /tmp/sock --selinux-label=system_u:object_r:svirt_socket_t:s0 ...
chcon system_u:object_r:svirt_image_t:s0 /tmp/sock
to allow qemu clients to connect to nbdkit when SELinux is enabled,
but only some of the time (like, it works fine without this on either
Fedora or RHEL, but not the other one, I forget which way round now).
IIRC this is handled by libvirt's security labelling code anyways.
(3) I'm wondering if, when libvirt's own debugging is
enabled, we
should also add -v (verbose) and maybe even these -D flags:
-v -D curl.verbose=1 # for nbdkit-curl-plugin
-v -D ssh.log=2 # for nbdkit-ssh-plugin
nbdkit will send debug messages to stderr or syslog (see
--log=stderr|syslog option), which may or may not be useful.
(4) case VIR_STORAGE_NET_PROTOCOL_NBD gives an error. Technically
nbdkit does support NBD forwarding (nbdkit-nbd-plugin), but I think
you're correct not to use this. It's better to have qemu connect
directly using its own NBD support.
That doesn't matter because nbdkit won't be even used for
VIR_STORAGE_NET_PROTOCOL_NBD disks and I don't see a point in doing so.
(5) Now we have the infrastructure, I think a massively valuable
feature would be VDDK support (nbdkit-vddk-plugin). Having libvirt
being able to boot qemu VMs backed directly by VMware servers would be
huge.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
_______________________________________________
Devel mailing list -- devel(a)lists.libvirt.org
To unsubscribe send an email to devel-leave(a)lists.libvirt.org
6:16 a.m.
New subject: Another comment on nbdkit support
On Fri, Jan 19, 2024 at 01:11:19PM +0100, Peter Krempa wrote:
On Fri, Jan 19, 2024 at 12:01:55 +0000, Richard W.M. Jones wrote:
> (2) I'm fairly sure you'll find you need to use --selinux-label at
> some point. This does some SELinux/sVirt voodoo on the socket. We
> found that this was necessary:
>
> nbdkit -U /tmp/sock --selinux-label=system_u:object_r:svirt_socket_t:s0 ...
> chcon system_u:object_r:svirt_image_t:s0 /tmp/sock
>
> to allow qemu clients to connect to nbdkit when SELinux is enabled,
> but only some of the time (like, it works fine without this on either
> Fedora or RHEL, but not the other one, I forget which way round now).
IIRC this is handled by libvirt's security labelling code anyways.
I think that's the labelling of the socket, but there's also the
labelling of the process (nbdkit) side of the socket? They are two
separate things, IIRC. Anyway just noting that we had trouble with
this in the past and the above ^^^ was found to be the solution.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
308
days inactive
308
days old
3 comments
2 participants
participants (2)
-
Peter Krempa -
Richard W.M. Jones