4:08 p.m.
On 7/18/19 2:58 PM, Daniel Henrique Barboza wrote:
On 7/18/19 2:18 PM, Laine Stump wrote:
> But to back up a bit - what is it about managed='yes' that makes you
> want to do it that way instead of managed='no'? Do you really ever
> need the devices to be binded to the host driver? Or are you just
> using managed='yes' because there's not a standard/concenient place
> to configure devices to be permanently binded to vfio-pci immediately
> when the host boots?
>
It's a case of user convenience for devices that has mixed usage, at
least in my opinion.
For example, I can say from personal experience dealing with devices
that will never be used directly by the host, such as NVIDIA GPUs that are
used only as hostdevs of guests, that this code I'm developing is
pointless. In that setup the GPUs are binded to vfio-pci right after the
host boots using a /etc/rc.d script (or something equivalent). Not sure
if this is the standard way of binding a device to vfio-pci, but it works
for that environment.
Yeah, the problem is that there really isn't a standard "this is *the
one correct way* to configure this" place for this config, so everybody
just does what works for them, making it difficult to provide a
"recommended solution" in the libvirt documentation that you (i.e. "I"
:-)) have a high level of confidence in.
The problem is with devices that the user expects to use both in
guests
and in the host. In that case, the user will need either to handle the
nodedev
detach/reattach manually or to use managed=true and let Libvirt re-attach
the devices every time the guest is destroyed. Even if the device is
going to
be used in the same or another guest right after (meaning that we
re-attached
the device back simply to detach it again right after), using managed=true
is convenient because the user doesn't need to think about the state of
the device.
Yeah, I agree that there are uses for managed='yes' and it's a good
thing to have. It's just that I think most of the time it's being used
when it isn't needed (and thus shouldn't be used).
> I think we should spend more time making it easier to have devices
> "pre-binded" to vfio-pci at boot time, so that we could discourage
> use of managed='yes'. (not "instead of" what you're doing, but
"in
> addition to" it).
>
>
I think managed=true use can be called a 'bad user habit' in that
sense. I can
think of some ways to alleviate it:
- an user setting in an conf file that changes how managed=true works.
Instead
of detach/re-attach the device, Libvirt will only detach the device,
leaving the
device bind to vfio-pci even after guest destroy
- same idea, but with a (yet another) XML attribute "re-attach=false"
in the
hostdev definition. I like this idea better because you can set customized
behavior for each device/guest instead of changing the managed mechanic to
everyone
I posted a patch to support that (with a new managed mode called
"detach", which would automatically bind the device to vfio-pci at geust
startup, and leave it binded to vfio-pci when the guest released it) a
couple years ago, and it was rejected upstream (after a lot of discussion):
https://www.redhat.com/archives/libvir-list/2016-March/msg00593.html
I believe the main reason was that it was "giving the consumer yet
another option that they probably wouldn't understand, and would make
the wrong choice on", or something like that...
I still like the idea, but it wasn't worth spending more time on the debate
- for boot time (daemon start time), one way I can think of is an XML
file with the hostdev devices that are supposed to be pre-binded to
vfio-pci
by libvirtd. Then the user can run the guests using managed=false in those
devices knowing that those devices are already taken care of. I don't know
how this idea interacts with the new "remember_owner" feature that
Michal implemented recently though .....
Back when I posted my patches, we discussed adding persistent config to
the nodedevice driver that would take care of binding certain devices to
vfio-pci. Unfortunately that misses one usage class - the case when a
device should *never* be bound to the host driver at all; those need to
be handled by something in the host boot process much earlier than libvirtd.
- we can make a 're-definition' of what managed=false means for PCI
hostdevs. Instead of failing to execute the guest if the device isn't
bind to
vfio-pci, managed=false means that Libvirt will detach the device from
the host if necessary, but it will not re-attach it back. If such a
redefinition
is a massive break to API and user expect behavior (I suspect it is
...) we can
create a "managed=mixed" with this mechanic
Yeah, changing the meaning of managed='no' would be a non-starter. (I
guess your "managed='mixed'" is pretty much what my 2016 patch did, only
with a different name).