1:06 p.m.
On 03/22/2016 07:03 PM, John Ferlan wrote:
On 03/14/2016 03:41 PM, Laine Stump wrote:
> Suggested by Alex Williamson.
>
> If you plan to assign a GPU to a virtual machine, but that GPU happens
> to be the host system console, you likely want it to start out using
> the host driver (so that boot messages/etc will be displayed), then
> later have the host driver replaced with vfio-pci for assignment to
> the virtual machine.
>
> However, in at least some cases (e.g. Intel i915) once the device has
> been detached from the host driver and attached to vfio-pci, attempts
> to reattach to the host driver only lead to "grief" (ask Alex for
> details). This means that simply using "managed='yes'" in libvirt
> won't work.
>
> And if you set "managed='no'" in libvirt then either you have to
> manually run virsh nodedev-detach prior to the first start of the
> guest, or you have to have a management application intelligent enough
> to know that it should detach from the host driver, but never reattach
> to it.
>
> This patch makes it simple/automatic to deal with such a case - it
> adds a third "managed" mode for assigned PCI devices, called
> "detach". It will detach ("unbind" in driver parlance) the device
from
> the host driver prior to assigning it to the guest, but when the guest
> is finished with the device, will leave it bound to vfio-pci. This
> allows re-using the device for another guest, without requiring
> initial out-of-band intervention to unbind the host driver.
> ---
>
> I'm sending this with the "RFC" tag because I'm concerned it might
be
> considered "feature creep" by some (although I think it makes at least
> as much sense as "managed='yes'") and also because, even once (if)
it
> is ACKed, I wouldn't want to push it until abologna is finished
> hacking around with the driver bind/unbind code - he has enough grief
> to deal with without me causing a bunch of merge conflicts :-)
>
[...]
Rather the burying this in one of the 3 other conversations that have
been taking place - I'll respond at the top level. I have been following
the conversation, but not at any great depth... Going to leave the
should we have "managed='detach'" conversation alone (at least for
now).
Not that I want to necessarily dip my toes into these shark infested
waters; however, one thing keeps gnawing at my fingers from hanging onto
the don't get involved in this one ledge ;-)... That one thing is the
problem to me is less libvirt's ability to manage whether the devices
are or aren't detached from the host, but rather that lower layers (e.g.
kernel) aren't happy over the frequency of such requests.
That is definitely one of the reasons we need to do it less. My
conclusion from all this discussion has been
1) lower layers are woefully lacking in ways available to specify what
driver to use for a specific device at boot time, making it essentially
impossible, for example, to have devices that will only be used for vfio
assignment to a guest be bound to the vfio-pci driver straight from boot
time. Instead they must be bound to pci-stub at boot, then later
re-bound to vfio-pci by "something, who knows, shuddup and do it
yerself!". And even that only works if you want *all* devices with the
same vendor/device id to be treated in the exact same manner. (and
blacklisting a particular driver is just hopelessly useless.
2) (1) isn't necessarily because the people in the lower layers are
being unhelpful jerks about it, it's because the way that things are
organized in the kernel and its boot process make it really difficult to
do it.
Thrashing for
any system isn't fun, but it's a lot easier to tell someone else to stop
doing it since it hurts when you do that. Tough to find a happy medium
between force user to detach rather than let libvirt manage and letting
libvirt be the culprit causing angst for the lower layers.
So, would it work to have some intermediary handle this thrashing by
creating some sort of "blob" that will accept responsibility for
reattaching the device to the host "at some point in time" as long as no
one else has requested to use it?
Why not add an attribute (e.g. delay='n') that is only valid when
managed='yes' to the device which means, rather than immediately
reattaching this to the host when the guest is destroy, libvirt will
delay the reattach by 'n' seconds. That way someone that knows they're
going to have a device used by multiple guests that could be thrashing
heavily in the detach -> reattach -> detach -> reattach -> etc loop
would be able to make use of an optimization of sorts that just places
the device back in the inactive list (as if it were detached manually),
then starts a thread that will reawaken when a timer fires to handle the
reattach. The thread would be destroyed in the event that something
codes along and uses (e.g. places back into the active list).
This sounds dangerous for two reasons: 1) someone may expect that
they'll be able to use the device on the host if it's not in use by a
guest, and this will be true only at undeterminable (from the outside)
times. 2) Rather than causing a failure immediately when the guest
terminates (or deletes the device), the failure will occur at some later
time, which could serve to obscure the source of the failure.
It's an interesting idea from the "this would be cool to code", but I
don't really see a practical use for it.
The caveats that come quickly to mind in using this is that devices that
were being managed could be left on the inactive list if the daemon dies
or is restarted, but I think it is detectable at restart so it may not
be totally bad. Also, failure to reattach is left in some thread which
has no one to message to other than libvirtd log files. Both would have
to be noted with any description of this.
Not all devices will want this delay logic and I think it's been pointed
out that there is a "known list" of them. In the long run it allows
some control by a user to decide how much rope they'd like to have to
hang themselves.
John
Not sure if Gerd, Alex, and Martin are regular libvir-list readers, so I
did CC them just in case so that it's easier for them to respond if they
so desire since they were at part of the discussions in this thread.
--
libvir-list mailing list
libvir-list(a)redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list