Re: [libvirt] [RFC PATCH] hostdev: add support for "managed='detach'"

On 03/15/2016 01:00 PM, Daniel P. Berrange wrote: >On Mon, Mar 14, 2016 at 03:41:48PM -0400, Laine Stump wrote: >>Suggested by Alex Williamson. >> >>If you plan to assign a GPU to a virtual machine, but that GPU happens >>to be the host system console, you likely want it to start out using >>the host driver (so that boot messages/etc will be displayed), then >>later have the host driver replaced with vfio-pci for assignment to >>the virtual machine. >> >>However, in at least some cases (e.g. Intel i915) once the device has >>been detached from the host driver and attached to vfio-pci, attempts >>to reattach to the host driver only lead to "grief" (ask Alex for >>details). This means that simply using "managed='yes'" in libvirt >>won't work. >> >>And if you set "managed='no'" in libvirt then either you have to >>manually run virsh nodedev-detach prior to the first start of the >>guest, or you have to have a management application intelligent enough >>to know that it should detach from the host driver, but never reattach >>to it. >> >>This patch makes it simple/automatic to deal with such a case - it >>adds a third "managed" mode for assigned PCI devices, called >>"detach". It will detach ("unbind" in driver parlance) the device from >>the host driver prior to assigning it to the guest, but when the guest >>is finished with the device, will leave it bound to vfio-pci. This >>allows re-using the device for another guest, without requiring >>initial out-of-band intervention to unbind the host driver. >You say that managed=yes causes pain upon re-attachment and that >apps should use managed=detach to avoid it, but how do management >apps know which devices are going to cause pain ? Libvirt isn't >providing any info on whether a particular device id needs to >use managed=yes vs managed=detach, and we don't want to be asking >the user to choose between modes in openstack/ovirt IMHO. I think >thats a fundamental problem with inventing a new value for managed >here. My suspicion is that in many/most cases users don't actually need for the device to be re-bound to the host driver after the guest is finished with it, because they're only going to use the device to assign to a different guest anyway. But because managed='yes' is what's supplied and is the easiest way to get it setup for assignment to a guest, that's what they use. As a matter of fact, all this extra churn of changing the driver back and forth for devices that are only actually used when they're bound to vfio-pci just wastes time, and makes it more likely that libvirt and its users will reveal and get caught up in the effects of some strange kernel driver loading/unloading bug (there was recently a bug reported like this; unfortunately the BZ record had customer info in it, so it's not publicly accessible :-( ) So beyond making this behavior available only when absolutely necessary, I think it is useful in other cases, at the user's discretion (and as I implied above, I think that if they understood the function and the tradeoffs, most people would choose to use managed='detach' rather than managed='yes')