Re: [libvirt] [RFC PATCH] hostdev: add support for "managed='detach'"
On 03/17/2016 02:32 PM, Daniel P. Berrange wrote:
On Thu, Mar 17, 2016 at 12:18:49PM -0600, Alex Williamson wrote:
> On Thu, 17 Mar 2016 17:59:53 +0000
> "Daniel P. Berrange" <berrange(a)redhat.com> wrote:
>
>> I don't think it is a significant burden really. Apps which want this
>> blacklisted forever likely want to setup the modprobe blacklist anyway
>> to stop the initial bind at boot up and instead permanently reserve
>> the device. This stops the device being used at startup - eg if we
>> have a bunch of NICs to be given to guests, you don't want the host
>> OS to automatically configure them and give them IP addresses on the
>> host before we start guests. So pre-reserving devices at the host OS
>> level is really want you want todo with data center / cloud management
>> apps like oVirt / OpenStack at least. They could easily use the
>> virNodeDeviceDetach API at the time they decide to assign a device
>> to a guest though.
> modprobe blacklist assumes that all devices managed by a given driver
> are reserved for VM use. That's very often not the case. Even with
> SR-IOV VFs, several vendors use the same driver for PF and VF, so
> that's just a poor solution. For GPU assignment we often recommend
> using pci-stub.ids on the kernel commandline to pre-load the pci-stub
> driver with PCI vendor and device IDs to claim to prevent host drivers
> from attaching, but that also assumes that you want to use everything
> matching those IDs for a VM, which users will quickly find fault with.
> Additionally, using either solution assumes that the device will be
> left entirely alone otherwise, which is also not true. If I blacklist
> i915 or using pci-stub.ids to make pci-stub claim it, then efifb or
> vesafb is more than happy to make use of it, so it's actually cleaner
> to let i915 grab the device and unbind it when ready. And of course
> the issue of assuming that the device can go without drivers, which may
> make your user run a headless system. This is really not the
> simplistic issue that it may seem. Thanks,
The issues you describe point towards the need for a better blacklisting
facility at the OS level IMHO. eg a way to tell the kernel module to
not automatically bind drivers for devices with a particular device ID.
To paraphrase an old saying: this is a nail, and libvirt is my hammer
:-) My first response to Alex when he asked about this feature was to
ask if he couldn't configure it somehow outside libvirt, and his
response (similar to above) made it fairly plain that you really can't,
at least not without adding custom startup scripts. Since the barrier to
entry for the kernel is much higher than for libvirt, this seems like a
way to actually get something that works.
Combined that with the virNodeDeviceDetach() API usage still feels
like
a better solution that adding a managed=detach flag to libvirt.
Again, assuming that there is a higher level management application that
is calling libvirt, maybe so. For anyone simply using virsh to
start/stop guests, it's a bit more cumbersome (but you could claim that
isn't a typical user, so that's okay).
But the same thing could be said about managed='yes' - why have that at
all when it could be handled purely with libvirt's nodedevice API? (and
would eliminate the ambiguity of error situations where, e.g. a device
had been successfully assigned to a guest, then detached from the guest,
but libvirt failed to rebind the device to the host driver - the device
*has* been successfully detached from the guest though, so should we
report success or failure?). (Seriously, I think we should consider
being more discouraging about use of managed='yes').