Re: [libvirt] Regarding PCI/PCIe Multifunction hotplug support in libvirt
On 04/15/2016 05:15 AM, Shivaprasad bhat wrote:
Hi All,
I am exploring how to go about supporting multi-function
hot-plug/unplug support from libvirt now that Qemu has enabled it.
The Qemu semantics is that, qemu queues the notification to guest
until function 0 is hot-
plugged and on function 0 hot-plug, all the previously added devices
in the slot are
notified to the guest.
For hot-unplug - On x86, unplug of any device in the slot would unplug
all the functions of
the slot. On PPC, unplug all non-zero functions first, then unplug the
function zero which
triggers the unplug of all the functions in the slot.
On Libvirt, I had a brief chat with Laine Stump on IRC and we "think"
the below semantics
would be appropriate.
1) Change the virDomainAttachDeviceFlags() to recognize multiple
devices in the XML
and the application should make one call to attach all the functions
for the slot at one
time.
2) The libvirt qemu driver to translate this into multiple attach
device commands to qemu
with the final operation being for function 0.
Some explanation of why I think the attach of all functions should be a
single operation in the libvirt API: mainly because it is a single
operation from the POV of the guest OS, and other hypervisors may
present it to libvirt in that manner as well. It would be more difficult
in such a case for libvirt to gather up the various attach calls to
squirt them out to the hypervisor when the final function attach is
requested. It would also lead to error recovery Hell if there was a
failure in the final step (doing the actual work once the info for all
the functions was saved up) - previous calls would have already returned
success, and now they suddenly become a failure; how would you report
that, and how would the management application recover?
(But of course I may be missing something stupid, so don't hesitate to
protest/discredit/etc :-)
Note that putting the XML for multiple functions that will be in the
same slot will allow management to let libvirt decide on the proper slot
as well as automatically setting the "multifunction" flag on function 0
(which should have always been done anyway. AFAIK, there isn't valid
case for having multiple functions in a slot where function 0 *doesn't*
have multifunction set).
3) The XML now needs to accept multiple devices and there is a need
for single parent
element. I am thinking if we should have all devices to be enclosed in
<device></device>
parent element. Note that this is only when all the devices should go
to the same slot. We
probably should disallow user attempting to hot-plug to different
slots with this.
4) For the hot-unplug, the application to send all the devices the
same way enclosed in
<device></devices>
"devices" in both cases, of course :-)
and libvirt to go ahead with unplug only if all the devices are
specified in the XML for the slot.
Want to know if you foresee any problem with using <device></device>
semantics Or you
have any different approach/suggestions. Any comments greatly appreciated.
Thanks and Regards,
Shivaprasad