On 04/15/2016 05:15 AM, Shivaprasad bhat wrote:

Hi All,

I am exploring how to go about supporting multi-function hot-plug/unplug support from libvirt now that Qemu has enabled it.

The Qemu semantics is that, qemu queues the notification to guest until function 0 is hot- plugged and on function 0 hot-plug, all the previously added devices in the slot are notified to the guest. For hot-unplug - On x86, unplug of any device in the slot would unplug all the functions of the slot. On PPC, unplug all non-zero functions first, then unplug the function zero which triggers the unplug of all the functions in the slot.

On Libvirt, I had a brief chat with Laine Stump on IRC and we "think" the below semantics would be appropriate. 1) Change the virDomainAttachDeviceFlags() to recognize multiple devices in the XML and the application should make one call to attach all the functions for the slot at one time. 2) The libvirt qemu driver to translate this into multiple attach device commands to qemu with the final operation being for function 0.

Some explanation of why I think the attach of all functions should be a single operation in the libvirt API: mainly because it is a single operation from the POV of the guest OS, and other hypervisors may present it to libvirt in that manner as well. It would be more difficult in such a case for libvirt to gather up the various attach calls to squirt them out to the hypervisor when the final function attach is requested. It would also lead to error recovery Hell if there was a failure in the final step (doing the actual work once the info for all the functions was saved up) - previous calls would have already returned success, and now they suddenly become a failure; how would you report that, and how would the management application recover? (But of course I may be missing something stupid, so don't hesitate to protest/discredit/etc :-) Note that putting the XML for multiple functions that will be in the same slot will allow management to let libvirt decide on the proper slot as well as automatically setting the "multifunction" flag on function 0 (which should have always been done anyway. AFAIK, there isn't valid case for having multiple functions in a slot where function 0 *doesn't* have multifunction set).

3) The XML now needs to accept multiple devices and there is a need for single parent element. I am thinking if we should have all devices to be enclosed in <device></device> parent element. Note that this is only when all the devices should go to the same slot. We probably should disallow user attempting to hot-plug to different slots with this. 4) For the hot-unplug, the application to send all the devices the same way enclosed in <device></devices>

"devices" in both cases, of course :-)

and libvirt to go ahead with unplug only if all the devices are specified in the XML for the slot.

Want to know if you foresee any problem with using <device></device> semantics Or you have any different approach/suggestions. Any comments greatly appreciated.

Thanks and Regards, Shivaprasad