Re: [libvirt] [Qemu-devel] [RFC 0/2] Attempt to implement the standby feature for assigned network devices
Quoting Sameeh Jubran (2018-10-25 13:01:10)
On Thu, Oct 25, 2018 at 5:06 PM Sameeh Jubran
<sameeh(a)daynix.com> wrote:
>
> From: Sameeh Jubran <sjubran(a)redhat.com>
>
> Hi all,
>
> Background:
>
> There has been a few attempts to implement the standby feature for vfio
> assigned devices which aims to enable the migration of such devices. This
> is another attempt.
>
> The series implements an infrastructure for hiding devices from the bus
> upon boot. What it does is the following:
>
> * In the first patch the infrastructure for hiding the device is added
> for the qbus and qdev APIs. A "hidden" boolean is added to the device
> state and it is set based on a callback to the standby device which
> registers itself for handling the assessment: "should the primary device
> be hidden?" by cross validating the ids of the devices.
>
> * In the second patch the virtio-net uses the API to hide the vfio
> device and unhides it when the feature is acked.
>
> Disclaimers:
>
> * I have only scratch tested this and from qemu side, it seems to be
> working.
> * This is an RFC so it lacks some proper error handling in few cases
> and proper resource freeing. I wanted to get some feedback first
> before it is finalized.
>
> Command line example:
>
> /home/sameeh/Builds/failover/qemu/x86_64-softmmu/qemu-system-x86_64 \
> -netdev
tap,id=hostnet0,script=world_bridge_standalone.sh,downscript=no,ifname=cc1_71 \
> -netdev
tap,vhost=on,id=hostnet1,script=world_bridge_standalone.sh,downscript=no,ifname=cc1_72,queues=4
\
> -device
virtio-net,host_mtu=1500,netdev=hostnet1,id=cc1_72,vectors=10,mq=on,primary=cc1_71 \
> -device e1000,netdev=hostnet0,id=cc1_71,standby=cc1_72 \
>
> Migration support:
>
> Pre migration or during setup phase of the migration we should send an
> unplug request to the guest to unplug the primary device. I haven't had
> the chance to implement that part yet but should do soon. Do you know
> what's the best approach to do so? I wanted to have a callback to the
> virtio-net device which tries to send an unplug request to the guest and
> if succeeds then the migration continues. It needs to handle the case where
> the migration fails and then it has to replug the primary device back.
I think that the "add_migration_state_change_notifier" API call can be used
from within the virtio-net device to achieve this, what do you think?
I think it would be good to hear from the libvirt folks (on Cc:) on this as
having QEMU unplug a device without libvirt's involvement seems like it
could cause issues. Personally I think it seems cleaner to just have QEMU
handle the 'hidden' aspects of the device and leave it to QMP/libvirt to do
the unplug beforehand. On the libvirt side I could imagine adding an option
like virsh migrate --switch-to-standby-networking or something along
that line to do it automatically (if we decide doing it automatically is
even needed on that end).
>
> The following terms are used as interchangeable:
> standby - virtio-net
> primary - vfio-device - physical device - assigned device
>
> Please share your thoughts and suggestions,
> Thanks!
>
> Sameeh Jubran (2):
> qdev/qbus: Add hidden device support
> virtio-net: Implement VIRTIO_NET_F_STANDBY feature
>
> hw/core/qdev.c | 48 +++++++++++++++++++++++++---
> hw/net/virtio-net.c | 25 +++++++++++++++
> hw/pci/pci.c | 1 +
> include/hw/pci/pci.h | 2 ++
> include/hw/qdev-core.h | 11 ++++++-
> include/hw/virtio/virtio-net.h | 5 +++
> qdev-monitor.c | 58 ++++++++++++++++++++++++++++++++--
> 7 files changed, 142 insertions(+), 8 deletions(-)
>
> --
> 2.17.0
>
--
Respectfully,
Sameeh Jubran
Linkedin
Software Engineer @ Daynix.