5:16 a.m.
Laine,
Please find my comments inline.
Regards,
Shradha Shah
On 09/11/2012 08:07 PM, Laine Stump wrote:
On 09/07/2012 12:12 PM, Shradha Shah wrote:
> This patch series adds the support for interface-type="hostdev-hybrid" and
> forward mode="hostdev-hybrid".
>
> The hostdev-hybrid mode makes migration possible along with PCI-passthrough.
> I had posted a RFC on the hostdev-hybrid methodology earlier on the libvirt
> mailing list.
>
> The RFC can be found here:
> https://www.redhat.com/archives/libvir-list/2012-February/msg00309.html
Before anything else, let me outline what I *think* happens with a
hostdev-hybrid device entry, and you can tell me how far off I am :-):
* Any hostdev-hybrid interface definition results in 2 PCI devices being
added to the guest:
a) a PCI passthrough of an SR-IOV VF (done essentially the same as
<interface type='hostdev'>
b) a virtio-net device which is connected via macvtap "bridge" mode
(? is that always the case) to the PF of the VF in (a)
Yes the virtio-net device is always connected via macvtap "bridge" mode
* Both of these devices are assigned the same MAC address.
Correct.
* Each of these occupies one PCI address on the guest, so a total of 2
PCI addresses is needed for each hostdev-hybrid "device". (The
redundancy in this statement is to be sure that I'm right, as that's an
important point :-)
Yes a total of 2 PCI addresses are needed for each hostdev-hybrid "device"
* On the guest, these two network devices with matching MAC addresses
are put together into a bond interface, with an extra driver that causes
the bond to prefer the pci-passthrough device when it is present. So,
under normal circumstances *all* traffic goes through the
pci-passthrough device.
Correct.
* At migration time, since guests with attached pci-passthrough devices
can't be migrated, the pci-passthrough device (which is found by
searching the hostdev array for items with the "ephemeral" flag set) is
detached. This reduces the bond interface on the guest to only having
the virtio-net device, so traffic now passes through that device - it's
slower, but connectivity is maintained.
Correct.
* on the destination, a new VF is found, setup with proper MAC address,
VLAN, and 802.1QbX port info. A virtio-net device attached to the PF
associated with this VF (via macvtap bridge mode) is also setup. The
qemu commandline includes an entry for both of these devices. (Question:
Is it the virtio-net device that uses the guest PCI address given in the
<interface> device info?)
Yes, The virtio-net device gets the guest PCI address given in the
<interface> device info.
The VF gets a separate guest PCI address .
(Question: actually, I guess the
pci-passthrough device won't be attached until after the guest actually
starts running on the destination host, correct?)
The Vf is not attached until the migration completes because during migration
qemu checks for PCI-passthrough devices on the source and on the
destination.
* When migration is finished, the guest is shut down on the source and
started up on the destination, leaving the new instance of the guest
temporarily with just a single (virtio-net) device in the bond.
Yes, The VF is hotplug into the guest in the qemuMigrationFinish Stage
* Finally, the pci-passthrough of the VF is attached to the guest, and
the guest's bond interface resumes preferring this device, thus
restoring full speed networking.
Is that all correct?
If so, one issue I have is that one of the devices (the
pci-passthrough?) doesn't have its guest-side PCI address visible
anywhere in the guest's XML, does it? This is problematic, because
management applications (and libvirt itself) expect to be able to scan
the list of devices to learn what PCI slots are occupied on the guest,
and where they can add new devices.
Actually the guest PCI address of the
pci-passthrough device i.e. The VF
is visible in the guest's XML when the guest is running. The VF will be plugged
into the guest only when the guest is running or when the guest is not being
migrated hence will be visible in the guest XML.
i.e At any moment if a VF is plugged into the guest, its guest address will be
visible in the guest XML and the PCI slot will be marked as being used.
When the VF is hot-unplugged the PCI slot is free and hence can be used for some
other device by management applications (libvirt).
I have other questions beyond that, but either don't understand the code
enough yet to verbalize them, or will ask them next to the associated code.
--
libvir-list mailing list
libvir-list(a)redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list