Re: device compatibility interface for live migration with assigned devices
hi All,
Per our previous discussion, there are two main concerns to the previous
proposal:
(1) it's currently hard for openstack to match mdev types.
(2) complicated.
so, we further propose below changes:
(1) requiring two compatible mdevs to have the same mdev type for now.
(though kernel still exposes compatible_type attributes for future use)
(2) requiring 1:1 match for other attributes under sysfs type node for now
(those attributes are specified via compatible_<attribute name> but
with only 1 value in it.)
(3) do not match attributes under device instance node.
rather, they are regarded as part of resource claiming process.
so src and dest values are ensured to be 1:1.
A dynamic_resources attribute under sysfs <type-id> node is added to
list the attributes under device instance that mgt tools need to
ensure 1:1 from src and dest.
the "aggregator" attribute under device instance node is such one that
needs to be listed.
Those listed attributes can actually be treated as device state set by
vendor driver during live migration. but we still want to ask for them to
be set by mgt tools before live migration starts, in oder to reduce the
chance of live migration failure.
do you like those changes?
after the changes, the sysfs interface would look like blow:
|- [parent physical device]
|--- Vendor-specific-attributes [optional]
|--- [mdev_supported_types]
| |--- [<type-id>]
| | |--- create
| | |--- name
| | |--- available_instances
| | |--- device_api
| | |--- software_version
| | |--- compatible_type
| | |--- compatible_<device_api_specific_field>
| | |--- compatible_<type_specific_field>
| | |--- dynamic_resources
| | |--- description
| | |--- [devices]
- device_api : exact match between src and dest is required.
its value can be one of
"vfio-pci", "vfio-platform", "vfio-amba",
"vfio-ccw", "vfio-ap"
- software_version: version of vendor driver.
in major.minor.bugfix scheme.
dest major should be equal to src major,
dest minor should be no less than src minor.
once migration stream related code changed, vendor
drivers need to bump the version.
- compatible_type: not used by mgt tools currently.
vendor drivers can provide this attribute, but need to
know that mgt apps would ignore it.
when in future mgt tools support this attribute, it
would allow migration across different mdev types,
so that devices of older generation may be able to
migrate to newer generations.
- compatible_<device_api_specific_field>: for device api specific attributes,
e.g. compatible_subchannel_type,
dest values should be superset of arc values.
vendor drivers can specify only one value in this attribute,
in order to do exact match between src and dest.
It's ok for mgt tools to only read one value in the
attribute so that src:dest values are 1:1.
- compatible_<type_specific_field>: for mdev type specific attributes,
e.g. compatible_pci_ids, compatible_chpid_type
dest values should be superset of arc values.
vendor drivers can specify only one value in the attribute
in order to do exact match between src and dest.
It's ok for mgt tools to only read one value in the
attribute so that src:dest values are 1:1.
- dynamic_resources: though defined statically under <type-id>,
this attribute lists attributes under device instance that
need to be set as part of claiming dest resources.
e.g. $cat dynamic_resources: aggregator, fps,...
then after dest device is created, values of its device
attributes need to be set to that of src device attributes.
Failure in syncing src device values to dest device
values is treated the same as failing to claiming
dest resources.
attributes under device instance that are not listed
in this attribute would not be part of resource checking in
mgt tools.
Thanks
Yan