[libvirt] [RFC 00/15] qmp: Report supported device types on 'query-machines'
The Problem =========== Currently management software has no way to find out which device types can be plugged in a machine, unless the machine is already initialized. Even after the machine is initialized, there's no way to map existing bus types to supported device types unless management software hardcodes the mapping between bus types and device types. Example: floppy support on q35 vs i440fx ---------------------------------------- There's no way for libvirt to find out that there's no floppy controller on pc-q35-* machine-types by default. With this series, pc-i440fx-* will report "floppy" as a supported device type, but pc-q35-* will not. Example: Legacy PCI vs vs PCIe devices -------------------------------------- Some devices require a PCIe bus to be available, others work on both legacy PCI and PCIe, while others work only on a legacy PCI bus. Currently management software has no way to know which devices can be added to a given machine, unless it hardcodes machine-type names and device-types names. The Proposed Interface ====================== This series adds a new field to the output of 'query-machines': 'supported-device-types'. It will contain a list of QOM type names, that can be used to find the list of device types that can be plugged in the machine by default. The type names reported on the new field can then be used as the 'implements' argument on the 'qom-list-types' command, to find out which device types can be plugged on the machine. Example output -------------- (QEMU) query-machines { "return": [ [...] { "supported-device-types": [ "sys-bus-device" ], "cpu-max": 1, "hotpluggable-cpus": false, "name": "none" }, [...] { "supported-device-types": [ "sys-bus-device" ], "cpu-max": 1, "hotpluggable-cpus": false, "name": "xenpv" }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "i2c-slave", "pci-device", "isa-device", "ide-device" ], "name": "pc-i440fx-2.8", "alias": "pc", "is-default": true, "cpu-max": 255, "hotpluggable-cpus": true }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "isa-device", "ide-device" ], "cpu-max": 1, "hotpluggable-cpus": true, "name": "isapc" }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "i2c-slave", "pci-device", "isa-device", "ide-device" ], "cpu-max": 128, "hotpluggable-cpus": true, "name": "xenfv" }, [...] { "alias": "q35", "supported-device-types": [ "sys-bus-device", "i2c-slave", "PCIE-device", "isa-device", "ide-device" ], "cpu-max": 288, "hotpluggable-cpus": true, "name": "pc-q35-2.8" }, [...] ] } Considered alternatives ======================= Indirect mapping (machine => bus => device) ------------------------------------------- This RFC implements a mechanism to implement ax machine-type => supported-device-types mapping. An alternative solution I considered was to expose an indirect mapping: machine-type => default-bus-types followed by bus-type => supported-device-types. But exposing only the resulting supported device-types list imposes less restrictions on how the device and bus type hierarchy is implemented inside QEMU. There's still a machine-type => bus-type => device-type mapping implemented internally, but it is an implementation detail on the current version, and not part of the externally-visible interface. The Implementation ================== This add a new field to MachineClass: default_bus_types, and a new field to BusClass: supported_device_type. The two fields are used to build the list of supported device types for a given machine. On most cases, the normal QOM type hierarchy is used to define the set of supported devices for a bus. On the case of PCIe buses, a INTERFACE_PCIE_DEVICE interface name was introduced, to indicate PCIe-capable devices. This means we are duplicating information in some cases: * BusClass::supported_device_type duplicates knowlege that is already encoded in DeviceClass::bus_type. To make sure both fields agree with each other, a new device_class_set_bus_type() wrapper was added, to perform additional validation. * MachineClass::default_bus_type duplicates knowledge that is already encoded in the machine init function. To make sure the information is correct, a qmp-machine-info.py test case is added, that will validate the supported-device-types field based on the buses created by the machine. * PCIDeviceClass::is_express and INTERFACE_PCIE_DEVICE both encode the same information about a PCI device class. A future version of this series may include a class_base_post_init hook that will allow TYPE_PCI_DEVICE to validate/update is_express and the interface list to ensure both are always consistent. Test Code --------- qdev-based test code for the new field was implemented in a Python script. Some extra support was implemented in tests/Makefile.include, scripts/qemu.py and scripts/qtest.py to allow the test to be implemented. Limitations =========== TYPE_SYS_BUS_DEVICE is too generic ---------------------------------- Currently all machines have a TYPE_SYS_BUS bus, meaning all TYPE_SYS_BUS_DEVICE subclasses are reported as supported. The current solution in this series is to report TYPE_SYS_BUS_DEVICE as supported by all machines. But we could gradually add arch-specific or machine-family-specific interface names that can be used on devices that are meant to work with only a subset of TYPE_SYS_BUS_DEVICE subclasses. A future version of this series may remove TYPE_SYS_BUS_DEVICE from the supported-device-types output, and return a arch-specific or machine-family-specific interface name to restrict management software to a subset of TYPE_SYS_BUS_DEVICE subclasses. PCI vs PCIe ----------- Machines with PCIe buses will report INTERFACE_PCIE_DEVICE on supported-device-types. Machines with legacy PCI buses will report TYPE_PCI_DEVICE on supported-device-types. The problem with the current approach is that PCIe devices are TYPE_PCI_DEVICE subclasses. The allows PCI device classes to indicate they are PCIe-capable, but there's no obvious way to indicate that a device is PCIe-only. This needs to be addressed in a future version of this series. Suggestions are welcome. Incomplete bus lists on some machines ------------------------------------- With this series, not all machines classes are changed to add the full list of device types on the 'supported-device-types'. To allow the code to be updated gradually, qmp-machine-info.py has a STRICT_ARCHES variable, that will make the test code require a complete device type list only on some architectures. Out of scope: Configurable buses -------------------------------- There's no way to map machine options like "usb=on|off" to device-types or buses. I plan to propose a new interface that allows machine options to be mapped to buses/device-types later. Out of scope: Deciding where to plug devices -------------------------------------------- Once management software discovers which devices can be plugged to a machine, it still has to discover or define where devices can/should/will be plugged. This is out of the scope of this series. Out of scope: Hotplug --------------------- The proposed interface is supposed to help management software find which device types can be used when creating the VM. Device hotplug is out of the scope of this series. However, the new 'device-types' QOM property on bus objects could be used to find out which device types can be plugged on the existing buses. --- Cc: libvir-list@redhat.com Cc: Laine Stump <laine@redhat.com> Eduardo Habkost (15): qemu.py: Make logging optional qtest.py: Support QTEST_LOG environment variable qtest.py: make logging optional qtest.py: Make 'binary' parameter optional tests: Add rules to non-gtester qtest test cases qdev: Add device_type field to BusClass machine: Add MachineClass::default_buses field qmp: Add 'supported-device-types' field to 'query-machines' pci: Introduce INTERFACE_PCIE_DEVICE interface name pc: Initialize default bus lists s390x: Initialize default bus lists arm: Initialize default bus lists mips: Initialize default bus lists ppc: Initialize default bus lists qdev: Add device_class_set_bus_type() function hw/arm/aspeed.c | 2 + hw/arm/collie.c | 1 + hw/arm/cubieboard.c | 1 + hw/arm/exynos4_boards.c | 5 ++ hw/arm/gumstix.c | 7 ++ hw/arm/highbank.c | 4 ++ hw/arm/imx25_pdk.c | 1 + hw/arm/kzm.c | 1 + hw/arm/musicpal.c | 1 + hw/arm/nseries.c | 2 + hw/arm/palm.c | 1 + hw/arm/realview.c | 1 + hw/arm/spitz.c | 10 +++ hw/arm/stellaris.c | 4 ++ hw/audio/intel-hda.c | 9 ++- hw/block/fdc.c | 17 +++-- hw/block/nvme.c | 4 ++ hw/char/virtio-serial-bus.c | 3 +- hw/core/bus.c | 9 +++ hw/core/machine.c | 18 ++++- hw/core/qdev.c | 8 +++ hw/core/sysbus.c | 3 +- hw/i2c/core.c | 9 ++- hw/i386/pc_piix.c | 13 ++++ hw/i386/pc_q35.c | 4 ++ hw/ide/qdev.c | 3 +- hw/input/adb.c | 9 ++- hw/ipack/ipack.c | 9 ++- hw/isa/isa-bus.c | 3 +- hw/mips/mips_malta.c | 7 ++ hw/mips/mips_r4k.c | 2 + hw/misc/auxbus.c | 3 +- hw/net/e1000e.c | 4 ++ hw/net/vmxnet3.c | 4 ++ hw/pci-bridge/ioh3420.c | 4 ++ hw/pci-bridge/xio3130_downstream.c | 4 ++ hw/pci/pci.c | 16 ++++- hw/ppc/e500plat.c | 3 + hw/ppc/mac_newworld.c | 4 ++ hw/ppc/mac_oldworld.c | 3 + hw/ppc/mpc8544ds.c | 4 ++ hw/ppc/ppc440_bamboo.c | 1 + hw/ppc/prep.c | 4 ++ hw/ppc/spapr_vio.c | 3 +- hw/s390x/css-bridge.c | 2 + hw/s390x/event-facility.c | 3 +- hw/s390x/s390-pci-bus.c | 9 ++- hw/s390x/s390-virtio-ccw.c | 6 ++ hw/s390x/virtio-ccw.c | 2 +- hw/scsi/megasas.c | 7 ++ hw/scsi/scsi-bus.c | 3 +- hw/scsi/vmw_pvscsi.c | 1 + hw/sd/core.c | 7 ++ hw/sd/sd.c | 2 +- hw/ssi/ssi.c | 9 ++- hw/usb/bus.c | 3 +- hw/usb/dev-smartcard-reader.c | 9 ++- hw/usb/hcd-xhci.c | 4 ++ hw/vfio/pci.c | 4 ++ hw/virtio/virtio-bus.c | 1 + hw/virtio/virtio-pci.c | 4 ++ hw/virtio/virtio.c | 2 +- include/hw/boards.h | 5 ++ include/hw/pci/pci.h | 3 + include/hw/qdev-core.h | 4 ++ qapi-schema.json | 9 ++- scripts/qemu.py | 25 +++++-- scripts/qtest.py | 15 +++- tests/Makefile.include | 39 ++++++++++- tests/qmp-machine-info.py | 138 +++++++++++++++++++++++++++++++++++++ vl.c | 11 +++ 71 files changed, 518 insertions(+), 37 deletions(-) create mode 100755 tests/qmp-machine-info.py -- 2.7.4
The new field will return a list of device type names that are compatible with the default bus configuration of the machine-type. The list can be easily built using the MachineClass::default_bus_types and BusClass::supported_device_type fields. The returned types can be used as the 'implements' parameter to 'qom-list-types', to find the list of devices that can be plugged on the machine. Note that some machine options may enable or disable some bus types and affect the set of compatible device types. Introspection of those options is out of the scope of this patch. Includes a qtest test case that will validate the returned by actually running each machine-type and checking the list of available buses. Cc: libvir-list@redhat.com Cc: Laine Stump <laine@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com> --- qapi-schema.json | 9 ++- tests/Makefile.include | 2 + tests/qmp-machine-info.py | 138 ++++++++++++++++++++++++++++++++++++++++++++++ vl.c | 11 ++++ 4 files changed, 159 insertions(+), 1 deletion(-) create mode 100755 tests/qmp-machine-info.py diff --git a/qapi-schema.json b/qapi-schema.json index f3e9bfc..6db397d 100644 --- a/qapi-schema.json +++ b/qapi-schema.json @@ -3178,12 +3178,19 @@ # # @hotpluggable-cpus: cpu hotplug via -device is supported (since 2.7.0) # +# @supported-device-types: List of QOM type names of devices that +# can be plugged on the machine by default. +# Note that some machine options may enable +# or disable some bus types and affect the +# set of compatible device types. (since 2.9) +# # Since: 1.2.0 ## { 'struct': 'MachineInfo', 'data': { 'name': 'str', '*alias': 'str', '*is-default': 'bool', 'cpu-max': 'int', - 'hotpluggable-cpus': 'bool'} } + 'hotpluggable-cpus': 'bool', + 'supported-device-types': [ 'str' ] } } ## # @query-machines: diff --git a/tests/Makefile.include b/tests/Makefile.include index eb1031b..7016737 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -315,6 +315,8 @@ check-qtest-s390x-y = tests/boot-serial-test$(EXESUF) check-qtest-generic-y += tests/qom-test$(EXESUF) +check-simpleqtest-generic-y += $(SRC_PATH)/tests/qmp-machine-info.py + qapi-schema += alternate-any.json qapi-schema += alternate-array.json qapi-schema += alternate-base.json diff --git a/tests/qmp-machine-info.py b/tests/qmp-machine-info.py new file mode 100755 index 0000000..5258434 --- /dev/null +++ b/tests/qmp-machine-info.py @@ -0,0 +1,138 @@ +#!/usr/bin/env python +import sys, os +sys.path.append(os.path.join(os.path.dirname(__file__), '..', 'scripts')) +import qtest +import unittest +import logging +import argparse + +logger = logging.getLogger('qemu.tests.machineinfo') + +# machines that we can't easily test because they can't run on all hosts: +BLACKLIST = set(['xenpv', 'xenfv']) + +# architectures where machines are expected to report all available buses: +STRICT_ARCHES = set() + +class QueryMachinesTest(unittest.TestCase): + def walkQOMTree(self, vm, path): + """Walk QOM tree recusrively, starting at path""" + children = vm.qmp('qom-list', path=path)['return'] + for c in children: + logging.debug('walking %s. child: %s', path, c) + if not c['type'].startswith('child<'): + continue + + cp = '%s/%s' % (path, c['name']) + yield cp + + for gc in self.walkQOMTree(vm, cp): + yield gc + + def findAllBuses(self, vm): + """Find all bus objects in the QOM tree""" + r = vm.qmp('qom-list-types', implements='bus') + bus_types = set([b['name'] for b in r['return']]) + for cp in self.walkQOMTree(vm, '/machine'): + t = vm.qmp('qom-get', path=cp, property='type')['return'] + if t in bus_types: + dt = vm.qmp('qom-get', path=cp, property='device-type')['return'] + yield dict(path=cp, type=t, device_type=dt) + + def validateSupportedDeviceTypes(self, machine): + """Validate 'supported-device-types' on 'query-machines'""" + if machine['name'] in BLACKLIST: + return + + vm = qtest.QEMUQtestMachine(args=['-S', '-machine', machine['name']], logging=False) + vm.launch() + try: + buses = list(self.findAllBuses(vm)) + bus_types = set([b['type'] for b in buses]) + device_types = set([b['device_type'] for b in buses]) + logger.debug("buses for machine %s: %s", machine['name'], + ' '.join(bus_types)) + logger.debug("device-type for machine %s: %s", machine['name'], + ' '.join(device_types)) + reported_types = set(machine['supported-device-types']) + extra_types = reported_types.difference(device_types) + missing_types = device_types.difference(reported_types) + # the machine MUST NOT report any types if the bus is not available + # by default (in other words, extra_types should be empty) + self.assertEquals(extra_types, set()) + # missing_types, on the other hand, may be empty. sometimes + # a bus is available but the machine doesn't report it yet + if missing_types: + logger.info("extra device types present on machine %s: %s", + machine['name'], ' '.join(missing_types)) + if vm.qmp('query-target')['return']['arch'] in STRICT_ARCHES: + self.fail("extra device types: %s" % (' '.join(missing_types))) + finally: + vm.shutdown() + + @classmethod + def addMachineTest(klass, method, machine): + """Dynamically add a testMachine_<name>_<machine> method to the class""" + def testMachine(self): + return method(self, machine) + machine_name = machine['name'].replace('-', '_').replace('.', '_') + method_name = 'testMachine_%s_%s' % (method.__name__, machine_name) + setattr(klass, method_name, testMachine) + + @classmethod + def discoverMachines(klass): + """Run query-machines + + This method is run before test cases are started, so we + can dynamically add test cases for each machine supported + by the binary. + """ + vm = qtest.QEMUQtestMachine(args=['-S', '-machine', 'none'], logging=False) + vm.launch() + try: + machines = vm.qmp('query-machines')['return'] + finally: + vm.shutdown() + return machines + + @classmethod + def addMachineTestCases(klass): + """Dynamically add test methods for each machine-type""" + machines = klass.discoverMachines() + for m in machines: + klass.addMachineTest(klass.validateSupportedDeviceTypes, m) + +def load_tests(loader, tests=None, pattern=None): + QueryMachinesTest.addMachineTestCases() + ts = unittest.TestSuite() + tests = loader.loadTestsFromTestCase(QueryMachinesTest) + ts.addTests(tests) + return ts + +def main(argv): + # we don't use unittest.main() because: + # 1) it doesn't configure logging level + # 2) it doesn't let us disable all output + parser = argparse.ArgumentParser() + parser.add_argument('--verbosity', help='Set verbosity', + default=1, type=int) + parser.add_argument('--quiet', '-q', help='Run tests quietly', + action='store_const', dest='verbosity', const=0) + parser.add_argument('--verbose', '-v', help='Run tests verbosely', + action='store_const', dest='verbosity', const=2) + parser.add_argument('--debug', '-d', help='Debug output', + action='store_const', dest='verbosity', const=4) + args = parser.parse_args(argv[1:]) + if args.verbosity == 0: + output = open('/dev/null', 'w') + else: + output = sys.stderr + if args.verbosity >= 4: + logging.basicConfig(level=logging.DEBUG) + elif args.verbosity >= 2: + logging.basicConfig(level=logging.INFO) + tests = load_tests(unittest.TestLoader(), None, None) + unittest.TextTestRunner(stream=output, verbosity=args.verbosity).run(tests) + +if __name__ == '__main__': + main(sys.argv) diff --git a/vl.c b/vl.c index d77dd86..af862db 100644 --- a/vl.c +++ b/vl.c @@ -1541,6 +1541,8 @@ MachineInfoList *qmp_query_machines(Error **errp) MachineClass *mc = el->data; MachineInfoList *entry; MachineInfo *info; + GList *bus; + strList **next_dev; info = g_malloc0(sizeof(*info)); if (mc->is_default) { @@ -1557,6 +1559,15 @@ MachineInfoList *qmp_query_machines(Error **errp) info->cpu_max = !mc->max_cpus ? 1 : mc->max_cpus; info->hotpluggable_cpus = !!mc->query_hotpluggable_cpus; + next_dev = &info->supported_device_types; + for (bus = mc->default_buses; bus; bus = bus->next) { + BusClass *bc = BUS_CLASS(object_class_by_name(bus->data)); + strList *new = g_new0(strList, 1); + new->value = g_strdup(bc->device_type); + *next_dev = new; + next_dev = &new->next; + } + entry = g_malloc0(sizeof(*entry)); entry->value = info; entry->next = mach_list; -- 2.7.4
Considered alternatives =======================
Indirect mapping (machine => bus => device) -------------------------------------------
This RFC implements a mechanism to implement ax machine-type => supported-device-types mapping. An alternative solution I considered was to expose an indirect mapping: machine-type => default-bus-types followed by bus-type => supported-device-types.
But exposing only the resulting supported device-types list imposes less restrictions on how the device and bus type hierarchy is implemented inside QEMU. There's still a machine-type => bus-type => device-type mapping implemented internally, but it is an implementation detail on the current version, and not part of the externally-visible interface.
The Implementation ==================
This add a new field to MachineClass: default_bus_types, and a new field to BusClass: supported_device_type.
Is it possible to modify a machine (setting some properties e.g. on the command line), that suddenly more devices are supported? Something like enabling an additional bus? (I assume so, because it is called "default bus types" :) ) If so, the indirect mapping could be of more benefit in the long run. Thinking about my machine at home: I just care about the available buses. If my machine doesn't have USB, but PCI, I can buy a USB PCI card and make it support USB. Then I can happily plug in USB devices. So the "default" state might at least no longer be sufficient when wanting to plug in a USB fan on a hot summer day ;) . But, with the indirect mapping, I guess we would need yet another qmp command ... -- David
On Tue, Nov 22, 2016 at 09:18:21AM +0100, David Hildenbrand wrote:
Considered alternatives =======================
Indirect mapping (machine => bus => device) -------------------------------------------
This RFC implements a mechanism to implement ax machine-type => supported-device-types mapping. An alternative solution I considered was to expose an indirect mapping: machine-type => default-bus-types followed by bus-type => supported-device-types.
But exposing only the resulting supported device-types list imposes less restrictions on how the device and bus type hierarchy is implemented inside QEMU. There's still a machine-type => bus-type => device-type mapping implemented internally, but it is an implementation detail on the current version, and not part of the externally-visible interface.
The Implementation ==================
This add a new field to MachineClass: default_bus_types, and a new field to BusClass: supported_device_type.
Is it possible to modify a machine (setting some properties e.g. on the command line), that suddenly more devices are supported? Something like enabling an additional bus? (I assume so, because it is called "default bus types" :) )
Yes. See the "Out of scope: Configurable buses" section on the cover letter.
If so, the indirect mapping could be of more benefit in the long run.
Yes, a new mapping may be necessary to find out which options (e.g. "usb=on") or device-types (e.g. "-device isa-fdc", "-device piix3-usb-uhci") will add new buses and make new device-types supported by the machine. But this will probably require new QMP commands, and I still want to keep the default case simple.
Thinking about my machine at home:
I just care about the available buses. If my machine doesn't have USB, but PCI, I can buy a USB PCI card and make it support USB. Then I can happily plug in USB devices. So the "default" state might at least no longer be sufficient when wanting to plug in a USB fan on a hot summer day ;) .
But, with the indirect mapping, I guess we would need yet another qmp command ...
--
David
-- Eduardo
(CCing people from the spapr PCI-express thread) On Mon, Nov 21, 2016 at 11:11:58PM -0200, Eduardo Habkost wrote:
The Problem ===========
Currently management software has no way to find out which device types can be plugged in a machine, unless the machine is already initialized.
Even after the machine is initialized, there's no way to map existing bus types to supported device types unless management software hardcodes the mapping between bus types and device types.
Example: floppy support on q35 vs i440fx ----------------------------------------
There's no way for libvirt to find out that there's no floppy controller on pc-q35-* machine-types by default.
With this series, pc-i440fx-* will report "floppy" as a supported device type, but pc-q35-* will not.
Example: Legacy PCI vs vs PCIe devices --------------------------------------
Some devices require a PCIe bus to be available, others work on both legacy PCI and PCIe, while others work only on a legacy PCI bus.
Currently management software has no way to know which devices can be added to a given machine, unless it hardcodes machine-type names and device-types names.
Another example: spapr and PCIe root bus ----------------------- See the thread at: Subject: [RFC PATCH qemu] spapr_pci: Create PCI-express root bus by default If we mak new spapr machine-type versions create a PCIe root bus, management software will need a way to find out: 1) The type of the default bus for the machine type; 2) The ID of the default bus for the machine type. Otherwise, management software will have to hardcode it based on machine-type version. The proposed interface should solve this problem. There are other comment below, at "Limitations" section:
The Proposed Interface ======================
This series adds a new field to the output of 'query-machines': 'supported-device-types'. It will contain a list of QOM type names, that can be used to find the list of device types that can be plugged in the machine by default. The type names reported on the new field can then be used as the 'implements' argument on the 'qom-list-types' command, to find out which device types can be plugged on the machine.
Example output --------------
(QEMU) query-machines { "return": [ [...] { "supported-device-types": [ "sys-bus-device" ], "cpu-max": 1, "hotpluggable-cpus": false, "name": "none" }, [...] { "supported-device-types": [ "sys-bus-device" ], "cpu-max": 1, "hotpluggable-cpus": false, "name": "xenpv" }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "i2c-slave", "pci-device", "isa-device", "ide-device" ], "name": "pc-i440fx-2.8", "alias": "pc", "is-default": true, "cpu-max": 255, "hotpluggable-cpus": true }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "isa-device", "ide-device" ], "cpu-max": 1, "hotpluggable-cpus": true, "name": "isapc" }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "i2c-slave", "pci-device", "isa-device", "ide-device" ], "cpu-max": 128, "hotpluggable-cpus": true, "name": "xenfv" }, [...] { "alias": "q35", "supported-device-types": [ "sys-bus-device", "i2c-slave", "PCIE-device", "isa-device", "ide-device" ], "cpu-max": 288, "hotpluggable-cpus": true, "name": "pc-q35-2.8" }, [...] ] }
Considered alternatives =======================
Indirect mapping (machine => bus => device) -------------------------------------------
This RFC implements a mechanism to implement ax machine-type => supported-device-types mapping. An alternative solution I considered was to expose an indirect mapping: machine-type => default-bus-types followed by bus-type => supported-device-types.
But exposing only the resulting supported device-types list imposes less restrictions on how the device and bus type hierarchy is implemented inside QEMU. There's still a machine-type => bus-type => device-type mapping implemented internally, but it is an implementation detail on the current version, and not part of the externally-visible interface.
The Implementation ==================
This add a new field to MachineClass: default_bus_types, and a new field to BusClass: supported_device_type.
The two fields are used to build the list of supported device types for a given machine. On most cases, the normal QOM type hierarchy is used to define the set of supported devices for a bus. On the case of PCIe buses, a INTERFACE_PCIE_DEVICE interface name was introduced, to indicate PCIe-capable devices.
This means we are duplicating information in some cases:
* BusClass::supported_device_type duplicates knowlege that is already encoded in DeviceClass::bus_type.
To make sure both fields agree with each other, a new device_class_set_bus_type() wrapper was added, to perform additional validation.
* MachineClass::default_bus_type duplicates knowledge that is already encoded in the machine init function.
To make sure the information is correct, a qmp-machine-info.py test case is added, that will validate the supported-device-types field based on the buses created by the machine.
* PCIDeviceClass::is_express and INTERFACE_PCIE_DEVICE both encode the same information about a PCI device class.
A future version of this series may include a class_base_post_init hook that will allow TYPE_PCI_DEVICE to validate/update is_express and the interface list to ensure both are always consistent.
Test Code ---------
qdev-based test code for the new field was implemented in a Python script. Some extra support was implemented in tests/Makefile.include, scripts/qemu.py and scripts/qtest.py to allow the test to be implemented.
Limitations ===========
"default defaults" vs "-nodefault defaults" ------------------------------------------- Two bad news: 1) We need to differentiate buses created by the machine with "-nodefaults" and buses that are created only without "-nodefaults". libvirt use -nodefaults when starting QEMU, so knowing which buses are available when using -nodefaults is more interesting for them. Other software, on the other hand, might be interested in the results without -nodefaults. We need to be able model both cases in the new interface. Suggestions are welcome. 2) A lot of machine-types won't start if using "-nodefaults -machine <machine>" without any extra devices or drives. Lots of machines require some drives or devices to be created (especially ARM machines that require a SD drive to be available). Some machines will make QEMU exit, some of them simply segfault. I am looking for ways to work around it so we can still validate -nodefaults-based info on the test code. Out of scope (in this version): bus IDs --------------------------------------- This series only returns info about supported device-types, but I plan to also report the bus IDs on a new version of this series. Knowing the bus IDs for a machine is also important for management software. (See spapr + PCIe root bus example above.) Additional comment about PCIe below:
TYPE_SYS_BUS_DEVICE is too generic ----------------------------------
Currently all machines have a TYPE_SYS_BUS bus, meaning all TYPE_SYS_BUS_DEVICE subclasses are reported as supported.
The current solution in this series is to report TYPE_SYS_BUS_DEVICE as supported by all machines. But we could gradually add arch-specific or machine-family-specific interface names that can be used on devices that are meant to work with only a subset of TYPE_SYS_BUS_DEVICE subclasses.
A future version of this series may remove TYPE_SYS_BUS_DEVICE from the supported-device-types output, and return a arch-specific or machine-family-specific interface name to restrict management software to a subset of TYPE_SYS_BUS_DEVICE subclasses.
PCI vs PCIe -----------
Machines with PCIe buses will report INTERFACE_PCIE_DEVICE on supported-device-types.
Machines with legacy PCI buses will report TYPE_PCI_DEVICE on supported-device-types.
The problem with the current approach is that PCIe devices are TYPE_PCI_DEVICE subclasses. The allows PCI device classes to indicate they are PCIe-capable, but there's no obvious way to indicate that a device is PCIe-only. This needs to be addressed in a future version of this series.
Suggestions are welcome.
I found out that this is a bit more complicated: not all TYPE_PCIE_BUS instances are created equal. Some of them will accept only PCIe devices, some of them can accept legacy PCI devices. The QMP interface I'm proposing shouldn't be affected by this, but the implementation needs to be changed, as it is currently based on an indirect machine-type => bus-types => device-types mapping, and a simple bus-type => device-type mapping won't work for PCIe. Probably I will replace MachineClass::default_buses with a MachineClass::default_device_types field. The 'device-type' QOM property behavior will probably change too, so each PCIe bus can report an appropriate device type as supported.
Incomplete bus lists on some machines -------------------------------------
With this series, not all machines classes are changed to add the full list of device types on the 'supported-device-types'. To allow the code to be updated gradually, qmp-machine-info.py has a STRICT_ARCHES variable, that will make the test code require a complete device type list only on some architectures.
Out of scope: Configurable buses --------------------------------
There's no way to map machine options like "usb=on|off" to device-types or buses. I plan to propose a new interface that allows machine options to be mapped to buses/device-types later.
Out of scope: Deciding where to plug devices --------------------------------------------
Once management software discovers which devices can be plugged to a machine, it still has to discover or define where devices can/should/will be plugged. This is out of the scope of this series.
Out of scope: Hotplug ---------------------
The proposed interface is supposed to help management software find which device types can be used when creating the VM. Device hotplug is out of the scope of this series. However, the new 'device-types' QOM property on bus objects could be used to find out which device types can be plugged on the existing buses.
--- Cc: libvir-list@redhat.com Cc: Laine Stump <laine@redhat.com>
Eduardo Habkost (15): qemu.py: Make logging optional qtest.py: Support QTEST_LOG environment variable qtest.py: make logging optional qtest.py: Make 'binary' parameter optional tests: Add rules to non-gtester qtest test cases qdev: Add device_type field to BusClass machine: Add MachineClass::default_buses field qmp: Add 'supported-device-types' field to 'query-machines' pci: Introduce INTERFACE_PCIE_DEVICE interface name pc: Initialize default bus lists s390x: Initialize default bus lists arm: Initialize default bus lists mips: Initialize default bus lists ppc: Initialize default bus lists qdev: Add device_class_set_bus_type() function
hw/arm/aspeed.c | 2 + hw/arm/collie.c | 1 + hw/arm/cubieboard.c | 1 + hw/arm/exynos4_boards.c | 5 ++ hw/arm/gumstix.c | 7 ++ hw/arm/highbank.c | 4 ++ hw/arm/imx25_pdk.c | 1 + hw/arm/kzm.c | 1 + hw/arm/musicpal.c | 1 + hw/arm/nseries.c | 2 + hw/arm/palm.c | 1 + hw/arm/realview.c | 1 + hw/arm/spitz.c | 10 +++ hw/arm/stellaris.c | 4 ++ hw/audio/intel-hda.c | 9 ++- hw/block/fdc.c | 17 +++-- hw/block/nvme.c | 4 ++ hw/char/virtio-serial-bus.c | 3 +- hw/core/bus.c | 9 +++ hw/core/machine.c | 18 ++++- hw/core/qdev.c | 8 +++ hw/core/sysbus.c | 3 +- hw/i2c/core.c | 9 ++- hw/i386/pc_piix.c | 13 ++++ hw/i386/pc_q35.c | 4 ++ hw/ide/qdev.c | 3 +- hw/input/adb.c | 9 ++- hw/ipack/ipack.c | 9 ++- hw/isa/isa-bus.c | 3 +- hw/mips/mips_malta.c | 7 ++ hw/mips/mips_r4k.c | 2 + hw/misc/auxbus.c | 3 +- hw/net/e1000e.c | 4 ++ hw/net/vmxnet3.c | 4 ++ hw/pci-bridge/ioh3420.c | 4 ++ hw/pci-bridge/xio3130_downstream.c | 4 ++ hw/pci/pci.c | 16 ++++- hw/ppc/e500plat.c | 3 + hw/ppc/mac_newworld.c | 4 ++ hw/ppc/mac_oldworld.c | 3 + hw/ppc/mpc8544ds.c | 4 ++ hw/ppc/ppc440_bamboo.c | 1 + hw/ppc/prep.c | 4 ++ hw/ppc/spapr_vio.c | 3 +- hw/s390x/css-bridge.c | 2 + hw/s390x/event-facility.c | 3 +- hw/s390x/s390-pci-bus.c | 9 ++- hw/s390x/s390-virtio-ccw.c | 6 ++ hw/s390x/virtio-ccw.c | 2 +- hw/scsi/megasas.c | 7 ++ hw/scsi/scsi-bus.c | 3 +- hw/scsi/vmw_pvscsi.c | 1 + hw/sd/core.c | 7 ++ hw/sd/sd.c | 2 +- hw/ssi/ssi.c | 9 ++- hw/usb/bus.c | 3 +- hw/usb/dev-smartcard-reader.c | 9 ++- hw/usb/hcd-xhci.c | 4 ++ hw/vfio/pci.c | 4 ++ hw/virtio/virtio-bus.c | 1 + hw/virtio/virtio-pci.c | 4 ++ hw/virtio/virtio.c | 2 +- include/hw/boards.h | 5 ++ include/hw/pci/pci.h | 3 + include/hw/qdev-core.h | 4 ++ qapi-schema.json | 9 ++- scripts/qemu.py | 25 +++++-- scripts/qtest.py | 15 +++- tests/Makefile.include | 39 ++++++++++- tests/qmp-machine-info.py | 138 +++++++++++++++++++++++++++++++++++++ vl.c | 11 +++ 71 files changed, 518 insertions(+), 37 deletions(-) create mode 100755 tests/qmp-machine-info.py
-- 2.7.4
-- Eduardo
(CCing the maintainers of the machines that crash when using -nodefaults) On Tue, Nov 22, 2016 at 08:34:50PM -0200, Eduardo Habkost wrote: [...]
"default defaults" vs "-nodefault defaults" -------------------------------------------
Two bad news:
1) We need to differentiate buses created by the machine with "-nodefaults" and buses that are created only without "-nodefaults".
libvirt use -nodefaults when starting QEMU, so knowing which buses are available when using -nodefaults is more interesting for them.
Other software, on the other hand, might be interested in the results without -nodefaults.
We need to be able model both cases in the new interface. Suggestions are welcome.
The good news is that the list is short. The only[1] machines where the list of buses seem to change when using -nodefaults are: * mpc8544ds * ppce500 * mpc8544ds * ppce500 * s390-ccw-virtio-* On all cases above, the only difference is that a virtio bus is available if not using -nodefaults. Considering that the list is short, I plan to rename 'supported-device-types' to 'always-available-buses', and document that it will include only the buses that are not disabled by -nodefaults. [1] I mean, the only ones from the set that don't crash with -nodefaults. The ones below could not be tested:
2) A lot of machine-types won't start if using "-nodefaults -machine <machine>" without any extra devices or drives.
Lots of machines require some drives or devices to be created (especially ARM machines that require a SD drive to be available).
Some machines will make QEMU exit, some of them simply segfault. I am looking for ways to work around it so we can still validate -nodefaults-based info on the test code.
The following machines won't work with -nodefaults: These make QEMU segfault: * cubieboard * petalogix-ml605 * or32-sim * virtex-ml507 * Niagara These exit with a "missing SecureDigital device" error: * akita * borzoi * cheetah * connex * mainstone * n800 * n810 * spitz * sx1 * sx1-v1 * terrier * tosa * verdex * z2 -- Eduardo
On Wed, Nov 23, 2016 at 03:10:47PM -0200, Eduardo Habkost wrote:
(CCing the maintainers of the machines that crash when using -nodefaults)
On Tue, Nov 22, 2016 at 08:34:50PM -0200, Eduardo Habkost wrote: [...]
"default defaults" vs "-nodefault defaults" -------------------------------------------
Two bad news:
1) We need to differentiate buses created by the machine with "-nodefaults" and buses that are created only without "-nodefaults".
libvirt use -nodefaults when starting QEMU, so knowing which buses are available when using -nodefaults is more interesting for them.
Other software, on the other hand, might be interested in the results without -nodefaults.
We need to be able model both cases in the new interface. Suggestions are welcome.
The good news is that the list is short. The only[1] machines where the list of buses seem to change when using -nodefaults are:
* mpc8544ds * ppce500 * mpc8544ds * ppce500 * s390-ccw-virtio-*
On all cases above, the only difference is that a virtio bus is available if not using -nodefaults.
Hrm.. that's odd. Well, it makes sense for the s390 which has special virtio arrangements. However, the others are all embedded ppc machines, whose virtio should be bog-standard virtio-pci. I'm wondering if the addition of the virtio "bus" is a side-effect of the NIC or storage device created without -nodefaults being virtio.
Considering that the list is short, I plan to rename 'supported-device-types' to 'always-available-buses', and document that it will include only the buses that are not disabled by -nodefaults.
[1] I mean, the only ones from the set that don't crash with -nodefaults. The ones below could not be tested:
2) A lot of machine-types won't start if using "-nodefaults -machine <machine>" without any extra devices or drives.
Lots of machines require some drives or devices to be created (especially ARM machines that require a SD drive to be available).
Some machines will make QEMU exit, some of them simply segfault. I am looking for ways to work around it so we can still validate -nodefaults-based info on the test code.
The following machines won't work with -nodefaults:
These make QEMU segfault: * cubieboard * petalogix-ml605 * or32-sim * virtex-ml507 * Niagara
These exit with a "missing SecureDigital device" error: * akita * borzoi * cheetah * connex * mainstone * n800 * n810 * spitz * sx1 * sx1-v1 * terrier * tosa * verdex * z2
-- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
On Thu, 24 Nov 2016 12:51:19 +1100 David Gibson <david@gibson.dropbear.id.au> wrote:
On Wed, Nov 23, 2016 at 03:10:47PM -0200, Eduardo Habkost wrote:
(CCing the maintainers of the machines that crash when using -nodefaults)
On Tue, Nov 22, 2016 at 08:34:50PM -0200, Eduardo Habkost wrote: [...]
"default defaults" vs "-nodefault defaults" -------------------------------------------
Two bad news:
1) We need to differentiate buses created by the machine with "-nodefaults" and buses that are created only without "-nodefaults".
libvirt use -nodefaults when starting QEMU, so knowing which buses are available when using -nodefaults is more interesting for them.
Other software, on the other hand, might be interested in the results without -nodefaults.
We need to be able model both cases in the new interface. Suggestions are welcome.
The good news is that the list is short. The only[1] machines where the list of buses seem to change when using -nodefaults are:
* mpc8544ds * ppce500 * mpc8544ds * ppce500 * s390-ccw-virtio-*
On all cases above, the only difference is that a virtio bus is available if not using -nodefaults.
Hrm.. that's odd. Well, it makes sense for the s390 which has special virtio arrangements.
I don't think it makes much sense for s390 either... is this a 'virtio' bus or a 'virtio-{pci,ccw}' bus? The transport bus should be present with -nodefaults; the virtio bus is basically a glue bus for virtio devices...
However, the others are all embedded ppc machines, whose virtio should be bog-standard virtio-pci. I'm wondering if the addition of the virtio "bus" is a side-effect of the NIC or storage device created without -nodefaults being virtio.
I'd suspect the default NICs (which are virtio at least in the s390 case).
On Thu, Nov 24, 2016 at 05:30:19PM +0100, Cornelia Huck wrote:
On Thu, 24 Nov 2016 12:51:19 +1100 David Gibson <david@gibson.dropbear.id.au> wrote:
On Wed, Nov 23, 2016 at 03:10:47PM -0200, Eduardo Habkost wrote:
(CCing the maintainers of the machines that crash when using -nodefaults)
On Tue, Nov 22, 2016 at 08:34:50PM -0200, Eduardo Habkost wrote: [...]
"default defaults" vs "-nodefault defaults" -------------------------------------------
Two bad news:
1) We need to differentiate buses created by the machine with "-nodefaults" and buses that are created only without "-nodefaults".
libvirt use -nodefaults when starting QEMU, so knowing which buses are available when using -nodefaults is more interesting for them.
Other software, on the other hand, might be interested in the results without -nodefaults.
We need to be able model both cases in the new interface. Suggestions are welcome.
The good news is that the list is short. The only[1] machines where the list of buses seem to change when using -nodefaults are:
* mpc8544ds * ppce500 * mpc8544ds * ppce500 * s390-ccw-virtio-*
On all cases above, the only difference is that a virtio bus is available if not using -nodefaults.
Hrm.. that's odd. Well, it makes sense for the s390 which has special virtio arrangements.
I don't think it makes much sense for s390 either... is this a 'virtio' bus or a 'virtio-{pci,ccw}' bus? The transport bus should be present with -nodefaults; the virtio bus is basically a glue bus for virtio devices...
I mean no device of type "virtio-ccw-bus" (which is a subtype of "virtio-bus") is present on the device tree. Is the TYPE_VIRTIO_BUS bus supposed to be user-visible, or is it just internal? -- Eduardo
Eduardo Habkost <ehabkost@redhat.com> writes:
(CCing the maintainers of the machines that crash when using -nodefaults)
On Tue, Nov 22, 2016 at 08:34:50PM -0200, Eduardo Habkost wrote: [...]
"default defaults" vs "-nodefault defaults" -------------------------------------------
Two bad news:
1) We need to differentiate buses created by the machine with "-nodefaults" and buses that are created only without "-nodefaults".
libvirt use -nodefaults when starting QEMU, so knowing which buses are available when using -nodefaults is more interesting for them.
Yes.
Other software, on the other hand, might be interested in the results without -nodefaults.
Maybe. Related: other machine options, such as usb=on.
We need to be able model both cases in the new interface. Suggestions are welcome.
The good news is that the list is short. The only[1] machines where the list of buses seem to change when using -nodefaults are:
* mpc8544ds * ppce500 * mpc8544ds * ppce500 * s390-ccw-virtio-*
On all cases above, the only difference is that a virtio bus is available if not using -nodefaults.
Considering that the list is short, I plan to rename 'supported-device-types' to 'always-available-buses', and document that it will include only the buses that are not disabled by -nodefaults.
[1] I mean, the only ones from the set that don't crash with -nodefaults. The ones below could not be tested:
2) A lot of machine-types won't start if using "-nodefaults -machine <machine>" without any extra devices or drives.
Lots of machines require some drives or devices to be created (especially ARM machines that require a SD drive to be available).
Some machines will make QEMU exit, some of them simply segfault. I am looking for ways to work around it so we can still validate -nodefaults-based info on the test code.
The following machines won't work with -nodefaults:
These make QEMU segfault: * cubieboard * petalogix-ml605 * or32-sim * virtex-ml507 * Niagara
Bugs.
These exit with a "missing SecureDigital device" error: * akita * borzoi * cheetah * connex * mainstone * n800 * n810 * spitz * sx1 * sx1-v1 * terrier * tosa * verdex * z2
Bugs only if there is no other way to provide the SD device. I believe some variation if -drive if=sd,... should do fine.
On 11/22/2016 03:11 AM, Eduardo Habkost wrote:
The Problem ===========
Currently management software has no way to find out which device types can be plugged in a machine, unless the machine is already initialized.
Hi Eduardo, Thank you for this interesting series. I think this is a problem worth addressing.
Even after the machine is initialized, there's no way to map existing bus types to supported device types unless management software hardcodes the mapping between bus types and device types.
Here I am a little lost. We are going for machine => supported devices or bus-type => supported devices?
Example: floppy support on q35 vs i440fx ----------------------------------------
There's no way for libvirt to find out that there's no floppy controller on pc-q35-* machine-types by default.
Again "by default". So do we want to query the init state of a machine? What devices are there? Or what devices *can be* there?
With this series, pc-i440fx-* will report "floppy" as a supported device type, but pc-q35-* will not.
Example: Legacy PCI vs vs PCIe devices --------------------------------------
Some devices require a PCIe bus to be available, others work on both legacy PCI and PCIe, while others work only on a legacy PCI bus.
Currently management software has no way to know which devices can be added to a given machine, unless it hardcodes machine-type names and device-types names.
Again it seems a double problem, machine => devices vs pci/pcie bus => devices. The bus => devices match is not related to a machine type.
The Proposed Interface ======================
This series adds a new field to the output of 'query-machines': 'supported-device-types'. It will contain a list of QOM type names, that can be used to find the list of device types that can be plugged in the machine by default.
What do you mean "by default"? Without bridges or part of the machine itself? The type names reported on
the new field can then be used as the 'implements' argument on the 'qom-list-types' command, to find out which device types can be plugged on the machine.
Example output --------------
(QEMU) query-machines { "return": [ [...] { "supported-device-types": [ "sys-bus-device"
I don't know how "sys-bus-device" can help us... :)
], "cpu-max": 1, "hotpluggable-cpus": false, "name": "none" }, [...] { "supported-device-types": [ "sys-bus-device" ], "cpu-max": 1, "hotpluggable-cpus": false, "name": "xenpv" }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "i2c-slave", "pci-device", "isa-device", "ide-device"
Is don't know is this high level classification is useful, here is an example: pvi-device is supported => then we look for all pci devices? But what if some pci devices make sense on a machine type, but not on another?
], "name": "pc-i440fx-2.8", "alias": "pc", "is-default": true, "cpu-max": 255, "hotpluggable-cpus": true }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "isa-device", "ide-device" ], "cpu-max": 1, "hotpluggable-cpus": true, "name": "isapc" }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "i2c-slave", "pci-device", "isa-device", "ide-device" ], "cpu-max": 128, "hotpluggable-cpus": true, "name": "xenfv" }, [...] { "alias": "q35", "supported-device-types": [ "sys-bus-device", "i2c-slave", "PCIE-device", "isa-device", "ide-device" ], "cpu-max": 288, "hotpluggable-cpus": true, "name": "pc-q35-2.8" }, [...] ] }
Considered alternatives =======================
Indirect mapping (machine => bus => device) -------------------------------------------
This RFC implements a mechanism to implement ax machine-type => supported-device-types mapping. An alternative solution I considered was to expose an indirect mapping: machine-type => default-bus-types followed by bus-type => supported-device-types.
As I previously stated, I don't know if it helps. bus-type can support different devices on different archs.
But exposing only the resulting supported device-types list imposes less restrictions on how the device and bus type hierarchy is implemented inside QEMU. There's still a machine-type => bus-type => device-type mapping implemented internally, but it is an implementation detail on the current version, and not part of the externally-visible interface.
Good, I personally don't like this "pass".
The Implementation ==================
This add a new field to MachineClass: default_bus_types, and a new field to BusClass: supported_device_type.
The two fields are used to build the list of supported device types for a given machine. On most cases, the normal QOM type hierarchy is used to define the set of supported devices for a bus. On the case of PCIe buses, a INTERFACE_PCIE_DEVICE interface name was introduced, to indicate PCIe-capable devices.
This means we are duplicating information in some cases:
* BusClass::supported_device_type duplicates knowlege that is already encoded in DeviceClass::bus_type.
To make sure both fields agree with each other, a new device_class_set_bus_type() wrapper was added, to perform additional validation.
* MachineClass::default_bus_type duplicates knowledge that is already encoded in the machine init function.
To make sure the information is correct, a qmp-machine-info.py test case is added, that will validate the supported-device-types field based on the buses created by the machine.
* PCIDeviceClass::is_express and INTERFACE_PCIE_DEVICE both encode the same information about a PCI device class.
A future version of this series may include a class_base_post_init hook that will allow TYPE_PCI_DEVICE to validate/update is_express and the interface list to ensure both are always consistent.
Test Code ---------
qdev-based test code for the new field was implemented in a Python script. Some extra support was implemented in tests/Makefile.include, scripts/qemu.py and scripts/qtest.py to allow the test to be implemented.
Limitations ===========
TYPE_SYS_BUS_DEVICE is too generic ----------------------------------
Currently all machines have a TYPE_SYS_BUS bus, meaning all TYPE_SYS_BUS_DEVICE subclasses are reported as supported.
Agreed, this is a problem.
The current solution in this series is to report TYPE_SYS_BUS_DEVICE as supported by all machines. But we could gradually add arch-specific or machine-family-specific interface names that can be used on devices that are meant to work with only a subset of TYPE_SYS_BUS_DEVICE subclasses.
A future version of this series may remove TYPE_SYS_BUS_DEVICE from the supported-device-types output, and return a arch-specific or machine-family-specific interface name to restrict management software to a subset of TYPE_SYS_BUS_DEVICE subclasses.
PCI vs PCIe -----------
Machines with PCIe buses will report INTERFACE_PCIE_DEVICE on supported-device-types.
Machines with legacy PCI buses will report TYPE_PCI_DEVICE on supported-device-types.
The problem with the current approach is that PCIe devices are TYPE_PCI_DEVICE subclasses. The allows PCI device classes to indicate they are PCIe-capable, but there's no obvious way to indicate that a device is PCIe-only. This needs to be addressed in a future version of this series.
Suggestions are welcome.
As we talked offline, I personally like an interface IPCIType with a field having 3 possible values {pci/pcie/hybrid} To understand how hybrid works we need some rules, like "pci on pci bus/pcie on pcie bus" "pcie on a non-root pcie bus/pcie otherwise I don't think we'll have a lot of rules, simple boolean fields for the interface should be enough. This still does not solve the problem that some devices makes sense only on a specific arch.
Incomplete bus lists on some machines -------------------------------------
With this series, not all machines classes are changed to add the full list of device types on the 'supported-device-types'. To allow the code to be updated gradually, qmp-machine-info.py has a STRICT_ARCHES variable, that will make the test code require a complete device type list only on some architectures.
Out of scope: Configurable buses --------------------------------
There's no way to map machine options like "usb=on|off" to device-types or buses. I plan to propose a new interface that allows machine options to be mapped to buses/device-types later.
Out of scope: Deciding where to plug devices --------------------------------------------
Once management software discovers which devices can be plugged to a machine, it still has to discover or define where devices can/should/will be plugged. This is out of the scope of this series.
That's a pitty :( I was hoping this series will solve this issue. But if it prepares the grounds for it is also a good step . Thanks, Marcel
Out of scope: Hotplug ---------------------
The proposed interface is supposed to help management software find which device types can be used when creating the VM. Device hotplug is out of the scope of this series. However, the new 'device-types' QOM property on bus objects could be used to find out which device types can be plugged on the existing buses.
--- Cc: libvir-list@redhat.com Cc: Laine Stump <laine@redhat.com>
Eduardo Habkost (15): qemu.py: Make logging optional qtest.py: Support QTEST_LOG environment variable qtest.py: make logging optional qtest.py: Make 'binary' parameter optional tests: Add rules to non-gtester qtest test cases qdev: Add device_type field to BusClass machine: Add MachineClass::default_buses field qmp: Add 'supported-device-types' field to 'query-machines' pci: Introduce INTERFACE_PCIE_DEVICE interface name pc: Initialize default bus lists s390x: Initialize default bus lists arm: Initialize default bus lists mips: Initialize default bus lists ppc: Initialize default bus lists qdev: Add device_class_set_bus_type() function
hw/arm/aspeed.c | 2 + hw/arm/collie.c | 1 + hw/arm/cubieboard.c | 1 + hw/arm/exynos4_boards.c | 5 ++ hw/arm/gumstix.c | 7 ++ hw/arm/highbank.c | 4 ++ hw/arm/imx25_pdk.c | 1 + hw/arm/kzm.c | 1 + hw/arm/musicpal.c | 1 + hw/arm/nseries.c | 2 + hw/arm/palm.c | 1 + hw/arm/realview.c | 1 + hw/arm/spitz.c | 10 +++ hw/arm/stellaris.c | 4 ++ hw/audio/intel-hda.c | 9 ++- hw/block/fdc.c | 17 +++-- hw/block/nvme.c | 4 ++ hw/char/virtio-serial-bus.c | 3 +- hw/core/bus.c | 9 +++ hw/core/machine.c | 18 ++++- hw/core/qdev.c | 8 +++ hw/core/sysbus.c | 3 +- hw/i2c/core.c | 9 ++- hw/i386/pc_piix.c | 13 ++++ hw/i386/pc_q35.c | 4 ++ hw/ide/qdev.c | 3 +- hw/input/adb.c | 9 ++- hw/ipack/ipack.c | 9 ++- hw/isa/isa-bus.c | 3 +- hw/mips/mips_malta.c | 7 ++ hw/mips/mips_r4k.c | 2 + hw/misc/auxbus.c | 3 +- hw/net/e1000e.c | 4 ++ hw/net/vmxnet3.c | 4 ++ hw/pci-bridge/ioh3420.c | 4 ++ hw/pci-bridge/xio3130_downstream.c | 4 ++ hw/pci/pci.c | 16 ++++- hw/ppc/e500plat.c | 3 + hw/ppc/mac_newworld.c | 4 ++ hw/ppc/mac_oldworld.c | 3 + hw/ppc/mpc8544ds.c | 4 ++ hw/ppc/ppc440_bamboo.c | 1 + hw/ppc/prep.c | 4 ++ hw/ppc/spapr_vio.c | 3 +- hw/s390x/css-bridge.c | 2 + hw/s390x/event-facility.c | 3 +- hw/s390x/s390-pci-bus.c | 9 ++- hw/s390x/s390-virtio-ccw.c | 6 ++ hw/s390x/virtio-ccw.c | 2 +- hw/scsi/megasas.c | 7 ++ hw/scsi/scsi-bus.c | 3 +- hw/scsi/vmw_pvscsi.c | 1 + hw/sd/core.c | 7 ++ hw/sd/sd.c | 2 +- hw/ssi/ssi.c | 9 ++- hw/usb/bus.c | 3 +- hw/usb/dev-smartcard-reader.c | 9 ++- hw/usb/hcd-xhci.c | 4 ++ hw/vfio/pci.c | 4 ++ hw/virtio/virtio-bus.c | 1 + hw/virtio/virtio-pci.c | 4 ++ hw/virtio/virtio.c | 2 +- include/hw/boards.h | 5 ++ include/hw/pci/pci.h | 3 + include/hw/qdev-core.h | 4 ++ qapi-schema.json | 9 ++- scripts/qemu.py | 25 +++++-- scripts/qtest.py | 15 +++- tests/Makefile.include | 39 ++++++++++- tests/qmp-machine-info.py | 138 +++++++++++++++++++++++++++++++++++++ vl.c | 11 +++ 71 files changed, 518 insertions(+), 37 deletions(-) create mode 100755 tests/qmp-machine-info.py
On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote:
On 11/22/2016 03:11 AM, Eduardo Habkost wrote:
The Problem ===========
Currently management software has no way to find out which device types can be plugged in a machine, unless the machine is already initialized.
Hi Eduardo, Thank you for this interesting series. I think this is a problem worth addressing.
Even after the machine is initialized, there's no way to map existing bus types to supported device types unless management software hardcodes the mapping between bus types and device types.
Here I am a little lost.
We are going for machine => supported devices or bus-type => supported devices?
On this series, we go for machine-type => supported-devices. A bus-type => supported-devices map wouldn't work because different PCIe bus instances might accept different types of devices (so supported-devices depend on the specific bus instance, not only on the bus-type). v2 will probably be more detailed. I plan to change it to: query-machine(machine-type) => list of BusInfo BusInfo would contain: * bus-type * bus-path * accepted-device-types (list of type/interface names)
Example: floppy support on q35 vs i440fx ----------------------------------------
There's no way for libvirt to find out that there's no floppy controller on pc-q35-* machine-types by default.
Again "by default". So do we want to query the init state of a machine? What devices are there? Or what devices *can be* there?
"by default" means what's present when using "-machine <machine>" with no extra -device arguments. We want to know what _buses_ are always there. Which in turn lets management know which _device_ types _can_ be plugged.
With this series, pc-i440fx-* will report "floppy" as a supported device type, but pc-q35-* will not.
Example: Legacy PCI vs vs PCIe devices --------------------------------------
Some devices require a PCIe bus to be available, others work on both legacy PCI and PCIe, while others work only on a legacy PCI bus.
Currently management software has no way to know which devices can be added to a given machine, unless it hardcodes machine-type names and device-types names.
Again it seems a double problem, machine => devices vs pci/pcie bus => devices. The bus => devices match is not related to a machine type.
A bus-type => device-type match would not depend on the machine-type, but it would not be useful: different bus instances can accept different device-types (and the way the bus topology is configured depend on the machine-type).
The Proposed Interface ======================
This series adds a new field to the output of 'query-machines': 'supported-device-types'. It will contain a list of QOM type names, that can be used to find the list of device types that can be plugged in the machine by default.
What do you mean "by default"? Without bridges or part of the machine itself?
I mean "when you just run -machine with no extra -device arguments".
The type names reported on
the new field can then be used as the 'implements' argument on the 'qom-list-types' command, to find out which device types can be plugged on the machine.
Example output --------------
(QEMU) query-machines { "return": [ [...] { "supported-device-types": [ "sys-bus-device"
I don't know how "sys-bus-device" can help us... :)
Yes, I added comments about it below. :)
], "cpu-max": 1, "hotpluggable-cpus": false, "name": "none" }, [...] { "supported-device-types": [ "sys-bus-device" ], "cpu-max": 1, "hotpluggable-cpus": false, "name": "xenpv" }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "i2c-slave", "pci-device", "isa-device", "ide-device"
Is don't know is this high level classification is useful, here is an example:
pvi-device is supported => then we look for all pci devices? But what if some pci devices make sense on a machine type, but not on another?
If not all pci devices are supported, then the machine must not return "pci-device" as supported. We need to define a new type/interface name that would be implemented only by the supported devices. e.g. "legacy-pci-device".
], "name": "pc-i440fx-2.8", "alias": "pc", "is-default": true, "cpu-max": 255, "hotpluggable-cpus": true }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "isa-device", "ide-device" ], "cpu-max": 1, "hotpluggable-cpus": true, "name": "isapc" }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "i2c-slave", "pci-device", "isa-device", "ide-device" ], "cpu-max": 128, "hotpluggable-cpus": true, "name": "xenfv" }, [...] { "alias": "q35", "supported-device-types": [ "sys-bus-device", "i2c-slave", "PCIE-device", "isa-device", "ide-device" ], "cpu-max": 288, "hotpluggable-cpus": true, "name": "pc-q35-2.8" }, [...] ] }
Considered alternatives =======================
Indirect mapping (machine => bus => device) -------------------------------------------
This RFC implements a mechanism to implement ax machine-type => supported-device-types mapping. An alternative solution I considered was to expose an indirect mapping: machine-type => default-bus-types followed by bus-type => supported-device-types.
As I previously stated, I don't know if it helps. bus-type can support different devices on different archs.
Yes. After learning a bit more about PCIe, I am convinced that a bus->type => device-types mapping won't help.
But exposing only the resulting supported device-types list imposes less restrictions on how the device and bus type hierarchy is implemented inside QEMU. There's still a machine-type => bus-type => device-type mapping implemented internally, but it is an implementation detail on the current version, and not part of the externally-visible interface.
Good, I personally don't like this "pass".
The Implementation ==================
This add a new field to MachineClass: default_bus_types, and a new field to BusClass: supported_device_type.
The two fields are used to build the list of supported device types for a given machine. On most cases, the normal QOM type hierarchy is used to define the set of supported devices for a bus. On the case of PCIe buses, a INTERFACE_PCIE_DEVICE interface name was introduced, to indicate PCIe-capable devices.
This means we are duplicating information in some cases:
* BusClass::supported_device_type duplicates knowlege that is already encoded in DeviceClass::bus_type.
To make sure both fields agree with each other, a new device_class_set_bus_type() wrapper was added, to perform additional validation.
* MachineClass::default_bus_type duplicates knowledge that is already encoded in the machine init function.
To make sure the information is correct, a qmp-machine-info.py test case is added, that will validate the supported-device-types field based on the buses created by the machine.
* PCIDeviceClass::is_express and INTERFACE_PCIE_DEVICE both encode the same information about a PCI device class.
A future version of this series may include a class_base_post_init hook that will allow TYPE_PCI_DEVICE to validate/update is_express and the interface list to ensure both are always consistent.
Test Code ---------
qdev-based test code for the new field was implemented in a Python script. Some extra support was implemented in tests/Makefile.include, scripts/qemu.py and scripts/qtest.py to allow the test to be implemented.
Limitations ===========
TYPE_SYS_BUS_DEVICE is too generic ----------------------------------
Currently all machines have a TYPE_SYS_BUS bus, meaning all TYPE_SYS_BUS_DEVICE subclasses are reported as supported.
Agreed, this is a problem.
The current solution in this series is to report TYPE_SYS_BUS_DEVICE as supported by all machines. But we could gradually add arch-specific or machine-family-specific interface names that can be used on devices that are meant to work with only a subset of TYPE_SYS_BUS_DEVICE subclasses.
A future version of this series may remove TYPE_SYS_BUS_DEVICE from the supported-device-types output, and return a arch-specific or machine-family-specific interface name to restrict management software to a subset of TYPE_SYS_BUS_DEVICE subclasses.
PCI vs PCIe -----------
Machines with PCIe buses will report INTERFACE_PCIE_DEVICE on supported-device-types.
Machines with legacy PCI buses will report TYPE_PCI_DEVICE on supported-device-types.
The problem with the current approach is that PCIe devices are TYPE_PCI_DEVICE subclasses. The allows PCI device classes to indicate they are PCIe-capable, but there's no obvious way to indicate that a device is PCIe-only. This needs to be addressed in a future version of this series.
Suggestions are welcome.
As we talked offline, I personally like an interface IPCIType with a field having 3 possible values {pci/pcie/hybrid}
To understand how hybrid works we need some rules, like "pci on pci bus/pcie on pcie bus" "pcie on a non-root pcie bus/pcie otherwise
I don't think we'll have a lot of rules, simple boolean fields for the interface should be enough.
What you propose makes sense, the only difference is that the boolean fields would be just interface names that can be used as the "implements" argument on qom-list-types. e.g.: * Hybrid PCI device-types would implement both "legacy-pci-device" and "pcie-device" interfaces. * PCIe-only device-types would implement only the "pcie-device" interface. * Legacy-PCI-only device-types would implement only the "legacy-pci-device" interface. Then, the bus instances would have a 'accepted-device-types' field: * A legacy PCI bus would accept only "legacy-pci-device" devices. * A PCIe-only bus would accept only "pcie-device" devices. * A PCIe bus that accepts legacy PCI devices (the root bus?) would accept both "legacy-pci-device" and "pcie-device" devices. Then, query-machines would return the list of bus instances that machine init creates, containing the bus ID, bus type, and accepted-device-types. Does that make sense?
This still does not solve the problem that some devices makes sense only on a specific arch.
Right now, we can simply compile out arch-specific devices that can never be plugged in a QEMU binary. But if we still want them compiled in for some reason, we can categorize them on a different type/interface name, and the corresponding machine-type would tell management that their buses accept only the arch-specific type/interface name.
Incomplete bus lists on some machines -------------------------------------
With this series, not all machines classes are changed to add the full list of device types on the 'supported-device-types'. To allow the code to be updated gradually, qmp-machine-info.py has a STRICT_ARCHES variable, that will make the test code require a complete device type list only on some architectures.
Out of scope: Configurable buses --------------------------------
There's no way to map machine options like "usb=on|off" to device-types or buses. I plan to propose a new interface that allows machine options to be mapped to buses/device-types later.
Out of scope: Deciding where to plug devices --------------------------------------------
Once management software discovers which devices can be plugged to a machine, it still has to discover or define where devices can/should/will be plugged. This is out of the scope of this series.
That's a pitty :( I was hoping this series will solve this issue. But if it prepares the grounds for it is also a good step .
The bus ID will be in the scope of v2.
Thanks, Marcel
Out of scope: Hotplug ---------------------
The proposed interface is supposed to help management software find which device types can be used when creating the VM. Device hotplug is out of the scope of this series. However, the new 'device-types' QOM property on bus objects could be used to find out which device types can be plugged on the existing buses.
--- Cc: libvir-list@redhat.com Cc: Laine Stump <laine@redhat.com>
Eduardo Habkost (15): qemu.py: Make logging optional qtest.py: Support QTEST_LOG environment variable qtest.py: make logging optional qtest.py: Make 'binary' parameter optional tests: Add rules to non-gtester qtest test cases qdev: Add device_type field to BusClass machine: Add MachineClass::default_buses field qmp: Add 'supported-device-types' field to 'query-machines' pci: Introduce INTERFACE_PCIE_DEVICE interface name pc: Initialize default bus lists s390x: Initialize default bus lists arm: Initialize default bus lists mips: Initialize default bus lists ppc: Initialize default bus lists qdev: Add device_class_set_bus_type() function
hw/arm/aspeed.c | 2 + hw/arm/collie.c | 1 + hw/arm/cubieboard.c | 1 + hw/arm/exynos4_boards.c | 5 ++ hw/arm/gumstix.c | 7 ++ hw/arm/highbank.c | 4 ++ hw/arm/imx25_pdk.c | 1 + hw/arm/kzm.c | 1 + hw/arm/musicpal.c | 1 + hw/arm/nseries.c | 2 + hw/arm/palm.c | 1 + hw/arm/realview.c | 1 + hw/arm/spitz.c | 10 +++ hw/arm/stellaris.c | 4 ++ hw/audio/intel-hda.c | 9 ++- hw/block/fdc.c | 17 +++-- hw/block/nvme.c | 4 ++ hw/char/virtio-serial-bus.c | 3 +- hw/core/bus.c | 9 +++ hw/core/machine.c | 18 ++++- hw/core/qdev.c | 8 +++ hw/core/sysbus.c | 3 +- hw/i2c/core.c | 9 ++- hw/i386/pc_piix.c | 13 ++++ hw/i386/pc_q35.c | 4 ++ hw/ide/qdev.c | 3 +- hw/input/adb.c | 9 ++- hw/ipack/ipack.c | 9 ++- hw/isa/isa-bus.c | 3 +- hw/mips/mips_malta.c | 7 ++ hw/mips/mips_r4k.c | 2 + hw/misc/auxbus.c | 3 +- hw/net/e1000e.c | 4 ++ hw/net/vmxnet3.c | 4 ++ hw/pci-bridge/ioh3420.c | 4 ++ hw/pci-bridge/xio3130_downstream.c | 4 ++ hw/pci/pci.c | 16 ++++- hw/ppc/e500plat.c | 3 + hw/ppc/mac_newworld.c | 4 ++ hw/ppc/mac_oldworld.c | 3 + hw/ppc/mpc8544ds.c | 4 ++ hw/ppc/ppc440_bamboo.c | 1 + hw/ppc/prep.c | 4 ++ hw/ppc/spapr_vio.c | 3 +- hw/s390x/css-bridge.c | 2 + hw/s390x/event-facility.c | 3 +- hw/s390x/s390-pci-bus.c | 9 ++- hw/s390x/s390-virtio-ccw.c | 6 ++ hw/s390x/virtio-ccw.c | 2 +- hw/scsi/megasas.c | 7 ++ hw/scsi/scsi-bus.c | 3 +- hw/scsi/vmw_pvscsi.c | 1 + hw/sd/core.c | 7 ++ hw/sd/sd.c | 2 +- hw/ssi/ssi.c | 9 ++- hw/usb/bus.c | 3 +- hw/usb/dev-smartcard-reader.c | 9 ++- hw/usb/hcd-xhci.c | 4 ++ hw/vfio/pci.c | 4 ++ hw/virtio/virtio-bus.c | 1 + hw/virtio/virtio-pci.c | 4 ++ hw/virtio/virtio.c | 2 +- include/hw/boards.h | 5 ++ include/hw/pci/pci.h | 3 + include/hw/qdev-core.h | 4 ++ qapi-schema.json | 9 ++- scripts/qemu.py | 25 +++++-- scripts/qtest.py | 15 +++- tests/Makefile.include | 39 ++++++++++- tests/qmp-machine-info.py | 138 +++++++++++++++++++++++++++++++++++++ vl.c | 11 +++ 71 files changed, 518 insertions(+), 37 deletions(-) create mode 100755 tests/qmp-machine-info.py
-- Eduardo
On 11/23/2016 07:35 PM, Eduardo Habkost wrote:
On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote:
On 11/22/2016 03:11 AM, Eduardo Habkost wrote:
The Problem ===========
Currently management software has no way to find out which device types can be plugged in a machine, unless the machine is already initialized.
Hi Eduardo, Thank you for this interesting series. I think this is a problem worth addressing.
[...]
PCI vs PCIe -----------
Machines with PCIe buses will report INTERFACE_PCIE_DEVICE on supported-device-types.
Machines with legacy PCI buses will report TYPE_PCI_DEVICE on supported-device-types.
The problem with the current approach is that PCIe devices are TYPE_PCI_DEVICE subclasses. The allows PCI device classes to indicate they are PCIe-capable, but there's no obvious way to indicate that a device is PCIe-only. This needs to be addressed in a future version of this series.
Suggestions are welcome.
As we talked offline, I personally like an interface IPCIType with a field having 3 possible values {pci/pcie/hybrid}
To understand how hybrid works we need some rules, like "pci on pci bus/pcie on pcie bus" "pcie on a non-root pcie bus/pcie otherwise
I don't think we'll have a lot of rules, simple boolean fields for the interface should be enough.
What you propose makes sense, the only difference is that the boolean fields would be just interface names that can be used as the "implements" argument on qom-list-types.
e.g.:
* Hybrid PCI device-types would implement both "legacy-pci-device" and "pcie-device" interfaces. * PCIe-only device-types would implement only the "pcie-device" interface. * Legacy-PCI-only device-types would implement only the "legacy-pci-device" interface.
Then, the bus instances would have a 'accepted-device-types' field:
* A legacy PCI bus would accept only "legacy-pci-device" devices. * A PCIe-only bus would accept only "pcie-device" devices. * A PCIe bus that accepts legacy PCI devices (the root bus?) would accept both "legacy-pci-device" and "pcie-device" devices.
Then, query-machines would return the list of bus instances that machine init creates, containing the bus ID, bus type, and accepted-device-types.
Does that make sense?
Sure, the described approach solves the problem.
This still does not solve the problem that some devices makes sense only on a specific arch.
Right now, we can simply compile out arch-specific devices that can never be plugged in a QEMU binary. But if we still want them compiled in for some reason, we can categorize them on a different type/interface name, and the corresponding machine-type would tell management that their buses accept only the arch-specific type/interface name.
Adding an Interface for each arch could work, yes.
Incomplete bus lists on some machines -------------------------------------
With this series, not all machines classes are changed to add the full list of device types on the 'supported-device-types'. To allow the code to be updated gradually, qmp-machine-info.py has a STRICT_ARCHES variable, that will make the test code require a complete device type list only on some architectures.
Out of scope: Configurable buses --------------------------------
There's no way to map machine options like "usb=on|off" to device-types or buses. I plan to propose a new interface that allows machine options to be mapped to buses/device-types later.
Out of scope: Deciding where to plug devices --------------------------------------------
Once management software discovers which devices can be plugged to a machine, it still has to discover or define where devices can/should/will be plugged. This is out of the scope of this series.
That's a pitty :( I was hoping this series will solve this issue. But if it prepares the grounds for it is also a good step .
The bus ID will be in the scope of v2.
Thanks, Marcel
Thanks, Marcel
[...]
Eduardo Habkost <ehabkost@redhat.com> writes:
On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote:
On 11/22/2016 03:11 AM, Eduardo Habkost wrote:
The Problem ===========
Currently management software has no way to find out which device types can be plugged in a machine, unless the machine is already initialized.
I doubt it can find the precise list of device types even for an initialized machine.
Hi Eduardo, Thank you for this interesting series. I think this is a problem worth addressing.
Even after the machine is initialized, there's no way to map existing bus types to supported device types unless management software hardcodes the mapping between bus types and device types.
Here I am a little lost.
We are going for machine => supported devices or bus-type => supported devices?
On this series, we go for machine-type => supported-devices.
A bus-type => supported-devices map wouldn't work because different PCIe bus instances might accept different types of devices (so supported-devices depend on the specific bus instance, not only on the bus-type).
v2 will probably be more detailed. I plan to change it to:
query-machine(machine-type) => list of BusInfo
BusInfo would contain: * bus-type * bus-path * accepted-device-types (list of type/interface names)
Let me take a step back and explore what questions a management application may want to ask. I think the basic questions are: * Which devices models could be plugged now? "Now" means in the current state of the machine. Depending on "now", this could be hot- or cold-plug. * Into which sockets could device model T be plugged now? Mind, I wrote "socket", not bus. Buses provide sockets, but they are not the only provider of sockets. We need a way to name sockets. (QOM path to device or bus, socket address on device or bus) could work. Are we on the same page so far? Related: same questions with "newly created machine of machine type M" instead of "now". This isn't basic, because you could always create the machine, then ask the basic question. But not having to create the machine is convenient, in particular as long as a single QEMU process can only ever create one machine. Related: same, with creation options. Your series provides a solution for a special non-basic case: newly created machine of machine type M with no optional parts, i.e. with -nodefaults and nothing else. Or maybe with the default set of optional parts, i.e. without -nodefaults, not sure. I'm not demanding you provide a solution for the general case first. It would be nice, but it could be too complex for a first step. I do recommend you design your solution for the special case with the general case in mind. Say, by trying to ensure the special case query returns something that will do or can be evolved to do for the general case, too.
Example: floppy support on q35 vs i440fx ----------------------------------------
There's no way for libvirt to find out that there's no floppy controller on pc-q35-* machine-types by default.
Again "by default". So do we want to query the init state of a machine? What devices are there? Or what devices *can be* there?
"by default" means what's present when using "-machine <machine>" with no extra -device arguments.
Not just -device. Some legacy options can also create buses. For instance, -device if=scsi,... creates a SCSI bus.
We want to know what _buses_ are always there. Which in turn lets management know which _device_ types _can_ be plugged.
With this series, pc-i440fx-* will report "floppy" as a supported device type, but pc-q35-* will not.
Example: Legacy PCI vs vs PCIe devices --------------------------------------
Some devices require a PCIe bus to be available, others work on both legacy PCI and PCIe, while others work only on a legacy PCI bus.
Currently management software has no way to know which devices can be added to a given machine, unless it hardcodes machine-type names and device-types names.
Again it seems a double problem, machine => devices vs pci/pcie bus => devices. The bus => devices match is not related to a machine type.
A bus-type => device-type match would not depend on the machine-type, but it would not be useful: different bus instances can accept different device-types (and the way the bus topology is configured depend on the machine-type).
The Proposed Interface ======================
This series adds a new field to the output of 'query-machines': 'supported-device-types'. It will contain a list of QOM type names, that can be used to find the list of device types that can be plugged in the machine by default.
What do you mean "by default"? Without bridges or part of the machine itself?
I mean "when you just run -machine with no extra -device arguments".
The type names reported on
the new field can then be used as the 'implements' argument on the 'qom-list-types' command, to find out which device types can be plugged on the machine.
Example output --------------
(QEMU) query-machines { "return": [ [...] { "supported-device-types": [ "sys-bus-device"
I don't know how "sys-bus-device" can help us... :)
Yes, I added comments about it below. :)
The sysbus has always been a cop-out. It's a catch-all pseudo-bus. The only thing sysbus devices really have in common is that they can't plug into any real bus. This used to be an academic issue until Alex made some sysbus devices pluggable. Before, "sysbus device" told you everything about pluggability you needed to know: not pluggable. Since then, it tells you nothing. We'll have to address this regression eventually. See "TYPE_SYS_BUS_DEVICE is too generic" below.
], "cpu-max": 1, "hotpluggable-cpus": false, "name": "none" }, [...] { "supported-device-types": [ "sys-bus-device" ], "cpu-max": 1, "hotpluggable-cpus": false, "name": "xenpv" }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "i2c-slave", "pci-device", "isa-device", "ide-device"
Is don't know is this high level classification is useful, here is an example:
pvi-device is supported => then we look for all pci devices? But what if some pci devices make sense on a machine type, but not on another?
If not all pci devices are supported, then the machine must not return "pci-device" as supported. We need to define a new type/interface name that would be implemented only by the supported devices. e.g. "legacy-pci-device".
Before PCIe, "PCI device" told you everything about pluggability you needed to know: can be plugged into a PCI bus. Since PCIe, it doesn't. We'll have to address this regression, too. See "PCI vs PCIe" below.
], "name": "pc-i440fx-2.8", "alias": "pc", "is-default": true, "cpu-max": 255, "hotpluggable-cpus": true }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "isa-device", "ide-device" ], "cpu-max": 1, "hotpluggable-cpus": true, "name": "isapc" }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "i2c-slave", "pci-device", "isa-device", "ide-device" ], "cpu-max": 128, "hotpluggable-cpus": true, "name": "xenfv" }, [...] { "alias": "q35", "supported-device-types": [ "sys-bus-device", "i2c-slave", "PCIE-device", "isa-device", "ide-device" ], "cpu-max": 288, "hotpluggable-cpus": true, "name": "pc-q35-2.8" }, [...] ] }
Considered alternatives =======================
Indirect mapping (machine => bus => device) -------------------------------------------
This RFC implements a mechanism to implement ax machine-type => supported-device-types mapping. An alternative solution I considered was to expose an indirect mapping: machine-type => default-bus-types followed by bus-type => supported-device-types.
As I previously stated, I don't know if it helps. bus-type can support different devices on different archs.
Yes. After learning a bit more about PCIe, I am convinced that a bus->type => device-types mapping won't help.
But exposing only the resulting supported device-types list imposes less restrictions on how the device and bus type hierarchy is implemented inside QEMU. There's still a machine-type => bus-type => device-type mapping implemented internally, but it is an implementation detail on the current version, and not part of the externally-visible interface.
Good, I personally don't like this "pass".
Point taken. The core idea here seems to be using the QOM type hierarchy for representing "pluggability".
The Implementation ================== [Skipping for now...]
Limitations ===========
TYPE_SYS_BUS_DEVICE is too generic ----------------------------------
Currently all machines have a TYPE_SYS_BUS bus, meaning all TYPE_SYS_BUS_DEVICE subclasses are reported as supported.
Agreed, this is a problem.
The current solution in this series is to report TYPE_SYS_BUS_DEVICE as supported by all machines.
Because all machines have this catch-all pseudo-bus. Reporting it exists is correct, but as is the information is useless.
But we could gradually add arch-specific or machine-family-specific interface names that can be used on devices that are meant to work with only a subset of TYPE_SYS_BUS_DEVICE subclasses.
Basically, encode pluggability information in the QOM type hierarchy. To gauge whether that would work I'd have to actually understand how exactly pluggable sysbus devices work, in particular what the conditions for accepting a plug are.
A future version of this series may remove TYPE_SYS_BUS_DEVICE from the supported-device-types output, and return a arch-specific or machine-family-specific interface name to restrict management software to a subset of TYPE_SYS_BUS_DEVICE subclasses.
PCI vs PCIe -----------
Machines with PCIe buses will report INTERFACE_PCIE_DEVICE on supported-device-types.
Machines with legacy PCI buses will report TYPE_PCI_DEVICE on supported-device-types.
The problem with the current approach is that PCIe devices are TYPE_PCI_DEVICE subclasses. The allows PCI device classes to indicate they are PCIe-capable, but there's no obvious way to indicate that a device is PCIe-only. This needs to be addressed in a future version of this series.
Suggestions are welcome.
As we talked offline, I personally like an interface IPCIType with a field having 3 possible values {pci/pcie/hybrid}
To understand how hybrid works we need some rules, like "pci on pci bus/pcie on pcie bus" "pcie on a non-root pcie bus/pcie otherwise
I don't think we'll have a lot of rules, simple boolean fields for the interface should be enough.
What you propose makes sense, the only difference is that the boolean fields would be just interface names that can be used as the "implements" argument on qom-list-types.
e.g.:
* Hybrid PCI device-types would implement both "legacy-pci-device" and "pcie-device" interfaces. * PCIe-only device-types would implement only the "pcie-device" interface. * Legacy-PCI-only device-types would implement only the "legacy-pci-device" interface.
In other words, encode "pluggability" in the QOM type hierarchy.
Then, the bus instances would have a 'accepted-device-types' field:
* A legacy PCI bus would accept only "legacy-pci-device" devices. * A PCIe-only bus would accept only "pcie-device" devices. * A PCIe bus that accepts legacy PCI devices (the root bus?) would accept both "legacy-pci-device" and "pcie-device" devices.
Then, query-machines would return the list of bus instances that machine init creates, containing the bus ID, bus type, and accepted-device-types.
Does that make sense?
Yes. Our decision to have hybrid PCI/PCIe devices and buses breeds considerable complexity. I wish we had avoided them, but I believe it's too late to change now.
This still does not solve the problem that some devices makes sense only on a specific arch.
Examples?
Right now, we can simply compile out arch-specific devices that can never be plugged in a QEMU binary. But if we still want them compiled in for some reason, we can categorize them on a different type/interface name, and the corresponding machine-type would tell management that their buses accept only the arch-specific type/interface name.
Could become messy. Can't say without looking at examples.
Incomplete bus lists on some machines -------------------------------------
With this series, not all machines classes are changed to add the full list of device types on the 'supported-device-types'. To allow the code to be updated gradually, qmp-machine-info.py has a STRICT_ARCHES variable, that will make the test code require a complete device type list only on some architectures.
Ideally, pluggability information would be derived from QOM. I figure that's the only practical way to solve the general case. For the special case "newly created machine", deriving from QOM may be impractical, say because you'd have to create the machine. Then you have to hard-code the result somehow. Tests to validate the hardcoded results match the results you get when you actually create the machine would be required, at least once you *can* get results for a created machine.
Out of scope: Configurable buses --------------------------------
There's no way to map machine options like "usb=on|off" to device-types or buses. I plan to propose a new interface that allows machine options to be mapped to buses/device-types later.
Out of scope: Deciding where to plug devices --------------------------------------------
Once management software discovers which devices can be plugged to a machine, it still has to discover or define where devices can/should/will be plugged. This is out of the scope of this series.
That's a pitty :( I was hoping this series will solve this issue. But if it prepares the grounds for it is also a good step .
The bus ID will be in the scope of v2.
I'm afraid "bus" is insufficiently general.
Thanks, Marcel
Out of scope: Hotplug ---------------------
The proposed interface is supposed to help management software find which device types can be used when creating the VM. Device hotplug is out of the scope of this series. However, the new 'device-types' QOM property on bus objects could be used to find out which device types can be plugged on the existing buses.
On Thu, Nov 24, 2016 at 02:34:05PM +0100, Markus Armbruster wrote:
Eduardo Habkost <ehabkost@redhat.com> writes:
On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote:
On 11/22/2016 03:11 AM, Eduardo Habkost wrote:
The Problem ===========
Currently management software has no way to find out which device types can be plugged in a machine, unless the machine is already initialized.
I doubt it can find the precise list of device types even for an initialized machine.
My plan is to make it possible. See: [RFC 06/15] qdev: Add device_type field to BusClass I will change how it looks like on v2 (mostly due to the PCI/PCIe issues), but basically it makes all bus objects able to tell which device types they accept.
Hi Eduardo, Thank you for this interesting series. I think this is a problem worth addressing.
Even after the machine is initialized, there's no way to map existing bus types to supported device types unless management software hardcodes the mapping between bus types and device types.
Here I am a little lost.
We are going for machine => supported devices or bus-type => supported devices?
On this series, we go for machine-type => supported-devices.
A bus-type => supported-devices map wouldn't work because different PCIe bus instances might accept different types of devices (so supported-devices depend on the specific bus instance, not only on the bus-type).
v2 will probably be more detailed. I plan to change it to:
query-machine(machine-type) => list of BusInfo
BusInfo would contain: * bus-type * bus-path * accepted-device-types (list of type/interface names)
Let me take a step back and explore what questions a management application may want to ask. I think the basic questions are:
* Which devices models could be plugged now?
"Now" means in the current state of the machine. Depending on "now", this could be hot- or cold-plug.
Yes.
* Into which sockets could device model T be plugged now?
Mind, I wrote "socket", not bus. Buses provide sockets, but they are not the only provider of sockets.
We need a way to name sockets. (QOM path to device or bus, socket address on device or bus) could work.
Naming "sockets" was not on my model, but I think we can do that. I will probably send RFC v2 without any socket abstraction, so other conceptual changes I am introducing can be reviewed (especially the PCI/PCIe mess I am diving into, right now). But I will try to document how the design can evolve to handle sockets.
Are we on the same page so far?
Yes.
Related: same questions with "newly created machine of machine type M" instead of "now". This isn't basic, because you could always create the machine, then ask the basic question. But not having to create the machine is convenient, in particular as long as a single QEMU process can only ever create one machine.
Related: same, with creation options.
Your series provides a solution for a special non-basic case: newly created machine of machine type M with no optional parts, i.e. with -nodefaults and nothing else. Or maybe with the default set of optional parts, i.e. without -nodefaults, not sure.
I'm not demanding you provide a solution for the general case first. It would be nice, but it could be too complex for a first step. I do recommend you design your solution for the special case with the general case in mind. Say, by trying to ensure the special case query returns something that will do or can be evolved to do for the general case, too.
Yep. I will keep that in mind.
Example: floppy support on q35 vs i440fx ----------------------------------------
There's no way for libvirt to find out that there's no floppy controller on pc-q35-* machine-types by default.
Again "by default". So do we want to query the init state of a machine? What devices are there? Or what devices *can be* there?
"by default" means what's present when using "-machine <machine>" with no extra -device arguments.
Not just -device. Some legacy options can also create buses. For instance, -device if=scsi,... creates a SCSI bus.
Right. I found that out when debugging the -nodefaults issues.
We want to know what _buses_ are always there. Which in turn lets management know which _device_ types _can_ be plugged.
With this series, pc-i440fx-* will report "floppy" as a supported device type, but pc-q35-* will not.
Example: Legacy PCI vs vs PCIe devices --------------------------------------
Some devices require a PCIe bus to be available, others work on both legacy PCI and PCIe, while others work only on a legacy PCI bus.
Currently management software has no way to know which devices can be added to a given machine, unless it hardcodes machine-type names and device-types names.
Again it seems a double problem, machine => devices vs pci/pcie bus => devices. The bus => devices match is not related to a machine type.
A bus-type => device-type match would not depend on the machine-type, but it would not be useful: different bus instances can accept different device-types (and the way the bus topology is configured depend on the machine-type).
The Proposed Interface ======================
This series adds a new field to the output of 'query-machines': 'supported-device-types'. It will contain a list of QOM type names, that can be used to find the list of device types that can be plugged in the machine by default.
What do you mean "by default"? Without bridges or part of the machine itself?
I mean "when you just run -machine with no extra -device arguments".
The type names reported on
the new field can then be used as the 'implements' argument on the 'qom-list-types' command, to find out which device types can be plugged on the machine.
Example output --------------
(QEMU) query-machines { "return": [ [...] { "supported-device-types": [ "sys-bus-device"
I don't know how "sys-bus-device" can help us... :)
Yes, I added comments about it below. :)
The sysbus has always been a cop-out. It's a catch-all pseudo-bus. The only thing sysbus devices really have in common is that they can't plug into any real bus.
This used to be an academic issue until Alex made some sysbus devices pluggable. Before, "sysbus device" told you everything about pluggability you needed to know: not pluggable. Since then, it tells you nothing. We'll have to address this regression eventually. See "TYPE_SYS_BUS_DEVICE is too generic" below.
], "cpu-max": 1, "hotpluggable-cpus": false, "name": "none" }, [...] { "supported-device-types": [ "sys-bus-device" ], "cpu-max": 1, "hotpluggable-cpus": false, "name": "xenpv" }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "i2c-slave", "pci-device", "isa-device", "ide-device"
Is don't know is this high level classification is useful, here is an example:
pvi-device is supported => then we look for all pci devices? But what if some pci devices make sense on a machine type, but not on another?
If not all pci devices are supported, then the machine must not return "pci-device" as supported. We need to define a new type/interface name that would be implemented only by the supported devices. e.g. "legacy-pci-device".
Before PCIe, "PCI device" told you everything about pluggability you needed to know: can be plugged into a PCI bus. Since PCIe, it doesn't. We'll have to address this regression, too. See "PCI vs PCIe" below.
I am trying to clean up the PCI/PCIe mess right now, for v2.
], "name": "pc-i440fx-2.8", "alias": "pc", "is-default": true, "cpu-max": 255, "hotpluggable-cpus": true }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "isa-device", "ide-device" ], "cpu-max": 1, "hotpluggable-cpus": true, "name": "isapc" }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "i2c-slave", "pci-device", "isa-device", "ide-device" ], "cpu-max": 128, "hotpluggable-cpus": true, "name": "xenfv" }, [...] { "alias": "q35", "supported-device-types": [ "sys-bus-device", "i2c-slave", "PCIE-device", "isa-device", "ide-device" ], "cpu-max": 288, "hotpluggable-cpus": true, "name": "pc-q35-2.8" }, [...] ] }
Considered alternatives =======================
Indirect mapping (machine => bus => device) -------------------------------------------
This RFC implements a mechanism to implement ax machine-type => supported-device-types mapping. An alternative solution I considered was to expose an indirect mapping: machine-type => default-bus-types followed by bus-type => supported-device-types.
As I previously stated, I don't know if it helps. bus-type can support different devices on different archs.
Yes. After learning a bit more about PCIe, I am convinced that a bus->type => device-types mapping won't help.
But exposing only the resulting supported device-types list imposes less restrictions on how the device and bus type hierarchy is implemented inside QEMU. There's still a machine-type => bus-type => device-type mapping implemented internally, but it is an implementation detail on the current version, and not part of the externally-visible interface.
Good, I personally don't like this "pass".
Point taken.
The core idea here seems to be using the QOM type hierarchy for representing "pluggability".
Exactly.
The Implementation ================== [Skipping for now...]
Limitations ===========
TYPE_SYS_BUS_DEVICE is too generic ----------------------------------
Currently all machines have a TYPE_SYS_BUS bus, meaning all TYPE_SYS_BUS_DEVICE subclasses are reported as supported.
Agreed, this is a problem.
The current solution in this series is to report TYPE_SYS_BUS_DEVICE as supported by all machines.
Because all machines have this catch-all pseudo-bus. Reporting it exists is correct, but as is the information is useless.
I will probably keep returning it by default (to make the existing information correct), but provide mechanisms for machine-types to return more precise information.
But we could gradually add arch-specific or machine-family-specific interface names that can be used on devices that are meant to work with only a subset of TYPE_SYS_BUS_DEVICE subclasses.
Basically, encode pluggability information in the QOM type hierarchy.
To gauge whether that would work I'd have to actually understand how exactly pluggable sysbus devices work, in particular what the conditions for accepting a plug are.
I can't evaluate that, either. But note that covering 100% of the pluggability decisions for sysbus is not one of the problems I am trying to solve right now. I mean: if the new mechanisms can cover it too, great, but they don't (and there's no obvious solution for it), it is out of scope.
A future version of this series may remove TYPE_SYS_BUS_DEVICE from the supported-device-types output, and return a arch-specific or machine-family-specific interface name to restrict management software to a subset of TYPE_SYS_BUS_DEVICE subclasses.
PCI vs PCIe -----------
Machines with PCIe buses will report INTERFACE_PCIE_DEVICE on supported-device-types.
Machines with legacy PCI buses will report TYPE_PCI_DEVICE on supported-device-types.
The problem with the current approach is that PCIe devices are TYPE_PCI_DEVICE subclasses. The allows PCI device classes to indicate they are PCIe-capable, but there's no obvious way to indicate that a device is PCIe-only. This needs to be addressed in a future version of this series.
Suggestions are welcome.
As we talked offline, I personally like an interface IPCIType with a field having 3 possible values {pci/pcie/hybrid}
To understand how hybrid works we need some rules, like "pci on pci bus/pcie on pcie bus" "pcie on a non-root pcie bus/pcie otherwise
I don't think we'll have a lot of rules, simple boolean fields for the interface should be enough.
What you propose makes sense, the only difference is that the boolean fields would be just interface names that can be used as the "implements" argument on qom-list-types.
e.g.:
* Hybrid PCI device-types would implement both "legacy-pci-device" and "pcie-device" interfaces. * PCIe-only device-types would implement only the "pcie-device" interface. * Legacy-PCI-only device-types would implement only the "legacy-pci-device" interface.
In other words, encode "pluggability" in the QOM type hierarchy.
Then, the bus instances would have a 'accepted-device-types' field:
* A legacy PCI bus would accept only "legacy-pci-device" devices. * A PCIe-only bus would accept only "pcie-device" devices. * A PCIe bus that accepts legacy PCI devices (the root bus?) would accept both "legacy-pci-device" and "pcie-device" devices.
Then, query-machines would return the list of bus instances that machine init creates, containing the bus ID, bus type, and accepted-device-types.
Does that make sense?
Yes.
Our decision to have hybrid PCI/PCIe devices and buses breeds considerable complexity. I wish we had avoided them, but I believe it's too late to change now.
I'm diving into it right now. v2 will have some attempts to sort it out.
This still does not solve the problem that some devices makes sense only on a specific arch.
Examples?
Right now, we can simply compile out arch-specific devices that can never be plugged in a QEMU binary. But if we still want them compiled in for some reason, we can categorize them on a different type/interface name, and the corresponding machine-type would tell management that their buses accept only the arch-specific type/interface name.
Could become messy. Can't say without looking at examples.
Can't say, either.
Incomplete bus lists on some machines -------------------------------------
With this series, not all machines classes are changed to add the full list of device types on the 'supported-device-types'. To allow the code to be updated gradually, qmp-machine-info.py has a STRICT_ARCHES variable, that will make the test code require a complete device type list only on some architectures.
Ideally, pluggability information would be derived from QOM. I figure that's the only practical way to solve the general case.
For the special case "newly created machine", deriving from QOM may be impractical, say because you'd have to create the machine. Then you have to hard-code the result somehow. Tests to validate the hardcoded results match the results you get when you actually create the machine would be required, at least once you *can* get results for a created machine.
Tests were implemented, see [RFC 08/15] qmp: Add 'supported-device-types' field to 'query-machines'.
Out of scope: Configurable buses --------------------------------
There's no way to map machine options like "usb=on|off" to device-types or buses. I plan to propose a new interface that allows machine options to be mapped to buses/device-types later.
Out of scope: Deciding where to plug devices --------------------------------------------
Once management software discovers which devices can be plugged to a machine, it still has to discover or define where devices can/should/will be plugged. This is out of the scope of this series.
That's a pitty :( I was hoping this series will solve this issue. But if it prepares the grounds for it is also a good step .
The bus ID will be in the scope of v2.
I'm afraid "bus" is insufficiently general.
Does "socket" sound general enough?
Thanks, Marcel
Out of scope: Hotplug ---------------------
The proposed interface is supposed to help management software find which device types can be used when creating the VM. Device hotplug is out of the scope of this series. However, the new 'device-types' QOM property on bus objects could be used to find out which device types can be plugged on the existing buses.
-- Eduardo
Eduardo Habkost <ehabkost@redhat.com> writes:
On Thu, Nov 24, 2016 at 02:34:05PM +0100, Markus Armbruster wrote:
Eduardo Habkost <ehabkost@redhat.com> writes:
On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote:
On 11/22/2016 03:11 AM, Eduardo Habkost wrote:
The Problem ===========
Currently management software has no way to find out which device types can be plugged in a machine, unless the machine is already initialized.
I doubt it can find the precise list of device types even for an initialized machine.
My plan is to make it possible. See: [RFC 06/15] qdev: Add device_type field to BusClass
I will change how it looks like on v2 (mostly due to the PCI/PCIe issues), but basically it makes all bus objects able to tell which device types they accept.
Sounds like a step forward.
Hi Eduardo, Thank you for this interesting series. I think this is a problem worth addressing.
Even after the machine is initialized, there's no way to map existing bus types to supported device types unless management software hardcodes the mapping between bus types and device types.
Here I am a little lost.
We are going for machine => supported devices or bus-type => supported devices?
On this series, we go for machine-type => supported-devices.
A bus-type => supported-devices map wouldn't work because different PCIe bus instances might accept different types of devices (so supported-devices depend on the specific bus instance, not only on the bus-type).
v2 will probably be more detailed. I plan to change it to:
query-machine(machine-type) => list of BusInfo
BusInfo would contain: * bus-type * bus-path * accepted-device-types (list of type/interface names)
Let me take a step back and explore what questions a management application may want to ask. I think the basic questions are:
* Which devices models could be plugged now?
"Now" means in the current state of the machine. Depending on "now", this could be hot- or cold-plug.
Yes.
* Into which sockets could device model T be plugged now?
Mind, I wrote "socket", not bus. Buses provide sockets, but they are not the only provider of sockets.
We need a way to name sockets. (QOM path to device or bus, socket address on device or bus) could work.
Naming "sockets" was not on my model, but I think we can do that.
I will probably send RFC v2 without any socket abstraction, so other conceptual changes I am introducing can be reviewed (especially the PCI/PCIe mess I am diving into, right now). But I will try to document how the design can evolve to handle sockets.
Makes sense.
Are we on the same page so far?
Yes.
Related: same questions with "newly created machine of machine type M" instead of "now". This isn't basic, because you could always create the machine, then ask the basic question. But not having to create the machine is convenient, in particular as long as a single QEMU process can only ever create one machine.
Related: same, with creation options.
Your series provides a solution for a special non-basic case: newly created machine of machine type M with no optional parts, i.e. with -nodefaults and nothing else. Or maybe with the default set of optional parts, i.e. without -nodefaults, not sure.
I'm not demanding you provide a solution for the general case first. It would be nice, but it could be too complex for a first step. I do recommend you design your solution for the special case with the general case in mind. Say, by trying to ensure the special case query returns something that will do or can be evolved to do for the general case, too.
Yep. I will keep that in mind.
Example: floppy support on q35 vs i440fx ----------------------------------------
There's no way for libvirt to find out that there's no floppy controller on pc-q35-* machine-types by default.
Again "by default". So do we want to query the init state of a machine? What devices are there? Or what devices *can be* there?
"by default" means what's present when using "-machine <machine>" with no extra -device arguments.
Not just -device. Some legacy options can also create buses. For instance, -device if=scsi,... creates a SCSI bus.
Right. I found that out when debugging the -nodefaults issues.
We want to know what _buses_ are always there. Which in turn lets management know which _device_ types _can_ be plugged.
With this series, pc-i440fx-* will report "floppy" as a supported device type, but pc-q35-* will not.
Example: Legacy PCI vs vs PCIe devices --------------------------------------
Some devices require a PCIe bus to be available, others work on both legacy PCI and PCIe, while others work only on a legacy PCI bus.
Currently management software has no way to know which devices can be added to a given machine, unless it hardcodes machine-type names and device-types names.
Again it seems a double problem, machine => devices vs pci/pcie bus => devices. The bus => devices match is not related to a machine type.
A bus-type => device-type match would not depend on the machine-type, but it would not be useful: different bus instances can accept different device-types (and the way the bus topology is configured depend on the machine-type).
The Proposed Interface ======================
This series adds a new field to the output of 'query-machines': 'supported-device-types'. It will contain a list of QOM type names, that can be used to find the list of device types that can be plugged in the machine by default.
What do you mean "by default"? Without bridges or part of the machine itself?
I mean "when you just run -machine with no extra -device arguments".
The type names reported on
the new field can then be used as the 'implements' argument on the 'qom-list-types' command, to find out which device types can be plugged on the machine.
Example output --------------
(QEMU) query-machines { "return": [ [...] { "supported-device-types": [ "sys-bus-device"
I don't know how "sys-bus-device" can help us... :)
Yes, I added comments about it below. :)
The sysbus has always been a cop-out. It's a catch-all pseudo-bus. The only thing sysbus devices really have in common is that they can't plug into any real bus.
This used to be an academic issue until Alex made some sysbus devices pluggable. Before, "sysbus device" told you everything about pluggability you needed to know: not pluggable. Since then, it tells you nothing. We'll have to address this regression eventually. See "TYPE_SYS_BUS_DEVICE is too generic" below.
], "cpu-max": 1, "hotpluggable-cpus": false, "name": "none" }, [...] { "supported-device-types": [ "sys-bus-device" ], "cpu-max": 1, "hotpluggable-cpus": false, "name": "xenpv" }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "i2c-slave", "pci-device", "isa-device", "ide-device"
Is don't know is this high level classification is useful, here is an example:
pvi-device is supported => then we look for all pci devices? But what if some pci devices make sense on a machine type, but not on another?
If not all pci devices are supported, then the machine must not return "pci-device" as supported. We need to define a new type/interface name that would be implemented only by the supported devices. e.g. "legacy-pci-device".
Before PCIe, "PCI device" told you everything about pluggability you needed to know: can be plugged into a PCI bus. Since PCIe, it doesn't. We'll have to address this regression, too. See "PCI vs PCIe" below.
I am trying to clean up the PCI/PCIe mess right now, for v2.
], "name": "pc-i440fx-2.8", "alias": "pc", "is-default": true, "cpu-max": 255, "hotpluggable-cpus": true }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "isa-device", "ide-device" ], "cpu-max": 1, "hotpluggable-cpus": true, "name": "isapc" }, [...] { "supported-device-types": [ "sys-bus-device", "floppy", "i2c-slave", "pci-device", "isa-device", "ide-device" ], "cpu-max": 128, "hotpluggable-cpus": true, "name": "xenfv" }, [...] { "alias": "q35", "supported-device-types": [ "sys-bus-device", "i2c-slave", "PCIE-device", "isa-device", "ide-device" ], "cpu-max": 288, "hotpluggable-cpus": true, "name": "pc-q35-2.8" }, [...] ] }
Considered alternatives =======================
Indirect mapping (machine => bus => device) -------------------------------------------
This RFC implements a mechanism to implement ax machine-type => supported-device-types mapping. An alternative solution I considered was to expose an indirect mapping: machine-type => default-bus-types followed by bus-type => supported-device-types.
As I previously stated, I don't know if it helps. bus-type can support different devices on different archs.
Yes. After learning a bit more about PCIe, I am convinced that a bus->type => device-types mapping won't help.
But exposing only the resulting supported device-types list imposes less restrictions on how the device and bus type hierarchy is implemented inside QEMU. There's still a machine-type => bus-type => device-type mapping implemented internally, but it is an implementation detail on the current version, and not part of the externally-visible interface.
Good, I personally don't like this "pass".
Point taken.
The core idea here seems to be using the QOM type hierarchy for representing "pluggability".
Exactly.
The Implementation ================== [Skipping for now...]
Limitations ===========
TYPE_SYS_BUS_DEVICE is too generic ----------------------------------
Currently all machines have a TYPE_SYS_BUS bus, meaning all TYPE_SYS_BUS_DEVICE subclasses are reported as supported.
Agreed, this is a problem.
The current solution in this series is to report TYPE_SYS_BUS_DEVICE as supported by all machines.
Because all machines have this catch-all pseudo-bus. Reporting it exists is correct, but as is the information is useless.
I will probably keep returning it by default (to make the existing information correct), but provide mechanisms for machine-types to return more precise information.
But we could gradually add arch-specific or machine-family-specific interface names that can be used on devices that are meant to work with only a subset of TYPE_SYS_BUS_DEVICE subclasses.
Basically, encode pluggability information in the QOM type hierarchy.
To gauge whether that would work I'd have to actually understand how exactly pluggable sysbus devices work, in particular what the conditions for accepting a plug are.
I can't evaluate that, either.
But note that covering 100% of the pluggability decisions for sysbus is not one of the problems I am trying to solve right now. I mean: if the new mechanisms can cover it too, great, but they don't (and there's no obvious solution for it), it is out of scope.
We don't have to solve all the problems right away. But the less we know about the problems we're not solving right away, the higher the risk of accidentally creating obstacles to solving them later.
A future version of this series may remove TYPE_SYS_BUS_DEVICE from the supported-device-types output, and return a arch-specific or machine-family-specific interface name to restrict management software to a subset of TYPE_SYS_BUS_DEVICE subclasses.
PCI vs PCIe -----------
Machines with PCIe buses will report INTERFACE_PCIE_DEVICE on supported-device-types.
Machines with legacy PCI buses will report TYPE_PCI_DEVICE on supported-device-types.
The problem with the current approach is that PCIe devices are TYPE_PCI_DEVICE subclasses. The allows PCI device classes to indicate they are PCIe-capable, but there's no obvious way to indicate that a device is PCIe-only. This needs to be addressed in a future version of this series.
Suggestions are welcome.
As we talked offline, I personally like an interface IPCIType with a field having 3 possible values {pci/pcie/hybrid}
To understand how hybrid works we need some rules, like "pci on pci bus/pcie on pcie bus" "pcie on a non-root pcie bus/pcie otherwise
I don't think we'll have a lot of rules, simple boolean fields for the interface should be enough.
What you propose makes sense, the only difference is that the boolean fields would be just interface names that can be used as the "implements" argument on qom-list-types.
e.g.:
* Hybrid PCI device-types would implement both "legacy-pci-device" and "pcie-device" interfaces. * PCIe-only device-types would implement only the "pcie-device" interface. * Legacy-PCI-only device-types would implement only the "legacy-pci-device" interface.
In other words, encode "pluggability" in the QOM type hierarchy.
Then, the bus instances would have a 'accepted-device-types' field:
* A legacy PCI bus would accept only "legacy-pci-device" devices. * A PCIe-only bus would accept only "pcie-device" devices. * A PCIe bus that accepts legacy PCI devices (the root bus?) would accept both "legacy-pci-device" and "pcie-device" devices.
Then, query-machines would return the list of bus instances that machine init creates, containing the bus ID, bus type, and accepted-device-types.
Does that make sense?
Yes.
Our decision to have hybrid PCI/PCIe devices and buses breeds considerable complexity. I wish we had avoided them, but I believe it's too late to change now.
I'm diving into it right now. v2 will have some attempts to sort it out.
This still does not solve the problem that some devices makes sense only on a specific arch.
Examples?
Right now, we can simply compile out arch-specific devices that can never be plugged in a QEMU binary. But if we still want them compiled in for some reason, we can categorize them on a different type/interface name, and the corresponding machine-type would tell management that their buses accept only the arch-specific type/interface name.
Could become messy. Can't say without looking at examples.
Can't say, either.
Incomplete bus lists on some machines -------------------------------------
With this series, not all machines classes are changed to add the full list of device types on the 'supported-device-types'. To allow the code to be updated gradually, qmp-machine-info.py has a STRICT_ARCHES variable, that will make the test code require a complete device type list only on some architectures.
Ideally, pluggability information would be derived from QOM. I figure that's the only practical way to solve the general case.
For the special case "newly created machine", deriving from QOM may be impractical, say because you'd have to create the machine. Then you have to hard-code the result somehow. Tests to validate the hardcoded results match the results you get when you actually create the machine would be required, at least once you *can* get results for a created machine.
Tests were implemented, see [RFC 08/15] qmp: Add 'supported-device-types' field to 'query-machines'.
Out of scope: Configurable buses --------------------------------
There's no way to map machine options like "usb=on|off" to device-types or buses. I plan to propose a new interface that allows machine options to be mapped to buses/device-types later.
Out of scope: Deciding where to plug devices --------------------------------------------
Once management software discovers which devices can be plugged to a machine, it still has to discover or define where devices can/should/will be plugged. This is out of the scope of this series.
That's a pitty :( I was hoping this series will solve this issue. But if it prepares the grounds for it is also a good step .
The bus ID will be in the scope of v2.
I'm afraid "bus" is insufficiently general.
Does "socket" sound general enough?
Yes, because "socket" isn't defined, yet, so we can make it whatever it needs to be :)
Thanks, Marcel
Out of scope: Hotplug ---------------------
The proposed interface is supposed to help management software find which device types can be used when creating the VM. Device hotplug is out of the scope of this series. However, the new 'device-types' QOM property on bus objects could be used to find out which device types can be plugged on the existing buses.
On 11/24/2016 03:34 PM, Markus Armbruster wrote:
Eduardo Habkost <ehabkost@redhat.com> writes:
On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote:
On 11/22/2016 03:11 AM, Eduardo Habkost wrote:
The Problem
[...]
Our decision to have hybrid PCI/PCIe devices and buses breeds considerable complexity. I wish we had avoided them, but I believe it's too late to change now.
This still does not solve the problem that some devices makes sense only on a specific arch.
Hi Markus,
Examples?
One quick example would be that we don't want to see Intel's IOH 3420 PCIe Root Port in an ARM machine, or a pxb on a Q35 machine (in this case we want pxb-pcie) I do believe there are other examples, I'll try to think of more. Thanks, Marcel [...]
Marcel Apfelbaum <marcel@redhat.com> writes:
On 11/24/2016 03:34 PM, Markus Armbruster wrote:
Eduardo Habkost <ehabkost@redhat.com> writes:
On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote:
On 11/22/2016 03:11 AM, Eduardo Habkost wrote:
The Problem
[...]
Our decision to have hybrid PCI/PCIe devices and buses breeds considerable complexity. I wish we had avoided them, but I believe it's too late to change now.
This still does not solve the problem that some devices makes sense only on a specific arch.
Hi Markus,
Examples?
One quick example would be that we don't want to see Intel's IOH 3420 PCIe Root Port in an ARM machine, or a pxb on a Q35 machine (in this case we want pxb-pcie)
Such a device would be weird. But would it be wrong? Wrong enough for QEMU to reject it? Unless QEMU rejects it, there's no reason not to list it as pluggable.
I do believe there are other examples, I'll try to think of more.
Thanks, Marcel
[...]
On 11/24/2016 05:41 PM, Markus Armbruster wrote:
Marcel Apfelbaum <marcel@redhat.com> writes:
On 11/24/2016 03:34 PM, Markus Armbruster wrote:
Eduardo Habkost <ehabkost@redhat.com> writes:
On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote:
On 11/22/2016 03:11 AM, Eduardo Habkost wrote:
The Problem
[...]
Our decision to have hybrid PCI/PCIe devices and buses breeds considerable complexity. I wish we had avoided them, but I believe it's too late to change now.
This still does not solve the problem that some devices makes sense only on a specific arch.
Hi Markus,
Examples?
One quick example would be that we don't want to see Intel's IOH 3420 PCIe Root Port in an ARM machine, or a pxb on a Q35 machine (in this case we want pxb-pcie)
Such a device would be weird. But would it be wrong?
Define wrong :) Wrong enough for
QEMU to reject it?
QEMU accepts them and they even function correctly as far as I know. Unless QEMU rejects it, there's no reason not to
list it as pluggable.
This is the gray area I can't argue. I do think that Eduardo's work may present an opportunity to change QEMU's mantra: "everything goes as long as it works" to "here is what this configuration supports". Thanks, Marcel
I do believe there are other examples, I'll try to think of more.
Thanks, Marcel
[...]
Marcel Apfelbaum <marcel@redhat.com> writes:
On 11/24/2016 05:41 PM, Markus Armbruster wrote:
Marcel Apfelbaum <marcel@redhat.com> writes:
On 11/24/2016 03:34 PM, Markus Armbruster wrote:
Eduardo Habkost <ehabkost@redhat.com> writes:
On Wed, Nov 23, 2016 at 06:43:16PM +0200, Marcel Apfelbaum wrote:
On 11/22/2016 03:11 AM, Eduardo Habkost wrote: > The Problem
[...]
Our decision to have hybrid PCI/PCIe devices and buses breeds considerable complexity. I wish we had avoided them, but I believe it's too late to change now.
This still does not solve the problem that some devices makes sense only on a specific arch.
Hi Markus,
Examples?
One quick example would be that we don't want to see Intel's IOH 3420 PCIe Root Port in an ARM machine, or a pxb on a Q35 machine (in this case we want pxb-pcie)
Such a device would be weird. But would it be wrong?
Define wrong :)
I do:
Wrong enough for
QEMU to reject it?
QEMU accepts them and they even function correctly as far as I know.
Unless QEMU rejects it, there's no reason not to list it as pluggable.
This is the gray area I can't argue. I do think that Eduardo's work may present an opportunity to change QEMU's mantra: "everything goes as long as it works" to "here is what this configuration supports".
I guess you argument is that in reality, the devices you quoted are always part of specific chipsets, so "we dont want to see" them elsewhere. If I understand you correctly, there are two cases you don't want to see elsewhere: (1) The real PCI device only ever exists as function of another device. (2) The real PCI device only ever exists on certain boards. We accept these devices elsewhere due to the way we model them. Because we conflate PCI functions and devices, we can't model (1) correctly. I think the appropriate solution would be modelling functions separate from devices, then provide the functions in question only as part of the devices where you want to see them, by making them not user-pluggable. Because we model a board's chipset as a set of independent devices instead of a composite device, we can't model (2) correctly. I think the appropriate solution to (2) is modelling composite chipset devices, then provide the devices in question only as part of the chipset devices where you want to see them, by making them not user-pluggable. Adding a bunch of special types to QOM so that introspection claims "you can't plug that thing" (even though you actually could) would be an inappropriate solution. As long as you can plug them, QEMU should be honest about it, even if we think you shouldn't plug them. "Can't" is mechanism. "Shouldn't" is policy. Baking policy into introspection by making it lie doesn't strike me as a good idea. [...]
participants (6)
-
Cornelia Huck -
David Gibson -
David Hildenbrand -
Eduardo Habkost -
Marcel Apfelbaum -
Markus Armbruster