Re: [PATCH RFC V6 22/24] monitor,qdev: Introduce 'device_set' to change admin state of existing devices
On Thu, 09 Oct 2025 10:55:40 +0200 Markus Armbruster <armbru@redhat.com> wrote:
salil.mehta@opnsrc.net writes:
From: Salil Mehta <salil.mehta@huawei.com>
This patch adds a "device_set" interface for modifying properties of devices that already exist in the guest topology. Unlike 'device_add'/'device_del' (hot-plug), 'device_set' does not create or destroy devices. It is intended for guest-visible hot-add semantics where hardware is provisioned at boot but logically enabled/disabled later via administrative policy.
Compared to the existing 'qom-set' command, which is less intuitive and works only with object IDs, device_set provides a more device-oriented interface. It can be invoked at the QEMU prompt using natural device arguments, and the new '-deviceset' CLI option allows properties to be set at boot time, similar to how '-device' specifies device creation.
Why can't we use -device?
that's was my concern/suggestion in reply to cover letter (as a place to put high level review and what can be done for the next revision) (PS: It looks like I'm having email receiving issues (i.e. not getting from mail list my own emails that it bonces to me, so threading is all broken on my side and I'm might miss replies). But on positive side it looks like my replies reach the list and CCed just fine)
While the initial implementation focuses on "admin-state" changes (e.g., enable/disable a CPU already described by ACPI/DT), the interface is designed to be generic. In future, it could be used for other per-device set/unset style controls — beyond administrative power-states — provided the target device explicitly allows such changes. This enables fine-grained runtime control of device properties.
Beware, designing a generic interface can be harder, sometimes much harder, than designing a specialized one.
device_add and qom-set are generic, and they have issues:
* device_add effectively bypasses QAPI by using 'gen': false.
This bypasses QAPI's enforcement of documentation. Property documentation is separate and poor.
It also defeats introspection with query-qmp-schema. You need to resort to other means instead, say QOM introspection (which is a bag of design flaws on its own), then map from QOM to qdev.
* device_add lets you specify any qdev property, even properties that are intended only for use by C code.
This results in accidental external interfaces.
We tend to name properties like "x-prop" to discourage external use, but I wouldn't bet my own money on us getting that always right. Moreover, there's beauties like "x-origin".
* qom-set & friends effectively bypass QAPI by using type 'any'.
Again, the bypass results in poor documentation and a defeat of query-qmp-schema.
* qom-set lets you mess with any QOM property with a setter callback.
Again, accidental external interfaces: most of these properties are not meant for use with qom-set. For some, qom-set works, for some it silently does nothing, and for some it crashes. A lot more dangerous than device_add.
The "x-" convention can't help here: some properties are intended for external use with object-add, but not with qom-set.
We should avoid such issues in new interfaces.
We'll examine how this applies to device_set when I review the QAPI schema.
Key pieces: * QMP: qmp_device_set() to update an existing device. The device can be located by "id" or via driver+property match using a DeviceListener callback (qdev_find_device()). * HMP: "device_set" command with tab-completion. Errors are surfaced via hmp_handle_error(). * CLI: "-deviceset" option for setting startup/admin properties at boot, including a JSON form. Options are parsed into qemu_deviceset_opts and applied after device creation. * Docs/help: HMP help text and qemu-options.hx additions explain usage and explicitly note that no hot-plug occurs. * Safety: disallowed during live migration (migration_is_idle() check).
Semantics: * Operates on an existing DeviceState; no enumeration/new device appears. * Complements device_add/device_del by providing state mutation only. * Backward compatible: no behavior change unless "device_set"/"-deviceset" is used.
Examples: HMP: (qemu) device_set host-arm-cpu,core-id=3,admin-state=enable
CLI (at boot): -smp cpus=4,maxcpus=4 \ -deviceset host-arm-cpu,core-id=2,admin-state=disable
QMP (JSON form): { "execute": "device_set", "arguments": { "driver": "host-arm-cpu", "core-id": 1, "admin-state": "disable" } }
{"error": {"class": "CommandNotFound", "desc": "The command device_set has not been found"}}
Clue below.
NOTE: The qdev_enable()/qdev_disable() hooks for acting on admin-state will be added in subsequent patches. Device classes must explicitly support any property they want to expose through device_set.
Signed-off-by: Salil Mehta <salil.mehta@huawei.com> --- hmp-commands.hx | 30 +++++++++ hw/arm/virt.c | 86 +++++++++++++++++++++++++ hw/core/cpu-common.c | 12 ++++ hw/core/qdev.c | 21 ++++++ include/hw/arm/virt.h | 1 + include/hw/core/cpu.h | 11 ++++ include/hw/qdev-core.h | 22 +++++++ include/monitor/hmp.h | 2 + include/monitor/qdev.h | 30 +++++++++ include/system/system.h | 1 + qemu-options.hx | 51 +++++++++++++-- system/qdev-monitor.c | 139 +++++++++++++++++++++++++++++++++++++++- system/vl.c | 39 +++++++++++ 13 files changed, 440 insertions(+), 5 deletions(-)
Clue: no update to the QAPI schema, i.e. the QMP command does not exist.
diff --git a/hmp-commands.hx b/hmp-commands.hx index d0e4f35a30..18056cf21d 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -707,6 +707,36 @@ SRST or a QOM object path. ERST
+{ + .name = "device_set", + .args_type = "device:O", + .params = "driver[,prop=value][,...]", + .help = "set/unset existing device property", + .cmd = hmp_device_set, + .command_completion = device_set_completion, +}, + +SRST +``device_set`` *driver[,prop=value][,...]* + Change the administrative power state of an existing device. + + This command enables or disables a known device (e.g., CPU) using the + "device_set" interface. It does not hotplug or add a new device. + + Depending on platform support (e.g., PSCI or ACPI), this may trigger + corresponding operational changes — such as powering down a CPU or + transitioning it to active use. + + Administrative state: + * *enabled* — Allows the guest to use the device (e.g., CPU_ON) + * *disabled* — Prevents guest use; device is powered off (e.g., CPU_OFF) + + Note: The device must already exist (be declared during machine creation). + + Example: + (qemu) device_set host-arm-cpu,core-id=3,admin-state=disabled +ERST
How exactly is the device selected? You provide a clue above: 'can be located by "id" or via driver+property match'.
I assume by "id" is just like device_del, i.e. by qdev ID or QOM path.
By "driver+property match" is not obvious. Which of the arguments are for matching, and which are for setting?
If "id" is specified, is there any matching?
The matching feature complicates this interface quite a bit. I doubt it's worth the complexity. If you think it is, please split it off into a separate patch.
It's likely /me who to blame for asking to invent generic device-set QMP command. I see another application (beside ARM CPU power-on/off) for it, PCI devices to simulate powering on/off them at runtime without actually removing device. wrt command, I'd use only 'id' with it to identify target device (i.e. no template matching nor QMP path either). To enforce rule, what user hasn't named explicitly by providing 'id' isn't meant to be accessed/manged by user later on. potentially we can invent specialized power_set/get command as an alternative if it makes design easier. But then we would be spawning similar commands for other things, where as device-set would cover it all. But then I might be over-complicating things by suggesting a generic approach.
Next question. Is there a way for management applications to detect whether a certain device supports device_set for a certain property?
is there some kind of QMP command to check what does a device support, or at least what properties it supports? Can we piggy-back on that?
Without that, what are management application supposed to do? Hard-code what works? Run the command and see whether it fails?
Adding libvirt list to discussion and possible ideas on what can be done here.
I understand right now the command supports just "admin-state" for a certain set of devices, so hard-coding would be possible. But every new (device, property) pair then requires management application updates, and the hard-coded information becomes version specific. This will become unworkable real quick. Not good enough for a command designed to be generic.
+ { .name = "cpu", .args_type = "index:i",
We still do have a few legacy uses of cpu index (CLI|HMP), but I'd avoid using cpu index or something similar in new interfaces.
[...]
diff --git a/qemu-options.hx b/qemu-options.hx index 83ccde341b..f517b91042 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -375,7 +375,10 @@ SRST This is different from CPU hotplug where additional CPUs are not even present in the system description. Administratively disabled CPUs appear in ACPI tables i.e. are provisioned, but cannot be used until explicitly - enabled via QMP/HMP or the deviceset API. + enabled via QMP/HMP or the deviceset API. On ACPI guests, each vCPU counted + by 'disabledcpus=' is provisioned with '\ ``_STA``\ ' reporting Present=1 + and Enabled=0 (present-offline) at boot; it becomes Enabled=1 when brought + online via 'device_set ... admin-state=enable'.
On boards supporting CPU hotplug, the optional '\ ``maxcpus``\ ' parameter can be set to enable further CPUs to be added at runtime. When both @@ -455,6 +458,15 @@ SRST
-smp 2
+ Note: The cluster topology will only be generated in ACPI and exposed + to guest if it's explicitly specified in -smp. + + Note: Administratively disabled CPUs (specified via 'disabledcpus=' and + '-deviceset' at CLI during boot) are especially useful for platforms like + ARM that lack native CPU hotplug support. These CPUs will appear to the + guest as unavailable, and any attempt to bring them online must go through + QMP/HMP commands like 'device_set'. + Examples using 'disabledcpus':
For a board without CPU hotplug, enable 4 CPUs at boot and provision @@ -472,9 +484,6 @@ SRST ::
-smp cpus=4,disabledcpus=2,maxcpus=8 - - Note: The cluster topology will only be generated in ACPI and exposed - to guest if it's explicitly specified in -smp. ERST
DEF("numa", HAS_ARG, QEMU_OPTION_numa, @@ -1281,6 +1290,40 @@ SRST
ERST
+DEF("deviceset", HAS_ARG, QEMU_OPTION_deviceset, + "-deviceset driver[,prop[=value]][,...]\n" + " Set administrative power state of an existing device.\n" + " Does not hotplug a new device. Can disable or enable\n" + " devices (such as CPUs) at boot based on policy.\n" + " Example:\n" + " -deviceset host-arm-cpu,core-id=2,admin-state=disabled\n" + " Use '-deviceset help' for supported drivers\n" + " Use '-deviceset driver,help' for driver-specific properties\n", + QEMU_ARCH_ALL) +SRST +``-deviceset driver[,prop[=value]][,...]`` + Configure an existing device's administrative power state or properties. + + Unlike ``-device``, this option does not create a new device. Instead, + it sets startup properties (such as administrative power state) for + a device already declared via -smp or other machine configuration. + + Example: + -smp cpus=4 + -deviceset host-arm-cpu,core-id=2,admin-state=disabled + + The above disables CPU core 2 at boot using administrative offlining. + The guest may later re-enable the core (if permitted by platform policy). + + ``state=enabled|disabled`` + Sets the administrative state of the device: + - ``enabled``: device is made available at boot + - ``disabled``: device is administratively disabled and powered off + + Use ``-deviceset help`` to view all supported drivers. + Use ``-deviceset driver,help`` for property-specific help. +ERST + DEF("name", HAS_ARG, QEMU_OPTION_name, "-name string1[,process=string2][,debug-threads=on|off]\n" " set the name of the guest\n" diff --git a/system/qdev-monitor.c b/system/qdev-monitor.c index 2ac92d0a07..1099b1237d 100644 --- a/system/qdev-monitor.c +++ b/system/qdev-monitor.c @@ -263,12 +263,20 @@ static DeviceClass *qdev_get_device_class(const char **driver, Error **errp) }
dc = DEVICE_CLASS(oc); - if (!dc->user_creatable) { + if (!dc->user_creatable && !dc->admin_power_state_supported) { error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "driver", "a pluggable device type"); return NULL; }
+ if (phase_check(PHASE_MACHINE_READY) && + (!dc->hotpluggable || !dc->admin_power_state_supported)) { + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "driver", + "a pluggable device type or which supports changing power-" + "state administratively"); + return NULL; + } + if (object_class_dynamic_cast(oc, TYPE_SYS_BUS_DEVICE)) { /* sysbus devices need to be allowed by the machine */ MachineClass *mc = MACHINE_CLASS(object_get_class(qdev_get_machine())); @@ -939,6 +947,76 @@ void qmp_device_del(const char *id, Error **errp) } }
+void qmp_device_set(const QDict *qdict, Error **errp) +{ + const char *state; + const char *driver; + DeviceState *dev; + DeviceClass *dc; + const char *id; + + driver = qdict_get_try_str(qdict, "driver"); + if (!driver) { + error_setg(errp, "Parameter 'driver' is missing"); + return; + } + + /* check driver exists and we are at the right phase of machine init */ + dc = qdev_get_device_class(&driver, errp); + if (!dc) {
Since qdev_get_device_class() sets an error when it fails, *errp is not null here, ...
+ error_setg(errp, "driver '%s' not supported", driver);
... which makes this wrong. Caught by error_setv()'s assertion.
Please test your error paths.
+ return; + } + + if (migration_is_running()) { + error_setg(errp, "device_set not allowed while migrating"); + return; + } + + id = qdict_get_try_str(qdict, "id"); + + if (id) { + /* Lookup by ID */ + dev = find_device_state(id, false, errp); + if (errp && *errp) { + error_prepend(errp, "Device lookup failed for ID '%s': ", id); + return; + } + } else { + /* Lookup using driver and properties */ + dev = qdev_find_device(qdict, errp); + if (errp && *errp) { + error_prepend(errp, "Device lookup for %s failed: ", driver); + return; + } + } + if (!dev) { + error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND, + "No device found for driver '%s'", driver); + return; + } + + state = qdict_get_try_str(qdict, "admin-state"); + if (!state) { + error_setg(errp, "no device state change specified for device %s ", + dev->id); + return; + } else if (!strcmp(state, "enable")) { + + if (!qdev_enable(dev, qdev_get_parent_bus(DEVICE(dev)), errp)) { + return; + } + } else if (!strcmp(state, "disable")) { + if (!qdev_disable(dev, qdev_get_parent_bus(DEVICE(dev)), errp)) { + return; + } + } else { + error_setg(errp, "unrecognized specified state *%s* for device %s", + state, dev->id); + return; + } +} + int qdev_sync_config(DeviceState *dev, Error **errp) { DeviceClass *dc = DEVICE_GET_CLASS(dev);
[...]
On Thu, Oct 09, 2025 at 02:51:25PM +0200, Igor Mammedov via Devel wrote:
On Thu, 09 Oct 2025 10:55:40 +0200 Markus Armbruster <armbru@redhat.com> wrote:
salil.mehta@opnsrc.net writes:
From: Salil Mehta <salil.mehta@huawei.com>
This patch adds a "device_set" interface for modifying properties of devices that already exist in the guest topology. Unlike 'device_add'/'device_del' (hot-plug), 'device_set' does not create or destroy devices. It is intended for guest-visible hot-add semantics where hardware is provisioned at boot but logically enabled/disabled later via administrative policy.
Compared to the existing 'qom-set' command, which is less intuitive and works only with object IDs, device_set provides a more device-oriented interface. It can be invoked at the QEMU prompt using natural device arguments, and the new '-deviceset' CLI option allows properties to be set at boot time, similar to how '-device' specifies device creation.
Why can't we use -device?
that's was my concern/suggestion in reply to cover letter (as a place to put high level review and what can be done for the next revision)
(PS: It looks like I'm having email receiving issues (i.e. not getting from mail list my own emails that it bonces to me, so threading is all broken on my side and I'm might miss replies). But on positive side it looks like my replies reach the list and CCed just fine)
While the initial implementation focuses on "admin-state" changes (e.g., enable/disable a CPU already described by ACPI/DT), the interface is designed to be generic. In future, it could be used for other per-device set/unset style controls — beyond administrative power-states — provided the target device explicitly allows such changes. This enables fine-grained runtime control of device properties.
Beware, designing a generic interface can be harder, sometimes much harder, than designing a specialized one.
device_add and qom-set are generic, and they have issues:
* device_add effectively bypasses QAPI by using 'gen': false.
This bypasses QAPI's enforcement of documentation. Property documentation is separate and poor.
It also defeats introspection with query-qmp-schema. You need to resort to other means instead, say QOM introspection (which is a bag of design flaws on its own), then map from QOM to qdev.
* device_add lets you specify any qdev property, even properties that are intended only for use by C code.
This results in accidental external interfaces.
We tend to name properties like "x-prop" to discourage external use, but I wouldn't bet my own money on us getting that always right. Moreover, there's beauties like "x-origin".
* qom-set & friends effectively bypass QAPI by using type 'any'.
Again, the bypass results in poor documentation and a defeat of query-qmp-schema.
* qom-set lets you mess with any QOM property with a setter callback.
Again, accidental external interfaces: most of these properties are not meant for use with qom-set. For some, qom-set works, for some it silently does nothing, and for some it crashes. A lot more dangerous than device_add.
The "x-" convention can't help here: some properties are intended for external use with object-add, but not with qom-set.
We should avoid such issues in new interfaces.
We'll examine how this applies to device_set when I review the QAPI schema.
Key pieces: * QMP: qmp_device_set() to update an existing device. The device can be located by "id" or via driver+property match using a DeviceListener callback (qdev_find_device()). * HMP: "device_set" command with tab-completion. Errors are surfaced via hmp_handle_error(). * CLI: "-deviceset" option for setting startup/admin properties at boot, including a JSON form. Options are parsed into qemu_deviceset_opts and applied after device creation. * Docs/help: HMP help text and qemu-options.hx additions explain usage and explicitly note that no hot-plug occurs. * Safety: disallowed during live migration (migration_is_idle() check).
Semantics: * Operates on an existing DeviceState; no enumeration/new device appears. * Complements device_add/device_del by providing state mutation only. * Backward compatible: no behavior change unless "device_set"/"-deviceset" is used.
Examples: HMP: (qemu) device_set host-arm-cpu,core-id=3,admin-state=enable
CLI (at boot): -smp cpus=4,maxcpus=4 \ -deviceset host-arm-cpu,core-id=2,admin-state=disable
QMP (JSON form): { "execute": "device_set", "arguments": { "driver": "host-arm-cpu", "core-id": 1, "admin-state": "disable" } }
{"error": {"class": "CommandNotFound", "desc": "The command device_set has not been found"}}
Clue below.
NOTE: The qdev_enable()/qdev_disable() hooks for acting on admin-state will be added in subsequent patches. Device classes must explicitly support any property they want to expose through device_set.
Signed-off-by: Salil Mehta <salil.mehta@huawei.com> --- hmp-commands.hx | 30 +++++++++ hw/arm/virt.c | 86 +++++++++++++++++++++++++ hw/core/cpu-common.c | 12 ++++ hw/core/qdev.c | 21 ++++++ include/hw/arm/virt.h | 1 + include/hw/core/cpu.h | 11 ++++ include/hw/qdev-core.h | 22 +++++++ include/monitor/hmp.h | 2 + include/monitor/qdev.h | 30 +++++++++ include/system/system.h | 1 + qemu-options.hx | 51 +++++++++++++-- system/qdev-monitor.c | 139 +++++++++++++++++++++++++++++++++++++++- system/vl.c | 39 +++++++++++ 13 files changed, 440 insertions(+), 5 deletions(-)
Clue: no update to the QAPI schema, i.e. the QMP command does not exist.
On that point... No new pure HMP commands please. We consider implementation of the QMP command to be the mandatory first step in any patch series. Any HMP command must follow and must be implemented by calling the QMP command handler.
diff --git a/hmp-commands.hx b/hmp-commands.hx index d0e4f35a30..18056cf21d 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -707,6 +707,36 @@ SRST or a QOM object path. ERST
+{ + .name = "device_set", + .args_type = "device:O", + .params = "driver[,prop=value][,...]", + .help = "set/unset existing device property", + .cmd = hmp_device_set, + .command_completion = device_set_completion, +}, + +SRST +``device_set`` *driver[,prop=value][,...]* + Change the administrative power state of an existing device. + + This command enables or disables a known device (e.g., CPU) using the + "device_set" interface. It does not hotplug or add a new device. + + Depending on platform support (e.g., PSCI or ACPI), this may trigger + corresponding operational changes — such as powering down a CPU or + transitioning it to active use. + + Administrative state: + * *enabled* — Allows the guest to use the device (e.g., CPU_ON) + * *disabled* — Prevents guest use; device is powered off (e.g., CPU_OFF) + + Note: The device must already exist (be declared during machine creation). + + Example: + (qemu) device_set host-arm-cpu,core-id=3,admin-state=disabled +ERST
How exactly is the device selected? You provide a clue above: 'can be located by "id" or via driver+property match'.
I assume by "id" is just like device_del, i.e. by qdev ID or QOM path.
By "driver+property match" is not obvious. Which of the arguments are for matching, and which are for setting?
If "id" is specified, is there any matching?
The matching feature complicates this interface quite a bit. I doubt it's worth the complexity. If you think it is, please split it off into a separate patch.
It's likely /me who to blame for asking to invent generic device-set QMP command. I see another application (beside ARM CPU power-on/off) for it, PCI devices to simulate powering on/off them at runtime without actually removing device.
wrt command, I'd use only 'id' with it to identify target device (i.e. no template matching nor QMP path either). To enforce rule, what user hasn't named explicitly by providing 'id' isn't meant to be accessed/manged by user later on.
potentially we can invent specialized power_set/get command as an alternative if it makes design easier. But then we would be spawning similar commands for other things, where as device-set would cover it all. But then I might be over-complicating things by suggesting a generic approach.
The generic set/get design feels convenient because you don't need to create new commands, but it has significant downsides both for QEMU and the users of QEMU. From a QEMU POV the main burden is that we loose understanding of how users of QEMU are consuming our interface / functionality at a conceptual level and at the low level. This in turns means we either struggle to offer a stable API, or our hands are tied behind our back for future changes. Consumers of QEMU are similarly exposed to the raw low level details which has many downsides * If QEMU ever changes impl, but retains the conceptual functionality, apps are broken. * If a given feature is more complex than a single property, apps will be invoking a whole set of commands to set many props to achieve a given task. * If certain sequences of prop changes are needed, apps have no guidance on the ordering dependancies - which might even change between QEMU versions * If setting one prop fails, apps may need to manually rollback previous prop changes they made * The schema is unable to describe what functionality is now available since device properties are invisible. * If two devices expose the same functionality, but via different properties, apps have inconsistent interfaces I'm highly sceptical that exposing 'device_set' is a good idea.
Next question. Is there a way for management applications to detect whether a certain device supports device_set for a certain property?
is there some kind of QMP command to check what does a device support, or at least what properties it supports? Can we piggy-back on that?
Note, querying whether a device supports a property is conceptually quite different from querying whether QEMU supports a given operation, because it requires apps to first connect the dots between the low level property change, and the conceptual effect the want to produce. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Igor Mammedov <imammedo@redhat.com> writes:
On Thu, 09 Oct 2025 10:55:40 +0200 Markus Armbruster <armbru@redhat.com> wrote:
salil.mehta@opnsrc.net writes:
From: Salil Mehta <salil.mehta@huawei.com>
This patch adds a "device_set" interface for modifying properties of devices that already exist in the guest topology. Unlike 'device_add'/'device_del' (hot-plug), 'device_set' does not create or destroy devices. It is intended for guest-visible hot-add semantics where hardware is provisioned at boot but logically enabled/disabled later via administrative policy.
Compared to the existing 'qom-set' command, which is less intuitive and works only with object IDs, device_set provides a more device-oriented interface. It can be invoked at the QEMU prompt using natural device arguments, and the new '-deviceset' CLI option allows properties to be set at boot time, similar to how '-device' specifies device creation.
Why can't we use -device?
that's was my concern/suggestion in reply to cover letter (as a place to put high level review and what can be done for the next revision)
Yes.
(PS: It looks like I'm having email receiving issues (i.e. not getting from mail list my own emails that it bonces to me, so threading is all broken on my side and I'm might miss replies). But on positive side it looks like my replies reach the list and CCed just fine)
For what it's worth, your replies arrive fine here.
While the initial implementation focuses on "admin-state" changes (e.g., enable/disable a CPU already described by ACPI/DT), the interface is designed to be generic. In future, it could be used for other per-device set/unset style controls — beyond administrative power-states — provided the target device explicitly allows such changes. This enables fine-grained runtime control of device properties.
Beware, designing a generic interface can be harder, sometimes much harder, than designing a specialized one.
device_add and qom-set are generic, and they have issues:
* device_add effectively bypasses QAPI by using 'gen': false.
This bypasses QAPI's enforcement of documentation. Property documentation is separate and poor.
It also defeats introspection with query-qmp-schema. You need to resort to other means instead, say QOM introspection (which is a bag of design flaws on its own), then map from QOM to qdev.
* device_add lets you specify any qdev property, even properties that are intended only for use by C code.
This results in accidental external interfaces.
We tend to name properties like "x-prop" to discourage external use, but I wouldn't bet my own money on us getting that always right. Moreover, there's beauties like "x-origin".
* qom-set & friends effectively bypass QAPI by using type 'any'.
Again, the bypass results in poor documentation and a defeat of query-qmp-schema.
* qom-set lets you mess with any QOM property with a setter callback.
Again, accidental external interfaces: most of these properties are not meant for use with qom-set. For some, qom-set works, for some it silently does nothing, and for some it crashes. A lot more dangerous than device_add.
The "x-" convention can't help here: some properties are intended for external use with object-add, but not with qom-set.
We should avoid such issues in new interfaces.
[...]
diff --git a/hmp-commands.hx b/hmp-commands.hx index d0e4f35a30..18056cf21d 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -707,6 +707,36 @@ SRST or a QOM object path. ERST
+{ + .name = "device_set", + .args_type = "device:O", + .params = "driver[,prop=value][,...]", + .help = "set/unset existing device property", + .cmd = hmp_device_set, + .command_completion = device_set_completion, +}, + +SRST +``device_set`` *driver[,prop=value][,...]* + Change the administrative power state of an existing device. + + This command enables or disables a known device (e.g., CPU) using the + "device_set" interface. It does not hotplug or add a new device. + + Depending on platform support (e.g., PSCI or ACPI), this may trigger + corresponding operational changes — such as powering down a CPU or + transitioning it to active use. + + Administrative state: + * *enabled* — Allows the guest to use the device (e.g., CPU_ON) + * *disabled* — Prevents guest use; device is powered off (e.g., CPU_OFF) + + Note: The device must already exist (be declared during machine creation). + + Example: + (qemu) device_set host-arm-cpu,core-id=3,admin-state=disabled +ERST
How exactly is the device selected? You provide a clue above: 'can be located by "id" or via driver+property match'.
I assume by "id" is just like device_del, i.e. by qdev ID or QOM path.
By "driver+property match" is not obvious. Which of the arguments are for matching, and which are for setting?
If "id" is specified, is there any matching?
The matching feature complicates this interface quite a bit. I doubt it's worth the complexity. If you think it is, please split it off into a separate patch.
It's likely /me who to blame for asking to invent generic device-set QMP command. I see another application (beside ARM CPU power-on/off) for it, PCI devices to simulate powering on/off them at runtime without actually removing device.
I prefer generic commands over collecting ad hoc single-purpose commands, too. Getting the design right can be difficult.
wrt command, I'd use only 'id' with it to identify target device (i.e. no template matching nor QMP path either). To enforce rule, what user hasn't named explicitly by providing 'id' isn't meant to be accessed/manged by user later on.
Works well, except when we need to access / manage onboard devices. That's still an unsolved problem.
potentially we can invent specialized power_set/get command as an alternative if it makes design easier. But then we would be spawning similar commands for other things, where as device-set would cover it all. But then I might be over-complicating things by suggesting a generic approach.
Unclear. I feel it's best to start the design process with ensvisaged uses. Can you tell me a bit more about the uses you have in mind?
Next question. Is there a way for management applications to detect whether a certain device supports device_set for a certain property?
is there some kind of QMP command to check what does a device support, or at least what properties it supports? Can we piggy-back on that?
Maybe. QAPI schema introspection (query-qmp-schema) has been a success. It has a reasonably expressive type system, deprecation information, and hides much implementation detail. Sadly, it doesn't cover most of QOM and all of qdev due to QAPI schema bypass. QOM type introspection (qom-list-types and qom-list-properties) is weak. You can retrieve a property's name and type. The latter is seriously underspecified, and somewhere between annoying and impossible to use reliably. Properties created in certain ways are not visible here. These are rare. QOM object introspection (qom-list) is the same for concrete objects rather than types. qdev introspection (device-list-properties) is like QOM type introspection. I'm not sure why it exists. Use QOM type introspection instead. QOM introspection is servicable for checking whether a certain property exists. Examining a property's type is unadvisable.
Without that, what are management application supposed to do? Hard-code what works? Run the command and see whether it fails?
Adding libvirt list to discussion and possible ideas on what can be done here.
I understand right now the command supports just "admin-state" for a certain set of devices, so hard-coding would be possible. But every new (device, property) pair then requires management application updates, and the hard-coded information becomes version specific. This will become unworkable real quick. Not good enough for a command designed to be generic.
[...]
On Thu, 9 Oct 2025 at 15:56, Markus Armbruster <armbru@redhat.com> wrote:
qdev introspection (device-list-properties) is like QOM type introspection. I'm not sure why it exists.
It exists because it is the older of the two interfaces: device-list-properties was added in 2012, whereas qom-list-properties was only added in 2018. device-list-properties also does some device-specific sanitization that may or may not be helpful: it won't let you try it on an abstract base class, for instance, and it won't list "legacy-" properties. One problem you don't mention with QOM introspection is that we have no marking for whether properties are intended to be user-facing knobs, configurable things to be set by other parts of QEMU, or purely details of the implementation. thanks -- PMM
Peter Maydell <peter.maydell@linaro.org> writes:
On Thu, 9 Oct 2025 at 15:56, Markus Armbruster <armbru@redhat.com> wrote:
qdev introspection (device-list-properties) is like QOM type introspection. I'm not sure why it exists.
It exists because it is the older of the two interfaces: device-list-properties was added in 2012, whereas qom-list-properties was only added in 2018.
I suspected it was, but didn't want to make unchecked claims. Thanks for checking!
device-list-properties also does some device-specific sanitization that may or may not be helpful: it won't let you try it on an abstract base class, for instance,
Introspecting abstract bases is probably not useful. But what harm could it do? Can't see why preventing it is worth the bother. Of course, changing it now is not worth the bother, either :)
and it won't list "legacy-" properties.
I remember these exist, but not what they're good for :) Should we deprecate device-list-properties in favour of qom-list-properties?
One problem you don't mention with QOM introspection is that we have no marking for whether properties are intended to be user-facing knobs, configurable things to be set by other parts of QEMU, or purely details of the implementation.
Yes. This is what I had in mind when I pointed out "accidental external interfaces".
On Thu, 09 Oct 2025 16:55:54 +0200 Markus Armbruster <armbru@redhat.com> wrote:
Igor Mammedov <imammedo@redhat.com> writes:
On Thu, 09 Oct 2025 10:55:40 +0200 Markus Armbruster <armbru@redhat.com> wrote:
salil.mehta@opnsrc.net writes:
From: Salil Mehta <salil.mehta@huawei.com>
This patch adds a "device_set" interface for modifying properties of devices that already exist in the guest topology. Unlike 'device_add'/'device_del' (hot-plug), 'device_set' does not create or destroy devices. It is intended for guest-visible hot-add semantics where hardware is provisioned at boot but logically enabled/disabled later via administrative policy.
Compared to the existing 'qom-set' command, which is less intuitive and works only with object IDs, device_set provides a more device-oriented interface. It can be invoked at the QEMU prompt using natural device arguments, and the new '-deviceset' CLI option allows properties to be set at boot time, similar to how '-device' specifies device creation.
Why can't we use -device?
that's was my concern/suggestion in reply to cover letter (as a place to put high level review and what can be done for the next revision)
Yes.
(PS: It looks like I'm having email receiving issues (i.e. not getting from mail list my own emails that it bonces to me, so threading is all broken on my side and I'm might miss replies). But on positive side it looks like my replies reach the list and CCed just fine)
For what it's worth, your replies arrive fine here.
While the initial implementation focuses on "admin-state" changes (e.g., enable/disable a CPU already described by ACPI/DT), the interface is designed to be generic. In future, it could be used for other per-device set/unset style controls — beyond administrative power-states — provided the target device explicitly allows such changes. This enables fine-grained runtime control of device properties.
Beware, designing a generic interface can be harder, sometimes much harder, than designing a specialized one.
device_add and qom-set are generic, and they have issues:
* device_add effectively bypasses QAPI by using 'gen': false.
This bypasses QAPI's enforcement of documentation. Property documentation is separate and poor.
It also defeats introspection with query-qmp-schema. You need to resort to other means instead, say QOM introspection (which is a bag of design flaws on its own), then map from QOM to qdev.
* device_add lets you specify any qdev property, even properties that are intended only for use by C code.
This results in accidental external interfaces.
We tend to name properties like "x-prop" to discourage external use, but I wouldn't bet my own money on us getting that always right. Moreover, there's beauties like "x-origin".
* qom-set & friends effectively bypass QAPI by using type 'any'.
Again, the bypass results in poor documentation and a defeat of query-qmp-schema.
* qom-set lets you mess with any QOM property with a setter callback.
Again, accidental external interfaces: most of these properties are not meant for use with qom-set. For some, qom-set works, for some it silently does nothing, and for some it crashes. A lot more dangerous than device_add.
The "x-" convention can't help here: some properties are intended for external use with object-add, but not with qom-set.
We should avoid such issues in new interfaces.
[...]
diff --git a/hmp-commands.hx b/hmp-commands.hx index d0e4f35a30..18056cf21d 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -707,6 +707,36 @@ SRST or a QOM object path. ERST
+{ + .name = "device_set", + .args_type = "device:O", + .params = "driver[,prop=value][,...]", + .help = "set/unset existing device property", + .cmd = hmp_device_set, + .command_completion = device_set_completion, +}, + +SRST +``device_set`` *driver[,prop=value][,...]* + Change the administrative power state of an existing device. + + This command enables or disables a known device (e.g., CPU) using the + "device_set" interface. It does not hotplug or add a new device. + + Depending on platform support (e.g., PSCI or ACPI), this may trigger + corresponding operational changes — such as powering down a CPU or + transitioning it to active use. + + Administrative state: + * *enabled* — Allows the guest to use the device (e.g., CPU_ON) + * *disabled* — Prevents guest use; device is powered off (e.g., CPU_OFF) + + Note: The device must already exist (be declared during machine creation). + + Example: + (qemu) device_set host-arm-cpu,core-id=3,admin-state=disabled +ERST
How exactly is the device selected? You provide a clue above: 'can be located by "id" or via driver+property match'.
I assume by "id" is just like device_del, i.e. by qdev ID or QOM path.
By "driver+property match" is not obvious. Which of the arguments are for matching, and which are for setting?
If "id" is specified, is there any matching?
The matching feature complicates this interface quite a bit. I doubt it's worth the complexity. If you think it is, please split it off into a separate patch.
It's likely /me who to blame for asking to invent generic device-set QMP command. I see another application (beside ARM CPU power-on/off) for it, PCI devices to simulate powering on/off them at runtime without actually removing device.
I prefer generic commands over collecting ad hoc single-purpose commands, too. Getting the design right can be difficult.
wrt command, I'd use only 'id' with it to identify target device (i.e. no template matching nor QMP path either). To enforce rule, what user hasn't named explicitly by providing 'id' isn't meant to be accessed/manged by user later on.
Works well, except when we need to access / manage onboard devices. That's still an unsolved problem.
potentially we can invent specialized power_set/get command as an alternative if it makes design easier. But then we would be spawning similar commands for other things, where as device-set would cover it all. But then I might be over-complicating things by suggesting a generic approach.
Unclear.
I feel it's best to start the design process with ensvisaged uses. Can you tell me a bit more about the uses you have in mind?
We have nic failover 'feature' https://www.qemu.org/docs/master/system/virtio-net-failover.html to make it work we do abuse hotplug and that poses problem during migration, since: - unplugging primary device releases resources (which might not be possible to claim back in case migration failure) - it's similar on destination side, where attempt to hotplug primary might fail die to insufficient resources leaving guest on 'degraded' virtio-net link. Idea was that instead of hotplug we can power off primary device, (it will still exist and keep resources), initiate migration, and then on target do the same starting with primary fully realized but powered of (and failing migration early if it can't claim resources, safely resuming QEMU on source incl. primary link), and then guest failover driver on destination would power primary on as part of switching to primary link. Above would require -device/device_add support for specifying device's power state as minimum.
Next question. Is there a way for management applications to detect whether a certain device supports device_set for a certain property?
is there some kind of QMP command to check what does a device support, or at least what properties it supports? Can we piggy-back on that?
Maybe.
QAPI schema introspection (query-qmp-schema) has been a success. It has a reasonably expressive type system, deprecation information, and hides much implementation detail. Sadly, it doesn't cover most of QOM and all of qdev due to QAPI schema bypass.
QOM type introspection (qom-list-types and qom-list-properties) is weak. You can retrieve a property's name and type. The latter is seriously underspecified, and somewhere between annoying and impossible to use reliably. Properties created in certain ways are not visible here. These are rare.
QOM object introspection (qom-list) is the same for concrete objects rather than types.
qdev introspection (device-list-properties) is like QOM type introspection. I'm not sure why it exists. Use QOM type introspection instead.
QOM introspection is servicable for checking whether a certain property exists. Examining a property's type is unadvisable.
Without that, what are management application supposed to do? Hard-code what works? Run the command and see whether it fails?
Adding libvirt list to discussion and possible ideas on what can be done here.
I understand right now the command supports just "admin-state" for a certain set of devices, so hard-coding would be possible. But every new (device, property) pair then requires management application updates, and the hard-coded information becomes version specific. This will become unworkable real quick. Not good enough for a command designed to be generic.
[...]
Igor Mammedov <imammedo@redhat.com> writes:
On Thu, 09 Oct 2025 16:55:54 +0200 Markus Armbruster <armbru@redhat.com> wrote:
Igor Mammedov <imammedo@redhat.com> writes:
[...]
It's likely /me who to blame for asking to invent generic device-set QMP command. I see another application (beside ARM CPU power-on/off) for it, PCI devices to simulate powering on/off them at runtime without actually removing device.
I prefer generic commands over collecting ad hoc single-purpose commands, too. Getting the design right can be difficult.
wrt command, I'd use only 'id' with it to identify target device (i.e. no template matching nor QMP path either). To enforce rule, what user hasn't named explicitly by providing 'id' isn't meant to be accessed/manged by user later on.
Works well, except when we need to access / manage onboard devices. That's still an unsolved problem.
potentially we can invent specialized power_set/get command as an alternative if it makes design easier. But then we would be spawning similar commands for other things, where as device-set would cover it all. But then I might be over-complicating things by suggesting a generic approach.
Unclear.
I feel it's best to start the design process with ensvisaged uses. Can you tell me a bit more about the uses you have in mind?
We have nic failover 'feature' https://www.qemu.org/docs/master/system/virtio-net-failover.html to make it work we do abuse hotplug and that poses problem during migration, since: - unplugging primary device releases resources (which might not be possible to claim back in case migration failure)
Serious reliability issue with no work-around.
- it's similar on destination side, where attempt to hotplug primary might fail die to insufficient resources leaving guest on 'degraded' virtio-net link.
Obvious work-around is failing the migration. Same as we do when we can't create devices.
Idea was that instead of hotplug we can power off primary device, (it will still exist and keep resources), initiate migration, and then on target do the same starting with primary fully realized but powered of (and failing migration early if it can't claim resources, safely resuming QEMU on source incl. primary link), and then guest failover driver on destination would power primary on as part of switching to primary link.
I can see how power on / off makes more sense than hot plug / unplug.
Above would require -device/device_add support for specifying device's power state as minimum.
The obvious way to control a device's power state with -device / device_add is a qdev property. Easy enough. Do we need to control a device's power state after it's created? If I understand your use case correctly, the answer is yes. -device / device_add can't do that. qom-set could, but friends don't let friends use it in production. Any other prior art for controlling device state at run time via QMP? [...]
On Mon, 20 Oct 2025 13:22:08 +0200 Markus Armbruster <armbru@redhat.com> wrote:
Igor Mammedov <imammedo@redhat.com> writes:
On Thu, 09 Oct 2025 16:55:54 +0200 Markus Armbruster <armbru@redhat.com> wrote:
Igor Mammedov <imammedo@redhat.com> writes:
[...]
It's likely /me who to blame for asking to invent generic device-set QMP command. I see another application (beside ARM CPU power-on/off) for it, PCI devices to simulate powering on/off them at runtime without actually removing device.
I prefer generic commands over collecting ad hoc single-purpose commands, too. Getting the design right can be difficult.
wrt command, I'd use only 'id' with it to identify target device (i.e. no template matching nor QMP path either). To enforce rule, what user hasn't named explicitly by providing 'id' isn't meant to be accessed/manged by user later on.
Works well, except when we need to access / manage onboard devices. That's still an unsolved problem.
potentially we can invent specialized power_set/get command as an alternative if it makes design easier. But then we would be spawning similar commands for other things, where as device-set would cover it all. But then I might be over-complicating things by suggesting a generic approach.
Unclear.
I feel it's best to start the design process with ensvisaged uses. Can you tell me a bit more about the uses you have in mind?
We have nic failover 'feature' https://www.qemu.org/docs/master/system/virtio-net-failover.html to make it work we do abuse hotplug and that poses problem during migration, since: - unplugging primary device releases resources (which might not be possible to claim back in case migration failure)
Serious reliability issue with no work-around.
- it's similar on destination side, where attempt to hotplug primary might fail die to insufficient resources leaving guest on 'degraded' virtio-net link.
Obvious work-around is failing the migration. Same as we do when we can't create devices.
Idea was that instead of hotplug we can power off primary device, (it will still exist and keep resources), initiate migration, and then on target do the same starting with primary fully realized but powered of (and failing migration early if it can't claim resources, safely resuming QEMU on source incl. primary link), and then guest failover driver on destination would power primary on as part of switching to primary link.
I can see how power on / off makes more sense than hot plug / unplug.
Above would require -device/device_add support for specifying device's power state as minimum.
The obvious way to control a device's power state with -device / device_add is a qdev property. Easy enough.
Do we need to control a device's power state after it's created? If I understand your use case correctly, the answer is yes. -device / device_add can't do that.
Could you elaborate why why -device/device_add can't do that?
qom-set could, but friends don't let friends use it in production.
Any other prior art for controlling device state at run time via QMP?
[...]
Igor Mammedov <imammedo@redhat.com> writes:
On Mon, 20 Oct 2025 13:22:08 +0200 Markus Armbruster <armbru@redhat.com> wrote:
Igor Mammedov <imammedo@redhat.com> writes:
On Thu, 09 Oct 2025 16:55:54 +0200 Markus Armbruster <armbru@redhat.com> wrote:
[...]
I feel it's best to start the design process with ensvisaged uses. Can you tell me a bit more about the uses you have in mind?
We have nic failover 'feature' https://www.qemu.org/docs/master/system/virtio-net-failover.html to make it work we do abuse hotplug and that poses problem during migration, since: - unplugging primary device releases resources (which might not be possible to claim back in case migration failure)
Serious reliability issue with no work-around.
- it's similar on destination side, where attempt to hotplug primary might fail die to insufficient resources leaving guest on 'degraded' virtio-net link.
Obvious work-around is failing the migration. Same as we do when we can't create devices.
Idea was that instead of hotplug we can power off primary device, (it will still exist and keep resources), initiate migration, and then on target do the same starting with primary fully realized but powered of (and failing migration early if it can't claim resources, safely resuming QEMU on source incl. primary link), and then guest failover driver on destination would power primary on as part of switching to primary link.
I can see how power on / off makes more sense than hot plug / unplug.
Above would require -device/device_add support for specifying device's power state as minimum.
The obvious way to control a device's power state with -device / device_add is a qdev property. Easy enough.
Do we need to control a device's power state after it's created? If I understand your use case correctly, the answer is yes. -device / device_add can't do that.
Could you elaborate why why -device/device_add can't do that?
-device / device_add create, configure, and realize a new device. They can't reconfigure an existing device. In particular, they can't be used to control an existing device's power state.
qom-set could, but friends don't let friends use it in production.
Any other prior art for controlling device state at run time via QMP?
[...]
On Wed, 29 Oct 2025 12:38:02 +0100 Markus Armbruster <armbru@redhat.com> wrote:
Igor Mammedov <imammedo@redhat.com> writes:
On Mon, 20 Oct 2025 13:22:08 +0200 Markus Armbruster <armbru@redhat.com> wrote:
Igor Mammedov <imammedo@redhat.com> writes:
On Thu, 09 Oct 2025 16:55:54 +0200 Markus Armbruster <armbru@redhat.com> wrote:
[...]
I feel it's best to start the design process with ensvisaged uses. Can you tell me a bit more about the uses you have in mind?
We have nic failover 'feature' https://www.qemu.org/docs/master/system/virtio-net-failover.html to make it work we do abuse hotplug and that poses problem during migration, since: - unplugging primary device releases resources (which might not be possible to claim back in case migration failure)
Serious reliability issue with no work-around.
- it's similar on destination side, where attempt to hotplug primary might fail die to insufficient resources leaving guest on 'degraded' virtio-net link.
Obvious work-around is failing the migration. Same as we do when we can't create devices.
Idea was that instead of hotplug we can power off primary device, (it will still exist and keep resources), initiate migration, and then on target do the same starting with primary fully realized but powered of (and failing migration early if it can't claim resources, safely resuming QEMU on source incl. primary link), and then guest failover driver on destination would power primary on as part of switching to primary link.
I can see how power on / off makes more sense than hot plug / unplug.
Above would require -device/device_add support for specifying device's power state as minimum.
The obvious way to control a device's power state with -device / device_add is a qdev property. Easy enough.
Do we need to control a device's power state after it's created? If I understand your use case correctly, the answer is yes. -device / device_add can't do that.
Could you elaborate why why -device/device_add can't do that?
-device / device_add create, configure, and realize a new device.
They can't reconfigure an existing device. In particular, they can't be used to control an existing device's power state.
Sorry, I've misread as we can't use both for creating device in powered off state. Perhaps we should consider a new specialized QMP command to manipulate runtime power state. (Like it was suggested by Daniel)
qom-set could, but friends don't let friends use it in production.
Any other prior art for controlling device state at run time via QMP?
[...]
participants (4)
-
Daniel P. Berrangé -
Igor Mammedov -
Markus Armbruster -
Peter Maydell