> QAPI/QMP introspection has all commands and events, and all types > reachable from them. query-qmp-schema returns an array, where each > array element describes one command, event, or type. When a command, > event, or type is conditional in the schema, the element is wrapped in > the #if generated for the condition. > After reading and answering to your valuable email, I definitely think the introspection schema we expose should be adapted, independently of how we build QAPI code (i.e. using #ifdef TARGET or not). Is it something technically hard to achieve?

Trouble is some uses of the second kind are in QAPI conditionals. I can see three options: (1) Drop these conditionals. (2) Replace them by run-time checks. (3) Have target-specific QAPI-generated code for multiple targets coexist in the single binary. As far as I can tell, your RFC series is an incomplete attempt at (2). I gather you considered (3), but you dislike it for its bloat and possibly other reasons. I sympathize; the QAPI-generated code is plenty bloated as it is, in good part to early design decisions (not mine). Your "no noticeable differences" goal precludes (1). Back to (2). In C, replacing compile-time conditionals by run-time checks means replacing #if FOO by if (foo). Such a transformation isn't possible in the QAPI schema. To make it possible, we need to evolve the QAPI schema language. docs/devel/qapi-code-gen.rst describes what we have: COND = STRING | { 'all: [ COND, ... ] } | { 'any: [ COND, ... ] } | { 'not': COND } [....] The C code generated for the definition will then be guarded by an #if preprocessing directive with an operand generated from that condition: * STRING will generate defined(STRING) * { 'all': [COND, ...] } will generate (COND && ...) * { 'any': [COND, ...] } will generate (COND || ...) * { 'not': COND } will generate !COND So, conditions are expression trees where the leaves are preprocessor symbols and the inner nodes are operators. It's not quite obvious to me how to best evolve this to support run-time checks.

Whatever we choose should support generating Rust and Go as well. Why? Rust usage in QEMU is growing, and we'll likely need to generate some Rust from the QAPI schema. Victor Toso has been working on Go bindings for use in Go QMP client software.

>> The build system treats QAPI modules qapi/*-target.json as >> target-specific. The .c files generated for them are compiled per >> target. See qapi/meson.build. >> >> Only such target-specific modules can can use target-specific QAPI >> conditionals. Use in target-independent modules will generate C that >> won't compile. >> >> Poisoned macros used in qapi/*-target.json: >> >> CONFIG_KVM >> TARGET_ARM >> TARGET_I386 >> TARGET_LOONGARCH64 >> TARGET_MIPS >> TARGET_PPC >> TARGET_RISCV >> TARGET_S390X >> >>> What we try to do here is to build them only once >>> instead. >> >> You're trying to eliminate target-specific QAPI conditionals. Correct? >> > > Yes, but without impacting the list of commands exposed. Thus, it would > be needed to select at runtime to expose/register commands. Conditionals affect more than just commands.

>>> In the past, we identied that the best approach to solve this is to expose code >>> for all targets (thus removing all #if clauses), and stub missing >>> symbols for concerned targets. >> >> This affects QAPI/QMP introspection, i.e. the value of query-qmp-schema. >> >> Management applications can no longer use introspection to find out >> whether target-specific things are available. >> > > As asked on my previous email answering Daniel, would that be possible > to build the schema dynamically, so we can decide what to expose or not > introspection wise? QAPI was designed to be compile-time static. Revising such fundamental design assumptions is always fraught. I can't give you a confident assessment now. All I can offer you is my willingness to explore solutions. See "really fancy" below. Fun fact: we used to generate the value of query-qmp-schema as a single string. We switched to the current, more bloated representation to support conditionals (commit 7d0f982bfbb).

>> For instance, query-cpu-definitions is implemented for targets arm, >> i386, loongarch, mips, ppc, riscv, and s390x. It initially was for >> fewer targets, and more targets followed one by one. Still more may >> follow in the future. Right now, management applications can use >> introspection to find out whether it is available. That stops working >> when you make it available for all targets, stubbed out for the ones >> that don't (yet) implement it. >> > > I will repeat, just to be clear, I don't think exposing all commands is > a good idea. > The current series *does not* do this, simply because I didn't want to > huge work for nothing. Got it. >> Management applications may have to be adjusted for this. >> >> This is not an attempt to shoot down your approach. I'm merely >> demonstrating limitations of your promise "if anyone notices a >> difference, it will be a bug." >> > > I stick to this promise :). > >> Now, we could get really fancy and try to keep introspection the same by >> applying conditionals dynamically somehow. I.e. have the single binary >> return different introspection values depending on the actual guest's >> target. >> >> This requires fixing the target before introspection. Unless this is >> somehow completely transparent (wrapper scripts, or awful hacks based on >> the binary's filename, perhaps), management applications may have to be >> adjusted to actually do that. >> >> Applies not just to introspection. Consider query-cpu-definitions >> again. It currently returns CPU definitions for *the* target. What >> would a single binary's query-cpu-definitions return? The CPU >> definitions for *all* its targets? Management applications then receive >> CPUs that won't work, which may upset them. To avoid noticable >> difference, we again have to fix the target before we look. >> >> Of course, "fixing the target" stops making sense once we move to >> heterogeneous machines with multiple targets. >> > > At this point, I don't have think about what should be the semantic when > we'll have multiple targets running simultaneously (expose the union, > restrict to the main arch, choose a third way). We have to unless we make query-cpu-definitions fail or impossible to send while the target is still undecided. Making it fail would violate the "no observable differences" goal. The only path to true "no observable differences" I can see is to fix the target before the management application interacts with QEMU in any way. This would make QMP commands (query-cpu-definitions, query-qmp-schema, ...) impossible to send before the target is fixed.

>>> This series build QAPI generated code once, by removing all TARGET_{arch} and >>> CONFIG_KVM clauses. What it does *not* at the moment is: >>> - prevent target specific commands to be visible for all targets >>> (see TODO comment on patch 2 explaining how to address this) >>> - nothing was done to hide all this from generated documentation >> >> For better or worse, generated documentation always contains everything. >> > > Fine for me, it makes sense, as the official documentation published, > which is what people will consume primarily, is for all targets. > >> An argument could be made for stripping out documentation for the stuff >> that isn't included in this build. >> >>> From what I understood, the only thing that matters is to limit qmp commands >>> visible. Exposing enums, structure, or events is not a problem, since they >>> won't be used/triggered for non concerned targets. Please correct me if this is >>> wrong, and if there are unexpected consequences for libvirt or other consumers. >> >> I'm not sure what you mean by "to limit qmp commands visible". >> >> QAPI/QMP introspection has all commands and events, and all types >> reachable from them. query-qmp-schema returns an array, where each >> array element describes one command, event, or type. When a command, >> event, or type is conditional in the schema, the element is wrapped in >> the #if generated for the condition. >> > > After reading and answering to your valuable email, I definitely think Thanks! > the introspection schema we expose should be adapted, independently of > how we build QAPI code (i.e. using #ifdef TARGET or not). > > Is it something technically hard to achieve? Unclear. See "fundamental design assumptions" and "need to evolve the QAPI schema language" above. If you want to learn more about introspection, I'd recommend docs/devel/qapi-code-gen.rst section "Client JSON Protocol introspection".

[...]

Pierrick Bouvier <pierrick.bouvier(a)linaro.org&gt; writes: > After looking at the introspection code, I don't see any major blocker. > We need to keep some of existing "if", as they are based on config-host, > and should apply. > We can introduce a new "available_if" (or any other name), which > generates a runtime check when building the schema, or when serializing > a struct. > > This way, by modifying the .json with: > - if: 'TARGET_I386' > + available_if: 'target_i386()' > > This way, we keep the possibility to have ifdef, and we can expose at > runtime based on available_if. So we can keep the exact same schema we > have today per target. The name is ugly. Naming is hard. No need to worry about it right now. Semantics of having both 'if' and 'available_if'? To work out an answer, let's consider how to convert conditionals: * 'if': STRING If STRING is a target-specific macro, replace by 'available_if': PRED, where PRED is the equivalent run-time predicate. Else, no change. * 'if': { 'all': [COND, ...] } If COND contains only target-specific macros, replace by 'available_if': { 'all': [PRED, ...] }, where the PRED are the equivalent run-time predicates. If COND contains no target-specific macros, no change. What if it contains both? - If each COND contains either only target-specific macros, or no target-specific macros, we could split the target-specific ones off into an additional 'available_if'. This requires defining the semantics of having both 'if' and 'available_if' as "both conditions must be satisfied". - What if this isn't the case? * 'if' { 'any': [COND, ...] } Similar, but to be able to split the COND we need "either condition must be satisfied." Even if we can make this work somehow, it would likely be a royal mess to explain in qapi-code-gen.rst. We currently don't have "mixed" conditionals. So we could sidestep the problem: you can have either 'if' or 'available_if', but not both. Feels like a cop out to me. What if we move the "is dynamic" bit from the root of the conditional to its leaves? So far, the leaves are macro names. What if we additionally permit a function name? Function name, not C expression, to not complicate generating code in languages other than C too much. Ignore the question of syntax for now, i.e. how to decide whether a leaf is a macro or a function name.

On Tue, Apr 29, 2025 at 09:43:24AM +0200, Markus Armbruster wrote: > Pierrick Bouvier <pierrick.bouvier(a)linaro.org&gt; writes: > > > After looking at the introspection code, I don't see any major blocker. > > We need to keep some of existing "if", as they are based on config-host, > > and should apply. > > We can introduce a new "available_if" (or any other name), which > > generates a runtime check when building the schema, or when serializing > > a struct. > > > > This way, by modifying the .json with: > > - if: 'TARGET_I386' > > + available_if: 'target_i386()' > > > > This way, we keep the possibility to have ifdef, and we can expose at > > runtime based on available_if. So we can keep the exact same schema we > > have today per target. > > The name is ugly. Naming is hard. No need to worry about it right now. > > Semantics of having both 'if' and 'available_if'? To work out an > answer, let's consider how to convert conditionals: > > * 'if': STRING > > If STRING is a target-specific macro, replace by 'available_if': PRED, > where PRED is the equivalent run-time predicate. > > Else, no change. > > * 'if': { 'all': [COND, ...] } > > If COND contains only target-specific macros, replace by > 'available_if': { 'all': [PRED, ...] }, where the PRED are the > equivalent run-time predicates. > > If COND contains no target-specific macros, no change. > > What if it contains both? > > - If each COND contains either only target-specific macros, or no > target-specific macros, we could split the target-specific ones off > into an additional 'available_if'. This requires defining the > semantics of having both 'if' and 'available_if' as "both conditions > must be satisfied". > > - What if this isn't the case? > > * 'if' { 'any': [COND, ...] } > > Similar, but to be able to split the COND we need "either condition > must be satisfied." > > Even if we can make this work somehow, it would likely be a royal mess > to explain in qapi-code-gen.rst. > > We currently don't have "mixed" conditionals. So we could sidestep the > problem: you can have either 'if' or 'available_if', but not both. > Feels like a cop out to me. > > What if we move the "is dynamic" bit from the root of the conditional to > its leaves? So far, the leaves are macro names. What if we > additionally permit a function name? > > Function name, not C expression, to not complicate generating code in > languages other than C too much. > > Ignore the question of syntax for now, i.e. how to decide whether a leaf > is a macro or a function name. I wonder if any of this is worth the pain in practice.....

Looking at the QAPI schema, we apply TARGET_xxxx conditions either to commands, or to structs/enums that are used in args/return of commands. We don't conditionalize individual fields, etc.

I tried to query our schema with 'jq' (incidentally rather tedious because of our JSON-but-not-JSON language[1]). If I select only commands we get: query-cpu-definitions => currently many arches query-cpu-model-expansion => currently many arches query-cpu-model-baseline => currently s390x only query-cpu-model-comparison => currently s390x only query-s390x-cpu-polarization => inherently s390x only query-gic-capabilities => inherently arm only query-sev => inherently x86 only query-sev-attestation-report => inherently x86 only query-sev-capabilities => inherently x86 only query-sev-launch-measure => inherently x86 only query-sgx => inherently x86 only query-sgx-capabilities => inherently x86 only rtc-reset-reinjection => inherently x86 only set-cpu-topology => inherently s390x only sev-inject-launch-secret => inherently x86 only xen-event-inject => currently x86 only xen-event-list => currently x86 only The two Xen commands are currently limited to x86, but if we ever extended Xen to arm, possibly they would make sense. IOW, conceptually a target conditional might be useful in future. The CPU model commands are the ones where having the target conditions visible in schema appears to add value, in that they'll allow a mgmt app to detect when we extend any of them to cover new architectures. Libvirt (and other mgmt apps) want to query the schema to see if commands exist in the QEMU they're using, before trying to invoke them. To some degree this is just a "nice to have" that improves error reporting/detection.

For the commands that are inherently arch specific, the mgmt app should conceptually already know what architectures these commands apply to. These target conditionals provide little (no) value when probing commands in the schema.

IOW, if we (for example) have a single binary for x86 and s390, it should be harmless if we report that 'query-sev' exists regardless of arch, as the mgmt app should none the less already know to only use it with x86. I don't know if libvirt correctly filters based on architecture in the case of SEV/SGX/GIC/RTC when probing & using these features, but if it does not, then I'd consider that a pre-existing bug that should be fixed.

Libvirt doesn't use the Xen commands. For query-cpu-model-comparison/baseline, libvirt already filters its usage of these based on s390 arch, so even if x86 reported them I believe it won't break libvirt today. If these commands are extended to other archs in future, libvirt might want a way to detect this. On the flipside it might not be the end of the world if we just expose them on all arches and simply have them return an error at runtime where non-applicable.

IOW, while the target conditions could theoretically provide value at some point in future, it does not look like they do anything /today/ for libvirt. Given that I wonder if we shouldn't just ignore the problem, and blindly remove all TARGET_nnn conditions from QAPI schema today. Let our future selves worry about it should this prove insufficient later.

With regards, Daniel [1] To use QAPI JSON with 'jq' you must convert ' to " and strip comments. eg cat *.json | sed -e "s/'/\"/g" -e 's/#.*//' | jq ...expression...

I don't think we should think too much ahead for languages other than C, for one, two, and even three reasons :)

- First, it's already broken because we rely on ifdef that won't be there in Rust or Go.

- Second, it's code, we can just change it later if needed.

- Third, those json are consumed only by QEMU (right?), so we are free to write/modify them as we want.

The only thing that must stay the same is what we expose to the consumer in the schema, and which commands we expose in qemu.

On Wed, May 07, 2025 at 09:55:13AM +0200, Markus Armbruster wrote: > Pierrick Bouvier <pierrick.bouvier(a)linaro.org&gt; writes: > > [...] > >> I don't think we should think too much ahead for languages other than C, >> for one, two, and even three reasons :) > > I agree that thinking ahead too much is a bad habit. So is thinking > ahead too little :) > >> - First, it's already broken because we rely on ifdef that won't be >> there in Rust or Go. > > I don't think it's broken. QAPI 'if' translates straightforwardly to C > #if, but that doesn't mean it cannot be translated to conditional > compilation / metaprogramming in other languages. > > In fact, the value of 'if' used to be C constant expressions suitable > for use with #if, and we changed it to its current form specifically to > enable Rust work, in merge commit c83fcfaf8a5. Marc-André's was trying > to develop Rust bindings back then, and if I remember correctly this > change was enough to let him implement 'if' with Rust. The usefulness of the conditions to non-C languages may well vary depended on intended use case. If I'm writing a mgmt app that can talk to QEMU, and I want to be able to talk to both x86 and s390 system emulators, I am unlikely to want the language bindings to omit features based on TARGET_xxx conditions. I won't want a separate API for each QEMU arch target, I'll want one API for all arches. Expanding this further, if I'm writing a mgmt app to talk to QEMU 9.2.0, I am also unlikely to want the language bindings to omit features based on CONFIG_xxxx conditions, because I likely want to talk to a QEMU 9.2.0 built by any OS distro, each of which may have chosen a different set of --enable flags for configure, and thus having different CONFIG_xxx conditions. TL;DR: in terms of code generation, the prime use case for conditionally generated code is for QEMU's own internal usage. For QMP bindings generated for 3rd party app usage, the use of conditions in generation feels like a niche requirment at best. IMHO the code for any public facing API derived from QMP schema should be invariant for any given QEMU release.

With regards, Daniel

On 4/25/25 08:38, Markus Armbruster wrote: > Pierrick Bouvier <pierrick.bouvier(a)linaro.org&gt; writes: > > > Note: This RFC was posted to trigger a discussion around this topic, and it's > > not expected to merge it as it is. > > > > Context > > ======= > > > > Linaro is working towards heterogeneous emulation, mixing several architectures > > in a single QEMU process. The first prerequisite is to be able to build such a > > binary, which we commonly name "single-binary" in our various series. > > An (incomplete) list of series is available here: > > https://patchew.org/search?q=project%3AQEMU+single-binary > > > > We don't expect to change existing command line interface or any observable > > behaviour, it should be identical to existing binaries. If anyone notices a > > difference, it will be a bug. > > Define "notice a difference" :) More on that below. > Given a single-binary *named* exactly like an existing qemu-system-X binary, any user or QEMU management layer should not be able to distinguish it from the real qemu-system-X binary. The new potential things will be: - introduction of an (optional) -target option, which allows to override/disambiguate default target detected. - potentially more boards/cpus/devices visible, once we start developing heterogeneous emulation. See it as a new CONFIG_{new_board} present. Out of that, once the current target is identified, based on argv[0], there should be absolutely no difference, whether in the behaviour, UI, command line, or the monitor interfaces.

Pierrick Bouvier <pierrick.bouvier(a)linaro.org&gt; writes:

To be precise: conditionals that use macros restricted to target-specific code, i.e. the ones poisoned by exec/poison.h. Let's call them target-specific QAPI conditionals. The QAPI generator is blissfully unaware of all this. The build system treats QAPI modules qapi/*-target.json as target-specific. The .c files generated for them are compiled per target. See qapi/meson.build. Only such target-specific modules can can use target-specific QAPI conditionals. Use in target-independent modules will generate C that won't compile. Poisoned macros used in qapi/*-target.json: CONFIG_KVM TARGET_ARM TARGET_I386 TARGET_LOONGARCH64 TARGET_MIPS TARGET_PPC TARGET_RISCV TARGET_S390X

> What we try to do here is to build them only once instead. You're trying to eliminate target-specific QAPI conditionals. Correct? > In the past, we identied that the best approach to solve this is to expose code > for all targets (thus removing all #if clauses), and stub missing > symbols for concerned targets. This affects QAPI/QMP introspection, i.e. the value of query-qmp-schema. Management applications can no longer use introspection to find out whether target-specific things are available.

For instance, query-cpu-definitions is implemented for targets arm, i386, loongarch, mips, ppc, riscv, and s390x. It initially was for fewer targets, and more targets followed one by one. Still more may follow in the future. Right now, management applications can use introspection to find out whether it is available. That stops working when you make it available for all targets, stubbed out for the ones that don't (yet) implement it. Management applications may have to be adjusted for this. This is not an attempt to shoot down your approach. I'm merely demonstrating limitations of your promise "if anyone notices a difference, it will be a bug." Now, we could get really fancy and try to keep introspection the same by applying conditionals dynamically somehow. I.e. have the single binary return different introspection values depending on the actual guest's target.

This requires fixing the target before introspection. Unless this is somehow completely transparent (wrapper scripts, or awful hacks based on the binary's filename, perhaps), management applications may have to be adjusted to actually do that.

Applies not just to introspection. Consider query-cpu-definitions again. It currently returns CPU definitions for *the* target. What would a single binary's query-cpu-definitions return? The CPU definitions for *all* its targets? Management applications then receive CPUs that won't work, which may upset them. To avoid noticable difference, we again have to fix the target before we look.

Of course, "fixing the target" stops making sense once we move to heterogeneous machines with multiple targets. > This series build QAPI generated code once, by removing all TARGET_{arch} and > CONFIG_KVM clauses. What it does *not* at the moment is: > - prevent target specific commands to be visible for all targets > (see TODO comment on patch 2 explaining how to address this) > - nothing was done to hide all this from generated documentation

Peter Krempa <pkrempa(a)redhat.com&gt; writes:

> The second thing that libvirt does after 'query-version' is > 'query-target'. > > So what should libvirt do once multiple targets are supported? > > How do we query CPUs for each of the supported targets? > > Will the result be the same if we query them one at a time or all at > once? Pierrick's stated goal is to have no noticable differences between the single binary and the qemu-system-<target> it covers. This is obviously impossible if we can interact with the single binary before the target is fixed.

On 4/28/25 4:07 AM, Markus Armbruster wrote: > Peter Krempa <pkrempa(a)redhat.com&gt; writes: > >> So what should libvirt do once multiple targets are supported? >> >> How do we query CPUs for each of the supported targets? >> It's kind of a similar question we have to solve now with QEMU code. What happens when a symbol is duplicated, and available only for several targets? In this case, we found various approaches to solve this: - unify this symbol for all targets (single implementation) - unify all targets to provide this symbol (multiple impl, all targets) - rename symbols adding {arch} suffix, so it's disambiguated by name - create a proper interface which an available function (multiple impl, selective targets) In the case of query-cpu-definitions, my intuition is that we want to have a single implementation, and that we return *all* the cpus, merging all architectures. In the end, we (and libvirt also) should think out of the "target" box. It's an implementation detail, based on the fact QEMU had 'targets' associated to various binaries for a long time and not a concept that should leak into all consumers. >> Will the result be the same if we query them one at a time or all at >> once? > > Pierrick's stated goal is to have no noticable differences between the > single binary and the qemu-system-<target> it covers. This is obviously > impossible if we can interact with the single binary before the target > is fixed. > Right. At this point, we can guarantee the target will be fixed before anything else, at the start of main(). It's obviously an implementation choice, but to be honest, I don't see what we would gain from having a "null" default QEMU target, unable to emulate anything. >>> This requires fixing the target before introspection. Unless this is >>> somehow completely transparent (wrapper scripts, or awful hacks based on >>> the binary's filename, perhaps), management applications may have to be >>> adjusted to actually do that. >> >> As noted filename will not work. Users can specify any filename and >> create override scripts or rename the binary. > > True. > I would prefer to not open this pandora box on this thread, but don't worry, the best will be done to support all those cases, including renaming the binary, allowing any prefix, suffix, as long as name stays unambiguous. If you rename it to qemu-ok, how can you expect anything? We can provide the possibility to have a "default" target set at compile time, for distributors creating their own specific QEMU binaries. But in the context of classical software distribution, it doesn't make any sense.

On 29/04/2025 10.23, Markus Armbruster wrote: ... > I don't wish to derail this thread, but we've been dancing around the > question of how to best fix the target for some time. I think we should > talk about it for real. > > Mind, this is not an objection to your larger "single binary" idea. It > could be only if it was an intractable problem, but I don't think it is. > > You want the single binary you're trying to create to be a drop-in > replacement for per-target binaries. > > "Drop-in replacement" means existing usage continues to work. > Additional interfaces are not a problem. > > To achieve "drop-in replacement", the target needs to be fixed > automatically, and before the management application can further > interact with it. > > If I understand you correctly, you're proposing to use argv[0] for that, > roughly like this: assume it's qemu-system-<target>, extract <target> > first thing in main(), done. > > What if it's not named that way? If I understand you correctly, you're > proposing to fall back to a compiled-in default target. > > I don't think this is going to fly. I tend to disagree. For normal users that consume QEMU via the distros, the check via argv[0] should be good enough. For developers, I think we can assume that they are adaptive enough to use an additional "-target" option in case they mis-named their binary in a bad way.

> Developers rename the binary all the time, and expect this not to change > behavior. For instance, I routinely rename qemu-FOO to qemu-FOO.old or > qemu-FOO.COMMIT-HASH to let me compare behavior easily. Developers should already be aware that this can cause trouble, since e.g. the qtests are deriving the target architecture from the binary name already. See the qtest_get_arch() function.

On 29/04/2025 10.23, Markus Armbruster wrote: ... > I don't wish to derail this thread, but we've been dancing around the > question of how to best fix the target for some time. I think we should > talk about it for real. >

> Mind, this is not an objection to your larger "single binary" idea. It > could be only if it was an intractable problem, but I don't think it is. > > You want the single binary you're trying to create to be a drop-in > replacement for per-target binaries. > > "Drop-in replacement" means existing usage continues to work. > Additional interfaces are not a problem. > > To achieve "drop-in replacement", the target needs to be fixed > automatically, and before the management application can further > interact with it. > > If I understand you correctly, you're proposing to use argv[0] for that, > roughly like this: assume it's qemu-system-<target>, extract <target> > first thing in main(), done. > > What if it's not named that way? If I understand you correctly, you're > proposing to fall back to a compiled-in default target. > > I don't think this is going to fly. I tend to disagree. For normal users that consume QEMU via the distros, the check via argv[0] should be good enough. For developers, I think we can assume that they are adaptive enough to use an additional "-target" option in case they mis-named their binary in a bad way. > Developers rename the binary all the time, and expect this not to change > behavior. For instance, I routinely rename qemu-FOO to qemu-FOO.old or > qemu-FOO.COMMIT-HASH to let me compare behavior easily. Developers should already be aware that this can cause trouble, since e.g. the qtests are deriving the target architecture from the binary name already. See the qtest_get_arch() function. > We could relax the assumption to support such renames. Developers then > need to be aware of what renames are supported. Meh. > > The more we relax the pattern, the likelier surprising behavior becomes. > > We could mitigate surprises by eliminating the built-in default target. Just print out a proper error message if the target cannot be derived from argv[0], pointing the users to use "-target", and we should be fine. And if someone renames their "qemu-sytem-aarch64" symlink to "qemu-system-x86_64" and still expect to run aarch64 images that way, that's just plain stupidity. > Users invoke their binaries with their own names, too. If Joe R. User > finds qemu-system-<joe's-fav-target> too much to type, and creates a > symlink named q to it, more power to him! They can also either use shell aliases or short shell scripts to achieve that goal, so that's not really a show stopper. > Distributions have packaged renamed binaries. qemu-kvm has been used > quite widely. Yes, and QEMU already checks for that naming in configure_accelerators() ... so that's rather another indicator that we can go with configuration via argv[0] :-) > In neither of these cases, relaxing the pattern helps. > > The least bad solution I can see so far is a new option -target. > > Instead of turning the target-specific binaries into links to / copies > of the single binary, they become wrappers that pass -target as the > first option. We need to make sure this option is honored in time then, > which should be easy enough. > > If you invoke the single binary directly, you need to pass -target > yourself. If you don't to pass it, or pass it late in the command line, > you open up a window for interaction with indeterminate target. > Target-specific interfaces could exhibit different behavior then, even > fail. That's fine under "additional interfaces are not a problem". > > Thoughts? Shell script wrappers always have the problem that they break the direct usage of debuggers like "valgrind" or "gdb" with the target binary, so that's also not the best solution. I'd go with Pierrick's idea to try to determine the target via argv[0]. And for people who really want to rename their binary in a way that makes it impossible to determine the target automatically, just provide the "-target" option as fallback solution, too. Thomas

On 29/04/2025 21.48, Pierrick Bouvier wrote: ... > I'm not keen to have a default target set, but it's a personal opinion based > on fear of "implicit smart choice hurts", so I'll be happy to change my mind > with a good argument for it. No default target, please! We've seen this with the default machines - it looks convenient first, but only gives troubles in the long run. Preferred defaults can change in the course of time, but if you have baked in the logic in hundreds of scripts out there, it's hard to change it afterwards again.

Thomas

On 29/4/25 10:23, Markus Armbruster wrote: > Pierrick Bouvier <pierrick.bouvier(a)linaro.org&gt; writes: > > > On 4/28/25 4:07 AM, Markus Armbruster wrote: > > > Peter Krempa <pkrempa(a)redhat.com&gt; writes: > > > > > > > So what should libvirt do once multiple targets are supported? > > > > > > > > How do we query CPUs for each of the supported targets? > > > > > > > > It's kind of a similar question we have to solve now with QEMU code. > > What happens when a symbol is duplicated, and available only for several > > targets? > > > > In this case, we found various approaches to solve this: > > - unify this symbol for all targets (single implementation) > > - unify all targets to provide this symbol (multiple impl, all targets) > > - rename symbols adding {arch} suffix, so it's disambiguated by name > > - create a proper interface which an available function (multiple impl, > > selective targets) > > > > In the case of query-cpu-definitions, my intuition is that we want to > > have a single implementation, and that we return *all* the cpus, merging > > all architectures. In the end, we (and libvirt also) should think out of > > the "target" box. It's an implementation detail, based on the fact QEMU > > had 'targets' associated to various binaries for a long time and not a > > concept that should leak into all consumers. > > > > > > Will the result be the same if we query them one at a time or all at > > > > once? > > > > > > Pierrick's stated goal is to have no noticable differences between the > > > single binary and the qemu-system-<target> it covers. This is obviously > > > impossible if we can interact with the single binary before the target > > > is fixed. > > > > > > > Right. > > At this point, we can guarantee the target will be fixed before anything > > else, at the start of main(). It's obviously an implementation choice, > > but to be honest, I don't see what we would gain from having a "null" > > default QEMU target, unable to emulate anything. > > > > > > > This requires fixing the target before introspection. Unless this is > > > > > somehow completely transparent (wrapper scripts, or awful hacks based on > > > > > the binary's filename, perhaps), management applications may have to be > > > > > adjusted to actually do that. > > > > > > > > As noted filename will not work. Users can specify any filename and > > > > create override scripts or rename the binary. > > > > > > True. > > > > > > > I would prefer to not open this pandora box on this thread, but don't > > worry, the best will be done to support all those cases, including > > renaming the binary, allowing any prefix, suffix, as long as name stays > > unambiguous. If you rename it to qemu-ok, how can you expect anything? > > > > We can provide the possibility to have a "default" target set at compile > > time, for distributors creating their own specific QEMU binaries. But in > > the context of classical software distribution, it doesn't make any sense. > > I don't wish to derail this thread, but we've been dancing around the > question of how to best fix the target for some time. I think we should > talk about it for real. > > Mind, this is not an objection to your larger "single binary" idea. It > could be only if it was an intractable problem, but I don't think it is. > > You want the single binary you're trying to create to be a drop-in > replacement for per-target binaries. > > "Drop-in replacement" means existing usage continues to work. > Additional interfaces are not a problem. > > To achieve "drop-in replacement", the target needs to be fixed > automatically, and before the management application can further > interact with it. > > If I understand you correctly, you're proposing to use argv[0] for that, > roughly like this: assume it's qemu-system-<target>, extract <target> > first thing in main(), done. > > What if it's not named that way? If I understand you correctly, you're > proposing to fall back to a compiled-in default target. > > I don't think this is going to fly. Rather than using non-constant argv[0] Pierrick suggested to add a single CLI option '-target' which selects the corresponding TargetInfo structure to use at runtime. I.e. for ARM: https://lore.kernel.org/qemu-devel/20250424222112.36194-12-philmd@linaro.... For distros qemu-system-arm could be a shell script prepending '-target arm' while passing the arguments calling qemu-system. If a distro wants to name a binary 'qemu-kvm' it can drop the -target option and hard-wire its target_info() to a distro-specific TargetInfo implementation, or &target_info_x86_64_system.

On 4/29/25 2:35 AM, Philippe Mathieu-Daudé wrote: > If a distro wants to name a binary 'qemu-kvm' it can drop the > -target option and hard-wire its target_info() to a distro-specific > TargetInfo implementation, or &target_info_x86_64_system. > Having updated my Debian stable to next stable (trixie) last week, I noticed that qemu-kvm was removed [1]. I don't know why, when or how, but it's just an example that things can change, and people can survive to it. [1] https://packages.debian.org/search?keywords=qemu-kvm For the concerned other distros, if at least one packager asks us to provide a "./configure --default-target", it will be an excellent reason and opportunity to do it. But before that, let's first build this single binary, let's see if it's useful, let's see how to use it, and eventually, let's see how to package this and cover corner cases.