On Thu, Oct 05, 2017 at 02:11 PM +0200, Jiri Denemark <jdenemar(a)redhat.com&gt; wrote: Right. Yep. So how do we deal with this? -- Beste Grüße / Kind regards Marc Hartmayer IBM Deutschland Research & Development GmbH Vorsitzende des Aufsichtsrats: Martina Koederitz Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft: Böblingen Registergericht: Amtsgericht Stuttgart, HRB 243294

On Thu, Oct 12, 2017 at 02:07 PM +0200, Jiri Denemark <jdenemar(a)redhat.com&gt; wrote: Hi all, we recently encountered the problem that the 'host-model' [1] has to be related to the machine type of a domain. We have following problem: Let's assume we've a z13 system with a QEMU 2.9 and we define a domain using the default s390-virtio-ccw machine together with the host-model CPU mode [1]. The definition will have the machine expanded to s390-virtio-ccw-2.9 but retain the host-model CPU mode in the domain definition. In a next step we upgrade to QEMU 2.10 (first version to recognize z14). Everything is still fine, even though the machine runs in 2.9 compatibility mode. Finally we upgrade to a z14. As a consequence it is not possible to start the domain anymore as the machine type doesn't support our CPU host model (which is expanded at start time of the domain). For determining the actual host-model the QMP command 'query-cpu-model-expansion' is used. This is done once per QEMU binary and the result of it is cached by libvirt. The problem with that is that libvirt queries with the newest machine type of the QEMU binary for the host CPU model. But now we realized that we have to consider the machine type of each domain for the determination of the host CPU model of a domain. We could now either probe the host CPU model for each QEMU binary + machine type combination and for this we've to start a new QEMU process each time. Or we can add a QMP command/parameter that allows us to determine the host CPU model(s) with respect to the passed machine type and/or all supported machine types. The QMP command query-cpu-model-expansion is currently described in qapi-schema.json as follows: <quotation> ## # @query-cpu-model-expansion: # # Expands a given CPU model (or a combination of CPU model + additional options) # to different granularities, allowing tooling to get an understanding what a # specific CPU model looks like in QEMU under a certain configuration. # # This interface can be used to query the "host" CPU model. # # The data returned by this command may be affected by: # # * QEMU version: CPU models may look different depending on the QEMU version. # (Except for CPU models reported as "static" in query-cpu-definitions.) # * machine-type: CPU model may look different depending on the machine-type. # (Except for CPU models reported as "static" in query-cpu-definitions.) # * machine options (including accelerator): in some architectures, CPU models # may look different depending on machine and accelerator options. (Except for # CPU models reported as "static" in query-cpu-definitions.) # * "-cpu" arguments and global properties: arguments to the -cpu option and # global properties may affect expansion of CPU models. Using # query-cpu-model-expansion while using these is not advised. # # Some architectures may not support all expansion types. s390x supports # "full" and "static". # # Returns: a CpuModelExpansionInfo. Returns an error if expanding CPU models is # not supported, if the model cannot be expanded, if the model contains # an unknown CPU definition name, unknown properties or properties # with a wrong type. Also returns an error if an expansion type is # not supported. # # Since: 2.8.0 ## { 'command': 'query-cpu-model-expansion', 'data': { 'type': 'CpuModelExpansionType', 'model': 'CpuModelInfo' }, 'returns': 'CpuModelExpansionInfo' } 4:46:25 PM ◄ qapi-schema.json </quotation> What do you think? Thank you. Marc [1] https://libvirt.org/formatdomain.html#elementsCPU -- Beste Grüße / Kind regards Marc Hartmayer IBM Deutschland Research & Development GmbH Vorsitzende des Aufsichtsrats: Martina Koederitz Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft: Böblingen Registergericht: Amtsgericht Stuttgart, HRB 243294

On Fri, Oct 20, 2017 at 13:37:42 +0200, David Hildenbrand wrote: The thing is libvirt calls query-cpu-model-expansion to check what the host CPU is. This 'host-model' CPU is replaced with the probed CPU model when a domain starts. The problem described by Marc is that the probed CPU model cannot be used universally with all machine types. So starting a domain on such host with machine type s390-virtio-ccw-2.10 works, but a domain with machine type s390-virtio-ccw-2.9 fails to start with the same probed CPU model. Jirka

On Fri, Oct 20, 2017 at 15:04:57 +0200, David Hildenbrand wrote: The CPU model is copied into the XML each time the domain starts. Even preserving the actual CPU model in the XML from the first start wouldn't solve the issue. You can always create a new domain with s390-virtio-ccw-2.9 machine type even on the upgraded host. Jirka

Actually, what is the cause of that problem? I assume it is the gs feature (gs_allowed)? We should really avoid such things (..._allowed) for CPU model features in the future and clue all new such stuff to cpumodel_allowed. -- Thanks, David

On 20.10.2017 15:36, Christian Borntraeger wrote: My point was, that the host model would have to be copied and _remain_ there when s390-ccw-virtio was expanded to s390-ccw-virtio-2.9. So really replacing <cpu mode='host-model'/> by the model z13-base.... This would at least fix this issue. Just like s390-ccw-virtio get's replaced and remains thats way. But this might for sure have other problems. Migration is a totally different story, as tooling has to make sure to find a CPU model that is compatible over all host models. So cpumodel_allowed would indeed work. (at least in my theory ;) ) I am not sure if that is the right thing to do. The documentation states clearly that the host model is copied. If that is not runnable, fix the setup. -- Thanks, David

On 20.10.2017 15:49, Christian Borntraeger wrote: Yes, as stated in the documentation "copying host CPU definition from capabilities XML" this can not work. And it works just as documented. Not saying this is a nice thing :) I think we should try to fix gs_allowed (if possible) and avoid something like that in the future. This would avoid other complexity involved when suddenly having X host models. -- Thanks, David