Re: [PATCH 5/5] qemu: Prefer -numa cpu over -numa node,cpus=

On Thu, 30 Sep 2021 14:08:34 +0200 Peter Krempa <pkrempa(a)redhat.com&gt; wrote: > On Tue, Sep 21, 2021 at 16:50:31 +0200, Michal Privoznik wrote: >> QEMU is trying to obsolete -numa node,cpus= because that uses >> ambiguous vCPU id to [socket, die, core, thread] mapping. The new >> form is: >> >> -numa cpu,node-id=N,socket-id=S,die-id=D,core-id=C,thread-id=T >> >> which is repeated for every vCPU and places it at [S, D, C, T] >> into guest NUMA node N. >> >> While in general this is magic mapping, we can deal with it. >> Firstly, with QEMU 2.7 or newer, libvirt ensures that if topology >> is given then maxvcpus must be sockets * dies * cores * threads >> (i.e. there are no 'holes'). >> Secondly, if no topology is given then libvirt itself places each >> vCPU into a different socket (basically, it fakes topology of: >> [maxvcpus, 1, 1, 1]) >> Thirdly, we can copy whatever QEMU is doing when mapping vCPUs >> onto topology, to make sure vCPUs don't start to move around. > > There's a problem with this premise though and unfortunately we don't > seem to have qemuxml2argvtest for it. > > On PPC64, in certain situations the CPU can be configured such that > threads are visible only to VMs. This has substantial impact on how CPUs > are configured using the modern parameters (until now used only for > cpu hotplug purposes, and that's the reason vCPU hotplug has such > complicated incantations when starting the VM). > > In the above situation a CPU with topology of: > sockets=1, cores=4, threads=8 (thus 32 cpus) > > will only expose 4 CPU "devices". > > core-id: 0, core-id: 8, core-id: 16 and core-id: 24 > > yet the guest will correctly see 32 cpus when used as such. > > You can see this in: > > tests/qemuhotplugtestcpus/ppc64-modern-individual-monitor.json > > Also note that the 'props' object does _not_ have any socket-id, and > management apps are supposed to pass in 'props' as is. (There's a bunch > of code to do that on hotplug). > > The problem is that you need to query the topology first (unless we want > to duplicate all of qemu code that has to do with topology state and > keep up with changes to it) to know how it's behaving on current > machine. This historically was not possible. The supposed solution for > this was the pre-config state where we'd be able to query and set it up > via QMP, but I was not keeping up sufficiently with that work, so I > don't know if it's possible. > > If preconfig is a viable option we IMO should start using it sooner > rather than later and avoid duplicating qemu's logic here. using preconfig is preferable variant otherwise libvirt would end up duplicating topology logic which differs not only between targets but also between machine/cpu types. Closest example how to use preconfig is in pc_dynamic_cpu_cfg() test case. Though it uses query-hotpluggable-cpus only for verification, but one can use the command at the preconfig stage to get topology for given -smp/-machine type combination.