4:58 a.m.
Am 15.01.2013 17:16, schrieb Juan Quintela:
* cpu hot plug
- use qdev propierties conected to a set of socket objects (anthony)
- cpusets are the wrong interface (anthony)
- make a link between cpu <-> socket instead of a propierty?
- how far are we from being able to describe a cpu with -device?
(didn't heare the answer, andreas?)
- perhaps the best approach?
- After soft-freeze, exceptions depend on the maintainer
- After hard-freeze, no exceptions
-device don't require a bus, just an implementation detail, we can change that
- use cpuset as an intermediate step until full vision is implemented
- several approaches from where we are now, to have something before
we get a full solution
At this point, Andreas agreed to write a better summary of the
discussion and suggestions O:-)
Got buried, here we go:
== vCPU hot-plug user interfaces ==
=== cpu_set ===
Previously available in qemu-kvm.git:
`cpu_set <n+1> online` via HMP
Pros:
* Hides QOM/qdev implementation details (afaerber)
* Thus: Doesn't depend on QOM CPUState refactoring (imammedo)
* Opens a fast route to implementing vCPU unplug in KVM (imammedo)
* Unintrusive to add and easy to obsolete/remove in future (imammedo)
* Existing virt-test cases (afaerber)
* Supported by libvirt (imammedo)
* Prevents confusing guests by hot-plugging random mix of CPUs (agraf)
Cons:
* Cannot express topologies (ehabkost)
=== device_add ===
`device_add driver=Haswell-x86_64-cpu id=qdevid`
[You can try this today and see it failing / not working.]
Pros:
* QMP/HMP command available today and known to users (afaerber)
* Unified command for device and CPU hot-plug (imammedo)
* Would allow first doing thread-level vCPU hotplug (imammedo)
* Could be extended to support socket-level hot-plug (aliguori/imammedo)
Cons:
* Operates on raw QOM type name unlike -cpu (afaerber)
* Needs support in libvirt for device_add driver=<CPU> (imammedo)
* libvirt needs means to enumerate CPU types (imammedo) => QMP? (AF)
Challenges:
* No CPU qbus (afaerber)
=> "should work" without (aliguori)
* CPU subclasses needed for identifying type name (afaerber/imammedo)
=> "Haswell-x86_64-cpu" does not exist yet, just "x86_64-cpu"
* CPU class_init for -cpu host requires KVM init (imammedo)
[suggestion by ehabkost to use kvm_arch_vcpu_init, WIP by afaerber]
* Conversion of CPU features to static properties needed (imammedo)
=> device_add driver=foo,level=x,xlevel=y,...
* Alternatively conversion to global properties (imammedo)
* Cements type names - rename for 1.4? (afaerber) => permissable (alig.)
[patches for arm, m68k, openrisc, unicore32 on list]
=== qom-set ===
`qom-set` via QMP w/ link<CPUSocket> property (aliguori)
Topology represented in QOM:
CPUSocket has-a CPUCore has-a CPUThread a.k.a. CPUState, or
CPUSocket links-to CPUCore links-to CPUThread a.k.a. CPUState
Challenges (afaerber):
* No CPUSocket/CPUCore objects yet and may take a while to get there...
topology fields being moved to CPUState for 1.4 [done, more WIP]
* No decisions on canonical paths for CPUs: CPU? machine? unassigned?
* Duality of thread-level device types and socket-level? (afaerber)
=> fine to have, e.g., quad-core Xeon 500 device (aliguori)
* CPUState is no_user (afaerber)
=> need to generally drop no_user for QOM (aliguori)
=== libvirt ===
libvirt's XML topology modelling is closer to today's -smp than to the
desired QOM modelling:
http://www.libvirt.org/formatcaps.html
`virsh setvcpus <domain> <n>`
http://libvirt.org/sources/virshcmdref/html/sect-setvcpus.html
== qom-cpu course of action (afaerber) ==
It was requested to have vCPU hot-plug in v1.5.
For device_add we need to move code from cpu_init() into QOM facilities.
=> QOM realize support would help [applied by aliguori]
=> cleanups piggy-backed onto CPU realizefn [applied to qom-cpu-next]
Agreement on goal of X86CPU subclasses, but conflicts how to get there:
* Refactor x86_def_t to X86CPUInfo for X86CPUClass class_init? (AF 2012)
* Refactor x86_def_t to X86CPU instance_init as done for arm?
* Refactor x86_def_t to class_inits? (afaerber)
-> heavy merge conflicts due to bug fixes / cleanups
Pro: We can get things into a consistent QOM'ish state across targets.
Con: We will refactor again on top for machine-compat properties.
* Keep x86_def_t within X86CPUClass as done for ppc? (WIP: afaerber)
=> smallest common denominator, separates x86 from cross-target work
APIC ID topology fixes are being reviewed for 1.4. [merged]
X86CPU wave 4 cleanups by Igor are being reviewed for 1.4. [merged]
Rename CPU types according to unified <name>-<arch>-cpu scheme for 1.4?
(aliguori: permissable) [patches on list]
VMState series by Juan being rebased - subset for 1.4, rest for 1.5.
[1.4 part on list, WIP for 1.5]
Remainder is considered 1.5 material, qom-cpu-next avail. during Freeze.
== Common issues (imammedo) ==
- back-port CPU hot-plug ACPI notification
- hot-plug is not allowed on SysBus:
- APIC that is created by CPU on hot-plug is a SysBus device,
so essentially APIC is hot-plugged to SysBus anyway which causes
abort in qdev_try_create() -> qdev_set_parent_bus(SysBus) ->
bus_add_child() [no hot-plug on SysBus], which is hard to fix
without allowing hot-plug on SysBus.
[AF: change the APIC to not be a SysBusDevice (like CPUState) then?]
- qdev_device_add() and qdev_try_create() assign SysBus by default to
device if no explicit bus was provided.
- qbus_find_recursive() in qdev_device_add() always returns SysBus
if last 2 args are NULL, which is case with busless CPU
- qdev_try_create() adds device SysBus if no bus was provided in
1st arg
- vapic is not working with hot-plug in current master
(do not know why yet)
Experimental CPU hot-add tree using device_add implemented only for
qemu64 CPU:
https://github.com/imammedo/qemu/tree/x86-cpu-hot-add.WIP
---
That completes Igor's and my notes, amended by status updates of mine.
Sorry for the delay, I did try to address some of the discussed issues
in form of patches in the meantime though. ;)
CC'ing s390x, ppc and arm KVM folks that were not on the call for
feedback on their requirements and thoughts.
Regards,
Andreas
--
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg