[Libvir] CPU pinning of domains at creation time
There are a few things I gathered on this issue. This affects
NUMA setups, where basically if a domain must be placed on a given cell
it is not good to let the hypervisor place it first with its own heuristics
and then later migrate it to a different set of CPU, but better to
instruct the hypervisor to start said domain on the given set.
- For Xen it is possible to instruct the hypervisor by passing
(cpus '2,3') in the SExpr where the argument is a list of
the physical processors allowed
- For KVM I think the standard way would be to select the
cpuset using sched_setaffinity() between the fork of the
current process and the exec of the qemu process
- there is no need (from a NUMA perspective) to do fine grained
allocation at that point, as long as the domain can be restricted
to a given cell at startup, then if needed virDomainPinVcpu() can be
used later to do more precise pinning in order to try to optimize
placement
- to be able to instruct the hypervisor at creation time adding the
information in the domain XML description looks the more natural way
(another option would be to force to use virDomainDefineXML, add a
call using the resulting virDomainPtr to define the set, and
then virDomainCreate would be used to do the actual start)
+ the good point of having this embedded in the XML is that
we still have all informations about the domain settings in
the XML, if we want to restart it later
+ the bad point is that we need to fetch and carry this extra
information when doing XML dumps to not loose it for example
when manipulating the domain to add or remove devices
- extracting a cpuset can still be an heavy operation, for example
if using xend on need one RPC per vcpu in the domain, the cpuset
being constructed by OR'ing logically all cpumaps used by the
vcpus of the domain (though in most case this will be the full
map after the first CPU and can be stopped immediately)
- for the mapping at the XML level I suggest to use a simple extension
to the <vcpu>n</vcpu> and extend it to
<vcpu cpuset='2,3'>n</vcpu>
with a limited syntax which is just the comma separated list of
allowed CPU numbers (if the code actually detects such a cpuset is
in effect i.e. in general this won't be added).
Internally implementing this should not be too hard, I would probably refactor
some of the existing parsing code, provide functions to get the cpuset and
the number of physical processors.
Does this sounds okay ?
Daniel
--
Red Hat Virtualization group http://redhat.com/virtualization/
Daniel Veillard | virtualization library http://libvirt.org/
veillard(a)redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/