* Daniel Veillard <veillard@redhat.com> [2007-11-08 15:27]:
On Thu, Nov 08, 2007 at 02:00:10PM -0600, Ryan Harper wrote:
* Daniel Veillard <veillard@redhat.com> [2007-11-08 10:08]:
I promised that mail for the beginning of the week but I still have I think tuning informations are that set of parameters associated to a domain or a host, which are not stricly needed to get the domain(s) working but improve their runtime behaviour. To me this includes: - scheduling parameters the scope may be host/hypervisor/domain - vcpu affinity i.e. to which set of physical CPU each of the vcpu may be bound - and possibly others ...
The problem: ------------ People would like to associate those to the XML domain informations, the goal being to be able to restore those informations when a domain (re-)starts. I have been objecting it so far because, I think those informations don't have the same lifetime and scope as the other domain informations saved in the XML. Since they are not needed to start the domain, and that once the domain is started the existing domain API can be used to change those informations, it is better to keep them separate.
For at least (maybe only) Xen NUMA systems, the application of "tuning" information after a domain is started does not achieve the same affect as including the information during the initial construction of the domain. In particular, Xen needs to know which physical cpus are being used to determine which cpus it from which numanode it will allocate memory. Adjusting affinity after the domain has allocated memory doesn't allow libvirt or any management app to control from which node domains pull memory.
yes, I understand and that's why I agreed to add the cpuset information at that point it's more than tunning because it may be irreversible for the lifetime of the domain, so this really should be in the XML. I'm not suggesting to go back about 'cpu affinity' i.e. to which physical CPUs a domain should be bound, but 'vcpu affinity' i.e. then how the virtual CPUs of the domain are mapped onto that cpu set, that can change
OK, I see your distinction here.
dynamically without (serious) performance penalty.
At least for Xen, the 'cpu' affinity specified with a domain is only accessible via the xen config file and is not enforced in any way such that it prevents from someone "tuning" a domain to use physical cpus outside of the specified cpumap. Users can can certainly specify a cpu outside of the original cpuset from the config file which in a NUMA scenario has the potential for serious performance penalties.
I don't have any objection to separating "tuning" information as long as we have the ability to merge permanent domain parameters with its "tuning" information prior to domain construction.
My point is that you don't need the tuning informations to create the domain, if you need them it's not tuning. When you say you want to merge them, do you want this to create the domain ? It should not be necessary (or I take a counter example that would help me), right ?
I agree here. I was lumping cpuset info into your tunable category but you clarified the distinction above. I just want to ensure that initial cpuset mapping is present prior to constructing a domain as that is integral for proper Xen NUMA memory allocation. -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx (512) 838-9253 T/L: 678-9253 ryanh@us.ibm.com