Hi sorry for the delay in responding.
Regarding the intel email footer, I understand that It might deter people from
responding.
it is automatically added if I email an external address by our exchange server.
I have asked for my account to be added to the exception whitelist
so this should be removed soon. If not I will follow up with my personal email account.
Regarding abstracting the implementation to a separate file, if the changes are small
enough
Perhaps the new code can be added util/virpci and util/virhostdev instead of creating
util/virinterupt
I can refactor the implementation to reflect whichever is preferred during code review.
I would prefer to use auto as the default also.
If the consensus is that it would be better not to enable this feature by default, then I
am happy to do that too.
From my initial investigation I have just looked at affinitizing
interrupts when the vm starts.
I agree that the final implementation should also
handle changes in the vm configuration (vpus, hostdevs plug/unplugged)
and changes In the vm power state also.
Regards
Sean
-----Original Message-----
From: Martin Kletzander [mailto:mkletzan@redhat.com]
Sent: Monday, August 25, 2014 11:35 AM
To: Mooney, Sean K
Cc: libvir-list(a)redhat.com
Subject: Re: [libvirt] Automatically affinitize hostdev interrupts to vm vCpus (qemu/kvm)
On Mon, Aug 18, 2014 at 07:01:40PM +0000, Mooney, Sean K wrote:
Hi
I would like to ask for comments and propose a possible new libvirt feature.
Hi, could you convince your e-mail agent to wrap long lines?
Problem statement:
At present when you boot a vm via libvirt it is possible to pin vm vCPUs to host CPUs to
improve performance in the guest under certain conditions.
If hostdev interrupts are not pined to a specific core/cpuset suboptimal processing of
irqs may occur reducing the performance of both guest and host.
I would like to propose extending libvirt to automatically pin interrupts for hostdev
devices if they are present in the guest.
By affinitizing interrupts, cache line sharing between the specified interrupt and guest
can be achieved.
If CPU affinity for the guest, as set by the cpuset parameter, is
intelligently chosen to place the guest on the same numa node as the hostdev, cross socket
traffic can be mitigated.
As a result, latency which would be introduce if the interrupt processing was scheduled to
a non-local numa node CPU can be reduced via interrupt pinning.
Proposed change:
* util/virpci and util/virhostdev will be extended to retrieve IRQ and
msi_interupt information from sysfs.
* util/virinterupt will be created
* util/virinterupt will implement managing interrupt affinity via
/proc/irq/x/smp_affinity
Nice that this will be abstracted out in a separate file, although if it's not huge,
it can be part of something else.
* qemuProcessInitCpuAffinity will be extended to conditionally
affinitize hostdev interrupts to vm vCpus when hostdevs are present in the vm definition.
So it will only affect hostdevs, OK, that means there should be less (or no)
disadvantages). Although beware that hostdevs can be plugged/unplugged, number of vCPUs
can be changed and, most importantly, the affinities (either set with sched_set_affinity
or using cgroups) can be changed during the lifetime of the VM, and the smp_affinity of
each IRQ should track such changes. Needless to say the affinity should be restored after
the machine dies/is turned off, etc., not just on hostdev unplug.
Alternative implementation:
In addition to the above changes the hostdev element could be extended to include a cpuset
attribute:
* if the cpuset is auto: the interrupts would be pinned to the same cpuset as the
vms vCPU
* if a cpuset is specified: the interrupts would be affinitized as per the set
cpuset
* if the cpuset is native: the interrupts will not be pinned and the current
behaviour will not be changed.
* If the cpuset attribute is not present either the behaviour of auto or native
could be used as a default.
o Using auto as the default would allow transparent use of the feature.
I like defaulting to auto also because the transparent use will have effect only in
existing deployments with vCPU pinning.
o Using native as the default would allow no changes to any existing
deployments unless the feature is requested.
Although this version might be preferred by others. This can, however, be discussed (and
changed) during reviews.
Any feedback is welcomed.
Regards
Sean.
--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263 Business address: Dromore House, East Park,
Shannon, Co. Clare
This e-mail and any attachments may contain confidential material for the sole use of the
intended recipient(s). Any review or distribution by others is strictly prohibited. If you
are not the intended recipient, please contact the sender and delete all copies.
Since this mailing-list is public and archived, such disclaimer is unenforceable. As an
upstream project, the development is done in the open. Such statements as the above one
may be the cause of nobody replying for some time to your e-mail.
The statement itself can be understood that even delivering the mail through a list is
prohibited. Please consider removing such statements in following e-mails (or use private
e-mail address if the disclaimer is enforced by your employer) as keeping them may result
in the rest of the community not being able to communicate with you.
Martin
--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare
This e-mail and any attachments may contain confidential material for the sole use of the
intended recipient(s). Any review or distribution by others is strictly prohibited. If you
are not the intended recipient, please contact the sender and delete all copies.