Re: Add iommu device when VM configured with > 255 vcpus
On 5/29/24 1:41 AM, Peter Krempa wrote:
On Tue, May 28, 2024 at 16:26:18 -0600, Jim Fehlig via Devel wrote:
> Hi All,
>
> I vaguely recall a discussion about $subject, but can't find it now. Perhaps
> buried in another thread. The topic has been raised internally again, and
> I'd like to gauge the community's interest in automatically adding the
> necessary devices/config when user has specified vcpus > 255.
> The comparison for prior art is a bit of a stretch, but we e.g. add <audio
> type='spice'/> when spice graphics is configured.
The thing about 'audio' "device" is that it's purely just backend
with
no impact on the VM ABI. In fact 'audio' and 'graphics' IMO should not
have been put under <devices> for that reason.
> I know libvirt has
> generally tried to avoid policy decisions, but it's not clear to me where we
> stand with cases such as this, where every x86 VM with > 255 vcpus needs a
> similarly configured iommu.
Adding the IOMMU would change the guest ABI, so libvirt can't auto-add
it, unless a VM with > 255 cpus would not start at all.
libvirt already prevents defining a VM with > 255 vcpus, but without a properly
configured iommu
error: unsupported configuration: more than 255 vCPUs require extended interrupt
mode enabled on the iommu device
It's possible to start such a VM using qemu directly, although the guest (linux
at least) does not make much progress
[ 0.095107][ T0] [Firmware Bug]: CPU 0: APIC ID mismatch. CPUID: 0x0001
APIC: 0x0000
[ 0.003921][ T0] [Firmware Bug]: CPU 2: APIC ID mismatch. CPUID: 0x0003
APIC: 0x0002
[ 0.003921][ T0] [Firmware Bug]: CPU 4: APIC ID mismatch. CPUID: 0x0005
APIC: 0x0004
[ 0.003921][ T0] [Firmware Bug]: CPU 6: APIC ID mismatch. CPUID: 0x0007
APIC: 0x0006
...
[ 0.003921][ T0] [Firmware Bug]: CPU 250: APIC ID mismatch. CPUID: 0x00fb
APIC: 0x00fa
[ 0.003921][ T0] [Firmware Bug]: CPU 252: APIC ID mismatch. CPUID: 0x00fd
APIC: 0x00fc
[ 0.003921][ T0] [Firmware Bug]: CPU 254: APIC ID mismatch. CPUID: 0x00ff
APIC: 0x00fe
Regards,
Jim