Info regarding AMX support and libvirt implications

Hello, I'm trying to use AMX in my virtual machines. More info on AMX: https://www.intel.com/content/www/us/en/products/docs/accelerator-engines/ad... My system in test is currently SLES 15 SP5. I'm also verifying in parallel with Suse (especially regarding the backported features in their 5.14 based kernel), but in the meantime I would like to understand implication, if any, of libvirt in the certification loop I have to analyse. From what I see we have in upstream: . support in the KVM kernel module since 5.17 . support of cpu model SapphireRapids, the first offering AMX as an ISA extension, in QEMU since 7.0 Is there any dependance to check on libvirt too? When I run virsh cpu-models x86_64 Is libvirt sw stack querying qemu directly? Or the kvm kernel module? Or any internal "database" file? From man page it is not clear to me what "known" means: " cpu-models Syntax: cpu-models arch Print the list of CPU models known by libvirt for the specified architecture. Whether a specific hypervisor is able to create a domain which uses any of the printed CPU models is a separate question which can be answered by looking at the domain capabilities XML returned by domcapabilities command. Moreover, for some architectures libvirt does not know any CPU models and the usable CPU models are only limited by the hypervisor. This command will print that all CPU models are accepted for these architectures and the actual list of supported CPU models can be checked in the domain capabilities XML. " In SLES 15 SP5 with: qemu-7.1.0-150500.49.9.2.x86_64 kernel-default-5.14.21-150500.55.49.1.x86_64 libvirtd-*-9.0.0-150500.6.11.1.x86_64 I get # virsh cpu-models x86_64 ... Cascadelake-Server Cascadelake-Server-noTSX Icelake-Client Icelake-Client-noTSX Icelake-Server Icelake-Server-noTSX Cooperlake Snowridge athlon phenom Opteron_G1 Opteron_G2 ... # virsh domcapabilities | grep -i sapphirerapid # In fedora39 with qemu-8.1.3-4.fc39.x86_64 kernel-6.7.5-200.fc39.x86_64 libvirt-*-9.7.0-2.fc39.x86_64 I get # virsh cpu-models x86_64 ... Cascadelake-Server Cascadelake-Server-noTSX Icelake-Client Icelake-Client-noTSX Icelake-Server Icelake-Server-noTSX Cooperlake Snowridge SapphireRapids athlon phenom Opteron_G1 Opteron_G2 ... # virsh domcapabilities | grep -i sapphirerapids <model usable='no' vendor='Intel'>SapphireRapids</model> # because I'm running on a client system without AMX support Thanks in advance, Gianluca

On 2/29/24 13:15, Gianluca Cecchi wrote:
Hello, I'm trying to use AMX in my virtual machines. More info on AMX: https://www.intel.com/content/www/us/en/products/docs/accelerator-engines/ad... <https://www.intel.com/content/www/us/en/products/docs/accelerator-engines/advanced-matrix-extensions/overview.html> My system in test is currently SLES 15 SP5. I'm also verifying in parallel with Suse (especially regarding the backported features in their 5.14 based kernel), but in the meantime I would like to understand implication, if any, of libvirt in the certification loop I have to analyse. From what I see we have in upstream: . support in the KVM kernel module since 5.17 . support of cpu model SapphireRapids, the first offering AMX as an ISA extension, in QEMU since 7.0
Is there any dependance to check on libvirt too?
Yes, support for the SapphireRapids CPU model was added in libvirt 9.4.0 https://gitlab.com/libvirt/libvirt/-/commit/96c8d39af007000daf3d5dfa845365f6...
When I run virsh cpu-models x86_64 Is libvirt sw stack querying qemu directly? Or the kvm kernel module? Or any internal "database" file?
From man page it is not clear to me what "known" means: " cpu-models Syntax:
cpu-models arch
Print the list of CPU models known by libvirt for the specified architecture. Whether a specific hypervisor is able to create a domain which uses any of the printed CPU models is a separate question which can be answered by looking at the domain capabilities XML returned by domcapabilities command. Moreover, for some architectures libvirt does not know any CPU models and the usable CPU models are only limited by the hypervisor. This command will print that all CPU models are accepted for these architectures and the actual list of supported CPU models can be checked in the domain capabilities XML. "
In SLES 15 SP5 with:
qemu-7.1.0-150500.49.9.2.x86_64 kernel-default-5.14.21-150500.55.49.1.x86_64 libvirtd-*-9.0.0-150500.6.11.1.x86_64
There are other requests to support SapphireRapids in SLES15 SP5, but it will take some time to backport patches for all these components and release updated packages. Regards, Jim

On Fri, Mar 1, 2024 at 12:47 AM Jim Fehlig <jfehlig@suse.com> wrote: [snip]
Yes, support for the SapphireRapids CPU model was added in libvirt 9.4.0
https://gitlab.com/libvirt/libvirt/-/commit/96c8d39af007000daf3d5dfa845365f6...
Thanks. Is there any virsh or another command or any file on a system that shows me the features supported for a certain cpu-model? Eg without going to the sources inside src/cpu_map/x86_SapphireRapids.xml? Without being on that cpu when running the command? [snip]
There are other requests to support SapphireRapids in SLES15 SP5, but it will take some time to backport patches for all these components and release updated packages.
Regards, Jim
Ok. Is it expected to have full support for SP6 in June? Any way to pre-test in case? Thanks, Gianluca

On 3/1/24 03:30, Gianluca Cecchi wrote:
On Fri, Mar 1, 2024 at 12:47 AM Jim Fehlig <jfehlig@suse.com <mailto:jfehlig@suse.com>> wrote: [snip]
Yes, support for the SapphireRapids CPU model was added in libvirt 9.4.0
https://gitlab.com/libvirt/libvirt/-/commit/96c8d39af007000daf3d5dfa845365f6... <https://gitlab.com/libvirt/libvirt/-/commit/96c8d39af007000daf3d5dfa845365f66379aaac>
Thanks. Is there any virsh or another command or any file on a system that shows me the features supported for a certain cpu-model?
Not that I'm aware of. Perhaps others can suggest something I'm not considering.
Eg without going to the sources inside src/cpu_map/x86_SapphireRapids.xml? Without being on that cpu when running the command?
I think that's it, which is better than looking at e.g. target/i386/cpu.c in the qemu sources :-). BTW, on a libvirt installation you'd likely find those files in /usr/share/libvirt/cpu_map/.
[snip]
There are other requests to support SapphireRapids in SLES15 SP5, but it will take some time to backport patches for all these components and release updated packages.
Regards, Jim
Ok. Is it expected to have full support for SP6 in June? Any way to pre-test in case?
Yes, SapphireRapids will be supported in SP6, which has libvirt 10.0.0. WRT pre-test, there's a SLE public beta program, but I'm not familiar with the details. IMO it would be easier to experiment with Leap 15.6, which shares virt-related packages with SLE15 SP6 https://get.opensuse.org/leap/15.6/#download https://download.opensuse.org/distribution/leap/15.6/ Please report any bugs you find to bugzilla.opensuse.org. Regards, Jim
participants (2)
-
Gianluca Cecchi
-
Jim Fehlig