[libvirt] inquiry, get native capabilities of the host

Hello, I'm interested in getting native capabilities of host CPU using libvirt. I've looked at .../src/cpu/cpu_map.xml. And it does describe native capabilities of host CPU very well. But, libvirt always returns a subset of the native capabilities which is far different from the feature lists in the xml file. (I'm testing with <model name='Westmere'>.) The sponsor of my project is interested is high-performance computing. And knowing native feature of the host CPU that can also be available to a virtual machine is very important for performance reason. I couldn't find how to get the native features of the host that is described in .../src/cpu/cpu_map.xml. Is there a way or libvirt API to get the native capabilities of host CPU that is well described in .../src/cpu/cpu_map.xml? Thanks, David.

Hi, On Thu, May 12, 2011 at 15:39:44 -0700, Dong-In David Kang wrote:
I'm interested in getting native capabilities of host CPU using libvirt. I've looked at .../src/cpu/cpu_map.xml. And it does describe native capabilities of host CPU very well. But, libvirt always returns a subset of the native capabilities which is far different from the feature lists in the xml file. (I'm testing with <model name='Westmere'>.) The sponsor of my project is interested is high-performance computing. And knowing native feature of the host CPU that can also be available to a virtual machine is very important for performance reason.
I couldn't find how to get the native features of the host that is described in .../src/cpu/cpu_map.xml. Is there a way or libvirt API to get the native capabilities of host CPU that is well described in .../src/cpu/cpu_map.xml?
I'm not sure I totally understand what you are looking for. Is the issue you have that virsh capabilities tells you something like this: <cpu> <arch>x86_64</arch> <model>Westmere</model> <vendor>Intel</vendor> <topology sockets='1' cores='2' threads='2'/> <feature name='rdtscp'/> <feature name='xtpr'/> <feature name='tm2'/> <feature name='est'/> <feature name='vmx'/> <feature name='ds_cpl'/> <feature name='monitor'/> <feature name='pbe'/> <feature name='tm'/> <feature name='ht'/> <feature name='ss'/> <feature name='acpi'/> <feature name='ds'/> <feature name='vme'/> </cpu> while in cpu_map.xml mentions much more features for Westmere model? If so, than the answer is quite simple. The thing is that the CPU description is additive, that is the above XML snippet says that host CPU supports all features that are defined by Westmere CPU model in cpu_map.xml *plus* the features that are explicitly listed in the above <cpu> element. So if you expand the model, you get a full list of features supported by the host CPU: <!-- from Nehalem which Westmere is derived from --> <feature name='sse2'/> <feature name='sse'/> <feature name='fxsr'/> <feature name='mmx'/> <feature name='pat'/> <feature name='cmov'/> <feature name='pge'/> <feature name='sep'/> <feature name='apic'/> <feature name='cx8'/> <feature name='mce'/> <feature name='pae'/> <feature name='msr'/> <feature name='tsc'/> <feature name='pse'/> <feature name='de'/> <feature name='fpu'/> <feature name='mtrr'/> <feature name='clflush'/> <feature name='mca'/> <feature name='pse36'/> <feature name='pni'/> <feature name='cx16'/> <feature name='ssse3'/> <feature name='sse4.1'/> <feature name='sse4.2'/> <feature name='popcnt'/> <feature name='lm'/> <feature name='syscall'/> <feature name='nx'/> <feature name='lahf_lm'/> <!-- from Westmere --> <feature name='aes'/> <!-- explicitly listed --> <feature name='rdtscp'/> <feature name='xtpr'/> <feature name='tm2'/> <feature name='est'/> <feature name='vmx'/> <feature name='ds_cpl'/> <feature name='monitor'/> <feature name='pbe'/> <feature name='tm'/> <feature name='ht'/> <feature name='ss'/> <feature name='acpi'/> <feature name='ds'/> <feature name='vme'/> Jirka

Hi Jiri, Thank you for the valuable information. That's exactly what I want to get. Let me tell you what I'm doing now. I'm extending OpenStack (cloud computing infrastructure) for high performance computing. Before spawning a cloud instance, I want to check the full capability of the host machine because I will use KVM. With the expanded feature list, a cloud user can decide which node to choose. For example, if a users needs sse4.1, the scheduler should search for a node having sse4.1 as its feature. OpensStack currently calls virConnectGetCapabilities at the start of a node to get its capability. And it returns not fully expanded feature list. Could you tell me how I can exand it fully? Should I patch libvirt? Or can it be done giving different arguments to libvirt API? Thank you, Dong-In ----- Original Message ----- From: "Jiri Denemark" <jdenemar@redhat.com> To: "Dong-In David Kang" <dkang@isi.edu> Cc: libvir-list@redhat.com Sent: Friday, May 13, 2011 5:29:09 AM Subject: Re: [libvirt] inquiry, get native capabilities of the host Hi, On Thu, May 12, 2011 at 15:39:44 -0700, Dong-In David Kang wrote:
I'm interested in getting native capabilities of host CPU using libvirt. I've looked at .../src/cpu/cpu_map.xml. And it does describe native capabilities of host CPU very well. But, libvirt always returns a subset of the native capabilities which is far different from the feature lists in the xml file. (I'm testing with <model name='Westmere'>.) The sponsor of my project is interested is high-performance computing. And knowing native feature of the host CPU that can also be available to a virtual machine is very important for performance reason.
I couldn't find how to get the native features of the host that is described in .../src/cpu/cpu_map.xml. Is there a way or libvirt API to get the native capabilities of host CPU that is well described in .../src/cpu/cpu_map.xml?
I'm not sure I totally understand what you are looking for. Is the issue you have that virsh capabilities tells you something like this: <cpu> <arch>x86_64</arch> <model>Westmere</model> <vendor>Intel</vendor> <topology sockets='1' cores='2' threads='2'/> <feature name='rdtscp'/> <feature name='xtpr'/> <feature name='tm2'/> <feature name='est'/> <feature name='vmx'/> <feature name='ds_cpl'/> <feature name='monitor'/> <feature name='pbe'/> <feature name='tm'/> <feature name='ht'/> <feature name='ss'/> <feature name='acpi'/> <feature name='ds'/> <feature name='vme'/> </cpu> while in cpu_map.xml mentions much more features for Westmere model? If so, than the answer is quite simple. The thing is that the CPU description is additive, that is the above XML snippet says that host CPU supports all features that are defined by Westmere CPU model in cpu_map.xml *plus* the features that are explicitly listed in the above <cpu> element. So if you expand the model, you get a full list of features supported by the host CPU: <!-- from Nehalem which Westmere is derived from --> <feature name='sse2'/> <feature name='sse'/> <feature name='fxsr'/> <feature name='mmx'/> <feature name='pat'/> <feature name='cmov'/> <feature name='pge'/> <feature name='sep'/> <feature name='apic'/> <feature name='cx8'/> <feature name='mce'/> <feature name='pae'/> <feature name='msr'/> <feature name='tsc'/> <feature name='pse'/> <feature name='de'/> <feature name='fpu'/> <feature name='mtrr'/> <feature name='clflush'/> <feature name='mca'/> <feature name='pse36'/> <feature name='pni'/> <feature name='cx16'/> <feature name='ssse3'/> <feature name='sse4.1'/> <feature name='sse4.2'/> <feature name='popcnt'/> <feature name='lm'/> <feature name='syscall'/> <feature name='nx'/> <feature name='lahf_lm'/> <!-- from Westmere --> <feature name='aes'/> <!-- explicitly listed --> <feature name='rdtscp'/> <feature name='xtpr'/> <feature name='tm2'/> <feature name='est'/> <feature name='vmx'/> <feature name='ds_cpl'/> <feature name='monitor'/> <feature name='pbe'/> <feature name='tm'/> <feature name='ht'/> <feature name='ss'/> <feature name='acpi'/> <feature name='ds'/> <feature name='vme'/> Jirka

Hi Dong-In,
Thank you for the valuable information. That's exactly what I want to get. Let me tell you what I'm doing now. I'm extending OpenStack (cloud computing infrastructure) for high performance computing. Before spawning a cloud instance, I want to check the full capability of the host machine because I will use KVM. With the expanded feature list, a cloud user can decide which node to choose. For example, if a users needs sse4.1, the scheduler should search for a node having sse4.1 as its feature.
As Daniel already said, you can use virConnectCompareCPU() with desired cpu xml and it will tell you if the desired cpu is compatible with host cpu on that node. However, you cannot specify features without also saying what cpu model you need. So in case you are only interested in sse4.1 feature regardless on cpu model you can use the following cpu xml: <cpu> <model>pentium</model> <feature name='sse4.1'/> </cpu> The 'pentium' model has almost no features and is most likely compatible with every modern cpu you may have.
OpensStack currently calls virConnectGetCapabilities at the start of a node to get its capability. And it returns not fully expanded feature list. Could you tell me how I can exand it fully?
Libvirt doesn't support expanding models since that only makes sense (to some extend) for certain cpu architecture. You could theoretically expand it yourself but cpu_map.xml is internal to libvirt and its schema may change any time. The idea (also already described by Daniel) is that applications should not really need to expand the features. An application should rather construct cpu xml according to user's needs and ask libvirt (via virConnectCompareCPU) if the requested cpu is compatible with host CPU. Jirka

Hi Jirka, Thank you for the answer. It is a very useful method to checking if the host can support a feature is useful. However, OpenStack uses libvirt in a different way. OpenStack collects the capabilities of a machine when the machine registers itself to the cloud as a compute node. virConnectGetCapabilities() is used to get the capabilities. After that, OpenStack does not inquire the compute node of its capabilities. OpenStack keeps that information of the compute nodes in the cloud (it can be in a database or in an ephemeral storage). When a request is sent to the scheduler with required features (such as sse4.1), the scheduler looks for a match. Here, the scheduler does not inquire the compute nodes if they support that feature. But, the scheduler uses the capabilities information it received from the compute nodes. (Changing this communication flow may need a big changes in OpenStack, which many people try to avoid.) In this model, OpenStack must get and keep the full feature list. And I thought it would be best to get the fully expanded feature list using virConnectGetCapabilities(). To that purpose, I've applied the following changes in the file libvirt-0.9.1/src/cpu/cpu_x86.c. Do you have any concern about the changes? Or could you suggest any better approach? --- libvirt-0.9.1/src/cpu/cpu_x86.c 2011-03-01 02:03:31.000000000 -0500 +++ cpu_x86.c 2011-05-18 09:42:32.909470642 -0400 @@ -477,8 +477,9 @@ if ((vendor = x86DataToVendor(copy, map)) && !(cpu->vendor = strdup(vendor->name))) goto no_memory; - +#if 0 // dkang: commented out to return full feature x86DataSubtract(copy, modelData); +#endif x86DataSubtract(modelData, data); /* because feature policy is ignored for host CPU */ Thanks, Dong-In ----- Original Message ----- From: "Jiri Denemark" <jdenemar@redhat.com> To: "Dong-In David Kang" <dkang@isi.edu> Cc: libvir-list@redhat.com Sent: Tuesday, May 17, 2011 2:21:18 PM Subject: Re: [libvirt] inquiry, get native capabilities of the host Hi Dong-In,
Thank you for the valuable information. That's exactly what I want to get. Let me tell you what I'm doing now. I'm extending OpenStack (cloud computing infrastructure) for high performance computing. Before spawning a cloud instance, I want to check the full capability of the host machine because I will use KVM. With the expanded feature list, a cloud user can decide which node to choose. For example, if a users needs sse4.1, the scheduler should search for a node having sse4.1 as its feature.
As Daniel already said, you can use virConnectCompareCPU() with desired cpu xml and it will tell you if the desired cpu is compatible with host cpu on that node. However, you cannot specify features without also saying what cpu model you need. So in case you are only interested in sse4.1 feature regardless on cpu model you can use the following cpu xml: <cpu> <model>pentium</model> <feature name='sse4.1'/> </cpu> The 'pentium' model has almost no features and is most likely compatible with every modern cpu you may have.
OpensStack currently calls virConnectGetCapabilities at the start of a node to get its capability. And it returns not fully expanded feature list. Could you tell me how I can exand it fully?
Libvirt doesn't support expanding models since that only makes sense (to some extend) for certain cpu architecture. You could theoretically expand it yourself but cpu_map.xml is internal to libvirt and its schema may change any time. The idea (also already described by Daniel) is that applications should not really need to expand the features. An application should rather construct cpu xml according to user's needs and ask libvirt (via virConnectCompareCPU) if the requested cpu is compatible with host CPU. Jirka

On Thu, May 12, 2011 at 03:39:44PM -0700, Dong-In David Kang wrote:
Hello,
I'm interested in getting native capabilities of host CPU using libvirt. I've looked at .../src/cpu/cpu_map.xml. And it does describe native capabilities of host CPU very well. But, libvirt always returns a subset of the native capabilities which is far different from the feature lists in the xml file. (I'm testing with <model name='Westmere'>.) The sponsor of my project is interested is high-performance computing. And knowing native feature of the host CPU that can also be available to a virtual machine is very important for performance reason.
I couldn't find how to get the native features of the host that is described in .../src/cpu/cpu_map.xml.
Libvirt provides a set of APIs for CPU compatibility comparisons, such that you don't actually need to know the precise CPU flags. The 'virsh capabilities' command returns an XML description of the host. This description takes the form of a CPU model that has the closest feature set to your physical CPU, plus a set of flags to make up the remainining delta. You can put that host CPU XML description unchanged in the guest XML and the guest will use the same set of features as the host. If you have a different CPU XML description for the guest CPU, you can also query whether it is supported by the host, eg cat > cpu.xml <cpu> <model>Penryn</model> <feature name='xtpr'/> <feature name='tm2'/> <feature name='est'/> <feature name='vmx'/> <feature name='ds_cpl'/> <feature name='monitor'/> <feature name='pbe'/> </cpu> <<EOF $ virsh cpu-compare cpu.xml CPU described in cpu.xml is incompatible with host CPU There is much more you can do besides this, see this guide http://berrange.com/posts/2010/02/15/guest-cpu-model-configuration-in-libvir... Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
participants (3)
-
Daniel P. Berrange
-
Dong-In David Kang
-
Jiri Denemark