Re: [libvirt-users] 'virsh capabilities' on Debian Wheezy-amd64 reports different cpu to Wheezy-i386 (on same hardware)

On Mon, Mar 03, 2014 at 11:15:51AM +0000, Struan Bartlett wrote: > On 03/03/2014 10:55, Martin Kletzander wrote: >> On Mon, Mar 03, 2014 at 10:47:03AM +0000, Struan Bartlett wrote: >>> On 03/03/2014 10:44, Martin Kletzander wrote: >>>> On Mon, Mar 03, 2014 at 10:30:11AM +0000, Struan Bartlett wrote: >>>>> Hi Martin >>>>> >>>>> Thanks for your response. Here's the output of that grep: >>>>> >>>>> # grep ^flags /proc/cpuinfo | sort -u >>>>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca >>>>> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall >>>>> nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl >>>>> xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor >>>>> ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 >>>>> x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat xsaveopt >>>>> pln pts dtherm tpr_shadow vnmi flexpriority ept vpid >>>>> >>>>> I've just managed to install the libvirt-bin:amd64 package on the same >>>>> machine, on the wheezy-i386 distribution. The output of 'virsh >>>>> capabilities' is now reported correctly, and my VMs that required >>>>> SandyBridge are now booting. >>>>> >>>>> Have you any further suggestions? >>>>> >>>> Oh, I missed the fact that it was 32 bit distro. In that case, I >>>> guess some cpu features won't be available. No other ideas. >>> Ok thanks Martin. If the amd64 and i386 versions of the package are >>> behaving differently in this respect, on otherwise the same hardware, >>> kernel and distro, then I can only presume this is a bug, so my next >>> step would be to raise this on the development list. >> Just a guess, but our CPUID instruction might be translated to >> assembly code differently. You can try running 'cpuid -ir' (or 'cpuid >> -ir1' if it's the same on all cpus) on both machines, that will give >> you the results libvirt is processing. If that output is different >> than there is nothing libvirt can do. If it's the same, it might be a >> libvirt bug. > You might be onto something here. The following are run on the same > physical machine: > > # apt-get install cpuid:i386 > # cpuid -ir1 >/tmp/cpuid-ir1-i386 > # apt-get install cpuid:amd64 > # cpuid -ir1 >/tmp/cpuid-ir1-amd64 > # diff -ubw /tmp/cpuid-ir1-i386 /tmp/cpuid-ir1-amd64 > --- /tmp/cpuid-ir1-i386 2014-03-03 11:13:00.839208222 +0000 > +++ /tmp/cpuid-ir1-amd64 2014-03-03 11:13:13.283307377 +0000 > @@ -10,11 +10,11 @@ > 00000008 00000000 00000000 00000000 00000000 > 00000009 00000000 00000000 00000000 00000000 > 0000000a 07300403 00000000 00000000 00000603 > -0000000b 00000000 00000000 000000e8 0000002c > +0000000b 00000000 00000000 0000006f 0000002c This one has *probably* no significance in libvirt code since we only call cpuid functions 0x00000001, 0x00000007 and 0x80000001 (which is enough for us to determine all we need). > 0000000c 00000000 00000000 00000000 00000000 > 0000000d 00000000 00000000 00000000 00000000 > 80000000 80000008 00000000 00000000 00000000 > -80000001 00000000 00000000 00000001 2c100000 > +80000001 00000000 00000000 00000001 2c100800 But this is something. This means in second run the CPU reports it has "syscall" and in the first one it doesn't. Since "syscall" is x86_64 instruction used instead of i386's "int 0x80" (IIRC) it makes sense that it is missing on 32-bit CPU (when the code is running in 32-bit mode). > 80000002 20202020 6e492020 286c6574 58202952 > 80000003 286e6f65 43202952 45205550 36322d35 > 80000004 204c3035 20402030 30382e31 007a4847 > > Could this be enough difference to throw cpu-detection off? > Even one bit is enough if that bit represents a flag which is not in the CPU you are looking for (i.e. SandyBridge). I must admin I was too lazy to compare the differences between the processor definitions you've sent (basically because I'm fairly certain we have no such mistakes in our code O:-) ). But I did that now and indeed, you can see that the difference is only in the "syscall" feature. You can do that yourselves using the info from "cpu_map.xml" if you don't believe me ;) Martin

> Struan > >>>> Martin >>>> >>>>> Struan >>>>> >>>>> On 03/03/2014 10:00, Martin Kletzander wrote: >>>>>> On Fri, Feb 28, 2014 at 03:45:01PM +0000, Struan Bartlett wrote: >>>>>>> Hi >>>>>>> >>>>>>> On a range of Dell servers containing Intel 64bit processors, 'virsh >>>>>>> capabilities' reports the cpu differently on Debian Wheezy-amd64 and >>>>>>> Wheezy-i386. The results given by the Wheezy-i386 version seem very >>>>>>> wrong (since n270 is an Atom processor). Apart from architecture, the >>>>>>> package versions of libvirt-bin are identical: 1.2.1-1~bpo70+1. >>>>>>> /usr/share/libvirt/cpu_map.xml files are identical. Is this a known >>>>>>> issue? Details for one server are: >>>>>>> >>>>>>> # cat /proc/cpuinfo| head -n 26 >>>>>>> processor : 0 >>>>>>> vendor_id : GenuineIntel >>>>>>> cpu family : 6 >>>>>>> model : 45 >>>>>>> model name : Intel(R) Xeon(R) CPU E5-2650L 0 @ 1.80GHz >>>>>>> stepping : 7 >>>>>>> microcode : 0x70d >>>>>>> cpu MHz : 1800.054 >>>>>>> cache size : 20480 KB >>>>>>> physical id : 0 >>>>>>> siblings : 16 >>>>>>> core id : 0 >>>>>>> cpu cores : 8 >>>>>>> apicid : 0 >>>>>>> initial apicid : 0 >>>>>>> fpu : yes >>>>>>> fpu_exception : yes >>>>>>> cpuid level : 13 >>>>>>> wp : yes >>>>>>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge >>>>>>> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe >>>>>>> syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good >>>>>>> nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 >>>>>>> monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 >>>>>>> sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat >>>>>>> xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid >>>>>>> bogomips : 3600.10 >>>>>>> clflush size : 64 >>>>>>> cache_alignment : 64 >>>>>>> address sizes : 46 bits physical, 48 bits virtual >>>>>>> power management: >>>>>>> >>>>>>> ... >>>>>>> </proc/cpuinfo for processors 1..31 snipped here for brevity> >>>>>>> >>>>>> Check if all 32 CPUs have *exactly* the same flags, I remember an >>>>>> issue when we reported a wrong cpu because the probing code was >>>>>> scheduled on one of them which had one flag missing. If package and >>>>>> cpu_map.xml are the same, this is the only thing I can think of. >>>>>> Simple 'grep ^flags /proc/cpuinfo | sort -u' should do. If only one >>>>>> line is printed out than I don't know where the problem might be... >>>>>> >>>>>> Martin >>>>>> >>>>>>> # Running Wheezy-amd64 libvirt-bin1.2.1-1~bpo70+1 >>>>>>> # virsh capabilities >>>>>>> >>>>>>> <cpu> >>>>>>> <arch>x86_64</arch> >>>>>>> <model>SandyBridge</model> >>>>>>> <vendor>Intel</vendor> >>>>>>> <topology sockets='2' cores='8' threads='2'/> >>>>>>> <feature name='pdpe1gb'/> >>>>>>> <feature name='osxsave'/> >>>>>>> <feature name='dca'/> >>>>>>> <feature name='pcid'/> >>>>>>> <feature name='pdcm'/> >>>>>>> <feature name='xtpr'/> >>>>>>> <feature name='tm2'/> >>>>>>> <feature name='est'/> >>>>>>> <feature name='smx'/> >>>>>>> <feature name='vmx'/> >>>>>>> <feature name='ds_cpl'/> >>>>>>> <feature name='monitor'/> >>>>>>> <feature name='dtes64'/> >>>>>>> <feature name='pbe'/> >>>>>>> <feature name='tm'/> >>>>>>> <feature name='ht'/> >>>>>>> <feature name='ss'/> >>>>>>> <feature name='acpi'/> >>>>>>> <feature name='ds'/> >>>>>>> <feature name='vme'/> >>>>>>> </cpu> >>>>>>> >>>>>>> >>>>>>> # Running Wheezy-i386 libvirt-bin1.2.1-1~bpo70+1 >>>>>>> # virsh capabilities >>>>>>> >>>>>>> <cpu> >>>>>>> <arch>x86_64</arch> >>>>>>> <model>n270</model> >>>>>>> <vendor>Intel</vendor> >>>>>>> <topology sockets='2' cores='8' threads='2'/> >>>>>>> <feature name='lahf_lm'/> >>>>>>> <feature name='lm'/> >>>>>>> <feature name='rdtscp'/> >>>>>>> <feature name='pdpe1gb'/> >>>>>>> <feature name='avx'/> >>>>>>> <feature name='osxsave'/> >>>>>>> <feature name='xsave'/> >>>>>>> <feature name='aes'/> >>>>>>> <feature name='tsc-deadline'/> >>>>>>> <feature name='popcnt'/> >>>>>>> <feature name='x2apic'/> >>>>>>> <feature name='sse4.2'/> >>>>>>> <feature name='sse4.1'/> >>>>>>> <feature name='dca'/> >>>>>>> <feature name='pcid'/> >>>>>>> <feature name='pdcm'/> >>>>>>> <feature name='xtpr'/> >>>>>>> <feature name='cx16'/> >>>>>>> <feature name='tm2'/> >>>>>>> <feature name='est'/> >>>>>>> <feature name='smx'/> >>>>>>> <feature name='vmx'/> >>>>>>> <feature name='ds_cpl'/> >>>>>>> <feature name='dtes64'/> >>>>>>> <feature name='pclmuldq'/> >>>>>>> <feature name='pbe'/> >>>>>>> <feature name='tm'/> >>>>>>> <feature name='ht'/> >>>>>>> <feature name='ss'/> >>>>>>> <feature name='acpi'/> >>>>>>> <feature name='ds'/> >>>>>>> <feature name='pse36'/> >>>>>>> </cpu> >>>>>>> >>>>>>> Kind regards >>>>>>> >>>>>>> Struan Bartlett >>>>>>> >>>>>>>