12:52 p.m.
Hi, some custom CPU models are reported from
virConnectGetDomainCapabilities as usable='yes' on a physical machine
while as usable='no' inside a VM running on the same machine. That's
not completely surprising.
But what surprises me is that those models are still reported from
virConnectCompareCPU as supported (VIR_CPU_COMPARE_SUPERSET) in the
nested environment and VMs can be started happily with them.
For instance, virConnectGetDomainCapabilities reports
<model usable='no'>Skylake-Client</model>
but when I try to use that model anyway, the VM starts fine with it:
<cpu mode='custom' match='exact' check='full'>
<model fallback='forbid'>Skylake-Client</model>
<topology sockets='16' cores='1' threads='1'/>
<feature policy='require' name='hypervisor'/>
<feature policy='disable' name='invpcid'/>
<numa>
<cell id='0' cpus='0' memory='524288'
unit='KiB'/>
</numa>
</cpu>
That's actually good news, but unexpected. Do I miss something?
Thanks,
Milan
11 a.m.
On Fri, Dec 07, 2018 at 11:52:38 +0100, Milan Zamazal wrote:
Hi, some custom CPU models are reported from
virConnectGetDomainCapabilities as usable='yes' on a physical machine
while as usable='no' inside a VM running on the same machine. That's
not completely surprising.
But what surprises me is that those models are still reported from
virConnectCompareCPU as supported (VIR_CPU_COMPARE_SUPERSET) in the
virConnectCompareCPU uses CPUID data for comparison, which is not the
same as a list of features QEMU/KVM can provide on the host. You should
use virConnectCompareHypervisorCPU to check whether a given CPU can be
used on the host.
nested environment and VMs can be started happily with them.
For instance, virConnectGetDomainCapabilities reports
<model usable='no'>Skylake-Client</model>
but when I try to use that model anyway, the VM starts fine with it:
<cpu mode='custom' match='exact' check='full'>
<model fallback='forbid'>Skylake-Client</model>
<topology sockets='16' cores='1' threads='1'/>
<feature policy='require' name='hypervisor'/>
<feature policy='disable' name='invpcid'/>
This is not the same as Skylake-Client, it's Skylake-Client without
invpcid. The usable='no' attribute says the Skylake-Client CPU model is
not usable unless you disable some features. You did that and it works.
If you asked for just Skylake-Client without any <feature> element, the
domain should fail to start.
Actually QEMU even reports what features need to be disabled to run each
CPU model, but I don't think that's really useful. You don't want to
disable all of them mechanically anyway since that can result in strange
CPU models which would confuse guests. That's why we only report the
usable=yes/no attribute.
Jirka
11:45 a.m.
Jiri Denemark <jdenemar(a)redhat.com> writes:
On Fri, Dec 07, 2018 at 11:52:38 +0100, Milan Zamazal wrote:
> Hi, some custom CPU models are reported from
> virConnectGetDomainCapabilities as usable='yes' on a physical machine
> while as usable='no' inside a VM running on the same machine. That's
> not completely surprising.
>
> But what surprises me is that those models are still reported from
> virConnectCompareCPU as supported (VIR_CPU_COMPARE_SUPERSET) in the
virConnectCompareCPU uses CPUID data for comparison, which is not the
same as a list of features QEMU/KVM can provide on the host. You should
use virConnectCompareHypervisorCPU to check whether a given CPU can be
used on the host.
> nested environment and VMs can be started happily with them.
>
> For instance, virConnectGetDomainCapabilities reports
>
> <model usable='no'>Skylake-Client</model>
>
> but when I try to use that model anyway, the VM starts fine with it:
>
> <cpu mode='custom' match='exact' check='full'>
> <model fallback='forbid'>Skylake-Client</model>
> <topology sockets='16' cores='1' threads='1'/>
> <feature policy='require' name='hypervisor'/>
> <feature policy='disable' name='invpcid'/>
This is not the same as Skylake-Client, it's Skylake-Client without
invpcid. The usable='no' attribute says the Skylake-Client CPU model is
not usable unless you disable some features. You did that and it works.
If you asked for just Skylake-Client without any <feature> element, the
domain should fail to start.
Thank you for explanation. However the behavior I observe is still not
clear to me. The <cpu> snippet above is from a running domain,
successfully started from this definition:
<cpu match="exact">
<model>Skylake-Client</model>
<topology cores="1" sockets="16" threads="1" />
<numa>
<cell cpus="0" id="0" memory="524288" />
</numa>
</cpu>
When this definition is fed to compare CPU, I get:
# virsh hypervisor-cpu-compare cpu.xml
CPU described in cpu.xml is incompatible with the CPU provided by hypervisor on the
host
# virsh cpu-compare cpu.xml
Host CPU is a superset of CPU described in cpu.xml
It's not clear to me:
- Why is the domain successfully started despite hypervisor-cpu-compare
rejects it?
- Why is `invpcid' disabled when `invpcid' is present in /proc/cpuinfo?
- What's the basic difference between virConnectCompareCPU and
virConnectCompareHypervisorCPU? Does "specific hypervisor and its
abilities" (as stated in the documentation) mean that the hypervisor
may extend CPU capabilities (by emulation), restrict CPU capabilities,
or both (depending or particular feature etc.)?
Actually QEMU even reports what features need to be disabled to run
each
CPU model, but I don't think that's really useful. You don't want to
disable all of them mechanically anyway since that can result in strange
CPU models which would confuse guests. That's why we only report the
usable=yes/no attribute.
Jirka
9:12 a.m.
On Tue, Dec 11, 2018 at 10:45:12 +0100, Milan Zamazal wrote:
Jiri Denemark <jdenemar(a)redhat.com> writes:
> On Fri, Dec 07, 2018 at 11:52:38 +0100, Milan Zamazal wrote:
>> Hi, some custom CPU models are reported from
>> virConnectGetDomainCapabilities as usable='yes' on a physical machine
>
>> while as usable='no' inside a VM running on the same machine.
That's
>> not completely surprising.
>>
>> But what surprises me is that those models are still reported from
>> virConnectCompareCPU as supported (VIR_CPU_COMPARE_SUPERSET) in the
>
> virConnectCompareCPU uses CPUID data for comparison, which is not the
> same as a list of features QEMU/KVM can provide on the host. You should
> use virConnectCompareHypervisorCPU to check whether a given CPU can be
> used on the host.
>
>> nested environment and VMs can be started happily with them.
>>
>> For instance, virConnectGetDomainCapabilities reports
>>
>> <model usable='no'>Skylake-Client</model>
>>
>> but when I try to use that model anyway, the VM starts fine with it:
>>
>> <cpu mode='custom' match='exact' check='full'>
>> <model fallback='forbid'>Skylake-Client</model>
>> <topology sockets='16' cores='1'
threads='1'/>
>> <feature policy='require' name='hypervisor'/>
>> <feature policy='disable' name='invpcid'/>
>
> This is not the same as Skylake-Client, it's Skylake-Client without
> invpcid. The usable='no' attribute says the Skylake-Client CPU model is
> not usable unless you disable some features. You did that and it works.
> If you asked for just Skylake-Client without any <feature> element, the
> domain should fail to start.
Thank you for explanation. However the behavior I observe is still not
clear to me. The <cpu> snippet above is from a running domain,
successfully started from this definition:
<cpu match="exact">
<model>Skylake-Client</model>
<topology cores="1" sockets="16" threads="1" />
<numa>
<cell cpus="0" id="0" memory="524288" />
</numa>
</cpu>
I see. When there's no check attribute in the <cpu> element, it's the
same as check='partial', which is a compatibility mode. Libvirt checks
whether either host CPU or QEMU is able to provide all required features
and starts the domain with the specified CPU model. Once the QEMU
process is running, libvirt will check which features were disabled or
enabled and updates the domain XML. Thus you can see the two extra
<feature> elements in the live XML.
If you want to get exactly the CPU model you asked for without QEMU
disabling anything, you can add check='full' attribute to the CPU
element. With this attribute set, you can only start a domain with a CPU
model which says usable='yes' or by explicitly disabling the features
which would QEMU disable.
When this definition is fed to compare CPU, I get:
# virsh hypervisor-cpu-compare cpu.xml
CPU described in cpu.xml is incompatible with the CPU provided by hypervisor on the
host
That's correct, but for compatibility reasons you will still be able to
start the domain (although with random features disabled by QEMU) unless
check='full' is used.
# virsh cpu-compare cpu.xml
Host CPU is a superset of CPU described in cpu.xml
Right, host CPU usually provides more features than what QEMU/KVM can
use.
It's not clear to me:
- Why is the domain successfully started despite hypervisor-cpu-compare
rejects it?
Because of the implicit check='partial' which enables backward
compatibility. Without us doing this, old domains would not be able to
start if QEMU could not provide some of the requested features. Before
QEMU was able to tell us what features can be enabled, we just asked for
some CPU model and QEMU disabled some features. Libvirt was only
checking the guest CPU definition with the host CPU.
- Why is `invpcid' disabled when `invpcid' is present in
/proc/cpuinfo?
No idea, this would be a question for QEMU folks. Perhaps it just cannot
be used in nested?
- What's the basic difference between virConnectCompareCPU and
virConnectCompareHypervisorCPU? Does "specific hypervisor and its
abilities" (as stated in the documentation) mean that the hypervisor
may extend CPU capabilities (by emulation), restrict CPU capabilities,
or both (depending or particular feature etc.)?
Both. That's why comparing directly to host CPU (virConnectCompareCPU)
is not very useful. Not to mention that it can only work for KVM, while
virConnectCompareHypervisorCPU can even be used with TCG, i.e., domain
type='qemu'.
Jirka
5:58 p.m.
Thank you for explanation, it's clear to me now.
One last question: Is there a way to get supported compatibility modes
on POWER? For instance, I get the following from domcapabilities on a
POWER9 machine:
<cpu>
<mode name='host-passthrough' supported='yes'/>
<mode name='host-model' supported='yes'>
<model fallback='allow'>POWER9</model>
<vendor>IBM</vendor>
</mode>
<mode name='custom' supported='no'/>
</cpu>
How can I find out that POWER8 guests are also supported on the machine?
Thanks,
Milan
2240
days inactive
2245
days old
4 comments
2 participants
participants (2)
-
Jiri Denemark -
Milan Zamazal