1:47 p.m.
Hi Jiri & Eduardo,
You might remember a discussion with David Hildenbrand of IBM on the Qemu
mailing list regarding a new Qemu<->Libvirt interface for cpu modeling. I am
picking up this work from David and I wanted to confirm that we are
still on the
same page as to the direction of that interface.
For your reference:
https://www.redhat.com/archives/libvir-list/2016-June/thread.html#01413
https://lists.gnu.org/archive/html/qemu-devel/2016-09/threads.html#00489
The first link is to the discussion you were directly involved in. The
second
link is to the final patch set posted to qemu-devel. The cover letter
gives a
good overview of the interface added to Qemu and the proposed use-case for
Libvirt to use this new cpu modeling support. I'll paste in the most
relevant
section for your convenience:
--------------------------------Libvirt usecase----------------------------
Testing for runability:
- Simply try to start QEMU with KVM, compat machine, CPU model
- Could be done using query-cpu-model-comparison in the future.
Identifying host model, e.g. "virsh capabilities"
- query-cpu-model-expansion on "host" with "-M none --enable-kvm"
<cpu mode='host-model'>:
- simply copy the identified host model
<cpu mode='host-passthrough'>:
- "-cpu host"
"virsh cpu-baseline":
- query-cpu-model-baseline on two models with "-M none"
"virsh cpu-compare":
- query-cpu-model-comparison on two models with "-M none"
There might be some cenarios where libvirt wants to convert another CPU
model to a static variant, this can be done using query-cpu-model-expansion
---------------------------------------------------------------------------
Now that I've hopefully refreshed your memory :) I just want to make
sure that
you are still on-board with this plan. I realize that this new approach does
things differently than Libvirt does today for other platforms. Especially
x86_64. The big differences are as follows:
1. We will invoke qemu to gather the host cpu data used for virsh
capabilities.
Today this data seems to be collected directly from the host hardware
for most
(all?) architectures.
2. virsh cpu-models {arch} will also use a Qemu invocation to gather cpu
model
data.
3. Most architectures seem to use a "map" (xml file containing cpu model
data
that ships with Libvirt) to satisfy #1 and #2 above. Our new method does
not use
this map as it gets all of the data directly from Qemu.
4. virsh cpu-baseline and cpu-compare will now invoke qemu directly as well.
Note: I'm not sure exactly how much all of this will apply just to s390 with
other architectures moving over to the new interface if/when they want
to. Or if
we will want to change all architectures to this new interface at the
same time.
Any guidance?
Thanks for your time and consideration.
--
-- Jason J. Herne (jjherne(a)linux.vnet.ibm.com)
2:01 p.m.
Hi!
On Thu, Sep 22, 2016 at 02:47:36PM -0400, Jason J. Herne wrote:
Hi Jiri & Eduardo,
You might remember a discussion with David Hildenbrand of IBM on the Qemu
mailing list regarding a new Qemu<->Libvirt interface for cpu modeling. I am
picking up this work from David and I wanted to confirm that we are still on
the
same page as to the direction of that interface.
I am, but I would like to hear Jiri's feedback in case there's
anything that is still lacking in the new interface.
Additionanl comments below:
For your reference:
https://www.redhat.com/archives/libvir-list/2016-June/thread.html#01413
https://lists.gnu.org/archive/html/qemu-devel/2016-09/threads.html#00489
The first link is to the discussion you were directly involved in. The
second
link is to the final patch set posted to qemu-devel. The cover letter gives
a
good overview of the interface added to Qemu and the proposed use-case for
Libvirt to use this new cpu modeling support. I'll paste in the most
relevant
section for your convenience:
--------------------------------Libvirt usecase----------------------------
Testing for runability:
- Simply try to start QEMU with KVM, compat machine, CPU model
The main problem here is for libvirt to get machine-friendly
error information from QEMU in case the VM fails to start. But if
libvirt avoids starting QEMU based on previously-collected
runnability information, we should be OK.
- Could be done using query-cpu-model-comparison in the future.
I sent a series adding runnability information to
query-cpu-definitions as well.
Identifying host model, e.g. "virsh capabilities"
- query-cpu-model-expansion on "host" with "-M none --enable-kvm"
Correct.
<cpu mode='host-model'>:
- simply copy the identified host model
<cpu mode='host-passthrough'>:
- "-cpu host"
Correct.
"virsh cpu-baseline":
- query-cpu-model-baseline on two models with "-M none"
"virsh cpu-compare":
- query-cpu-model-comparison on two models with "-M none"
Correct.
There might be some cenarios where libvirt wants to convert another CPU
model to a static variant, this can be done using query-cpu-model-expansion
---------------------------------------------------------------------------
Now that I've hopefully refreshed your memory :) I just want to make sure
that
you are still on-board with this plan. I realize that this new approach does
things differently than Libvirt does today for other platforms. Especially
x86_64. The big differences are as follows:
1. We will invoke qemu to gather the host cpu data used for virsh
capabilities.
Today this data seems to be collected directly from the host hardware for
most
(all?) architectures.
I believe libvirt developers want to change this, because the
current approach is wrong for x86 as well.
2. virsh cpu-models {arch} will also use a Qemu invocation to gather cpu
model
data.
I believe libvirt can collect this data once, and cache it (like
it does today).
3. Most architectures seem to use a "map" (xml file containing cpu model
data
that ships with Libvirt) to satisfy #1 and #2 above. Our new method does not
use
this map as it gets all of the data directly from Qemu.
I believe the goal is to make cpu_map.xml unnecessary on x86 as
well.
4. virsh cpu-baseline and cpu-compare will now invoke qemu directly as well.
Note: I'm not sure exactly how much all of this will apply just to s390 with
other architectures moving over to the new interface if/when they want to.
Or if
we will want to change all architectures to this new interface at the same
time.
Any guidance?
I am not sure about this part, because I don't know what are the
existing use cases of cpu-baseline and cpu-compare. But I believe
this shouldn't be a problem as long as the function is not called
very often and/or results are cached by libvirt.
--
Eduardo
7:05 a.m.
On Thu, Sep 22, 2016 at 14:47:36 -0400, Jason J. Herne wrote:
Testing for runability:
- Simply try to start QEMU with KVM, compat machine, CPU model
Yes, if the domain XML explicitly requests a specific CPU model.
Additionally, we need to make sure a CPU model chosen by libvirt
(host-model) is runnable, but that should be handled by
query-cpu-model-expansion.
Identifying host model, e.g. "virsh capabilities"
- query-cpu-model-expansion on "host" with "-M none --enable-kvm"
Right, except that it doesn't seem to work on x86_64 now.
It's not critical for query-cpu-model-expansion, but do we have an
interface we can use to check whether the new commands are supported by
a QEMU binary? Trying and checking for
{"error": {"class": "GenericError", "desc":
"this feature or command
is not currently supported"}}
is not really possible. At least we'd need a specific error class.
<cpu mode='host-model'>:
- simply copy the identified host model
Yes.
<cpu mode='host-passthrough'>:
- "-cpu host"
Exactly.
...
1. We will invoke qemu to gather the host cpu data used for virsh
capabilities. Today this data seems to be collected directly from the
host hardware for most (all?) architectures.
Not really, virsh capabilities will still contain data gathered directly
from the host hardware. But, we have virsh domcapabilities for
displaying data tight to a specific QEMU binary. Since yesterday
afternoon we have support for displaying CPU related data in the domain
capabilities XML; see
http://libvirt.org/formatdomaincaps.html#elementsCPU
The host-model part of the XML will show the result of
query-cpu-model-expansion on "host" model, or the result of querying the
hardware directly if we can't ask QEMU for that (which is the current
state).
2. virsh cpu-models {arch} will also use a Qemu invocation to gather
cpu model data.
No, virsh cpu-models will just list CPU models supported by libvirt (or
an empty list if libvirt supports all models supported by QEMU). The CPU
model data from QEMU can be found in domain capabilities XML.
3. Most architectures seem to use a "map" (xml file
containing cpu
model data that ships with Libvirt) to satisfy #1 and #2 above. Our
new method does not use this map as it gets all of the data directly
from Qemu.
Even if we switch some CPU operations to work through QEMU, we may still
need to use the cpu_map.xml file for some other operations, or other
hypervisor driver. It all depends on what we need to do for a given
architecture. For example, ARM does not use cpu_map.xml at all even now.
Slightly related, I don't think we have a way to list CPU features
supported by QEMU over QMP, do we? "-cpu ?" will show them all, but I
couldn't find a QMP command that would give me the same list.
4. virsh cpu-baseline and cpu-compare will now invoke qemu directly
as
well.
Perhaps, but before we can do that, we'll probably need to come up with
new APIs. It don't think it's critical, though.
Note: I'm not sure exactly how much all of this will apply just
to
s390 with other architectures moving over to the new interface if/when
they want to. Or if we will want to change all architectures to this
new interface at the same time.
Well, IIUC the new commands are not supported on any other architecture
right now, are they? Anyway, I don't think we need to switch everything
at the same time. If we have a way to detect what commands are supported
for a specific QEMU binary, we can implement the code in libvirt and use
when we can or falling back to the current way.
Jirka
1:23 p.m.
On Fri, Sep 23, 2016 at 02:05:59PM +0200, Jiri Denemark wrote:
On Thu, Sep 22, 2016 at 14:47:36 -0400, Jason J. Herne wrote:
> Testing for runability:
> - Simply try to start QEMU with KVM, compat machine, CPU model
Yes, if the domain XML explicitly requests a specific CPU model.
Additionally, we need to make sure a CPU model chosen by libvirt
(host-model) is runnable, but that should be handled by
query-cpu-model-expansion.
> Identifying host model, e.g. "virsh capabilities"
> - query-cpu-model-expansion on "host" with "-M none
--enable-kvm"
Right, except that it doesn't seem to work on x86_64 now.
Yeah, I still need to implement it.
It's not critical for query-cpu-model-expansion, but do we have an
interface we can use to check whether the new commands are supported by
a QEMU binary? Trying and checking for
{"error": {"class": "GenericError", "desc":
"this feature or command
is not currently supported"}}
is not really possible. At least we'd need a specific error class.
I believe query-qmp-schema and/or query-commands can be used for
that.
But you may still have the command returning the error above if
it is simply not implemented in that architecture. I will start a
qemu-devel thread to look for better solutions.
> <cpu mode='host-model'>:
> - simply copy the identified host model
Yes.
> <cpu mode='host-passthrough'>:
> - "-cpu host"
Exactly.
...
> 1. We will invoke qemu to gather the host cpu data used for virsh
> capabilities. Today this data seems to be collected directly from the
> host hardware for most (all?) architectures.
Not really, virsh capabilities will still contain data gathered directly
from the host hardware. But, we have virsh domcapabilities for
displaying data tight to a specific QEMU binary. Since yesterday
afternoon we have support for displaying CPU related data in the domain
capabilities XML; see
http://libvirt.org/formatdomaincaps.html#elementsCPU
The host-model part of the XML will show the result of
query-cpu-model-expansion on "host" model, or the result of querying the
hardware directly if we can't ask QEMU for that (which is the current
state).
> 2. virsh cpu-models {arch} will also use a Qemu invocation to gather
> cpu model data.
No, virsh cpu-models will just list CPU models supported by libvirt (or
an empty list if libvirt supports all models supported by QEMU). The CPU
model data from QEMU can be found in domain capabilities XML.
That's interesting to know. I was often confused about the
interfaces that seem to be generic (not depend on a specific QEMU
binary) but seemed to depend on information from the QEMU binary
(virsh capabilities and virsh cpu-models). It looks like I was
looking at the wrong commands.
> 3. Most architectures seem to use a "map" (xml file containing cpu
> model data that ships with Libvirt) to satisfy #1 and #2 above. Our
> new method does not use this map as it gets all of the data directly
> from Qemu.
Even if we switch some CPU operations to work through QEMU, we may still
need to use the cpu_map.xml file for some other operations, or other
hypervisor driver. It all depends on what we need to do for a given
architecture. For example, ARM does not use cpu_map.xml at all even now.
Slightly related, I don't think we have a way to list CPU features
supported by QEMU over QMP, do we? "-cpu ?" will show them all, but I
couldn't find a QMP command that would give me the same list.
device-list-properties should return all the properties you can
set. But I recommend that you don't make libvirt set any of the
properties in the list unless: 1) you know what it does; or 2) it
is returned by a query-cpu-model-expansion call.
> 4. virsh cpu-baseline and cpu-compare will now invoke qemu directly as
> well.
Perhaps, but before we can do that, we'll probably need to come up with
new APIs. It don't think it's critical, though.
> Note: I'm not sure exactly how much all of this will apply just to
> s390 with other architectures moving over to the new interface if/when
> they want to. Or if we will want to change all architectures to this
> new interface at the same time.
Well, IIUC the new commands are not supported on any other architecture
right now, are they? Anyway, I don't think we need to switch everything
at the same time. If we have a way to detect what commands are supported
for a specific QEMU binary, we can implement the code in libvirt and use
when we can or falling back to the current way.
Right now the commands are available only on s390x. I plan to
have them implemented in x86 before QEMU 2.8 is out.
--
Eduardo
9:21 a.m.
On Fri, Sep 23, 2016 at 15:23:45 -0300, Eduardo Habkost wrote:
On Fri, Sep 23, 2016 at 02:05:59PM +0200, Jiri Denemark wrote:
> It's not critical for query-cpu-model-expansion, but do we have an
> interface we can use to check whether the new commands are supported by
> a QEMU binary? Trying and checking for
>
> {"error": {"class": "GenericError",
"desc": "this feature or command
> is not currently supported"}}
>
> is not really possible. At least we'd need a specific error class.
I believe query-qmp-schema and/or query-commands can be used for
that.
But you may still have the command returning the error above if
it is simply not implemented in that architecture. I will start a
qemu-devel thread to look for better solutions.
Yes, that's my concern.
...
> No, virsh cpu-models will just list CPU models supported by
libvirt (or
> an empty list if libvirt supports all models supported by QEMU). The CPU
> model data from QEMU can be found in domain capabilities XML.
That's interesting to know. I was often confused about the
interfaces that seem to be generic (not depend on a specific QEMU
binary) but seemed to depend on information from the QEMU binary
(virsh capabilities and virsh cpu-models). It looks like I was
looking at the wrong commands.
Those are just old interfaces. We may add newer versions for some of
them that would take more input parameters, but I don't feel a strong
need for it right now.
And the domain capabilities XML gained support for showing CPU related
stuff only recently (it will be included in the upcoming 2.3.0 release),
so it's not like you had a choice where to look at :-)
...
> Slightly related, I don't think we have a way to list CPU
features
> supported by QEMU over QMP, do we? "-cpu ?" will show them all, but I
> couldn't find a QMP command that would give me the same list.
device-list-properties should return all the properties you can
set. But I recommend that you don't make libvirt set any of the
properties in the list unless: 1) you know what it does; or 2) it
is returned by a query-cpu-model-expansion call.
The use case was for pre-checking whether all CPU features specified in
domain XML are supported by current QEMU binary. What parameters would
I need to use for device-list-properties? And would it work with
-machine none?
...
> Well, IIUC the new commands are not supported on any other
architecture
> right now, are they? Anyway, I don't think we need to switch everything
> at the same time. If we have a way to detect what commands are supported
> for a specific QEMU binary, we can implement the code in libvirt and use
> when we can or falling back to the current way.
Right now the commands are available only on s390x. I plan to
have them implemented in x86 before QEMU 2.8 is out.
That would be nice.
Jirka
1:51 p.m.
On Thu, Sep 29, 2016 at 04:21:07PM +0200, Jiri Denemark wrote:
[...]
> > Slightly related, I don't think we have a way to list
CPU features
> > supported by QEMU over QMP, do we? "-cpu ?" will show them all, but
I
> > couldn't find a QMP command that would give me the same list.
>
> device-list-properties should return all the properties you can
> set. But I recommend that you don't make libvirt set any of the
> properties in the list unless: 1) you know what it does; or 2) it
> is returned by a query-cpu-model-expansion call.
The use case was for pre-checking whether all CPU features specified in
domain XML are supported by current QEMU binary. What parameters would
I need to use for device-list-properties? And would it work with
-machine none?
You just need the right QOM type name (in x86 it is
"<model>-x86_64-cpu" or "<model>-i386-cpu"). I will send a
patch
that includes a "qom-type" field in CpuDefinitionInfo
(query-cpu-definitions) so you know what's the right QOM type
name to query.
I think device-list-properties won't work with an abstract base
type like "x86_64-cpu" or "i386-cpu". But I think it's better
query the right class name for the CPU model you plan to use,
anyway (this way architecture code will be able to have different
CPU models supporting different sets of options, if necessary).
--
Eduardo
2 a.m.
On Thu, Sep 29, 2016 at 15:51:58 -0300, Eduardo Habkost wrote:
On Thu, Sep 29, 2016 at 04:21:07PM +0200, Jiri Denemark wrote:
[...]
> > > Slightly related, I don't think we have a way to list CPU features
> > > supported by QEMU over QMP, do we? "-cpu ?" will show them all,
but I
> > > couldn't find a QMP command that would give me the same list.
> >
> > device-list-properties should return all the properties you can
> > set. But I recommend that you don't make libvirt set any of the
> > properties in the list unless: 1) you know what it does; or 2) it
> > is returned by a query-cpu-model-expansion call.
>
> The use case was for pre-checking whether all CPU features specified in
> domain XML are supported by current QEMU binary. What parameters would
> I need to use for device-list-properties? And would it work with
> -machine none?
You just need the right QOM type name (in x86 it is
"<model>-x86_64-cpu" or "<model>-i386-cpu"). I will send
a patch
that includes a "qom-type" field in CpuDefinitionInfo
(query-cpu-definitions) so you know what's the right QOM type
name to query.
Hmm, it doesn't seem to work (I tried both -machine none and -machine
pc):
(QEMU) device-list-properties typename=SandyBridge-x86_64-cpu
{"error": {"class": "GenericError", "desc":
"Can't list properties of
device 'SandyBridge-x86_64-cpu'"}}
Anyway, is the list of properties expected to depend on the CPU model?
If not, wouldn't a simple query-cpu-properties returning a list of
properties QEMU understands be good enough? The list would be pretty
similar to what -cpu ? returns. It would help us provide a better error
in case a user asks for a CPU features that is not supported by their
QEMU binary, and it would also help us translate the result of
query-cpu-model-expansion; as you probably know, we use older spellings
for some CPU features (e.g., pclmulqdq vs. pclmuldq).
I think device-list-properties won't work with an abstract base
type like "x86_64-cpu" or "i386-cpu".
Right:
(QEMU) device-list-properties typename=x86_64-cpu
{"error": {"class": "GenericError", "desc":
"Parameter 'name' expects
non-abstract device type"}}
But I think it's better query the right class name for the CPU
model
you plan to use, anyway (this way architecture code will be able to
have different CPU models supporting different sets of options, if
necessary).
I see, this make sense. Although a generic query-cpu-properties would
still be useful for giving us the translation table between new and old
features names.
Jirka
8:49 a.m.
On Fri, Sep 30, 2016 at 09:00:15AM +0200, Jiri Denemark wrote:
On Thu, Sep 29, 2016 at 15:51:58 -0300, Eduardo Habkost wrote:
> On Thu, Sep 29, 2016 at 04:21:07PM +0200, Jiri Denemark wrote:
> [...]
> > > > Slightly related, I don't think we have a way to list CPU
features
> > > > supported by QEMU over QMP, do we? "-cpu ?" will show them
all, but I
> > > > couldn't find a QMP command that would give me the same list.
> > >
> > > device-list-properties should return all the properties you can
> > > set. But I recommend that you don't make libvirt set any of the
> > > properties in the list unless: 1) you know what it does; or 2) it
> > > is returned by a query-cpu-model-expansion call.
> >
> > The use case was for pre-checking whether all CPU features specified in
> > domain XML are supported by current QEMU binary. What parameters would
> > I need to use for device-list-properties? And would it work with
> > -machine none?
>
> You just need the right QOM type name (in x86 it is
> "<model>-x86_64-cpu" or "<model>-i386-cpu"). I will
send a patch
> that includes a "qom-type" field in CpuDefinitionInfo
> (query-cpu-definitions) so you know what's the right QOM type
> name to query.
Hmm, it doesn't seem to work (I tried both -machine none and -machine
pc):
(QEMU) device-list-properties typename=SandyBridge-x86_64-cpu
{"error": {"class": "GenericError", "desc":
"Can't list properties of
device 'SandyBridge-x86_64-cpu'"}}
Oops, we need to remove cannot_destroy_with_object_finalize_yet
from the CPU classes, and then this will work.
Anyway, is the list of properties expected to depend on the CPU model?
If not, wouldn't a simple query-cpu-properties returning a list of
properties QEMU understands be good enough? The list would be pretty
similar to what -cpu ? returns. It would help us provide a better error
in case a user asks for a CPU features that is not supported by their
QEMU binary, and it would also help us translate the result of
query-cpu-model-expansion; as you probably know, we use older spellings
for some CPU features (e.g., pclmulqdq vs. pclmuldq).
I see. I will check if I can fit this information inside
query-command-line-options.
> I think device-list-properties won't work with an abstract base
> type like "x86_64-cpu" or "i386-cpu".
Right:
(QEMU) device-list-properties typename=x86_64-cpu
{"error": {"class": "GenericError", "desc":
"Parameter 'name' expects
non-abstract device type"}}
> But I think it's better query the right class name for the CPU model
> you plan to use, anyway (this way architecture code will be able to
> have different CPU models supporting different sets of options, if
> necessary).
I see, this make sense. Although a generic query-cpu-properties would
still be useful for giving us the translation table between new and old
features names.
If we add the aliases for old names like you sugested on
qemu-devel, the old names should be available in the QOM queries,
too.
If we fix device-list-properties to work with abstract types and
add the old names to the QOM property list as aliases (both
things we should do eventually, anyway), then a specific
query-cpu-properties or query-command-line-options support may be
unnecessary.
--
Eduardo
2:51 p.m.
On 09/23/2016 08:05 AM, Jiri Denemark wrote:
On Thu, Sep 22, 2016 at 14:47:36 -0400, Jason J. Herne wrote:
...
> 1. We will invoke qemu to gather the host cpu data used for
virsh
> capabilities. Today this data seems to be collected directly from the
> host hardware for most (all?) architectures.
Not really, virsh capabilities will still contain data gathered directly
from the host hardware. But, we have virsh domcapabilities for
displaying data tight to a specific QEMU binary. Since yesterday
afternoon we have support for displaying CPU related data in the domain
capabilities XML; see
http://libvirt.org/formatdomaincaps.html#elementsCPU
The host-model part of the XML will show the result of
query-cpu-model-expansion on "host" model, or the result of querying the
hardware directly if we can't ask QEMU for that (which is the current
state).
So the expectation here is that virsh capabilities" reports actual host
cpu data. So for S390, we can leave our implementation of this "as-is"
and not report any features here, I'm guessing. And the <cpu>
section from virsh domcapabilities will contain the Qemu specific
supported cpu modeling info. As you stated <mode name='host-model'>
will contain the name (and possibly feature details?) of the model
returned by qmp query-model-expansion on 'host'. Furthermore, the
<mode name='custom'> section will list all supported model names, as
indicated by qmp query-cpu-definitions. Do I have it right?
> 2. virsh cpu-models {arch} will also use a Qemu invocation to
gather
> cpu model data.
No, virsh cpu-models will just list CPU models supported by libvirt
So, in other words we just spit out whatever models Libvirt managed to
grab, and cache, from a call to qmp query-cpu-definitions? Or am I
missing something?
(or an empty list if libvirt supports all models supported by QEMU).
Can you clarify? It seems odd that an empty list would be returned here.
I thought the point was to list all supported cpu models. For x86 today
I imagine it is the list found in cpu_map.xml. For s390, this will be
the list we get back from qmp query-cpu-definitions, which as you mention
below, is found in the domain capabilities XML.
The CPU model data from QEMU can be found in domain capabilities XML.
> 3. Most architectures seem to use a "map" (xml file containing cpu
> model data that ships with Libvirt) to satisfy #1 and #2 above. Our
> new method does not use this map as it gets all of the data directly
> from Qemu.
Even if we switch some CPU operations to work through QEMU, we may still
need to use the cpu_map.xml file for some other operations, or other
hypervisor driver. It all depends on what we need to do for a given
architecture. For example, ARM does not use cpu_map.xml at all even now.
Slightly related, I don't think we have a way to list CPU features
supported by QEMU over QMP, do we? "-cpu ?" will show them all, but I
couldn't find a QMP command that would give me the same list.
query-model-expansion will return all features of a given model name. Model
names can be enumerated via query-cpu-definitions.
> 4. virsh cpu-baseline and cpu-compare will now invoke qemu
directly as
> well.
Perhaps, but before we can do that, we'll probably need to come up with
new APIs. It don't think it's critical, though.
Do you mind elaborating on this a bit? Which APIs do we want to look at
here? Are you talking about new commands/calls to replace cpu-baseline
and cpu-compare?
Thanks again for your time.
--
-- Jason J. Herne (jjherne(a)linux.vnet.ibm.com)
7:38 a.m.
Polite Ping? :)
On 09/23/2016 03:51 PM, Jason J. Herne wrote:
On 09/23/2016 08:05 AM, Jiri Denemark wrote:
> On Thu, Sep 22, 2016 at 14:47:36 -0400, Jason J. Herne wrote:
...
>> 1. We will invoke qemu to gather the host cpu data used for virsh
>> capabilities. Today this data seems to be collected directly from the
>> host hardware for most (all?) architectures.
>
> Not really, virsh capabilities will still contain data gathered directly
> from the host hardware. But, we have virsh domcapabilities for
> displaying data tight to a specific QEMU binary. Since yesterday
> afternoon we have support for displaying CPU related data in the domain
> capabilities XML; see
> http://libvirt.org/formatdomaincaps.html#elementsCPU
> The host-model part of the XML will show the result of
> query-cpu-model-expansion on "host" model, or the result of querying the
> hardware directly if we can't ask QEMU for that (which is the current
> state).
So the expectation here is that virsh capabilities" reports actual host
cpu data. So for S390, we can leave our implementation of this "as-is"
and not report any features here, I'm guessing. And the <cpu>
section from virsh domcapabilities will contain the Qemu specific
supported cpu modeling info. As you stated <mode name='host-model'>
will contain the name (and possibly feature details?) of the model
returned by qmp query-model-expansion on 'host'. Furthermore, the
<mode name='custom'> section will list all supported model names, as
indicated by qmp query-cpu-definitions. Do I have it right?
>> 2. virsh cpu-models {arch} will also use a Qemu invocation to gather
>> cpu model data.
>
> No, virsh cpu-models will just list CPU models supported by libvirt
So, in other words we just spit out whatever models Libvirt managed to
grab, and cache, from a call to qmp query-cpu-definitions? Or am I
missing something?
> (or an empty list if libvirt supports all models supported by QEMU).
Can you clarify? It seems odd that an empty list would be returned here.
I thought the point was to list all supported cpu models. For x86 today
I imagine it is the list found in cpu_map.xml. For s390, this will be
the list we get back from qmp query-cpu-definitions, which as you mention
below, is found in the domain capabilities XML.
> The CPU model data from QEMU can be found in domain capabilities XML.
>
>> 3. Most architectures seem to use a "map" (xml file containing cpu
>> model data that ships with Libvirt) to satisfy #1 and #2 above. Our
>> new method does not use this map as it gets all of the data directly
>> from Qemu.
>
> Even if we switch some CPU operations to work through QEMU, we may still
> need to use the cpu_map.xml file for some other operations, or other
> hypervisor driver. It all depends on what we need to do for a given
> architecture. For example, ARM does not use cpu_map.xml at all even now.
>
> Slightly related, I don't think we have a way to list CPU features
> supported by QEMU over QMP, do we? "-cpu ?" will show them all, but I
> couldn't find a QMP command that would give me the same list.
>
query-model-expansion will return all features of a given model name. Model
names can be enumerated via query-cpu-definitions.
>> 4. virsh cpu-baseline and cpu-compare will now invoke qemu directly as
>> well.
>
> Perhaps, but before we can do that, we'll probably need to come up with
> new APIs. It don't think it's critical, though.
Do you mind elaborating on this a bit? Which APIs do we want to look at
here? Are you talking about new commands/calls to replace cpu-baseline
and cpu-compare?
Thanks again for your time.
--
-- Jason J. Herne (jjherne(a)linux.vnet.ibm.com)
10:15 a.m.
On Fri, Sep 23, 2016 at 15:51:12 -0400, Jason J. Herne wrote:
On 09/23/2016 08:05 AM, Jiri Denemark wrote:
> The host-model part of the XML will show the result of
> query-cpu-model-expansion on "host" model, or the result of querying the
> hardware directly if we can't ask QEMU for that (which is the current
> state).
So the expectation here is that virsh capabilities" reports actual host
cpu data. So for S390, we can leave our implementation of this "as-is"
and not report any features here, I'm guessing.
Yes.
And the <cpu> section from virsh domcapabilities will contain
the Qemu
specific supported cpu modeling info. As you stated <mode
name='host-model'> will contain the name (and possibly feature
details?) of the model returned by qmp query-model-expansion on
'host'.
Right, the host-model part should basically contain the CPU
configuration which libvirt will use if a user asks for host-model. For
example (on x86_64, since I have no experience with s390), the following
XML
<mode name='host-model' supported='yes'>
<model fallback='forbid'>Skylake-Client</model>
<vendor>Intel</vendor>
<feature policy='require' name='vmx'/>
<feature policy='require' name='tsc_adjust'/>
</mode>
would result in "-cpu Skylake-Client,+vmx,+tsc_adjust" QEMU command line.
Furthermore, the <mode name='custom'> section will list
all supported
model names, as indicated by qmp query-cpu-definitions. Do I have it
right?
Mostly, the list of supported CPUs is filtered by libvirt so on some
architectures, the <mode name='custom'> section will list fewer models
than what you can get from QEMU.
>> 2. virsh cpu-models {arch} will also use a Qemu invocation
to gather
>> cpu model data.
>
> No, virsh cpu-models will just list CPU models supported by libvirt
So, in other words we just spit out whatever models Libvirt managed to
grab, and cache, from a call to qmp query-cpu-definitions?
No. CPU models supported by libvirt are listed in cpu_map.xml. So we
have two basic sets of CPU models:
- A: models returned query-cpu-definitions
- B: models found in cpu_map.xml
Various interfaces report different sets:
- virsh cpu-models reports B
- <mode name='custom'> section of domain capabilities XML reports A ∩ B
> (or an empty list if libvirt supports all models supported by
QEMU).
Can you clarify?
This is a special case of B being empty, which means libvirt does not
filter CPU models and any model supported by QEMU can be used with
libvirt. In this case, <mode name='custom'> section of domain
capabilities XML will contain A. But virsh cpu-models will report an
empty list.
...
>> 4. virsh cpu-baseline and cpu-compare will now invoke qemu
directly as
>> well.
>
> Perhaps, but before we can do that, we'll probably need to come up with
> new APIs. It don't think it's critical, though.
Do you mind elaborating on this a bit? Which APIs do we want to look at
here? Are you talking about new commands/calls to replace cpu-baseline
and cpu-compare?
The virConnectCompareCPU, virConnectBaselineCPU don't have enough
parameters to be really useful. We'd need them to take more data (such
as emulator binary, domain type, and machine type) into account when
computing the results. But since we now provide CPU related data in
domain capabilities, I don't think it's critical to create new public
APIs for comparing CPUs. I think we may focus on the internals first.
Jirka
12:55 p.m.
On 09/29/2016 11:15 AM, Jiri Denemark wrote:
On Fri, Sep 23, 2016 at 15:51:12 -0400, Jason J. Herne wrote:
> On 09/23/2016 08:05 AM, Jiri Denemark wrote:
>> The host-model part of the XML will show the result of
>> query-cpu-model-expansion on "host" model, or the result of querying
the
>> hardware directly if we can't ask QEMU for that (which is the current
>> state).
>
> So the expectation here is that virsh capabilities" reports actual host
> cpu data. So for S390, we can leave our implementation of this "as-is"
> and not report any features here, I'm guessing.
Yes.
> And the <cpu> section from virsh domcapabilities will contain the Qemu
> specific supported cpu modeling info. As you stated <mode
> name='host-model'> will contain the name (and possibly feature
> details?) of the model returned by qmp query-model-expansion on
> 'host'.
Right, the host-model part should basically contain the CPU
configuration which libvirt will use if a user asks for host-model. For
example (on x86_64, since I have no experience with s390), the following
XML
<mode name='host-model' supported='yes'>
<model fallback='forbid'>Skylake-Client</model>
<vendor>Intel</vendor>
<feature policy='require' name='vmx'/>
<feature policy='require' name='tsc_adjust'/>
</mode>
would result in "-cpu Skylake-Client,+vmx,+tsc_adjust" QEMU command line.
Jiri,
In order to properly obtain the host cpu model data for virsh
capabilities on
s390 we will need to run a Qemu process. There is no precedent for doing
this
in a cpu connection driver today. I can imagine two ways to do this:
1. The s390 cpu connection driver (src/cpu/cpu_s390.c) can just
privately make
use of the default Qemu binary and the appropriate qmp calls can be made to
get the model name and possibly features.
2. We can extend the existing interface somehow so all archs could make
use of
a hypervisor instance for nodeData() and decode() operations. Perhaps a
new set
of callbacks? nodeDataWithHypervisor(), decodeWithHypervisor() ? We'd
have to
modify the generic versions of these functions to try one set of interfaces,
then the other in case the first set is not supported for a given arch.
Do either of these sound sane to you, or am I way off on this?
I'm trying to get the host cpu model for virCapsPtr caps, as it is filled in
via virQEMUCapsInit --> virQEMUCapsInitCPU. And subsequently referenced for
the output of virsh capabilities and copied into qemuCaps for reference by
virsh domcapabilities.
--
-- Jason J. Herne (jjherne(a)linux.vnet.ibm.com)
11:41 a.m.
On Wed, Oct 12, 2016 at 13:55:49 -0400, Jason J. Herne wrote:
In order to properly obtain the host cpu model data for virsh
capabilities on s390 we will need to run a Qemu process. There is no
precedent for doing this in a cpu connection driver today.
There are two reasons for this:
1) it was not required so far
2) it doesn't belong to the CPU driver
1. The s390 cpu connection driver (src/cpu/cpu_s390.c) can just
privately make use of the default Qemu binary and the appropriate qmp
calls can be made to get the model name and possibly features.
No, the CPU driver is supposed to do generic stuff independently on any
hypervisor.
2. We can extend the existing interface somehow so all archs could
make use of a hypervisor instance for nodeData() and decode()
operations. Perhaps a new set of callbacks? nodeDataWithHypervisor(),
decodeWithHypervisor() ?
Why? The hypervisor code that calls CPU driver APIs can just decide to
do something else for certain architectures.
I'm trying to get the host cpu model for virCapsPtr caps, as it
is
filled in via virQEMUCapsInit --> virQEMUCapsInitCPU. And subsequently
referenced for the output of virsh capabilities and copied into
qemuCaps for reference by virsh domcapabilities.
Anyway, host CPU capabilities in virCapsPtr is not supposed to contain
the CPU description queried from QEMU. If we have no generic code to
detect host CPU ourselves, we should just keep host CPU model empty.
The host CPU model queried from QEMU should be placed in virQEMUCapsPtr.
When probing QEMU, we cache data we get from QEMU and the host CPU model
is supposed to be computed virQEMUCapsInitHostCPUModel from the cached
data. (Currently, the function copies the model from virCapsPtr, but
that's only because we don't query QEMU for the right stuff yet.)
Jirka
3:13 a.m.
CCing Christian
On Thu, Sep 22, 2016 at 14:47:36 -0400, Jason J. Herne wrote:
> Testing for runability:
> - Simply try to start QEMU with KVM, compat machine, CPU model
Yes, if the domain XML explicitly requests a specific CPU model.
Additionally, we need to make sure a CPU model chosen by libvirt
(host-model) is runnable, but that should be handled by
query-cpu-model-expansion.
> Identifying host model, e.g. "virsh capabilities"
> - query-cpu-model-expansion on "host" with "-M none
--enable-kvm"
Right, except that it doesn't seem to work on x86_64 now.
It's not critical for query-cpu-model-expansion, but do we have an
interface we can use to check whether the new commands are supported by
a QEMU binary? Trying and checking for
{"error": {"class": "GenericError", "desc":
"this feature or command
is not currently supported"}}
is not really possible. At least we'd need a specific error class.
> <cpu mode='host-model'>:
> - simply copy the identified host model
Yes.
> <cpu mode='host-passthrough'>:
> - "-cpu host"
Exactly.
...
> 1. We will invoke qemu to gather the host cpu data used for virsh
> capabilities. Today this data seems to be collected directly from the
> host hardware for most (all?) architectures.
Not really, virsh capabilities will still contain data gathered directly
from the host hardware. But, we have virsh domcapabilities for
displaying data tight to a specific QEMU binary. Since yesterday
afternoon we have support for displaying CPU related data in the domain
capabilities XML; see
http://libvirt.org/formatdomaincaps.html#elementsCPU
The host-model part of the XML will show the result of
query-cpu-model-expansion on "host" model, or the result of querying the
hardware directly if we can't ask QEMU for that (which is the current
state).
> 2. virsh cpu-models {arch} will also use a Qemu invocation to gather
> cpu model data.
No, virsh cpu-models will just list CPU models supported by libvirt (or
an empty list if libvirt supports all models supported by QEMU). The CPU
model data from QEMU can be found in domain capabilities XML.
> 3. Most architectures seem to use a "map" (xml file containing cpu
> model data that ships with Libvirt) to satisfy #1 and #2 above. Our
> new method does not use this map as it gets all of the data directly
> from Qemu.
Even if we switch some CPU operations to work through QEMU, we may still
need to use the cpu_map.xml file for some other operations, or other
hypervisor driver. It all depends on what we need to do for a given
architecture. For example, ARM does not use cpu_map.xml at all even now.
Slightly related, I don't think we have a way to list CPU features
supported by QEMU over QMP, do we? "-cpu ?" will show them all, but I
couldn't find a QMP command that would give me the same list.
> 4. virsh cpu-baseline and cpu-compare will now invoke qemu directly as
> well.
Perhaps, but before we can do that, we'll probably need to come up with
new APIs. It don't think it's critical, though.
> Note: I'm not sure exactly how much all of this will apply just to
> s390 with other architectures moving over to the new interface if/when
> they want to. Or if we will want to change all architectures to this
> new interface at the same time.
Well, IIUC the new commands are not supported on any other architecture
right now, are they? Anyway, I don't think we need to switch everything
at the same time. If we have a way to detect what commands are supported
for a specific QEMU binary, we can implement the code in libvirt and use
when we can or falling back to the current way.
Jirka
David
3033
days inactive
3054
days old
13 comments
4 participants
participants (4)
-
David Hildenbrand -
Eduardo Habkost -
Jason J. Herne -
Jiri Denemark