12:04 p.m.
I've seen examples in the wild of the cluster attribute having
non-zero value on x86_64.
This is obviously quite confusing, but it's the information that
Linux exposes to userspace and we don't really have a way to tell
apart a valid die/cluster ID from a dummy one.
What ultimately matters is that the underlying assumptions about
topology are respected, which they are: in the x86_64 cases that
I have analyzed, for example, each "cluster" contained exactly
one core, so any program that would use this information to
influence guest topology decisions would be unaffected by the
additional level showing up in the hierarchy.
In an attempt to reduce confusion, document that the value for
these attributes is not necessarily going to be zero.
Signed-off-by: Andrea Bolognani <abologna(a)redhat.com>
---
docs/formatcaps.rst | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/docs/formatcaps.rst b/docs/formatcaps.rst
index f37532296f..c15d391b63 100644
--- a/docs/formatcaps.rst
+++ b/docs/formatcaps.rst
@@ -74,14 +74,18 @@ The ``<host/>`` element consists of the following child
elements:
``die_id``
Identifier for the die the CPU is in.
- Note that not all architectures support CPU dies: if the current
- architecture doesn't, the value will be 0 for all CPUs.
+ Note that, while not all architectures support CPU dies, this attribute
+ will always be present in the capabilities XML. If the architecture
+ doesn't support them, the value will likely be 0 for all CPUs, but it
+ could also be some other arbitrary value.
``cluster_id``
Identifier for the cluster the CPU is in.
- Note that not all architectures support CPU clusters: if the current
- architecture doesn't, the value will be 0 for all CPUs.
+ Note that, while not all architectures support CPU clusters, this
+ attribute will always be present in the capabilities XML. If the
+ architecture doesn't support them, the value will likely be 0 for all
+ CPUs, but it could also be some other arbitrary value.
``core_id``
Identifier for the core the CPU is in.
--
2.43.0
12:42 p.m.
On Tue, Feb 06, 2024 at 07:04:30PM +0100, Andrea Bolognani wrote:
I've seen examples in the wild of the cluster attribute having
non-zero value on x86_64.
This is obviously quite confusing, but it's the information that
Linux exposes to userspace and we don't really have a way to tell
apart a valid die/cluster ID from a dummy one.
What ultimately matters is that the underlying assumptions about
topology are respected, which they are: in the x86_64 cases that
I have analyzed, for example, each "cluster" contained exactly
one core, so any program that would use this information to
influence guest topology decisions would be unaffected by the
additional level showing up in the hierarchy.
In an attempt to reduce confusion, document that the value for
these attributes is not necessarily going to be zero.
Signed-off-by: Andrea Bolognani <abologna(a)redhat.com>
---
docs/formatcaps.rst | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/docs/formatcaps.rst b/docs/formatcaps.rst
index f37532296f..c15d391b63 100644
--- a/docs/formatcaps.rst
+++ b/docs/formatcaps.rst
@@ -74,14 +74,18 @@ The ``<host/>`` element consists of the following child
elements:
``die_id``
Identifier for the die the CPU is in.
- Note that not all architectures support CPU dies: if the current
- architecture doesn't, the value will be 0 for all CPUs.
+ Note that, while not all architectures support CPU dies, this attribute
+ will always be present in the capabilities XML. If the architecture
+ doesn't support them, the value will likely be 0 for all CPUs, but it
+ could also be some other arbitrary value.
I would remove the caveat about "not all architectures support CPU dies"
entirely, and about special die id values. I tend to say that every CPU
every manufactured supports "dies", but that most CPUs dont support more
than 1 die :-)
The purpose of exposing this info is primarily to help apps / admins
in placing guests, and matching host / guest topology where applicable.
To do that they don't need to care if a CPU supports more than 1 die
or not, they just need to see the topology reported.
If they do want to detect >1 die for some reason though, they should
not try to look for special 'die_id' values, instead look to see if
there are more than 1 *distinct* die_id within a single socket_id.
``cluster_id``
Identifier for the cluster the CPU is in.
- Note that not all architectures support CPU clusters: if the current
- architecture doesn't, the value will be 0 for all CPUs.
+ Note that, while not all architectures support CPU clusters, this
+ attribute will always be present in the capabilities XML. If the
+ architecture doesn't support them, the value will likely be 0 for all
+ CPUs, but it could also be some other arbitrary value.
Same comment as above.
``core_id``
Identifier for the core the CPU is in.
--
2.43.0
_______________________________________________
Devel mailing list -- devel(a)lists.libvirt.org
To unsubscribe send an email to devel-leave(a)lists.libvirt.org
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
9:37 a.m.
On Tue, Feb 06, 2024 at 06:42:22PM +0000, Daniel P. Berrangé wrote:
On Tue, Feb 06, 2024 at 07:04:30PM +0100, Andrea Bolognani wrote:
> ``die_id``
> Identifier for the die the CPU is in.
>
> + Note that, while not all architectures support CPU dies, this attribute
> + will always be present in the capabilities XML. If the architecture
> + doesn't support them, the value will likely be 0 for all CPUs, but it
> + could also be some other arbitrary value.
I would remove the caveat about "not all architectures support CPU dies"
entirely, and about special die id values. I tend to say that every CPU
every manufactured supports "dies", but that most CPUs dont support more
than 1 die :-)
The purpose of exposing this info is primarily to help apps / admins
in placing guests, and matching host / guest topology where applicable.
To do that they don't need to care if a CPU supports more than 1 die
or not, they just need to see the topology reported.
If they do want to detect >1 die for some reason though, they should
not try to look for special 'die_id' values, instead look to see if
there are more than 1 *distinct* die_id within a single socket_id.
That's fair, but I think it could still be useful to highlight that
the presence of the attribute in the *host* topology doesn't indicate
that attempting to use it in the *guest* topology will work, just to
avoid confusion.
How about
Note that this attribute is always present, even when the
architecture doesn't support guests with multiple CPU dies.
(and the same for clusters, of course)?
--
Andrea Bolognani / Red Hat / Virtualization
9:47 a.m.
On Thu, Feb 08, 2024 at 07:37:00AM -0800, Andrea Bolognani wrote:
On Tue, Feb 06, 2024 at 06:42:22PM +0000, Daniel P. Berrangé wrote:
> On Tue, Feb 06, 2024 at 07:04:30PM +0100, Andrea Bolognani wrote:
> > ``die_id``
> > Identifier for the die the CPU is in.
> >
> > + Note that, while not all architectures support CPU dies, this attribute
> > + will always be present in the capabilities XML. If the architecture
> > + doesn't support them, the value will likely be 0 for all CPUs, but
it
> > + could also be some other arbitrary value.
>
> I would remove the caveat about "not all architectures support CPU dies"
> entirely, and about special die id values. I tend to say that every CPU
> every manufactured supports "dies", but that most CPUs dont support more
> than 1 die :-)
>
> The purpose of exposing this info is primarily to help apps / admins
> in placing guests, and matching host / guest topology where applicable.
> To do that they don't need to care if a CPU supports more than 1 die
> or not, they just need to see the topology reported.
>
> If they do want to detect >1 die for some reason though, they should
> not try to look for special 'die_id' values, instead look to see if
> there are more than 1 *distinct* die_id within a single socket_id.
That's fair, but I think it could still be useful to highlight that
the presence of the attribute in the *host* topology doesn't indicate
that attempting to use it in the *guest* topology will work, just to
avoid confusion.
How about
Note that this attribute is always present, even when the
architecture doesn't support guests with multiple CPU dies.
The host topology info says nothing about what's possible in the
guest. The only requirement for the guest is that QEMU is new
enough to configure this. Aside from that any combo of sockets/
clusters/dies/cores/threads can be configured for the guest,
regardless of what host topology reports.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
9:52 a.m.
On Thu, Feb 08, 2024 at 03:47:06PM +0000, Daniel P. Berrangé wrote:
On Thu, Feb 08, 2024 at 07:37:00AM -0800, Andrea Bolognani wrote:
> On Tue, Feb 06, 2024 at 06:42:22PM +0000, Daniel P. Berrangé wrote:
> > On Tue, Feb 06, 2024 at 07:04:30PM +0100, Andrea Bolognani wrote:
> > > ``die_id``
> > > Identifier for the die the CPU is in.
> > >
> > > + Note that, while not all architectures support CPU dies, this
attribute
> > > + will always be present in the capabilities XML. If the architecture
> > > + doesn't support them, the value will likely be 0 for all CPUs,
but it
> > > + could also be some other arbitrary value.
> >
> > I would remove the caveat about "not all architectures support CPU
dies"
> > entirely, and about special die id values. I tend to say that every CPU
> > every manufactured supports "dies", but that most CPUs dont support
more
> > than 1 die :-)
> >
> > The purpose of exposing this info is primarily to help apps / admins
> > in placing guests, and matching host / guest topology where applicable.
> > To do that they don't need to care if a CPU supports more than 1 die
> > or not, they just need to see the topology reported.
> >
> > If they do want to detect >1 die for some reason though, they should
> > not try to look for special 'die_id' values, instead look to see if
> > there are more than 1 *distinct* die_id within a single socket_id.
>
> That's fair, but I think it could still be useful to highlight that
> the presence of the attribute in the *host* topology doesn't indicate
> that attempting to use it in the *guest* topology will work, just to
> avoid confusion.
>
> How about
>
> Note that this attribute is always present, even when the
> architecture doesn't support guests with multiple CPU dies.
The host topology info says nothing about what's possible in the
guest. The only requirement for the guest is that QEMU is new
enough to configure this. Aside from that any combo of sockets/
clusters/dies/cores/threads can be configured for the guest,
regardless of what host topology reports.
That's not correct. Even the latest version of QEMU only allows
configuring multiple dies and clusters for certain architectures.
$ qemu-system-x86_64 -M q35 -smp clusters=2
qemu-system-x86_64: clusters not supported by this machine's CPU topology
$ qemu-system-aarch64 -M virt -smp dies=2
qemu-system-aarch64: dies not supported by this machine's CPU topology
--
Andrea Bolognani / Red Hat / Virtualization
9:54 a.m.
On Thu, Feb 08, 2024 at 07:52:00AM -0800, Andrea Bolognani wrote:
On Thu, Feb 08, 2024 at 03:47:06PM +0000, Daniel P. Berrangé wrote:
> On Thu, Feb 08, 2024 at 07:37:00AM -0800, Andrea Bolognani wrote:
> > On Tue, Feb 06, 2024 at 06:42:22PM +0000, Daniel P. Berrangé wrote:
> > > On Tue, Feb 06, 2024 at 07:04:30PM +0100, Andrea Bolognani wrote:
> > > > ``die_id``
> > > > Identifier for the die the CPU is in.
> > > >
> > > > + Note that, while not all architectures support CPU dies, this
attribute
> > > > + will always be present in the capabilities XML. If the
architecture
> > > > + doesn't support them, the value will likely be 0 for all
CPUs, but it
> > > > + could also be some other arbitrary value.
> > >
> > > I would remove the caveat about "not all architectures support CPU
dies"
> > > entirely, and about special die id values. I tend to say that every CPU
> > > every manufactured supports "dies", but that most CPUs dont
support more
> > > than 1 die :-)
> > >
> > > The purpose of exposing this info is primarily to help apps / admins
> > > in placing guests, and matching host / guest topology where applicable.
> > > To do that they don't need to care if a CPU supports more than 1 die
> > > or not, they just need to see the topology reported.
> > >
> > > If they do want to detect >1 die for some reason though, they should
> > > not try to look for special 'die_id' values, instead look to see
if
> > > there are more than 1 *distinct* die_id within a single socket_id.
> >
> > That's fair, but I think it could still be useful to highlight that
> > the presence of the attribute in the *host* topology doesn't indicate
> > that attempting to use it in the *guest* topology will work, just to
> > avoid confusion.
> >
> > How about
> >
> > Note that this attribute is always present, even when the
> > architecture doesn't support guests with multiple CPU dies.
>
> The host topology info says nothing about what's possible in the
> guest. The only requirement for the guest is that QEMU is new
> enough to configure this. Aside from that any combo of sockets/
> clusters/dies/cores/threads can be configured for the guest,
> regardless of what host topology reports.
That's not correct. Even the latest version of QEMU only allows
configuring multiple dies and clusters for certain architectures.
Yes but that's a constraint of the guest machine type choice, related
to the host capabilities info.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
10:13 a.m.
On Thu, Feb 08, 2024 at 03:54:20PM +0000, Daniel P. Berrangé wrote:
On Thu, Feb 08, 2024 at 07:52:00AM -0800, Andrea Bolognani wrote:
> > > That's fair, but I think it could still be useful to highlight that
> > > the presence of the attribute in the *host* topology doesn't indicate
> > > that attempting to use it in the *guest* topology will work, just to
> > > avoid confusion.
> > >
> > > How about
> > >
> > > Note that this attribute is always present, even when the
> > > architecture doesn't support guests with multiple CPU dies.
> >
> > The host topology info says nothing about what's possible in the
> > guest. The only requirement for the guest is that QEMU is new
> > enough to configure this. Aside from that any combo of sockets/
> > clusters/dies/cores/threads can be configured for the guest,
> > regardless of what host topology reports.
>
> That's not correct. Even the latest version of QEMU only allows
> configuring multiple dies and clusters for certain architectures.
Yes but that's a constraint of the guest machine type choice, related
to the host capabilities info.
AFAICT there's no x86_64 (or ppc64, or s390x, or ...) machine type
that can use CPU clusters.
So if you're planning to run a same-architecture VM (most common use
case) you are not going to be able to use CPU clusters, which might
be confusing considering that CPU clusters appear to be supported by
your host CPU according to the capabilities XML.
This is the potential source of confusion that I'm trying to address
with the patch. I don't see how *not* mentioning this is going to be
more helpful to users.
--
Andrea Bolognani / Red Hat / Virtualization
10:22 a.m.
New subject: [PATCH] docs: Improve documentation for dies and
clusters
On Thu, Feb 08, 2024 at 08:13:26AM -0800, Andrea Bolognani wrote:
On Thu, Feb 08, 2024 at 03:54:20PM +0000, Daniel P. Berrangé wrote:
> On Thu, Feb 08, 2024 at 07:52:00AM -0800, Andrea Bolognani wrote:
> > > > That's fair, but I think it could still be useful to highlight
that
> > > > the presence of the attribute in the *host* topology doesn't
indicate
> > > > that attempting to use it in the *guest* topology will work, just to
> > > > avoid confusion.
> > > >
> > > > How about
> > > >
> > > > Note that this attribute is always present, even when the
> > > > architecture doesn't support guests with multiple CPU dies.
> > >
> > > The host topology info says nothing about what's possible in the
> > > guest. The only requirement for the guest is that QEMU is new
> > > enough to configure this. Aside from that any combo of sockets/
> > > clusters/dies/cores/threads can be configured for the guest,
> > > regardless of what host topology reports.
> >
> > That's not correct. Even the latest version of QEMU only allows
> > configuring multiple dies and clusters for certain architectures.
>
> Yes but that's a constraint of the guest machine type choice, related
> to the host capabilities info.
AFAICT there's no x86_64 (or ppc64, or s390x, or ...) machine type
that can use CPU clusters.
I believe x86 should be gaining CPU cluster support in QEMU, though
the current patch is suggesting to call them 'modules' when configuring
the guest, just to absolutely maximise confusion when linux calls
them 'clusters'.
So if you're planning to run a same-architecture VM (most common
use
case) you are not going to be able to use CPU clusters, which might
be confusing considering that CPU clusters appear to be supported by
your host CPU according to the capabilities XML.
On x86 AMD CPUs will report multiple dies while new Intel CPUs will
report multiple clusters. In both cases, you'll be able to run a
guest with multiple dies, if desire, and in future, with multiple
clusters.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
10:35 a.m.
On Thu, Feb 08, 2024 at 04:22:11PM +0000, Daniel P. Berrangé wrote:
On Thu, Feb 08, 2024 at 08:13:26AM -0800, Andrea Bolognani wrote:
> AFAICT there's no x86_64 (or ppc64, or s390x, or ...) machine type
> that can use CPU clusters.
I believe x86 should be gaining CPU cluster support in QEMU, though
the current patch is suggesting to call them 'modules' when configuring
the guest, just to absolutely maximise confusion when linux calls
them 'clusters'.
Lovely. Any chance we're still in time to prevent that specific
trainwreck from happening?
> So if you're planning to run a same-architecture VM (most
common use
> case) you are not going to be able to use CPU clusters, which might
> be confusing considering that CPU clusters appear to be supported by
> your host CPU according to the capabilities XML.
On x86 AMD CPUs will report multiple dies while new Intel CPUs will
report multiple clusters. In both cases, you'll be able to run a
guest with multiple dies, if desire, and in future, with multiple
clusters.
Are you aware of similar plans on the Arm side, to allow the use of
multiple dies for guests? What about ppc64, s390x, riscv64, ...?
Let's try one more time:
Note that, even if this attribute is present, you might not be able
to define guests with multiple CPU dies.
This should accurately describe both today's situation, as well as
whatever might come in the future to whatever architecture.
--
Andrea Bolognani / Red Hat / Virtualization
10:49 a.m.
New subject: [PATCH] docs: Improve documentation for dies and
clusters
On Thu, Feb 08, 2024 at 08:35:12AM -0800, Andrea Bolognani wrote:
On Thu, Feb 08, 2024 at 04:22:11PM +0000, Daniel P. Berrangé wrote:
> On Thu, Feb 08, 2024 at 08:13:26AM -0800, Andrea Bolognani wrote:
> > AFAICT there's no x86_64 (or ppc64, or s390x, or ...) machine type
> > that can use CPU clusters.
>
> I believe x86 should be gaining CPU cluster support in QEMU, though
> the current patch is suggesting to call them 'modules' when configuring
> the guest, just to absolutely maximise confusion when linux calls
> them 'clusters'.
Lovely. Any chance we're still in time to prevent that specific
trainwreck from happening?
I'm trying to discourage it, though their latest reply actually
indicates they want to report 'clusters' and 'modules' as distinct
concepts, so we've got even more to do on libvirt side :-)
https://lists.nongnu.org/archive/html/qemu-devel/2024-02/msg00052.html
> > So if you're planning to run a same-architecture VM
(most common use
> > case) you are not going to be able to use CPU clusters, which might
> > be confusing considering that CPU clusters appear to be supported by
> > your host CPU according to the capabilities XML.
>
> On x86 AMD CPUs will report multiple dies while new Intel CPUs will
> report multiple clusters. In both cases, you'll be able to run a
> guest with multiple dies, if desire, and in future, with multiple
> clusters.
Are you aware of similar plans on the Arm side, to allow the use of
multiple dies for guests? What about ppc64, s390x, riscv64, ...?
Honestly don't know, its all a matter for what HW vendors
choose todo.
Let's try one more time:
Note that, even if this attribute is present, you might not be able
to define guests with multiple CPU dies.
This should accurately describe both today's situation, as well as
whatever might come in the future to whatever architecture.
Sure.
Mostly what this points out is that we don't have any reporting
of available topology features in domain capabilities.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
11:37 a.m.
New subject: [PATCH] docs: Improve documentation for dies
and clusters
On Thu, Feb 08, 2024 at 04:49:44PM +0000, Daniel P. Berrangé wrote:
On Thu, Feb 08, 2024 at 08:35:12AM -0800, Andrea Bolognani wrote:
> > > AFAICT there's no x86_64 (or ppc64, or s390x, or ...) machine type
> > > that can use CPU clusters.
> >
> > I believe x86 should be gaining CPU cluster support in QEMU, though
> > the current patch is suggesting to call them 'modules' when
configuring
> > the guest, just to absolutely maximise confusion when linux calls
> > them 'clusters'.
>
> Lovely. Any chance we're still in time to prevent that specific
> trainwreck from happening?
I'm trying to discourage it, though their latest reply actually
indicates they want to report 'clusters' and 'modules' as distinct
concepts, so we've got even more to do on libvirt side :-)
https://lists.nongnu.org/archive/html/qemu-devel/2024-02/msg00052.html
It doesn't sound quite as bad then. Better than overloading an
existing term with new meaning, that's for sure. The whole CPU
topology hierarchy has gotten way beyond silly a long time ago
anyway :)
--
Andrea Bolognani / Red Hat / Virtualization
287
days inactive
289
days old
10 comments
2 participants
participants (2)
-
Andrea Bolognani -
Daniel P. Berrangé