On 09/19/2017 03:37 PM, Eduardo Habkost wrote: > Cache mode=passthrough can result in a broken cache topology if > the domain topology is not exactly the same as the host topology. > Warn about that in the documentation. > > Bug report for reference: > https://bugzilla.redhat.com/show_bug.cgi?id=1184125 > > Signed-off-by: Eduardo Habkost <ehabkost(a)redhat.com&gt; > --- > docs/formatdomain.html.in | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in > index 57ec2ff34..9c21892f3 100644 > --- a/docs/formatdomain.html.in > +++ b/docs/formatdomain.html.in > @@ -1478,7 +1478,9 @@ > > <dt><code>passthrough</code></dt> > <dd>The real CPU cache data reported by the host CPU will be > - passed through to the virtual CPU.</dd> > + passed through to the virtual CPU. Using this mode is not > + recommended unless the domain CPU and NUMA topology is exactly > + the same as the host CPU and NUMA topology.</dd> To me this sounds like it should be forbidden by libvirt, rather than just documented as "bad". (I haven't followed any previous discussion on the topic though, so maybe I'm over-reacting).

On Thu, Sep 21, 2017 at 01:14:04PM -0400, Laine Stump wrote: > On 09/19/2017 03:37 PM, Eduardo Habkost wrote: > > Cache mode=passthrough can result in a broken cache topology if > > the domain topology is not exactly the same as the host topology. > > Warn about that in the documentation. > > > > Bug report for reference: > > https://bugzilla.redhat.com/show_bug.cgi?id=1184125 > > > > Signed-off-by: Eduardo Habkost <ehabkost(a)redhat.com&gt; > > --- > > docs/formatdomain.html.in | 4 +++- > > 1 file changed, 3 insertions(+), 1 deletion(-) > > > > diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in > > index 57ec2ff34..9c21892f3 100644 > > --- a/docs/formatdomain.html.in > > +++ b/docs/formatdomain.html.in > > @@ -1478,7 +1478,9 @@ > > > > <dt><code>passthrough</code></dt> > > <dd>The real CPU cache data reported by the host CPU will be > > - passed through to the virtual CPU.</dd> > > + passed through to the virtual CPU. Using this mode is not > > + recommended unless the domain CPU and NUMA topology is exactly > > + the same as the host CPU and NUMA topology.</dd> > > To me this sounds like it should be forbidden by libvirt, rather than > just documented as "bad". (I haven't followed any previous discussion on > the topic though, so maybe I'm over-reacting). In high performance setups, people pin guest vCPUs to host pCPUs and set the vCPU topology to match the host pCPU topology they've pinned to. So ohaving a cache mode that matches this topology is just fine. It simply isn't something you want as a default for the more typical floating vCPUs scenarios.

On Mon, Nov 06, 2017 at 11:10:00AM -0200, Eduardo Habkost wrote: > On Thu, Sep 28, 2017 at 09:21:41AM +0100, Daniel P. Berrange wrote: > > On Thu, Sep 21, 2017 at 01:14:04PM -0400, Laine Stump wrote: > > > On 09/19/2017 03:37 PM, Eduardo Habkost wrote: > > > > Cache mode=passthrough can result in a broken cache topology if > > > > the domain topology is not exactly the same as the host topology. > > > > Warn about that in the documentation. > > > > > > > > Bug report for reference: > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1184125 > > > > > > > > Signed-off-by: Eduardo Habkost <ehabkost(a)redhat.com&gt; > > > > --- > > > > docs/formatdomain.html.in | 4 +++- > > > > 1 file changed, 3 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in > > > > index 57ec2ff34..9c21892f3 100644 > > > > --- a/docs/formatdomain.html.in > > > > +++ b/docs/formatdomain.html.in > > > > @@ -1478,7 +1478,9 @@ > > > > > > > > <dt><code>passthrough</code></dt> > > > > <dd>The real CPU cache data reported by the host CPU will be > > > > - passed through to the virtual CPU.</dd> > > > > + passed through to the virtual CPU. Using this mode is not > > > > + recommended unless the domain CPU and NUMA topology is exactly > > > > + the same as the host CPU and NUMA topology.</dd> > > > > > > To me this sounds like it should be forbidden by libvirt, rather than > > > just documented as "bad". (I haven't followed any previous discussion on > > > the topic though, so maybe I'm over-reacting). > > > > In high performance setups, people pin guest vCPUs to host pCPUs and > > set the vCPU topology to match the host pCPU topology they've pinned > > to. So ohaving a cache mode that matches this topology is just fine. > > It simply isn't something you want as a default for the more typical > > floating vCPUs scenarios. > > So, should this patch be applied? We could take a patch that describes more clearly when it is reasonable to use the passthrough mode.

On Mon, Nov 06, 2017 at 11:43:49AM -0200, Eduardo Habkost wrote: > On Mon, Nov 06, 2017 at 01:17:02PM +0000, Daniel P. Berrange wrote: > > On Mon, Nov 06, 2017 at 11:10:00AM -0200, Eduardo Habkost wrote: > > > On Thu, Sep 28, 2017 at 09:21:41AM +0100, Daniel P. Berrange wrote: > > > > On Thu, Sep 21, 2017 at 01:14:04PM -0400, Laine Stump wrote: > > > > > On 09/19/2017 03:37 PM, Eduardo Habkost wrote: > > > > > > Cache mode=passthrough can result in a broken cache topology if > > > > > > the domain topology is not exactly the same as the host topology. > > > > > > Warn about that in the documentation. > > > > > > > > > > > > Bug report for reference: > > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1184125 > > > > > > > > > > > > Signed-off-by: Eduardo Habkost <ehabkost(a)redhat.com&gt; > > > > > > --- > > > > > > docs/formatdomain.html.in | 4 +++- > > > > > > 1 file changed, 3 insertions(+), 1 deletion(-) > > > > > > > > > > > > diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in > > > > > > index 57ec2ff34..9c21892f3 100644 > > > > > > --- a/docs/formatdomain.html.in > > > > > > +++ b/docs/formatdomain.html.in > > > > > > @@ -1478,7 +1478,9 @@ > > > > > > > > > > > > <dt><code>passthrough</code></dt> > > > > > > <dd>The real CPU cache data reported by the host CPU will be > > > > > > - passed through to the virtual CPU.</dd> > > > > > > + passed through to the virtual CPU. Using this mode is not > > > > > > + recommended unless the domain CPU and NUMA topology is exactly > > > > > > + the same as the host CPU and NUMA topology.</dd> > > > > > > > > > > To me this sounds like it should be forbidden by libvirt, rather than > > > > > just documented as "bad". (I haven't followed any previous discussion on > > > > > the topic though, so maybe I'm over-reacting). > > > > > > > > In high performance setups, people pin guest vCPUs to host pCPUs and > > > > set the vCPU topology to match the host pCPU topology they've pinned > > > > to. So ohaving a cache mode that matches this topology is just fine. > > > > It simply isn't something you want as a default for the more typical > > > > floating vCPUs scenarios. > > > > > > So, should this patch be applied? > > > > We could take a patch that describes more clearly when it is reasonable > > to use the passthrough mode. > > Why "unless the domain CPU and NUMA topology is exactly the same > as the host CPU and NUMA topology" isn't a clear description? Just matching topology is not useful unless you've also pinned the guest CPUs to host CPUs. So I think it'd be clearer to say something like "If using 'passthrough' mode, it is recommended to explicitly pin each virtual CPU to a dedicated host CPU, and setup the guest CPU and NUMA topology to match that of the host. Mis-matched topology or freely floating CPUs will result in unpredictable performance, so should be avoided."