Hi, Apparently, reporting a level 3 cache on a virtual CPU can dramatically increase performance in some use cases [1]. The interesting part is that l3-cache=on does not provide the real CPU cache data, it's just making it up. Anyway, we should be able to enable this via libvirt. And since there is another property which enables real CPU cache data to be passed to a guest, I suggest the following /domain/cpu/cache element equivalent to l3-cache=on: <cache level='3' mode='emulate'/> If we need to add support for passing the real CPU cache data, we can do that with <cache level='3' mode='passthrough'/> or we can even make the level attribute optional and support <cache mode='passthrough'/> Missing cache element means default behaviour of the hypervisor and we can eventually add <cache mode='disable'/> to turn off passing any CPU cache info to the guest. But I think we should now focus only on <cache level='3' mode='emulate'/> and leaving the rest for the future when we actually need it. This is how a more complete example would look like: <cpu mode='custom' match='exact'> <model>Broadwell</model> <cache level='3' mode='emulate'/> </cpu> And libvirt would translate it into -cpu Broadwell,l3-cache=on. Do you have any thoughts about the XML schema or naming?

On Thu, Apr 20, 2017 at 10:02:38 +0100, Daniel P. Berrange wrote: > On Tue, Apr 18, 2017 at 04:22:36PM +0200, Jiri Denemark wrote: > > Hi, > > > > Apparently, reporting a level 3 cache on a virtual CPU can dramatically > > increase performance in some use cases [1]. The interesting part is that > > l3-cache=on does not provide the real CPU cache data, it's just making > > it up. Anyway, we should be able to enable this via libvirt. And since > > there is another property which enables real CPU cache data to be passed > > to a guest, I suggest the following /domain/cpu/cache element equivalent > > to l3-cache=on: > > > > <cache level='3' mode='emulate'/> > > > > If we need to add support for passing the real CPU cache data, we can do > > that with > > > > <cache level='3' mode='passthrough'/> > > > > or we can even make the level attribute optional and support > > > > <cache mode='passthrough'/> > > > > Missing cache element means default behaviour of the hypervisor and we > > can eventually add <cache mode='disable'/> to turn off passing any CPU > > cache info to the guest. > > > > But I think we should now focus only on <cache level='3' mode='emulate'/> > > and leaving the rest for the future when we actually need it. > > > > This is how a more complete example would look like: > > > > <cpu mode='custom' match='exact'> > > <model>Broadwell</model> > > <cache level='3' mode='emulate'/> > > </cpu> > > > > And libvirt would translate it into -cpu Broadwell,l3-cache=on. > > > > Do you have any thoughts about the XML schema or naming? > > The second QEMU property 'host-cache-info' causes the guest to see > the host cache topology. This affects L1, L2 and L3 caches all at > once. > > We could allow use of '<cache>' without specifying a level. ie > > <cache mode="passthrough"/> Yes, this is what I suggested a few lines above. However, it seems host-cache-info is only supported for -cpu host.

> to indicate passthrough of L1,L2 & L3 cache all together, mapping > to host-cache-info=on. and the <cache level=3 mode=emulate> > mapping to the l3-cache=on. > > These two elements would need to be mutually exclusive. Mixing them could allow host cache info to be passed through with specific level-N cache data overridden by additional <cache level='N'/> element. QEMU does not complain of both host-cache-info and l3-cache are enabled. But anyway, making them mutually exclusive is probably the

safest option. However, what would happen if <cpu mode='host-passthrough'> <cache level='3' mode='emulate'/> </cpu> is specified in domain XML? -cpu host,l3-cache=on or -cpu host,host-cache-info=off,l3-cache=on

And similarly, I guess <cache mode="passthrough"/> should just unconditionally disable l3-cache if we want to be sure they are always mutually exclusive.

On Thu, Apr 20, 2017 at 11:07:27 +0100, Daniel P. Berrange wrote: > On Thu, Apr 20, 2017 at 11:53:21AM +0200, Jiri Denemark wrote: > > On Thu, Apr 20, 2017 at 10:02:38 +0100, Daniel P. Berrange wrote: > > > On Tue, Apr 18, 2017 at 04:22:36PM +0200, Jiri Denemark wrote: > > > > Hi, > > > > > > > > Apparently, reporting a level 3 cache on a virtual CPU can dramatically > > > > increase performance in some use cases [1]. The interesting part is that > > > > l3-cache=on does not provide the real CPU cache data, it's just making > > > > it up. Anyway, we should be able to enable this via libvirt. And since > > > > there is another property which enables real CPU cache data to be passed > > > > to a guest, I suggest the following /domain/cpu/cache element equivalent > > > > to l3-cache=on: > > > > > > > > <cache level='3' mode='emulate'/> > > > > > > > > If we need to add support for passing the real CPU cache data, we can do > > > > that with > > > > > > > > <cache level='3' mode='passthrough'/> > > > > > > > > or we can even make the level attribute optional and support > > > > > > > > <cache mode='passthrough'/> > > > > > > > > Missing cache element means default behaviour of the hypervisor and we > > > > can eventually add <cache mode='disable'/> to turn off passing any CPU > > > > cache info to the guest. > > > > > > > > But I think we should now focus only on <cache level='3' mode='emulate'/> > > > > and leaving the rest for the future when we actually need it. > > > > > > > > This is how a more complete example would look like: > > > > > > > > <cpu mode='custom' match='exact'> > > > > <model>Broadwell</model> > > > > <cache level='3' mode='emulate'/> > > > > </cpu> > > > > > > > > And libvirt would translate it into -cpu Broadwell,l3-cache=on. > > > > > > > > Do you have any thoughts about the XML schema or naming? > > > > > > The second QEMU property 'host-cache-info' causes the guest to see > > > the host cache topology. This affects L1, L2 and L3 caches all at > > > once. > > > > > > We could allow use of '<cache>' without specifying a level. ie > > > > > > <cache mode="passthrough"/> > > > > Yes, this is what I suggested a few lines above. However, it seems > > host-cache-info is only supported for -cpu host. > > I don't think there's any dependancy on -cpu host actually. > > Semantically host-cache-info=on, only makes sense if the sockets/core/threads > topology of the guest, matches the host. What other CPUID bits you choose is > tangential. QEMU currently supports host-cache-info only for -cpu host. "Property '.host-cache-info' not found" if you try with any other CPU model. And device-list-properties QMP command confirms that, only host-x86_64-cpu type lists host-cache-info property. We'd probably need to check whether the property is available for the specific CPU model :(