Re: [libvirt] Yet another RFC for CAT

On Thu, Sep 07, 2017 at 11:02:21AM +0800, 乔立勇(Eli Qiao) wrote: > > I'm concerned about the idea of not checking 'from' for collisions, > > if there's allowed a mix of guests with & within 'from'. > > eg consider > > > > * Initially 24 MB of cache is free, starting at 8MB > > * run guest A from=8M, size=8M > > * run guest B size=8M > > => libvirt sets from=16M, so doesn't clash with A > > * stop guest A > > * run guest C size=8M > > => libvirt sets from=8M, so doesn't clash with B > > * restart guest A > > => now clashes with guest C, whereas if you had > > left guest A running, then C would have > > got from=24MB and avoided clash > > > > IOW, if we're to allow users to set 'from', I think we need to > > have an explicit flag to indicate whether this is an exclusive > > or shared allocation. That way guest A would set 'exclusive', > > and so at least see an error when it got a clash with guest > > C in the example. > > > > +1 > OK, I didn't like the exclusive/shared allocation at first when I thought about it, but getting back to it it looks like it could save us from unwanted behaviour. I think that if you are setting up stuff like this you should already know what you will be running and where. I thought that specifying 'from' also means that it can be shared because you have an idea about the machines running on the host. But that's not nice and user friendly. What I'm concerned about is the difference between the following scenarios: * run guest A from=0M size=8M allocation=exclusive * run guest B from=0M size=8M allocation=shared and * run guest A from=0M size=8M allocation=shared * run guest B from=0M size=8M allocation=shared When starting guest B, how do you know whether to error out or not? I'm not considering collecting information on all domains as that "does not scale" (as the cool kids would say it these days). The only idea I have is naming the group accordingly, e.g.: libvirt-qemu-domain-3-testvm-vcpu0-3+emu+io2-7+shared/ gross name, but users immediately know what it is for.

> > > - After starting a domain, fill in any missing information about the > > > allocation (I'm generalizing here, but fro now it would only be the > > > optional "from" attribute) > > > > > > - Add settings not only for vCPUs, but also for other threads as we do > > > with pinning, schedulers, etc. > > > > > Thanks Martin to propose this again. > > I have started this RFC since the beginning of the year, and made several > junior patches, but fail to get merged. > > While recently I (together with my team) have started a software "Resource > Management Daemon" to manage resource like last level cache, do cache > allocation and cache usage monitor, it's accept tcp/unix socket REST API > request and talk with /sys/fs/resctrl interface to manage all CAT stuff. > > RMD will hidden the complexity usage in CAT and it support not only VM > but also other applications and containers. > > RMD will open source soon in weeks, and could be leveraged in libvirt > or other management software which want to have control of fine granularity > resource. > > We have done an integration POC with OpenStack Nova, and would like > to get into integrate too. > > Would like to see if libvirt can integrate with RMD too. > I'm afraid there was such effort from Marcelo called resctrltool, however it was denied for some reason. Daniel could elaborate if you'd like to know more, I can't really recall the reasoning behind it.