Re: [PATCH v2 4/4] virtio-net: Add support for USO features

On Tue, Jul 30, 2024 at 07:46:12PM +0100, Daniel P. Berrangé wrote: > On Tue, Jul 30, 2024 at 02:13:51PM -0400, Peter Xu wrote: >> On Mon, Jul 29, 2024 at 06:26:41PM +0100, Daniel P. Berrangé wrote: >>> On Mon, Jul 29, 2024 at 01:00:30PM -0400, Peter Xu wrote: >>>> On Mon, Jul 29, 2024 at 04:58:03PM +0100, Daniel P. Berrangé wrote: >>>>> >>>>> We've got two mutually conflicting goals with the machine type >>>>> definitions. >>>>> >>>>> Primarily we use them to ensure stable ABI, but an important >>>>> secondary goal is to enable new tunables to have new defaults >>>>> set, without having to update every mgmt app. The latter >>>>> works very well when the defaults have no dependancy on the >>>>> platform kernel/OS, but breaks migration when they do have a >>>>> platform dependancy. >>>>> >>>>>> - Firstly, never quietly flipping any bit that affects the ABI... >>>>>> >>>>>> - Have a default value of off, then QEMU will always allow the VM to boot >>>>>> by default, while advanced users can opt-in on new features. We can't >>>>>> make this ON by default otherwise some VMs can already fail to boot, >>>>>> >>>>>> - If the host doesn't support the feature while the cmdline enabled it, >>>>>> it needs to fail QEMU boot rather than flipping, so that it says "hey, >>>>>> this host does not support running such VM specified, due to XXX >>>>>> feature missing". >>>>>> >>>>>> That's the only way an user could understand what happened, and IMHO that's >>>>>> a clean way that we stick with QEMU cmdline on defining the guest ABI, >>>>>> while in which the machine type is the fundation of such definition, as the >>>>>> machine type can decides many of the rest compat properties. And that's >>>>>> the whole point of the compat properties too (to make sure the guest ABI is >>>>>> stable). >>>>>> >>>>>> If kernel breaks it easily, all compat property things that we maintain can >>>>>> already stop making sense in general, because it didn't define the whole >>>>>> guest ABI.. >>>>>> >>>>>> So AFAIU that's really what we used for years, I hope I didn't overlook >>>>>> somehting. And maybe we don't yet need the "-platform" layer if we can >>>>>> keep up with this rule? >>>>> >>>>> We've failed at this for years wrt enabling use of new defaults that have >>>>> a platform depedancy, so historical practice isn't a good reference. >>>>> >>>>> There are 100's (possibly 1000's) of tunables set implicitly as part of >>>>> the machine type, and of those, libvirt likely only exposes a few 10's >>>>> of tunables. The vast majority are low level details that no mgmt app >>>>> wants to know about, they just want to accept QEMU's new defaults, >>>>> while preserving machine ABI. This is a good thing. No one wants the >>>>> burden of wiring up every single tunable into libvirt and mgmt apps. >>>>> >>>>> This is what the "-platform" concept would be intended to preserve. It >>>>> would allow a way to enable groups of settings that have a platform level >>>>> dependancy, without ever having to teach either libvirt or the mgmt apps >>>>> about the individual tunables. >>>> >>>> Do you think we can achieve similar goal by simply turning the feature to >>>> ON only after a few QEMU releases? I also mentioned that idea below. >>>> >>>> https://lore.kernel.org/r/ZqQNKZ9_OPhDq2AK@x1n >>>> >>>> So far it really sounds like the right thing to do to me to fix all similar >>>> issues, even without introducing anything new we need to maintain. >>> >>> Turning a feature with a platform dependency to "on" implies that >>> the machine type will cease to work out of the box for platforms >>> which lack the feature. IMHO that's not acceptable behaviour for >>> any of our supported platforms. >> >> Right, that's why I was thinking whether we should just always be on the >> safe side, even if I just replied in the other email to Akihiko, that we do >> have the option to make this more aggresive by turning those to ON after >> even 1-2 years or even less.. and we have control of how aggressive this >> can be. >> >>> >>> IOW, "after a few QEMU releases" implies a delay of as much as >>> 5 years, while we wait for platforms which don't support the >>> feature to drop out of our supported targets list. I don't >>> think that'll satisfy the desire to get the new feature >>> available to users as soon as practical for their particular >>> platform. >> >> The feature is always available since the 1st day, right? We just need the >> user to opt-in, by specifying ON in the cmdline. >> >> That'll be my take on this that QEMU's default VM setup should be always >> bootable, migratable, and so on. Then user opt-in on stuff like this one, >> where there's implication on the ABIs. The "user" can also include >> Libvirt. I mean when something is really important, Libvirt should, IMHO, >> opt-in by treating that similarly like many cpu properties, and by probing >> the host first. >> >> IIUC there aren't a lot of things like that (part of guest ABI & host >> kernel / HW dependent), am I right? Otherwise I would expect more failures >> like this one, but it isn't as much as that yet. IIUC it means the efforts >> to make Libvirt get involved should be hopefully under control too. The >> worst case is Libvirt doesn't auto-on it, but again the user should always >> have the option to turn it on when it's necessary. > > If it is left to libvirt, then it would very likely end up being a user > opt-in, not auto-enabled. Not sure whether there's other opinions, but that's definitely fine by me. I think it even makes more sense, as even if Libvirt probed the host and auto-on the feature, it also means Libvirt made a decision for the user, saying "having a better performance" is more important than "being able to migrate this VM everywhere". I don't see a way that can make such fair decision besides requesting the user to opt-in always for those, then the user is fully aware what is enabled, with the hope that when a migration fails later with "target host doesn't support feature XXX" the user is crystal clear on what happened.