Re: [PATCH] docs: outline bug expectations wrt automated tools / AI agents

On Fri, Jun 06, 2025 at 11:05:23AM +0200, Peter Krempa wrote: > On Fri, Jun 06, 2025 at 09:52:49 +0100, Daniel P. Berrangé via Devel wrote: > > From: Daniel P. Berrangé <berrange(a)redhat.com&gt; > > > > Bug reports from automated tools and AI agents are time consuming to > > triage and have poor signal/noise ratio. Set strong expectations for > > any reporters using such tools, in a (likely doomed) attempt to stem

> > the flow of poor quality reports.

> > +Use of automated tools / AI agents > > +---------------------------------- > > + > > +If any automated tool / AI agent is used to identify a bug / security > > +flaw, the following additional expectations apply when filing a report: > > + > > +- The tool / agent used **MUST** be clearly declared in the description > > +- All stated facts **MUST** be validated as correct and free from AI > > + hallucinations prior to filing > > +- The problem **MUST** be described against an upstream release that is > > + no more than 3 months old. > > +- The problem **SHOULD** be analysed and accompanied with a proposed > > + patch that can be directly applied to current git > > I'd also like to prohibit/avoid vague and too general statements. > In the few last reports that were low quality that I've seen, the > problem statement and reproducer were true because they were too vague. > > E.g. saying that "if you call this function with NULL argument it will > crash" can be true, but if we're making sure that it can't happen > elsewhere it's quite useless. > > I'm not sure though how to formulate that. I figure that kind of vague / have-wavy nonsense is often a characteristic of AI output. By requiring use of AI to be declared upfront, when we see such vague statements, we can just dismiss the bug or require the reporter to explain properly.