Also, Linus is also comfortable closing 100 PRs as "this looks like garbage". Many communities are softer and try to be more helpful with new contributors.
> Also, Linus is also comfortable closing 100 PRs as "this looks like garbage".
The problem isn't just the code - the documentation for the change explaining what it fixes, how, why, what testing has been done, and so forth is often either lacking, far to verbose, or both (massively verbose without actually containing much useful information or other sense).
You raise a valid point for the future: when the average quality of submissions is good enough that it is impossible to tell the difference between human (or human aided by AI but giving its output sufficient review/rewrite) and LLM/agent submissions, then the issue will have gone away. But, given the problems projects like this are seeing it is obvious that the technology (or the people using it, probably both) are not nearly there yet.
> The best LLM policy…
That is a great policy. If only the entire world would follow it, or in fact even ever read it.
If the only reliable way for now to not be suffocated under a mass of bad AI submissions is to refuse anything that even vaguely sniffs of AI, then that is what many projects are going to have to do. If that offends you, then fork the project and patch your changes in, or vide code one from scratch yourself, then you can keep maintaining that along with all the submissions you get for fixes. Let time tell which the right decision is :-)
If the author really did one-shot the PR, then the author must have also put in the extremely detailed and tedious work which would be required to tame LLM output for this specific repo, in which case, they actually put in real effort, applyed expertise and thoughtfulness, which is the whole issue at hand.
They don’t need to know. The policy is not a linter that will scan each PR. Just like the law does not monitor everyone 24/7 for criminal behavior. It’s enough that the rule is in place and it’s enforced when they catch an infraction. No need to be perfect as long as it lowers the pain.
> AI stream is a lot higher than what human review can take seriously, leading to either a longer list of un-reviewed PR's or not fully reviewed code and quality issues. Both thing we don't want in the project.
It’s not about detecting AI, it’s about reducing the amount of code to review.
This sort of thing, in the minds of not just you but many other people, is exactly the problem. On behalf of the projects that you have communicated with this way: thanks a bundle.
That fairly obviously doesn't turn into "bugs per hour" to verify, but if you find 10 candidates in an hour and then get reproducers of them out over the next few days, they are real bugs.
Like sure, "not properly reviewing" could be a thing but if I'm looking at a hard crash on my console, that's not behavior as designed.
You don’t know the model and prompt they used to generate the report so you can’t even take the model quality into account