upvote
The review burden problem mirrors what happens with internal tools generally. Teams use AI to spin up an internal system in a weekend, everyone's impressed, then six months later someone's spending half their time maintaining it. The build was never the expensive part. The review, the edge cases, the ongoing maintenance - that's where the real cost lives, whether it's OSS contributions or internal tooling
reply
deleted
reply
For well-intended open source contributions using GenAI, my current rules of thumb are:

* Prefer an issue over a PR (after iterating on the issue, either you or the maintainer can use it as a prompt)

* Only open a PR if the review effort is less than the implementation effort.

Whether the latter is feasible depends on the project, but in one of the projects I'm involved in it's fairly obvious: it's a package manager where the work is typically verifying dependencies and constraints; links to upstream commits etc are a great shortcut for reviewers.

reply
Unfortunately, LLMs generate useless word salad and nonsense even when working on issues text, you absolutely have to reword the writing from scratch otherwise it's just an annoyance and a complete waste of time. Even a good prompt doesn't help this all that much since it's just how the tool works under the hood: it doesn't have a goal of saying anything specific in the clearest possible way and inwardly rewording it until it does, it just writes stuff out that will hopefully end up seeming at least half-coherent. And their code is orders of magnitude worse than even their terrible English prose.
reply
Another great take I found online: "Don't send us a PR, send us the prompt you used to generate the PR."
reply
What I've been begging for every time someone wants me to read their AI "edited" wall of text.
reply
My rules of thumb is much shorter: don't.

The open source world has already been ripped off by AI the last thing they need is for AI to pollute the pedigree of the codebase.

reply
Suppose almost all work in the future is done via LLMs, just like almost all transportation is done today via cars instead of horses.

Do you think your worldview is still a reasonable one under those conditions?

reply
But all work isn't done by LLMs at the moment and we can't be sure that it will be so the question is ridiculous.

Maybe one day it will be.. And then people can reevaluate their stance then. Until that time, it's entirely reasonable to hold the position that you jhust don't

This is especially true with how LLM generated code may affect licensing and other things. There's a lot of unknowns there and it's entirely reasonable to not what to risk your projects license over some contributions.

I use them all the time at work because, rightly or wrongly, my company has decided that's the direction they want to go.

For open source, I'm not going to make that choice for them. If they explicitly allow for LLM generated code, then I'll use it, but if not I'm not going to assume that the project maintainers are willing to deal with the potential issues it creates.

For my own open source projects, I'm not interested in using LLM generated code. I mostly work on open source projects that I enjoy or in a specific area that I want to learn more about. The fact that it's functional software is great, but is only one of many goals of the project. AI generated code runs counter to all the other goals I have.

reply
That would only be a world where the copyright uncertainties around the output (and training!) of LLMs were a solved and known question. So that's not the world we currently live in.
reply
I say let people hold this stance. We, agentic coders, can easily enough fork their project and add whatever the features or refinements we wanted, and use that fork for ourselves, but also make it available for others in case other people want to use it for the extra features and polish as well. With AI, it's very easy to form a good architectural understanding of a large code base and figure out how to modify it in a sane, solid way that matches the existing patterns. And it's also very easy to resolve conflicts when you rebase your changes on top of whatever is new from upstream. So, maintaining a fork is really not that serious of and endeavor anymore. I'm actually maintaining a fork of Zed with several additional features (Claude Code style skills and slash commands, as well as a global agents.md file, instead of the annoying rules library system, which I removed, as well as the ability to choose models for sub-agents instead of always inheriting the model from the parent thread; and yes, master branch Zed has subagents! and another tool, jjdag)

That seems like a win-win in a sense: let the agentic coders do their thing, and the artisanal coders do their thing, and we'll see who wins in the long run.

reply
That's a pretty good framework!

Prompts from issue text makes a lot of sense.

reply
The problem was already there with lazy bug reports and inflammatory feature requests. Now there is a lazy (or inflammatory) accompanying code. But there were also well-written bug reports with no code attached due to lack of time/skills that now can potentially become useful PRs if handled with application and engineering knowledge and good faith and will.
reply
> It doesn't really matter what your stance on AI is, the problem is the increased review burden on OSS maintainers.

But the maintainers can use AI too, for their reviewing.

reply
Yes, but LLM-based reviews are not nearly a compensation for human review, so it doesn't change much.
reply
Isn't the obvious solution to not accept drive by changes?
reply
That's eliminating of an important part of open source culture.
reply
I don't think it really is - drive-by changes have been a net burden on maintainers long before LLMs started writing code. Someone who wants to put in the work to become a repeat contributor to a project is a different story.
reply
I've gotta disagree with you here - it's not uncommon for me to be diving into a library I'm using at work, find a small issue or something that could be improved (measurably, not stylistically), and open a PR to fix it. No big rewrites or anything crazy, but it would definitely fit the definition of "drive by change" that _thus far_ has been welcomed.
reply
>find a small issue

>No big rewrites or anything crazy

I think those are the key points why they've been welcomed.

reply
How to differentiate between a drive-by contribution and a first contribution from a potentially long-time contrubutor.

And I would say especially for operating systems if it gets any adoption irregular contributions are pretty legit. E.g. when someone wants just one specific piece of hardware supported that no one else has or needs without being employed by the vendor.

reply
This sounds complicated in theory, but it's easier in practice.

Potential long time contributor is somebody who was already asking annoying questions in the irc channel for a few months and helped with other stuff before shooting off th e PR. If the PR is the first time you hear from a person -- that's pretty drive-by ish.

reply
Sounds like a better way to make sure you have to be part of a clique to get your changed reviewed. I’ve been a long-time bug fixer in a few projects over the years without participating in IRC. I like the software and want it you work, but have no interest in conversing about it at that level, especially when I was conversing about software constantly at work.

I always provided well-documented PRs with a narrow scope and an obvious purpose.

reply
Why would I ask annoying questions when I can identify, reproduce, pinpoint the bug, locate it in code, and fix it? Doing it alone should make it clear I don't need to ask to understand it. And why would I be interested in small talk? Doubt many people are when they patch up their work tools. It's a dispassionate kind of kindness.

Not to mention LLMs can be annoying, too. Demand this, and you'll only be inviting bots to pester devs on IRC.

reply
Because you may misinterpret the correct fix or not know that your implementation doesn't fit the project's plans. Worse if it's LLM-generated.
reply
Hard disagree. Drive by's were the easiest to deal with, and the most welcome. Especially when the community tilted more to the side of non-amateurs and passionate people.
reply
I can understand drive-by features can be a net burden, but what is wrong with a drive-by bugfix?
reply
how in the heck do you disambiguate a first time long term contributor and a first time drive by contributor?
reply
Mostly by whether they check in first to see if the fix is actually welcome?

Drive-by folks tend to blindly fix the issue they care about, without regard to how/whether it fits into the overall project direction

reply
Your open source experience is very different from my open source experience.
reply
Sure - and I suspect we will see that soon enough. But it has downsides too, and finding the right way to vet potential contributors is tricky.
reply
> Even if we assume LLMs would consistently generate good enough quality code, code submitted by someone untrusted would still need detailed review for many reasons - so even in that case it would like be faster for the maintainers to just use the tools themselves, rather than reviewing someone else's use of the same tools.

Wouldn't an agent run by a maintainer require the same scrutiny? An agent is imo "someone else" and not a trusted maintainer.

reply
Yes, I agree. It was just me playing with a hypothetical (but in my view not imminent) future where vibe-coding without review would somehow be good enough.
reply
Project maintainers will always have the right to decide how to maintain their projects, and "owe" nothing to no one.

That being said, to outright ban a technology in 2026 on pure "vibes" is not something I'd say is reasonable. Others have already commented that it's likely unenforceable, but I'd also say it's unreasonable for the sake of utility. It leaves stuff on the table in a time where they really shouldn't. Things like documentation tracking, regression tracking, security, feature parity, etc. can all be enhanced with carefully orchestrated assistance. To simply ban this is ... a choice, I guess. But it's not reasonable, in my book. It's like saying we won't use ci/cd, because it's automated stuff, we're purely manual here.

I think a lot of projects will find ways to adapt. Create good guidelines, help the community to use the best tools for the best tasks, and use automation wherever it makes sense.

At the end of the day slop is slop. You can always refuse to even look at something if you don't like the presentation. Or if the code is a mess. Or if it doesn't follow conventions. Or if a PR is +203323 lines, and so on. But attaching "LLMs aka AI" to the reasoning only invites drama, if anything it makes the effort of distinguishing good content from good looking content even harder, and so on. In the long run it won't be viable. If there's a good way to optimise a piece of code, it won't matter where that optimisation came from, as long as it can be proved it's good.

tl;dr; focus on better verification instead of better identification; prove that a change is good instead of focusing where it came from; test, learn and adapt. Dogma was never good.

reply
Your analogy with CI/CD is flawed because while not all were convinced of the merits of CI/CD, it's also not technology built on vast energy use and copyright violation at a scale unseen in all of history, which has upended the hardware market, shaken the idea of job security for developers to its very foundation and done it while offering no really obvious benefits to groups wishing to produce really solid software. Maybe that comes eventually, but not at this level of maturity.

But you're right it's probably unenforceable. They will probably end up accepting PRs which were written with LLM assistance, but if they do it will be because it's well-written code that the contributor can explain in a way that doesn't sound to the maintainers like an LLM is answering their questions. And maybe at that point the community as a whole would have less to worry about - if we're still assuming that we're not setting ourselves up for horrible licence violation problems in the future when it turns out an LLM spat out something verbatim from a GPLed project.

reply
> Or if the code is a mess. Or if it doesn't follow conventions.

In my experience these things are very easily fixable by ai, I just ask it to follow the patterns found and conventions used in the code and it does that pretty well.

reply
I've recently worked extensively with "prompt coding", and the model we're using is very good at following such instructions early on. However after deep reasoning around problems, it tends to focus more on solving the problem at hand than following established guidelines.

Still haven't found a good way to keep it on course other than "Hey, remember that thing that you're required to do? Still do that please."

reply
A separate pre-planning step, so the context window doesn’t get too full too early on.

Off the shelf agentic coding tools should be doing this for you.

reply
At the moment verification at scale is an unsolved problem, though. As mentioned, I think this will act as a rough filter for now, but probably not work forever - and denying contributions from non-vetted contributors will likely end up being the new default.

Once outside contributions are rejected by default, the maintainers can of course choose whether or not to use LLMs or not.

I do think that it is a misconception that OSS software needs to "viable". OSS maintainers can have many motivations to build something, and just shipping a product might not be at the top of that list at all, and they certainly don't have that obligation. Personally, I use OSS as a way to build and design software with a level of gold plating that is not possible in most work settings, for the feeling that _I_ built something, and the pure joy of coding - using LLMs to write code would work directly against those goals. Whether LLMs are essential in more competitive environments is also something that there are mixed opinions on, but in those cases being dogmatic is certainly more risky.

reply
> That being said, to outright ban a technology in 2026 on pure "vibes" is not something I'd say is reasonable.

To outright accept LLM contributions would be as much "pure vibes" as banning it.

The thing is, those that maintain open source projects have to make a decision where they want to spend their time. It's open source, they are not being paid for it, they should and will decide what it acceptable and what is not.

If you dislike it, you are free to fork it and make a "LLM's welcome" fork. If, as you imply, the LLM contributions are invaluable, your fork should eventually become the better choice.

Or you can complain to the void that open source maintainers don't want to deal with low effort vibe coded bullshit PRs.

reply
Your reply is based on a 100% bad-faith, intellectually dishonest interpretation of the comment to which you’re replying. You know that. Nobody claimed that LLM code should be outright accepted. Also, nobody claimed that open source maintainers have the right to accept or decline based on whichever criteria they choose. To always come back to this point is so…American. It’s a cop-out. It’s a thought-terminating cliche. If you aren’t interested in discussing the merits of the decision, don’t bother joining the conversation. The world doesn’t need you to explain what consent is.

Most of all, I’m sick of the patronising “don’t forget that you can fork the project!” What’s the point of saying this? We all know. Nobody needs to be reminded. Nobody isn’t aware. You aren’t being clever. You aren’t adding anything to the conversation. You’re being snarky.

reply
> Nobody claimed that LLM code should be outright accepted

Not directly, but that's the implication.

I just did not pretend that was not the implication.

> always come back to this point is so…American

I am not American.

To be frank, this was the most insulting thing someone ever told me online. Congratulations. I feel insulted. You win this one.

> If you aren’t interested in discussing the merits of the decision, don’t bother joining the conversation.

I will join whatever conversation I want, and to my desires I adressed the merits of the discussion perfectly.

You are not the judge here, your opinion is as meaningless as mine.

> Most of all, I’m sick of the patronising “don’t forget that you can fork the project!” What’s the point of saying this?

That sounds like a "you" problem. You will be sick of it until the end of time, because that's the final right answer to any complaints of open source project governance.

> You aren’t adding anything to the conversation. You’re being snarky.

I disagree. In fact, I contributed more than you. I adressed arguments. You went on a whinging session about me.

reply
owing "nothing to no one" means you are allowed to be unreasonable...
reply
> That being said, to outright ban a technology in 2026 on pure "vibes" is not something I'd say is reasonable.

The response to a large enough amount of data is always vibes. You cannot analyze it all so you offload it to your intuition.

> It leaves stuff on the table in a time where they really shouldn't. Things like documentation tracking, regression tracking, security, feature parity, etc. can all be enhanced with carefully orchestrated assistance.

What’s stopping the maintainers themselves from doing just that? Nothing.

Producing it through their own pipeline means they don’t have to guess at the intentions of someone else.

Maintainers just doing it themselves is just the logical conclusion. Why go through the process of vetting the contribution of some random person who says that they’ve used AI “a little” to check if it was maybe really 90%, whether they have ulterior motives... just do it yourself.

reply
I feel like the pattern here is donate compute, not code. If agents are writing most of the software anyway, why deal with the overhead of reviewing other people's PRs? You're basically reviewing someone else's agent output when you could just run your own.

Maintainers could just accept feature requests, point their own agents at them using donated compute, and skip the whole review dance. You get code that actually matches the project's style and conventions, and nobody has to spend time cleaning up after a stranger's slightly-off take on how things should work.

reply
This is an interesting framing but it assumes maintainers want to use agents at all. Most OSS maintainers we've talked to build things because they enjoy the craft. Donating compute to replace that craft is like offering a chef a microwave.
reply
Well, it's not quite that easy because someone still has to test the agent's output and make sure it works as expected, which it often doesn't. In many cases, they still need to read the code and make sure that it does what it's supposed to do. Or they may need to spend time coming up with an effective prompt, which can be harder than it sounds for complicated projects where models will fail if you ask them to implement a feature without giving them detailed guidance on how to do so.
reply
Definitely, but that's kind of my point: the maintainers are still going to be way better at all of that than some random contributor who just wants a feature, vibe codes it, and barely tests it. The maintainers already know the codebase, they understand the implications of changes, and they can write much better plans for the agent to follow, which they can verify against. Having a great plan written down that you can verify against drastically lowers the risk of LLM-generated code
reply
Who reviews the correctness of the second agents' review?
reply
Or even more efficient: the model we already have. Donate money and let the maintainer decide whether to convert it into tokens or mash the keys themself.
reply
So your proposed solution to AI slop PRs is to "donate" compute, so the maintainers can waste their time by generating the AI slop themselves?
reply
The point isn't that agent output is magically better; it's that reviewing your own agent's output is way cheaper (intellectually) than reviewing a stranger's, because you've written the plan by yourself. And 'slop' is mostly what you get when you don't have a clear plan to verify against. Maintainers writing detailed specs for their own agents is a very different thing from someone vibe coding a feature request
reply
You’re assuming that maintainers have a desire to use agentic coding in the first place.

Secondly, it would seem that such contributions would contribute little value, if the maintainers have to write up the detailed plans by themselves, basically have to do all the work to implement the change by themselves.

reply
Open-source maintainers have no investors to placate, no competition to outrun, why would they want to use agentic coding in the first place?
reply