upvote
Just ban large prs. They can always be split up, and usually as well as helping review it makes them less risky and easier to find bugs and roll back if there was a problem.

The discipline of splitting them up helps in almost every dimension.

reply
Fully agree with a twist - ban dumb large PRs.

To add to my openspec shilling above, I've been known to "fix" large PRs by reverse engineering specs from them and building each spec out in a separate PR.

Last week me and a coworker turned a 15k line PR into 10 x 1.5k line ones that went in smoothly.

reply
Yes I do like to make a big pr first sometimes then it's usually pretty easy to split it into small ones and it's a helpful process in many ways. You can even present both together so people can see it either way, but I think in most systems it's better to have smaller prs merged one at a time.

For a rewrite, I guess sometimes it's better to have a spec then write to that, and PRs might be larger.

reply
So what do you think of Discord rewriting in rust, and AWS shipping an emulator that was ai generated? Do you think those were done with small prs? Do you think that it's bad they did those things?
reply
Please add your own thoughts to the conversation rather than aggressively asking rhetorical questions. "Counterexamples exist" is hardly a valid criticism of another human's perspective.
reply
If someone says PRs need to be small, and I say here are some examples of huge PRs that I think are good, do you think these are still bad? That is not remotely invalid.
reply
Where did grey-area say they need to be small. As in an absolute? They didn't, you just assumed that's what they meant and then went off the rails in your response.
reply
The first 4 words? "Just ban large prs." A ban is pretty absolute language. He would have banned my example PRs clearly. Or maybe he wouldn't which is why I asked about them as examples. Because that is the obvious follow up. If you ban large PRs what are you losing? You're loosing big swing aggressive efforts to make sweeping progress.
reply
> The first 4 words? "Just ban large prs."

That was a comment written in a particular context, where the original commenter was concerned with large PRs.

>> how can we review gigantic PR generated by AI.

It was not a comment written in a vacuum saying that all PRs should be small (or not large, whatever that means). Though grey-area does provide some reasonable commentary for why it's good to shrink them (easier to roll back, easier to isolate bugs, etc.). Maybe grey-area does mean to ban all large PRs, but that's not evident in this thread where they wrote a response to a specific comment offering their take on how to resolve the problem raised in that first comment.

> You're loosing big swing aggressive efforts to make sweeping progress.

Just a tip: "losing" not "loosing". We loose arrows at our enemies, we let loose the dogs of war, we loosen our purse strings. But we do not "loose big swing aggressive efforts", we "lose big swing aggressive efforts". It's funny that you get it right once, but mix the two up in the very next sentence.

reply
It's like responding to "you should wear a seatbelt, statistically they keep you safer in a crash" with "here are two people who got in a crash without a seatbelt and survived"
reply
I, personally, would certainly argue that those should be done in small PRs. If you have a working application and want to rewrite in rust, I would expect it to be done by replacing individual components one at a time while making sure relevant tests keep passing. If you're making a new application, I would expect it to be built incrementally by adding parts incrementally.
reply
Yeah, if you ended up doing it as a single massive PR how can you ever be sure that existing functionality was preserved?
reply
I think they could do those things with small PRs if they chose. If their agents are not intelligent enough to make small prs, I guess they can YOLO and assume everything is just fine with what the agents did. I do think small PRs would be better for agent review as well though, because they avoid any problems with context being overwhelmed and things being missed/forgotten.

What are the PRs you're thinking of though. If you mean a full clean rewrite, they'd probably proceed with spec first then just write it in bits to the spec. No reason it has to be large prs or smaller that's a design choice.

Often an exercise I find useful is do a quick big PR, then split it into small chunks that make sense independently for review. Or you can chain prs. Both those things can be done.

You're welcome to do large PRs, I'm not a fan of huge PRs though and would not pretend to review them (as pretending is what it would be), nor would I like to work somewhere where this is the culture.

reply
> One way could be to ask the submitter to give their prompt as part of their contribution

Zed's delta (https://delta.dev/) seems to be solving this problem by extending the version control to the conversation layer. I don't know if I'd want all my conversation to be tracked, some stuff belongs to the draft sheet.

reply
Look into [openspec](https://openspec.dev). My team is routinely passing 1-2-5k line PRs through review, with the following caveats:

1. PRs come with "the prompt". In openspec parlance this is a set of docs: a proposal, design, specs, task list.

2. I have a tiny skill to slice and dice PRs, so I can review each part of the system separately by rewriting commits in order of reviewal: specs/docs, production code, tests, evidence. I sometimes pull DB queries to a separate commit for focused review.

3. The evidence bit is built by another skill: go over all added spec scenarios and produce a markdown document with links to the code implementing the scenario and the tests proving the implementation works. Emacs helps me jump back and forth via those links and read the parts I'm interested in.

My next goal is to extend openspec schemas with custom workflow steps to make PRs land in my preferred review structure and contain the evidence report, so I don't have to build it.

reply
Your workflow looks really neat, I'll definitely give it a try.
reply
So now I get to review the prompts and a slopped PR-description? Oh man, that sounds nice.
reply
Happy to hear how you do it. We're all figuring that out.
reply
Up until you notice that in the spec an agent wrote "All operations should be atomic". But, the you observe that it keeps modifying the same value across multiple threads because there's another spec that states "No perf regression, pls".
reply
Do most people generate PRs from a one-shot prompt? I have a whole pipeline of prompts, tests and evals, which loop back and potentially restart if the result is inadequate. How would I include that in the PR?
reply
At least for my workflow, which is fairly similar, the result of the first few stages is a refined single prompt, which I then run and the result is a PR. so I, at least, would include that final single prompt over the prior intermediate results.

I also think that might not be everything you need to really vet/recreate those PRs since there's a lot of strategic and operational stuff that goes into the pre-composition runs (sometimes I actually throw everything away and restart), so something a little more sophisticated would be interesting.

reply
Good question, and I think that this is a problem that needs to be solved. Just like a commit repository, it would be lit to record the chain of thoughts of the agents you're using.
reply
Would have to include the full conversion history. But who is gonna review that it's probably as big as some of these big PRs are in the first place.
reply
Yep true, I'm still thinking about this as of today so my thoughts on this are not really mature and there are definitely a lot of caveats in that approach I didn't see.

However, having a rigorous spec process which ends up giving lots of details to the agent seems to me the right way to build stuff with LLMs. Rather than iterating and piling changes after changes.

reply
How does reviewing the long discussion the author had with the LLM prove the code is correct?
reply
Because, unless you are changing some code by hand. Your prompt and the context your giving to your agent is the seed for the outcome of the agent itself. No agent as far as I know can generate content ex-nihilo. So having a look at what people are telling their agent to do can definitely help understanding the result of the same agent in my opinion.
reply
If a PR is gigantic it means the team lead screwed up slicing the work.
reply
That is patently false in the real world we live in currently. I constantly see lazy coworkers post 2000 line PRs for a trivial change that their LLM generated when 500 lines might accomplish the same in a better way. It's almost always a lazy human accepting what Claude did by default without any manual review or asking the LLM to do anything differently.

Claude absolutely loves to reimplement the wheel locally instead of just calling a method in a library we already depend on. Every doc comment and README.md update is bloated by 2-5x what's needed to communicate intent or to actually be read by a human later. It makes random drive-by edits that aren't necessary and weren't asked for in the prompt. It's a well oiled machine for producing astronomical bloat by default if you don't supervise and rein it in!

reply
Yeah true, actually my phrasing was poor. I came to that idea through debate with other coworkers. This is not a solution for gigantic PRs but rather a method on how people should be reviewing AI changes.
reply
It also means the engineer didn't check in with the lead as their PR became unwieldy.
reply
Why, yes, blame someone else, and not the person that shat out the PR :)

I had a recent case where I, being the lead, sliced up the work real good into an epic and what not and a guy on the team just did a "hurr durr, Claude, go wild with all that" and threw a huge PR over the wall linked to 5+ stories.

Well, we got to have "the conversation" after that and learned how to fix these kinds of PRs together...

reply