upvote
Like before AI, the scrutiny varies with the sensitivity of the area being edited.

Simple UI change? I do an AI review, but otherwise neither read nor write the code. The models are good enough they write better UI code than me, 9 out of 10 times. Not always the more idiomatic, but usually safer and more correct.

Change to our core data plane? I might spend 2-3 times more effort reviewing it than before AI. Yes, I go more slowly than pre-AI. Many more reviews, many more angles considered, including both human and (lots of) AI review cycles.

Most code is not that critical, and AI is also scarily good at writing tests. We also spend considerably more time paying down tech debt and testing thanks to AI, now that the cost is near-zero.

Net: I spend 10-25X less time on low-risk changes. I often direct (or at least approve) the implementation approach, but I rarely read this code. I spend 2-3X more time on high-risk changes. In both cases, I never write code "by hand". Since about November, I've had no reason to actually edit code in a code editor (perhaps maybe except .env files, which we don't allow agents to edit for obvious reasons).

AI is a tool. You can use it to go fast recklessly, or you can use it to go slow with confidence. Just like before AI... the skill and art of engineering is knowing when to do which.

reply
> AI is also scarily good at writing tests

:-) I hope you read those tests before claiming it's "scary good"

reply
Indeed, much of the scariness is how fearlessly and confidently it writes them with little regard to their actual usefulness or value. When I find it adding a lot of tests, I often say something like: "audit each test carefully, and consider whether the test is testing a meaningful boundary or is more ceremonial. delete low-value tests and add new tests to cover meaningful boundaries not exercised by the gaps you identify". Without fail, this always produces some decent results.

Having said that, in truth, I almost never read the unit tests. Before AI, we had almost none (see: several person game studio) so the tradeoff is not "AI-generated tests" vs "human written ones", it's whether we have tests at all. So, I take them for what they're worth - not much - but if it catches an extra regression before it ships every now and then, it was worth it for the price (~free).

reply
Lots of unit test do not add value in the traditional sense, but they do help the llm to understand the code.
reply
I notice that the tests are very often for its own benefit. Like, you’ll ask it to stop doing something, literally to just remove code, and it will write a test to verify the behavior is not there.

I can’t imagine a more useless test, but I get that it wants to verify that it actually made the change. I just delete the test when it’s done.

reply
What does your workflow look like tool wise? Are you still using IDE?
reply
No. I used to use Cursor, but now my workflow is that I use an inhouse CLI tool I wrote called "bud" that wraps/seeds the harnesses per-worktree, and boots a full copy of the game so each worktree can work independently. If git worktrees solve the problem of code isolation, bud solves the problem of isolating everything else. It's about 15K lines of rust, and I use it 100 times a day or so. It's sort of a layer on top of a harness like codex/claude code.

I have 10+ of these workspaces in parallel, and I context switch between them as I get blocked on things. I manage the workspaces using `herder`, which is a terrific tmux-like tool that allows me to keep those workspaces on a nixOS machine I have at home that I SSH into via tailscale, so my agents don't stop working every time I close my laptop (it also lets me leverage that machine's computing resources instead of running dozens of servers and harnesses on my poor MacBook).

reply
Not parent poster but: I probably spend at least 2/3 of my tokens on code review & QA. At least at my workplace, that's the culture.
reply
Same. AI has found so many bugs I would have shipped a year ago.
reply