upvote
> AI is also scarily good at writing tests

:-) I hope you read those tests before claiming it's "scary good"

reply
Indeed, much of the scariness is how fearlessly and confidently it writes them with little regard to their actual usefulness or value. When I find it adding a lot of tests, I often say something like: "audit each test carefully, and consider whether the test is testing a meaningful boundary or is more ceremonial. delete low-value tests and add new tests to cover meaningful boundaries not exercised by the gaps you identify". Without fail, this always produces some decent results.

Having said that, in truth, I almost never read the unit tests. Before AI, we had almost none (see: several person game studio) so the tradeoff is not "AI-generated tests" vs "human written ones", it's whether we have tests at all. So, I take them for what they're worth - not much - but if it catches an extra regression before it ships every now and then, it was worth it for the price (~free).

reply
Lots of unit test do not add value in the traditional sense, but they do help the llm to understand the code.
reply
I notice that the tests are very often for its own benefit. Like, you’ll ask it to stop doing something, literally to just remove code, and it will write a test to verify the behavior is not there.

I can’t imagine a more useless test, but I get that it wants to verify that it actually made the change. I just delete the test when it’s done.

reply
What does your workflow look like tool wise? Are you still using IDE?
reply
No. I used to use Cursor, but now my workflow is that I use an inhouse CLI tool I wrote called "bud" that wraps/seeds the harnesses per-worktree, and boots a full copy of the game so each worktree can work independently. If git worktrees solve the problem of code isolation, bud solves the problem of isolating everything else. It's about 15K lines of rust, and I use it 100 times a day or so. It's sort of a layer on top of a harness like codex/claude code.

I have 10+ of these workspaces in parallel, and I context switch between them as I get blocked on things. I manage the workspaces using `herder`, which is a terrific tmux-like tool that allows me to keep those workspaces on a nixOS machine I have at home that I SSH into via tailscale, so my agents don't stop working every time I close my laptop (it also lets me leverage that machine's computing resources instead of running dozens of servers and harnesses on my poor MacBook).

reply