I was a bit hazy on numbers because I don't scientifically record them, but I guess what I'll say is the fixing and verifying takes a lot longer than writing the initial code. This was true before LLMs, but when writing code by hand I had the context of the code in my head, so potential issues, blast radius, etc. was a lot more obvious. By definition most of the things that break are things that are not trivially testable. Unfortunately, it's not as easy as saying "Claude, the thing broke, plz fix"; I've had to spend a lot of time recently helping it with context from debuggers, or just debugging myself manually, adding log statements, etc.

Am I giving the agent enough context? Well, I'm giving it as much as I can. Each submodule has an AGENTS.md, I have the agents add gotchas and instructions for some feature work when I discover where an agent went wrong, the codebase has a lot of comments along the lines of "if you edit this section, you need to also edit XYZ", and I lean on the type system as much as I can to make wrong-code not compile. It has access to playwright for driving the UI if it wants to. (Weirdly, I've found that Claude is really inconsistent about using these tools -- even though the instructions make it clear that it's allowed and encouraged. I think if your workflow differs from the models training and thusly you have to tell it so in AGENTS.md/CLAUDE.md, then it's very inconsistent about following those instructions. For instance, I don't want Claude to commit and I don't want it to sign commit messages, and it still does that all the time even though it's my like #1 directive of "don't mess with my git history")

There are some things though that are very hard for it to test. I'm exporting essentially a programming language to three game engine runtimes. They all have automated tests, but, I think people that have worked in video games know that games are very hard to automate testing on. This isn't really the fault of the agent I would say, just the nature of the problem, but it is worth noting.

I guess this is a long winded way of saying, even with LLMs tech debt is a thing you have to manage, and I think managing tech debt becomes even more important when you're dealing with LLMs, not less important.

"40 hours" in his context here is actually a work day, so 7-8hrs.

He says "40 hours" because he feels like he's managed to do 40 hours worth of work in this time, but then has to spend another "40 hours" (actually: 1 day) just going around kicking tyres.

Obviously the implication is that it's a net gain of some kind, but he's unsure if he caught everything.

(sorry to reiterate the GP, but I feel like you missed the important nuance that it's not a real 40 hours of time).