I think it probably is domain related. My main work for agents at the moment is on improving a crufty human-written JIT recompiler, and the number of mistakes and oversights even the very best models make is substantial. I would say maybe 50% of turns are correct semantically and don't regress performance, and that's after writing a lot of documentation, a lot of tests, and splitting up the codebase into smaller modules with well defined interfaces to reduce context size for any one change.

The first attempts I made to one shot improvements to the recompiler before doing the refactoring and test framework were basically all wrong and introduced lots of bugs, a per turn success rate in the single digit percentages. Each of the non-functional changes I made improved the performance substantially as you might expect. The real eye opener though was the refactoring into smaller units - that bumped up the success rate much more than I was expecting.

Don't get me wrong, the success rate of human changes is abysmal on these tasks, which is why the recompiler was in such a sorry state already. There are edge cases and gotchas everywhere. Opus, Astra and Gemini 4 all reason for hours on individual changes, and lesser models can churn for much longer and still fail.

Interestingly the code itself isn't really all that big - about 45000 lines of C all told.