upvote
Can you please share, if you are comfortable of course, what did the model(s) mess up? what were you using codex/cc/pi? did the project have a solid agent.md/claude.md? I am genuinely curious whenever someone have such a low success rate with models what is happening because it could be fixed maybe? From my own experience using agents for the past year or so. The rate if I have to guess, is well above 70%. I mainly use claude (opus) on typescript react projects that are well setup with minimal plugins/MCPs!

happy to share more if you are interested.

reply
Just earlier Opus 5 was supposed to migrate a Python codebase from kafka-python client-per-send to a client singleton. It decided it should add a partition key "that nobody asked for" which it admitted to at the end of the implementation.

It randomly decided going from no explicit partition key which would randomly add to a Kafka partition to partioning key company was the right solution.

In fact, that was fairly counterproductive since this codebase dumps events to the topic per company so you'd get a lumpy distribution where only one partition is ever active at once (starving other consumers)

The fact it decided it should add this without prompting was a bit shocking and I'll probably try to make it explicitly clear in the context not to "do random shit that wasn't asked for"

reply
> For instance the task naming in the task file starts with an optimistic 1, 2, 3, 5, 5a but then eventually gets to 8a, 8a1, and then ends up with 8b2c2b3 and “8b2c2b2b checkpoint1”. The code that it produced got ever more wild. I don’t want to bore you with what it tried to build, but here are some example pieces of the interpreter changes:

  Hardcoded constants everywhere
  Multiple same-line macro invocations in C
  Random indexes in production code
  Hideous tokenizer code in C
https://lucumr.pocoo.org/2026/9/7/astra-why/
reply
The reasons are highly project specific. The closer your project is to CRUD, the higher the chance of success.
reply
yes ... 'the closer it is to normative, the higher the chances of success'.
reply
If it works but the design is terrible - is that still success?
reply
In the short term yes. Long term, no. But I guess you can say the same thing about a team of engineers taking shortcuts
reply