upvote
It should be expected that more tokens give diminishing returns. Minimally, there's no limit on tokens but there is on quality of output (you can't reach negative bugs, or negative execution time). The graphs I've seen show a curved "frontier" of the tradeoff, and that line has improved over model generations.

That said, the companies are incentivized to sell you tokens, and therefore to have the models use as many tokens as they think you'll let them get away with for a given task / level of performance.

reply
Happens with humans too! My senior colleagues check in with me significantly less often and cost significantly more in the meantime!
reply
This is my experience too, but even worse. Opus 5 finished the task, I then asked it to code review it, 61 agents later it came back with a bunch of errors that needed fixing. The first pass had tests, they passed, they were just wrong. I wish more people started reviewing their AI output, because I see a worrying trend of ”we have all these tests the agent wrote so it has to be good”, which is not surprising because understanding tests is not a trivial skill.
reply