This is totally a thing I noticed myself about 3 months ago. Medium thinking effort is ideal for most tasks. At high and above, models tend to generate more output in the form of comments or code for the same problem with no real benefit. Its a self-feeding loop: more output becomes more input, which then becomes more output. High is the highest I go. If I need more intelligence, it's better to use a more powerful model with less thinking effort or break the problem into phases. Much better result.
replyThis version of Opus "max" apparently has even higher thinking output than Qwen "max", which is infamous for its thinking streams where it constantly second-guesses itself, then third-guesses, fourth-guesses and generally nth-guesses itself for arbitrarily large n. Of course, we aren't actually seeing Claude's raw thinking output: all we get is the after-the-fact prettified "summary". One wonders how much of that is a coincidence, or whether there's a reason behind that.
replyThere’s enough thinking leakage from the recent paper and just generally catching things on Reddit. Claude models overthink and self-doubt itself just as much as Qwen, but the summariser hides much of that.
replyPersonally I use everything in low reasoning. Maybe I'm wrong but I think that the higher reasoning settings are almost never worth it, it's marginal gains for a much higher budget.
replyI also switch to a better model for more complex tasks, also in low settings
"This is a classic test request..."
replyI know there's been discussion about whether pelicanmaxxing is happening, but this is at least evidence that Claude was explicitly exposed to this problem.
Of course it was exposed - not sure it's explicit or not. Why wouldn't HackerNews comments be part of the training data? And Simon's blog and the many discussions about Pelicans? It'd be hard to miss. Doesn't mean Anthropic has made this an explicit goal in training.
replyIt would be genuinely shocking at this point if any of the frontier models weren't well exposed to the problem.
replySee here for more discussion of that: https://news.ycombinator.com/item?id=49803892#49804881
replyJust want to say: you’re such a legend, please do not stop sharing your pelicans, it’s always fun to see how they change over the months :)
replyLets start frog riding motorcycle trend until they frogmaxx, or cat driving convertible.
replyThe model recognizing the task doesn't mean it was benchmaxxed (RLVR-trained) to solve it. It might simply recognize it from pre-training on Internet text.
replyFor people with any kind of budget, Opus 5.5's [Medium] actually can make sense dollar per intelligence/dollar per task wise. Heck, it puts some other models to shame. [Max]'s cost is completely unhinged.
replyMy most exciting recent release is actually 5.6 Luna, not because it is the best on any index, but the dollar per work is insane value for money. I find myself more exciting by "value" than hypothetical ceilings because I'm just not in that budget category.
That was true for me four weeks ago, but 2-3 weeks ago Luna turned into drivel in essentially the same complexity of task. I feel it came back somewhat in recent days but does feel like it's being manipulated.
replyInteresting, I have noticed so such collapse.
replyHave you ruled out the possibility that your system prompt, AGENTS.md, or increasing codebase complexity are not to blame?
I have experienced this with open weight models too. "Max" is for benchmaxxing the intelligence metric and is not meant for use in productive work. Like drawing pelicans.
replyHey! From the Artificial Analysis team. We also have a model releases page which shows all reasoning efforts (not just max), including the trade off curves
https://artificialanalysis.ai/models/releases/claude-opus-5-...
replyI'm amazed they didn't test xhigh thinking mode explicitly to ensure it didn't exceed the 128k thinking budget allocation. I guess pace of development gets away from everyone, even OpenAI.
replyI tried to replicate your test but after 8 minutes and more than 50 lines of "thinking" by dumping seemingly random loading-screen strings like "Placing the sun, clouds, seagulls, and sea backdrop" and "Positioning the tail feathers and calculating handlebar geometry" I gave up and cancelled the task.
replyI've asked Opus 5 Max for what I thought were easy tasks at work to be completed. It always fails after reaching a tool limit.
replyI asked Opus 5 High for the same task and requested it to minimize tool usage. It produced an answer in a few minutes that I was deploying to my target platform about 30 minutes later.
How did you get the reasoning trace? Is it the actual one or the summarized one?
replyIt's the summarized one returned by their API.
replyPiping the visible reasoning trace through their token counter API (I use https://tools.simonwillison.net/claude-token-counter for that) counts 27,888 tokens, so it's definitely a summary of the 128,000 actual token trace.
I am very interested in why it was able to overthink that much. In the 20-30mins of Max reasoning I've had so far, I'm not having the same issues (yet).
reply