I think we just need to decouple "doing better on benchmarks" and "actually more useful to me", since they've clearly diverged

Working with models that aren’t heavily shaped by weird regulatory targeting is wholesale different. I suspect a lot of the safety cycles are ruinous. I’m not sure if it’s the model or the harness, but I care less and less.

The commercial companies and their models are too weird, like their CEOs. I joked recently that it’s so much better when I don’t know, by first name, the CEO of the model I’m using. I really don’t want to bank on the guiding hand of Sam, Dario, Elon, or Sergey (not CEO but he’s baaaack).

Open source models will tear down the proprietary frontier configs.

The economics catching up with the providers in regards to how much compute they can burn per request and have it make sense for them financially?

A sort of model collapse where Opus 5 seems to love throwing out long paragraphs of text and it needs to be "fixed" by changing the output style and other patches.

I'm not sure, it might also catch up to Kimi K3 and GLM 5.3 and the models that I'm moving to from Anthropic.

models are incentivized by their makers to burn through as many tokens as they possibly can, so long as the customer doesn't cancel.

>Opus 4.8 and Opus 5 seems worse models than Opus 4.6

After all we've heard about benchmark cheating, I'm earnestly not sure which or whether benchmarks are reliable anymore. But, beyond the models, I wonder if changes to their harnesses and/or instructions dumb them down. I have noticed models change their behavior, even when using the same version/effort. Sometimes for better. Sometimes for worse.

And, I have noticed a model go from really good to struggling. On 4.8 things were going well for a good stretch, so I did not switch to 5 when it came out. Even after hearing complaints about 5, 4.8 was still going well. Then, suddenly over the last few days, 4.8 seems to have nosedived. It feels similar now to the complaints I hear about 5.

In my case, it suddenly started ignoring my design guide, and introducing new fonts etc. It would even use several different fonts and sizes, as well as different margins for similar elements within the same page. It abandoned classes and started inlining styles. It started feeling random and, even after it realized it needed to go back to the design guide, it just continued with more of the same.

There seems to be something that happens after new model releases in both quality and behavior of previous models. It may not be immediately, but eventually there is frequently some regression.

I was a 4.6 acolyte from April til the fable drop, lost that quick, cancelled and took a break, came back a month later, tried opus 5 and liked it, so unpinned 4.6.

Results were great at first, and they're still not terrible, but I have noticed a regression in accuracy, so to speak, where I am pointing out issues that are quite obvious in review.

I pretty much use sonnet 5 low/medium when I have a plan to solve a simple problem and depending on scope, opus low/medium for more complex/bigger scope implementation, and only go high when it's very complex or I'm spitballing architecture/solutions and iterating plan. Never go xhigh or max.

The verbosity is insane though, opus 5 documents everything and just regurgitates whatever lead it to the design choice in there, which makes it more opaque because it's talking about something that was discussed once in a session that no one else can see (except their backend ofc)

I don't even try to steer it away from that with harness, because it doesn't work and just ends up agonizing over whether it should write some comment. Three paragraphs waffling on that on verbose output

I did however have it write a script that basically is git add -A -p for comments though, haha.

I'm $20/month, have all my telemetry toggles off, don't really over engineer prompt/context, just some basic skills for repeated patterns.

It was the wrong time for the GP to drop that subscription from $200 to $20, because $200 gets you a metric assload of cognition while $20 gets you nothing beyond what a local model running on your own graphics card can deliver.

You ship 3 hours in three with AI. The old number is meaningless now.