First, it does not really reflect my experience. I find modern models to be largely interchangeable when it comes to common software engineering tasks, including some of the more recent open weight ones. The strongest models become interesting for Ph.D. level of work for me, which is not something I deal with regularly anymore.
None of them are perfect; they all have their strengths and shortcomings, but I can reasonably work with all of them and it honestly doesn't matter much anymore. I do sometimes wonder what exactly people are doing to see such a difference; at a minimum, their workflows must be very different from mine.
Second, you're arguing based on subscription pricing. But Anthropic makes the bulk of its revenue (and has its biggest margins) from enterprise/API pricing, where they are the most expensive provider by far. Anthropic isn't going to make a lot of money off your $20 subscription. With a subscription, you currently get more usage per dollar out of Anthropic's than OpenAI's plans, which approximately offsets the difference in API pricing.
If you think that, it means you're not demanding enough from your models.
Opus 5.5 still makes a lot of mistakes. Even if it can technically do all tasks you throw at it, there's no way it can do it at 100% reliability.
To be clear, AI writes 100% of my code since last year. But it still produces bugs, needs multiple prompts, and has less intuition than a human.
My usual workflow (not counting throwaway code) is to use agents as virtual pair programmers, which gives me a good balance of quality and velocity.
Each interaction with the agent remains individually comprehensible and makes progress towards one of the following goals:
- Understanding the problem/solution space.
- Implementing a solution.
- Understanding the solution, especially any agent-written parts and interaction with existing code.
- Evaluating the solution (tests, code analysis, what-if scenarios, consider edge cases).
This does not just involve the model generating code, but also explaining or analyzing it as needed.
This process allows me to retain a mental model in lockstep with developing the written code and have some assurance both of its functionality and non-functional requirements. It also fits in nicely with most existing team-level workflows.
But it's also something that I can do with pretty much any of the modern models.
There's two thresholds.
1. Tasks models can do. 2. Tasks models can do with enough reliability that you can YOLO merge without looking at the code.
Most models can do most of 1. That's no longer the interesting part.
2. is quickly expand. I can trust opus 5.5 enough with easy to medium tasks that I can YOLO merge them. For open models, I can only trust them with easy tasks if I want to YOLO merge.
I'd love to have a stronger model that I can trust to YOLO merge hard tasks.
The recent update to SlopCodeBench that includes Astra and Opus 5.5 (including the insight that higher reasoning levels may not improve the situation) seems to match my intuition. [1]
[1] https://github.com/humanlayer/advanced-context-engineering-f...
I struggle to find myself ever feeling comfortable doing this in a production environment at my day job.
These days your average Opus 5.5 PR with multiple passes of review by Opus 5.5 and Sol 6.1 agents is much less likely to contain bugs than your pre-AI PR.
Having said that, I was working on an AI for a certain video game, and I had Astra 6 and Sol 6/6.1 work for 30 hours straight, with some human steering to get it unstuck every now and then. It could not get it to work right, and the code it created was a mess.
When Opus 5.5 came out in like 4 hours it cracked it.
I didn't try but I'm very confident if Astra failed, the open models would've failed too.
Personal efficiency is noted as never hitting my limit despite it doing huge amounts of work at a time (ex: rewrite my notes app in swift for ios and install on my phone, 2 hours later its opened on my phone, looks nice, is working (base set of features). No follow up prompts, around 2 hours of work. 45% of my 5-hour limit.
Work efficiency is spend, task complexity, code quality (how often i have to steer - rare now), and wiring it up to more stuff (ex having it setup metrics and dashboards etc etc).
tl;dr: You should try Opus 5.5 and report back.
All indications are the models are increasingly worth less and less and rapidly approaching zero. Even worse, everyone is rapidly shifting to “router” architecture where they switch models constantly. LLMs are perhaps the least sticky product in the history of tech.
The only money to be made is selling compute to run models on. The only real out I see for Anthropic is if somehow compute becomes so constrained that they can flip to sell all their committed compute at a markup to run commodity models. Absent something like that it would appear they don’t have a viable business plan. As many have been pointing out, they casually left Q3 off their charts. Put that back in and suddenly the charts look very worrisome.
What many of us experienced this year is that Fable 5 and Opus 5.5 are good enough and future model improvements may have diminishing returns even if benchmarks improve.
I’m very excited for smarter models but what I really want now is not capability but speed, cost and open weights.
Every model release people say the model is good enough, yet Opus and Sol are still more popular than Deepseek. The frontier is shortening, but there are still considerable room for models to improve. SlopCodeBench is new and not close to being saturated.
This is great for us, but that behavior is the behavior of a commoditized product.
Right now OpenAI and Claude (and maybe Gemini) are over the finish line with reasonable prices for many tasks whereas the rest are still in the race.
But what happens when the open weight models cross the line and it just comes down to price?
Somewhere in there is a definition of commoditization.
Probably meant Gemini Argon