But the improvements in how well the model does the work are noticeably better with every good model release. As of now, orchestration of agent pipelines is ALMOST so good that I can point it at a problem, and it will solve it safely and reliably without making a mess. But it requires an agent pipeline with context gathering, refactoring, attack sessions, a sprinkle of Sol 6.1 for bug hunting, and Sol 6.0 for Daybrak security audits, and a final de-bloat/clean up pass. I’m feeling that we’re getting close to doing a lot of the work fully agentically. But I can absolutely "feel" that we are not there yet when it comes to the intelligence I need. I need moar!
I am also happy with this current state, as I believe it will at some point allow me to transition to open models once they reach the level of opus 5.5. high. But its probably futile to try predict the future in these times.
Feels like we are getting further into the adoption curve and the higher end models will now need to get into market segmentation to win.
That's true till it isn't, though. It's why it's always worth trying stronger models when they come out.
In my experience Fable is also better at findings bugs, so I go to it when I need to review critical code.
There's a trend on twitter going around telling you to paste the following into your agent, and it's the kind of work I go straight to fable+astra for:
Find all of the implicit state machines in my app. Make recommendations for changes that will elevate all three: the user experience (UX), developer experience (DX), and agent experience (AX) always, without breaking anything. Look at the code pathways for anything you find. Look through the git history for each one. Understand why code was written the way it was. Revisit each hypothesis and give me a full plan again after you're done.Opus 5.5 very frequently gets into rabbit holes and imagined edge cases which I have to intervene and tell it to ignore. It ends up taking longer and costing more tokens. It also very frequently requires me to agree on how to move forward multiple times, whereas fable just drafts the PR and tells me its ready. I have steering included in the skills to avoid some of this but Ive seen these same patterns frequently in regular opus 5.5 use in other areas as well
It could be flat web app with frontend -> api -> db later, or some very complicated system with many components and edge cases. Imo, in second case, Opus still have lots of struggle to understand system as a whole.
I guess I do have an outstanding hard problem where the SQP solver crawls across a flat-landscape. Opus 5.5 correctly tells me that is the problem, but isn't very useful in solving the problem (it has helped a bit with solver tuning, but it doesn't really know how to fix it). I have a hundred other bugs and refactoring I can do with Opus 5.5, though, where it happily chews through pretty complicated tasks for me with reasonably light hand-holding. Maybe this Saturday I take whatever credits I have left for the week and throw Fable at it and watch them evaporate.
It looks like you are working on some experimental system which is not critical atm, so you can mess around, create/fix bug, build prototypes and check them, do research of approaches, etc.
Now, imagine you have similarly complex system, but every second of downtime or bug produces lots of damage. In this case, you can't afford low intelligence vibe coding.
The fact that you deal with hundred of bugs, means that your Op5.5 won't work for second case.
For Fable, I really like to think of it as the heavyweight reviewer whenever I'm about to start so I save myself some security and/or glaring technical issue.
Eg. https://tfmbot.com is an implementation of a popular board game. Opus and even sonnet handle that part. Each week i spend my remaining credits on fable to think how to improve the ai opponents. It plays at a very high level (can get into the top 100 on BGA if i let it). Opus alone couldn’t get it to the level it’s reached.
I usually have Fable review what Opus produces. The edits are almost always valuable. For complex or high-stakes work, too, I'll have Fable review and revise my prompt before passing it to Opus and Sonnet swarms.
Why do I care? It's obvious that models base their performance from code in the vicinity, and ergo, if you have style/design/bug issues, gradually your code base will tend to get worse over time until it's unmaintainable.
Therefore, if I use an LLM, I want the output to be as pristine as possible. The first Fable release was exactly that. Since then, something happened, and I can't get the magic back.
This isn't to say the models aren't incredibly useful, of course they still are-- it's just that they need constant review and supervision if you don't want to end up with total garbage long term. I suspect the real magic is not being shared with us plebs.
Also "code review" above is often others' LLM skills reviewing the code and often flagging based on total falsehoods. I've taken to running a `/code-review` skill myself to preempt these claims (since findings by the same model tend to be pretty deterministic in my experience, even the ones that end up being wrong), usually at Opus 5.5, to preempt others' slop, and then end up arguing 90% of the points to confirm it wasn't accurate based on our usage of the system.
This is all horribly inefficient, and many aspects of our codebases across the monorepo have proliferated into spaghetti code by LLM-accelerated developers, but Fable often helps with analyzing the harder questions and saves me time on occasion arguing with (my or others') Opus findings, which are often still inaccurate for a sufficiently large interconnected system.
edit: I honestly don't know what percentile of "complexity" our typical usage is where I work, but I suspect it's a bit higher than the average. Monorepos, or at least companies with codebase sizes and demand to justify using them, probably make it worse. Opus has worked well enough for me on my personal projects, even some of the ones that have grown decently in size and complexity. I suspect part of this is an LLM extension of the old "my code makes sense, everybody else's doesn't" trend that necessitated stronger style guides in the past