undefined

Methodology leaves a lot to be desired in terms of understanding the tasks you've used. Being detailed about why they're more meaningful tests than the long horizon and coding tests used by other rankings is important.

False positives and poorly defined tasks/acceptance criteria have let some models have insanely inflated scores on bad benchmarks.

And sure, you can say they're not disclosed to prevent gaming, but if you're the only one who can review them then the might as well be a random number generator display with an unreadable UI.

by greenavocado18 hours ago|

parent|

[-]

You're not wrong, but the scores track with my experience switching between the proposed top variants. So there's my unscientific "evidence."

by nrmitchi18 hours ago|

parent|

prev|

[-]

I don't know how fast they reacted, but shortly after their documented time I started getting opus availability errors from fable requests, which seemed odd.

I'd also think that they would transparently degrade, just to prevent production outages for clients that are requesting Fable explicitly.

by steve_adams_8618 hours ago|

parent|

[-]

I mean hard to say on such short notice because they can swap out models without any notice. In terms of performance, I'm not asking it to do anything crazy so I think results would be similar across both models.

It did just use a small harness to run docker compose with different envs and other settings to validate a very small change, so... Feels like Fable

by nrmitchi17 hours ago|

parent|

[-]

No, I mean I was using fable (or, trying) and got an api error "Error: claude-opus-4-8[1m] is temporarily unavailable"

by re-thc18 hours ago|

parent|

prev|

[-]

> Maybe it's silently degrading? It's hard to say.

Opus 4.8 spams a lot more text. It'd be obvious.

by blueaquilae17 hours ago|

parent|

prev|

[-]

But token price is still fable level?

by sothatsit17 hours ago|

prev|

[-]

It is gone for me now.

> There's an issue with the selected model (claude-fable-5). It may not exist or you may not have access to it.

by AnotherGoodName17 hours ago|

parent|

[-]

Yep took a while but it's down. It's still in the model picker but it's broken

by waffletower16 hours ago|

parent|

[-]

Restart Claude Code and pick up the update to see the acknowledgement that Fable is gone.

by kip_18 hours ago|

prev|

[-]

I hadn't, but then 2.1.177 dropped in on auto-update and I assumed that was going to be the end of Fable for me, but I'm still on it. At least that's what the model picker is continuing to say along with the header.

    Claude Code v2.1.177
  Fable 5 with low effort · Claude Max
       ~/testing

Never mind, it failed a few minutes later with: There's an issue with the selected model (claude-fable-5). It may not exist or you may not have access to it. Run /model to pick a different model.

And now we're done. Oh well.

by guybedo18 hours ago|

prev|

[-]

ssshhhh don't tell anybody it's still working, i have some stuff to do :-)

by danso18 hours ago|

prev|

[-]

I was using Fable to review my codebase and came back from the gym an hour later to find that I had suddenly used up my entire Max plan quota for the next 5 hours

(I have never had an agent do enough to burn up the 5 hour quota on Max)

(edit: just switched my CC model to 4.8 and my 5-hr cycle reset back to 0%, even though it previously had 2 more hours to go)

by Tiberium18 hours ago|

prev|

[-]

I still also have access, so either they silently reroute Fable 5 to Opus 4.8 or hasn't actually pulled the switch yet.

by SXX18 hours ago|

parent|

[-]

You'll never know. They'll just silently sabotage if you're foreign national.

by reneberlin17 hours ago|

parent|