Even if it vaguely is, why do people think subjective anecdotal thoughts about which model is better are a good discussion topic. If there’s an objective evaluation idea that would be useful.
I run evals for a very specific domain. Opus 5.5 is still the industry leader (we can’t use Fable class yet).
It's relevant because there's only so many "yep it's down" comments that are interesting. They have a technology problem which isn't fixed. They're about to go IPO. Their marketing is clearly still working, but subjective experience with both contradicts the marketing.
> I run evals for a very specific domain.
What do your evals test for?
you'd rather have a subject that does exactly as told, or that does what's right?