Use Opus 4.8. 5 is absolute garbage.
replyDon't use DS4 Flash in max effort mode. It's just spinning its wheels, in my experience (I have a harness for testing models with 25 real bugs/features/etc from my real projects that I measure outcomes against) DS4 flash does _worse_ with max effort. It will literally have the right approach and reason itself away from it.
I don’t know how 5 can do so much better in benchmarks but absolutely suck to use in practice compared to 4.X. Fable feels better, Kimi and GLM also feel better sometimes but tbh all of them make plenty of annoying mistakes.
replyThe prompts in most of the benchmarks match what you have to write to obtain good performance from Opus 5. Reading benchmarks is very revealing.
replyi have heard that for max effort in flash and it can be true, but overall it still performs better then the high, i run a mixed q2q4 quant.
reply