upvote
Remember when OpenAI models loved talking about goblins and whatnot due to the RL?

https://openai.com/index/where-the-goblins-came-from/

Small quirks can quickly add up in posttraining if not caught. Although TBH with how obvious Claude language is, I do feel like this is something Anthropic probably noticed and just assumed people would not care about. Now that people have obviously cared, they're probably actively looking to alleviate it

reply
Can it be that now they are getting optimized against benchmarks that are valuing logics, rather than human appreciation? (I am not an expert at all, just an idea)
reply
It's the reinforcement learning rather than supervised learning.
reply
It is the switch from RLHF to RLVR. It benchmaxes better, but benchmarks don't cover human usability.
reply
Maybe it's the time period we're in, maybe I'm just grumpy, but it bugs me that they release a new model every single week and the new one is just a fine-tuned version of the "old" one. If 5.5 performs similar to Fable and really does cost 40% less, then 5.5 really should've just been Opus 5. And they're essentially admitting that they are shipping slop.
reply