Confidently yes. OpenAI for sure has been training larger models internally and distilling.
replyPre-training scaling laws all support larger models being more cost effeceint to train then smaller models. And distillation is comparably cheap. So you can get the most juice by training the biggest model you can and distilling it.
If there isn’t they’ve obviously missed an important and lucrative market.
replyIn fact, there should be more and more secret tiers for bigger and bigger money.
Ohhhh. I get it now. OpenAI is open in the sense that it's open to the public, unlike Anthropic, with special VIP access to models, like a nightclub.
replyThere's really no evidence that OpenAI has shared everything they have.
reply