upvote
Is it conceivable that in 10 years time we’ll have 7B models that have the same level performance as modern frontier ones?
reply
In all likelihood in a few years we'll get ~200-400bA~4-6 MoE models that are on chip, and they'll be better than the current frontier.
reply