The Chinese just saved the world economy by draining their absurdly enormous oil storage reserves nobody knew they had, wouldn't surprise me if they had lots of hidden compute too.
replyIt's not like same parameter count models are identical, so that doesn't appear to be an indicator for quality, or even compute requirements?
replyThere seems to be more to producing a better model than brute forcing parameter count after all.
Training and serving large models does require increasingly more compute, though. (The Chinese labs have clearly found some massive optimizations, but my point was that you'd think at some point even those optimizations wouldn't be enough to keep up with exponentially increasing model sizes.)
reply