upvote

points

by Chamix6 hours ago |
comments
upvoteby jychang5 hours ago|
next
[-]
Nobody is running 10s of trillion param models in 2026. That's ridiculous.

Opus is 2T-3T in size at most.

reply
upvoteby johndough3 hours ago|
parent|
[-]
Do you have any clues to guess the total model size? I do not see any limitations to making models ridiculously large (besides training), and the Scaling Law paper showed that more parameters = more better, so it would be a safe bet for companies that have more money than innovative spirit.
reply
upvoteby magicalhippo2 hours ago|
parent|
[-]
> I do not see any limitations to making models ridiculously large (besides training)

From my understanding, the "besides training" is a big issue. As I noted earlier[1], Qwen3 was much better than Qwen2.5, but the main difference was just more and better training data. The Qwen3.5-397B-A17B beat their 1T-parameter Qwen3-Max-Base, again a large change was more and better training data.

[1]: https://news.ycombinator.com/item?id=47089780

reply
upvoteby aurareturn6 hours ago|
prev|
[-]
China is targeting H20 because that's all they were officially allowed to buy.
reply
upvoteby Chamix6 hours ago|
parent|
[-]
I generally agree, back of the napkin math shows H20 cluster of 8gpu * 96gb = 768gb = 768B parameters on FP8 (no NVFP4 on Hopper), which lines up pretty nicely with the sizes of recent open source Chinese models.

However, I'd say its relatively well assumed in realpolitik land that Chinese labs managed to acquire plenty of H100/200 clusters and even meaningful numbers of B200 systems semi-illicitly before the regulations and anti-smuggling measures really started to crack down.

This does somewhat beg the question of how nicely the closed source variants, of undisclosed parameter counts, fit within the 1.1tb of H200 or 1.5tb of B200 systems.

reply
upvoteby aurareturn5 hours ago|
parent|
[-]
They do not have enough H200 or Blackwell systems to server 1.6 billion people and the world so I doubt it's in any meaningful number.
reply
upvoteby Chamix5 hours ago|
parent|
[-]
I assure you, the number of people paying to use Qwen3-Max or other similar proprietary endpoints is far less than 1.6 billion.
reply
upvoteby aurareturn5 hours ago|
parent|
[-]
You don't need to assure me. It's a theoretical maximum.
reply