> A lot more compute a lot more vram bandwidth.

But this gives you significantly more memory capacity for a compute/vram bandwidth tradeoff (i.e. arithmetic intensity) that's ultimately comparable to Blackwell, perhaps even more skewed towards compute. This seems good if you care about being able to store larger sparse models and more KV caches because that ultimately bottlenecks your ability to do inference.

Maybe. I don't know power for the MI350, but the RTX PRO 6000 is normally a 200-300W card, so you pay 2x the power for 3/4th the RAM and 3x the bandwidth. The performance frontier looks very different in terms of power to hold a mid-size model and throughout per user or across users. Price and inference stack would be big factors in what I would choose.

A great opportunity for huawei

Nobody helps China like American corps helps China. Domestic anticompetitive behavior spurs demand for international competition

I think the way it works is that the weights have to get streamed through the compute and the weights can be split very easily. With multiple cards you do have a coordination/communication phase after each layer of weights but that's usually much less significant than the compute phase.

It depends on how you slice the weights. You can split along layers which means each GPU waits for the previous one to finish or you can split the tensors across gpus.

Splitting the tensors is closer to n times the effective vram bandwidth but there are still overhead, sync waits, and reduce steps that eat away at the overall performance.