The methods for splitting weights across multiple chips are well established. Groq/Cerebras can't hold a model on one chip either.

They pipeline-parallelize across multiple chips. DeepSeek v4 Pro will be 30 chips.

I think it's enough that a single layer fits on each chip if you can daisy-chain them with good interconnects.