You need VRAM for the whole thing for optimal performance. Activation is chosen "randomly" for each token. PCIe becomes bottleneck, so much that just doing computation on CPU is likely faster.

But given it's only 6B, out of which only ~2.4B seem to be actually routed ("selected at random per token"), you could get reasonable performance with experts on CPU (still haven't tested, but 20-30 for dual channel DDR5 and 4 bpw quant).

I believe that at minimum, for usable performance, you need to be able to hold the 125B params + 51B ngrams in some sort of RAM.

Ideally VRAM, but the benefit of the MoE design is better performance with unified memory since most of that RAM is not read for every single token. So you could potentially have the model loaded in CPU RAM, and let unified memory systems page the relevant chunks on demand to VRAM, or run on a fully unified memory system and be able to achieve good speeds even with the limited memory bandwidth most of them have.