upvote
Yes, it runs in vllm happily. It gets >900TPS output reliably on a single 5090 with the nvfp4 model.

It's clearly worse than vanilla 26B-A4B, and lacks some things like structured outputs, and gets some tool calls wrong.

So you have to find a usecase or a hand rolled harness that leverages the cerebras-level TPS while not going off track during (even short) tasks.

reply