I feel the same too. I like the speed.
I'm also a big fan of glm 5.2 fast. I can't wait for like 2000 t/s on these haha.
replyI wouldn't exactly call it snappy, but faster than Pro, yes.
replySingle request depth on vllm with dspark, I'm getting ~200 tps, I'd say it's pretty snappy.
replyWell sure but you're running on tens of thousands of dollars of hardware.
reply