yes, I average 80-120 tok/s on my RTX 3080 with gemma 4 and faster with Qwen 3.5. The main use-case here is just code-monkey agents. I'm not looking for architectural guidance, but an agent to take a spec and complete it.
replyAnd is this using conventional model loading (all in VRAM), or are you streaming it in some way?
replyin theory, QWEN3.6-35B-A3B should run as fast as a 3B model; and in practice, it can be just as dumb.
reply