> in practice the MOE models with few active parameters are just faster and easier to run without too much loss of intelligence
replyIs this true for coding tasks? Genuinely asking because I don't yet have the hardware to be able to run/compare. I thought I remembered the sentiment that the 27B dense was better than the 35B MoE
The dense is better in its result, but is out of reach for most hardware compared to the MoE models, and the MoE models produce output tokens much much faster.
reply