Mistral 4 IMHO is not that good. I have run the full NVFP4 weight, and I couldn't stomach it after 3 days of driving it. I went back to other models, I have more fun bouncing between DeepSeek, GLM, MiMo, Hy4, MiniMax & Qwen. If Thinkingmachine's self reported benchmark is true, I'm certain it's going to get into heavy rotation of my models.
replyI played around with Le Chonk on a toy project and it was ... very okay. Felt like a throwback to the Opus 4 era. It committed when I had told it not to, and it spent a lot of time arguing with itself going in circles.
replyI feel like the gap between Opus and Grok is about the same as the gap between Grok and Mistral. I didn't revert its code or anything, but I also didn't refill the initial OpenRouter budget when it ran out.
At the same time, I guess Opus 4 level is pretty good for a Western open weights model. 2027 is going to be amazing.