I was so excited when I saw "blaizzy" in the domain, because Prince Canuma's work around MLX has been of such uniquely high quality.
reply[flagged]
replyHi Simon, sorry to spam your comments but 7 months ago you asked me for a media report to back a claim I made and this week it finally arrived. perhaps a little late to be very useful to anyone who maintains that gating function, but here nonetheless:
replyhttps://ntindependent.com.au/scientist-says-ministers-pole-f...
I would also note that for people who want to download these models, you can find MLX versions of just about everything popular on huggingface these days. For instance go look at the "main" page for Qwen 3.6 35B-A3B and then follow the link to quantizations, and pick one of the more popular/reputable MLX variants.
reply Would be totally unsurprised if those modalities and models get integrated into this UI.
Support for dedicated audio-only and image-generation-only models is coming soon.
Thanks for that. My first question was “What does this do that Unsloth doesn’t?”
replyPrince CacheAsNUMA? Nice :)
replyLM Studio is trash on Windows / Linux... guess that makes sense...
replySwitching to mlx-vlm is basically harmful since it (like vllm and sglang) have such garbage support for modern samplers. To be clear, I am one of the authors on the min_p paper, and if min_p is the best you have (when llamacpp supports the far superior top-n-sigma), than I have no reason to switch even if you are somehow faster.
reply(https://arxiv.org/abs/2411.07641)
And if you do care to support modern samplers, you can start with the following:
1. https://arxiv.org/abs/2509.23234
Thanks for bringing this up. You're 100% right. But most people, even technical ones are oblivious to how much of a difference modern samplers and higher quality quantization algorithms make for on-device LLM inference and are stuck with good old top-p, top-k samplers and RTN quantization.
replyTBF, mlx-vlm does support min-p sampling, but none of the other modern samplers that you list. Ollama and LM Studio are even worse with only top-p and top-k samplers.
its a wrapper around mlx, so thats gonna be the portability bottleneck
reply