upvote
What's the sota? The classic vLLM/llama.cpp? LM Studio? Unsloth studio any good?
reply
How they have any credibility after they told everyone they could run Deepseek R1 on their laptop (by giving the Qwen 7B distill the `deepseek` moniker)...
reply
And last time I have checked, you still can't run rerankers with it, yet you can download the models. See issue 3368, 2years old now.
reply