Note that 3.5 9B cannot do thinking (while 3.6 27B can, pretty effectively, quite verbosely).
reply3.5 9B can do thinking. Its just disabled by default in its gguf chat template.
replyIt is disabled because it doesn't work :) Try it and see the doom loop it gets itself in.
replyIs that a 1-bit LLM? I don’t understand the connection with this article.
replyOh, I don't actually know the difference if you want to explain it
replyThe title says it's 27B grade running on a phone and what I was comparing it to in my mind was a model that runs at 35B grade that could presumably run on a phone "better"?
edit: I asked AI for the difference and understand a little better, thanks for the heads up to learn the difference between models... I think the thing was, although ornith was created for a specific agentic purpose, it was still outperforming a previous generalist model I had running locally (so in my mind I thought it was still a better local model) - I'd like to try bonsai out if I can figure out how to run it lol
Orinth was not impressive in my vibes testing, I just completed my first grid analysis with real evals on qwen 27b. I can now scale that grid analysis and intend to include the qwen 9b ftunes I've seen going around. They were actually a main motivation because so many claim this or that one is better, but very little in the way of evals
replyI tried it, too, and it got stuck in some loops where it couldn’t recover. Shame, it was promising for the same reason as Bonsai’s models.
replycheck out geyron-9b, I've only used it a bit, but seems better than orinth on vibe evals
replyhuggingface.co/Tivaphraen/Geryon-9B-v1