upvote
Seconding this. If you know how to build with a dumber model it scales crazy well. And Qwen 3.8 Flash Next is just wildly good even at low quants. I'm trying to find it's limits and I've been using it for over a week building and haven't hit a wall yet.
reply
Are you using it for coding? I'm looking for a good coding model to use for Xcode and NextJS apps
reply
Really? I hit a wall immediately, though I'm using it to implement things originally written in a proof language, and wasn't that good at prompting to begin with.
reply
deleted
reply
What are you disagreeing with exactly? Qwen 3.8 flash costs a few cents per million tokens from cloud providers. You can get effectively unlimited usage on a $6/month plan. You were okay with buying a $5000+ Mac Studio for it because you have the money, but you are in the tiny minority.
reply
I disagree with your statement saying there's no difference between self-hosting local AI and self-hosting regular software. Regular services (which I guess is what you're referring to) require uptime, to be accesible from the Internet, backups, security patches, the list goes on. The main argument against self-hosting has never been cost, it is a complex task that requires knowledge and dedication. On the other hand, you can self-host on a Raspberry Pi or ancient laptop for close to nothing.

A local AI can be as simple as a program that runs in your computer [0]. Any machine with 12GB+ of VRAM can run the same model I'm using. I has a chat interface, can see images, control your browser, generate artifacts, work on projects and workspaces. You name it.

[0] https://stratallm.org

reply
How much was your Mac Studio?
reply
This is not about cost. Any machine with 12GB+ of VRAM can run Qwen3.8-Flash-Next at usable speeds using Strata
reply