upvote
No third party can ever provide the privacy and sovereignty of being able to run locally with abliterated models. I would do it this way even if the providers were much cheaper as I value those properties a -lot-.

In practice though, I can produce about 3B tokens/month of Qwen 3.8 Flash Next which is the model I crank almost 24/7. That would cost something like 1k/mo on OpenRouter. My setup costs about $300/mo in electricity, though getting solar soon to drop that to zero.

reply
It doesn’t really drop the cost of electricity to zero; you have to amortize the cost of the panels over their lifetime, plus the opportunity cost of the capital outlay (or interest on the loan)
reply
That is fair. Still, dramatically reduced cost and reliance on one less third party.
reply
Running locally is not a problem if you allow it to be slow. I get about 3.5 tps with Qwen3.8-27B:UD-Q4_K_XL on my 64 GB laptop and about the same tps with Qwen3.8-Flash-Next:UD-IQ4_XS (IIRC, or UD-Q4_K_XL) on my 128 GB desktop with a 16 GB GPU.
reply