I'm really hoping locally hosted llms get to the point of competing with current-day frontier models so that we all have "unlimited" usage.
replyThis is the bet of many of the big AI companies, and why they're subsidizing majorly the calls. With the latest cracks by the US gov, it seems Anthropic is starting to reduce those subsidies given their edge in the game. I am starting to consider local models more seriously beside just testing, but nowadays the ram/gpu market is bloated.
replyLocal models just don't seem that useful for me for these particular tasks yet - the most recent versions of Codex and Claude Opus are the first time I've found them to be particularly useful in a "real engineering" context that isn't just vibe coding.
replyGoogle's TurboQuant might help address this, but it also might just widen the gap even further.
I am far on the skeptic edge when it comes to the generative AI side of ML tools though, so do take my opinion with that weight.