Open weight models that run under your desk are not frontier model level, but they are getting closer. Improvements in agentic post training and things like TurboQuant mean that even if all frontier labs pull the plug tomorrow, we will still have agents to work with.
replyTurboQuant is not a step change, it's more of a smaller incremental improvement to KV quantization, and possibly (unsure) to quantization more generally. I'm actually more positive about SSD weights offload, which opens up very large local models for slow inference (good enough for slow chat) to virtually any hardware or amount of RAM.
replyI'm definitely looking forward to that, as I really want people to control their own tools.
reply