Inference costs will go down massively once they use the upcoming GPUs. I estimated that a model like GLM5.2 will be around 0.03USD/M output tokens in 2 years when the Feynman GPUs will be available in 2028. And this did not even consider architectural efficiency improvements. In mid 2027 we will already see a 10x reduction once everyone has switched to the Ruby architecture.
replyIt will be feasible for everyone to have 20 different agents running at all times. A new world is coming
Yeah I had to check, I'm paying 0.08usd per kwh. This is in the US PNW with quite a bit of local hydro power.
replyIt's all relative. On the opposite coast, Maine it is ~ $0.28 cents kwh including getting it there. (~ 50% energy, 50% delivery). It's too darn expensive here.
reply