> as input tokens that are already in the KV cache are practically free for the provider,
replynot at today's RAM prices.
> How can you be sure
replyYou are free to believe that they are doing all this. Or you can simply believe the intuition that models are getting cheaper by the day. I can run Gemma 4 31B from my laptop today.
> Or you can simply believe the intuition
replySure, you can believe you intuition as much as you want, but telling strangers over the internet that they are wrong because “I trust my intuition” is… awkward.
At some point it does come to intuition. Even if the companies IPO and share their financials, you can always argue that they might be lying.
replyAgain, there's a difference between relying on intuition in your life (which we all do, lacking perfect information that would allow us to avoid relying on it), and telling someone they are wrong because your intuition says so.
reply