The amazing part of DeepSeek's caching is that the TTL is (although not guaranteed) like 8-24 hours. Makes a big difference to how much time I can spend reviewing code, when most everyone else offers maybe 1 hour (for subscribers - often minutes for API users).

I mean, memory prices are high. They ain't getting cheaper. And KVCache is held in memory. The more people are using cache, the more memory you need. It's not free to cache it.

Plus there's complicated cluster-wide KVCache distribution schemes, built around RDMA networking, etc etc.

By far the most expensive hardware cost now of inference is the memory cost. KVCache is still using it. Though obv GPU compute is using more energy

It's not just like... "stick it in Redis, all done!"