Read the tech report and see how laser-focused they've been on compressing the disk KV footprint specifically. 890 bytes/token is absurd, and is probably the reason I regularly get ~0.5 Mtok request full cache hits after an hour. On their end that's just yanking a 414 MiB file off an SSD, then doing a little bit of compute (bounded SWA replay). Nobody else seems to be serving their model as well as they do.

It will of course depend on what you’re doing with it, but right now my session at work has a 99.8% cache hit rate, and I’ve been running this session for hours with 23M tokens read and 713K tokens written (Opus 5.5 in this case though)

cache hit is in fact that good.