If you read DeepSeek's papers, you'll find a litany of architectural features that allow for a greatly reduced cache hit price by shrinking the size of the KV-cache.
replyHow come no other big model seems to be able to deliver the same type of extremely low cache cost though, if their techniques are public?
replyI think the "architectural features" are part of the model, not the kv cache. So implementing it would be difficult and expensive.
replyDeepseek V4 paper is just ~three months old
replyMany of these techniques haven't been published very long ago - it often takes a good 6-8 months for techniques to percolate. But also, they come at a complexity cost and, seemingly, also at a stability cost.
replyAlso potentially a performance (in terms of output quality) cost. DeepSeek is cheap on a per token basis but lags behind in the benchmarks, perhaps it was a calculated tradeoff.
replyWhat provider are you using?
replyDeepSeek's own API
replyAny way to avoid China sales tax or is that just the cost of doing business?
replyhttps://openrouter.ai/deepseek/deepseek-v4-pro#providers
replyLook through the provider list for a company you are willing to do business with?
Good grief, the sales tax is only 6% on a service that's already extremely affordable.
replyFireworks.ai
reply