Hacker News
new
past
comments
ask
show
jobs
points
by
cortesoft
9 hours ago
|
comments
by
vanviegen
5 hours ago
|
[-]
Wrong on both counts. The kv-cache is likely to be offloaded to RAM or disk. What you have locally is just the log of messages. The kv-cache is the internal LLM state after having processed these messages, and it is
a lot
bigger.
reply