No it doesn't. They get added to its context, and it reads them afresh when answering the next question. That's not remembering.
If your short-term memory completely malfunctioned one day, so you had no ability to remember what was said to you a minute ago, then you would have to find workarounds. For example, you could write down everything someone says to you, then read your notes of the previous exchanges in that conversation in order to continue the conversation. That would be a good way to work around the fact that your short-term memory was broken. And if your notes were invisible to other people and you could read them really fast, then you could even make most people believe that you remembered what they said a minute ago. But you don't actually have a working memory, you're just writing down what they said and re-reading it while coming up with your next response.
That's exactly what LLMs do. That's not memory.
Attention over KV cache allows past behavior and past inputs to influence future inputs and future behavior. In LLMs.
Until the cache runs out, that is. But even then, you could totally use any of 9000 methods of cache compression, truncation, dropping or streaming and get away with it.
The difference between continuous learning and in-context learning seems to be in capacity, not in principle. Both are doing a similar thing, but one has more length and depth to it.
The model takes in the context, encodes it into a "memory" (the KV cache), and accesses that memory later. That fact doesn't change just because the KV cache grows in size with the context.
I don't know what memory would look like other than an encode-retrieve loop.
Relevant: Transformers are Multi-State RNNs - https://arxiv.org/abs/2401.06104
I don't think physical integration within one contained is relevant to system level behavior.
It still generates every response using the model's pristine state with every new API call; whether the context is provided from the client or from a colocated cache server doesn't really change that.
Christ HN isn't what it used to be