Consciousness is moment to moment and fleeting. There are people with brain defects that don’t let them form new memories. They have no memory about what happened a minute ago in their own consciousness. Still we would say that they are conscious, even if it’s only momentary. LLMs could conceivably have something like that within their CoT/MoE loops.
replyhow would you build memory into the weights? and why is RAG not enough? Our hippocampus is at a bit of a distance from our frontal cortex.
replyYeah it's a good question, I've also been thinking about harnesses and all these tacked on things we've done to add persistent memory, what makes that different, I don't know the answer, I guess that still 'feels' different than what we have, but it's hard to articulate how. As for the memory into weights thing, I meant along the lines of the Google TITANS/MIRAS papers that were released I think late last year.
reply