Beads kind of does "LLM memory over CLI", or there is https://github.com/wedow/ticket which is a minimal and sane implementation of the same idea.
I haven't measured, but documenting bug fixes and architecture seems to help, along with TDD patterns, including integration tests.
I would probably add it to Claude.md to look for all of the above when tackling a new bug.
While you can document everything and use git history, I think that having short entries in a kind of memory to remember past decisions, how issues were solved would be much more token efficient than reading lots of documentation and looking at git history and past code.