“Come as you are, as you were
As I want you to be…”
Memoria
Kurt Cobain

Cobain’s lyrics in this iconic Nirvana song point to a core insight about human memory: It isn’t fixed. We show up as we are, as we were, and sometimes as others expect us to be – a reminder that memory is not a perfect recording, but something constantly rewritten.

By contrast, the role of memory in computing systems has been much simpler than in human cognition. At the hardware level, the fastest memory sits right next to the processor, and slower memory sits farther away, with layers of cache in between to smooth things out. The hardware historically has been mostly agnostic about whether it stored code or data, and various databases management systems and caches optimized retrieval of information based on complex query criteria. While the implementation of these systems was complex, the important role they played was nevertheless concrete and straightforward.

Human memory is more complex and not as well understood. Working memory functions as a very limited scratchpad that helps us navigate complex reasoning processes, while other short-term memory holds important context. Long-term visual memory is used not just to remember people and places but, with training effort, abstractions like chess positions about which we reason in detail. Memories can also be multi-sensory; the sounds of a song or the smell of mountain air can evoke vivid emotions tied to important events in our lives. Unfortunately, as Kurt Cobain suggests – and many studies have confirmed – human memory is profoundly fallible and subjective.

One such study examining the reliability of eyewitness testimony found that early, uncontaminated recall can be remarkably accurate, yet memory quickly becomes distorted when exposed to suggestion, repetition, or biased questioning. Even the simple act of remembering alters the memory itself, strengthening some details and weakening others, much like editing a document every time you open it.

What role will memory play in AI?

Today, we have feature stores that serve data to models, context windows that allow models to carry on complex conversations, and vector databases that find objects similar to a presented target object. But AI struggles when context windows overflow, and reasoning systems are largely separate from memory systems.

We’ve already seen reports of AI models drifting – becoming more certain in wrong answers over time – a risk noted in studies like The Curse of Recursion. A confident answer might be grounded in training data, inferred from context, or produced by a prior hallucination. As models become more autonomous and long-lived, it won’t be enough to know what they recall; we’ll need ways to know how much to trust that recall, whether the information has been distorted by previous interactions, and whether the system is pulling from real observations or its own recycled errors.

We have much to learn about how human memories crystallize into wisdom that improves task performance – or how this process changes over our life cycle. No wonder we are still in the very early innings of designing memory systems for AI that will help it become continuously more effective the longer you work with it on a task and carry that learning forward to other tasks.

I am incredibly lucky to get to meet and sometimes partner with brilliant technologists and entrepreneurs solving some of the toughest problems in technology. I believe the problem of memory of AI systems is a deep and fascinating one that will play an important role on the journey to superintelligence.

I would love to hear what approaches you think are promising in this space.