AWS re:Invent 2025 - Beyond a Chatbot: How to Build Intelligent Agent Memory (AIM284)

Building Intelligent Agent Memory (AIM284)

Understanding the Limitations of Language Models (LMs)

• LMs are powerful reasoners but have fundamental limitations in memory and knowledge retention
• LMs can only store information in their parametric memory (model weights) and cannot learn or accumulate experience after training
• LMs have a temporary context window, and anything outside that window disappears after generation unless explicitly persisted

The Need for Memory Engineering

• Studies show that 95% of organizations see little to no value from AI agents, often due to issues with agent memory and continuity
• Agents frequently lose track of user goals, forget earlier details, provide inconsistent answers, and repeat work
• This is not a reasoning failure, but a memory failure - most agents today are stateless systems pretending to be stateful

The Role of Context Engineering

• Context engineering is the discipline of controlling what the LM sees at inference time, as the model's internal memory is limited and volatile
• Retrieval pipelines transform raw data into context the model can use, including chunking, embedding, and storing in a vector index
• Techniques like query augmentation, rewriting, expansion, and decomposition can dramatically enhance retrieval quality and reduce hallucinations

The Hierarchy of Agent Memory

• Short-term memory: The agent's working space, holding information for the current step or immediate sequence
• Long-term memory: Durable knowledge stored outside the model, enabling the agent to remember facts, decisions, and experiences
• Shared memory: A global layer that every agent in a multi-agent system reads and writes from, ensuring coordination and consistency

The Memory Engineering Lifecycle

1. Consolidation and Summarization: Extracting meaningful parts of an event, preserving context, and removing irrelevant details
2. Intelligent Forgetting: Removing low-value, outdated, or overridden information to keep the memory coherent and efficient
3. Retrieval: Ensuring the agent can find, rank, and select the right pieces of memory at the right time

Evaluating Agent Memory

• Reliability: Agents that don't lose track mid-task and have repeatable, safe reasoning
• Believability: Agents that feel consistent with personas that persist in interactions and adapt
• Capability: Agents that expand skills over time by remembering tools, workflows, and outcomes

Implementing Memory Engineering with MongoDB

• MongoDB provides a flexible operational database, native vector search for semantic retrieval, and high-quality embeddings and re-ranking models through Voyage
• Agents need schemas that match memory types, which can be easily evolved as the agent learns new things
• MongoDB supports the entire memory engineering lifecycle, from consolidation and summarization to retrieval and evaluation

Key Takeaways

1. Context is not memory - visibility and durability must be separated
2. Memory must be engineered through schemas, pipelines, and feedback loops
3. Agents only become reliable and useful when their memory systems are reliable and useful