AWS re:Invent 2025 - Build Resilient GenAI & Agentic Apps with Intelligent Memory (DAT105)

Building Resilient GenAI & Agentic Apps with Intelligent Memory

Key Trends and Challenges

• User behavior has shifted from searching for information to expecting direct answers and assistive experiences
• 70% of enterprises are running legacy infrastructure, making 50% of GenAI initiatives not ready or struggling
• 95% of GenAI projects are stuck in PoCs and not reaching production
• Enterprises need to build scalable, resilient, and cost-optimized GenAI systems for millions of users
• Avoiding "model hallucinations" and ensuring accurate, contextual, and actionable information is critical

Cockroach DB for Scalable and Resilient GenAI

• Cockroach DB is a distributed SQL database that provides:
  • Horizontal scalability by adding more nodes
  • High availability and resilience through data replication across regions and clouds
  • Ability to pin data to specific regions for compliance
  • Reduced latency by keeping data close to users
• Cockroach DB supports not just SQL transactions, but also documents, spatial data, and vector data

Vector Capabilities for GenAI

• Cockroach DB's distributed vector index addresses key challenges with vector data:
  • Reducing database bloat by 94% through quantization
  • Enabling per-user and per-region vector indexes for scalable and accurate vector searches
  • Avoiding data skew by keeping user data isolated during vector searches

GenAI Use Cases with Cockroach DB

1. RAG (Retrieval-Augmented Generation):

   • Implemented a banking assistant that uses Cockroach DB's vector search capabilities to retrieve relevant expense data
   • Leveraged caching to optimize costs and response times for frequently asked queries

2. Agentic Systems:

   • Demonstrated a workflow with multiple agents (receipt processing, fraud detection, budgeting) working together
   • Agents have access to tools and can take actions based on the data and memories stored in Cockroach DB

3. Intelligent Memory:

   • Stored user actions, preferences, and decisions in Cockroach DB to enable personalized and contextual responses
   • Used the distributed vector index to efficiently search and retrieve relevant memories for each user

Resilience and Scalability in Action

• Demonstrated Cockroach DB's ability to maintain availability and continue processing transactions even when a region was taken offline
• Used a data generator to simulate high-volume workloads and observed Cockroach DB's ability to handle the load without issues

Conclusion and Resources

• Cockroach DB provides a robust data layer foundation for building modern, scalable, and resilient GenAI applications
• Offers various deployment options, including a SaaS offering (Cockroach Cloud) and self-hosted versions
• Encourages attendees to explore Cockroach DB and provides a QR code for accessing more information and free credits