AWS re:Invent 2025 - Deep dive into Amazon DocumentDB and its innovations (DAT444)

AWS re:Invent 2025 - Deep Dive into Amazon DocumentDB and Its Innovations

Architecture Overview

• DocumentDB is a purpose-built database designed for managing document database workloads at enterprise scale
• Key features:
  • Separation of compute and storage
  • Distributed storage volume with 6 copies of data across 3 availability zones for high durability
  • Data is log-structured and continuously streamed to S3 for backups
  • Storage blocks are 10GB in size and self-healing, allowing for faster recovery times
  • Parallel checkpointing and write-ahead logs enable faster recovery and lower replication lag

Enhanced Compression

• Introduced Zstandard dictionary compression, which provides significant storage savings compared to the previous LZ4 compression
• Zstandard is more effective for smaller documents by analyzing common field patterns and creating a dictionary
• Results:
  • 17% increase in CPU utilization for compression/decompression
  • 63% reduction in storage usage (from 340GB to 125GB)
  • 40% decrease in throughput, but 55% increase in insert operations per second
• Compression works best for collections with repetitive field names and consistent schemas
• Can enable cost savings by reducing storage requirements and allowing use of lower-cost storage tiers

Graviton 4 Instances

• Graviton 4 instances provide up to 30% performance improvement over Graviton 3
• In-memory workloads see over 100% price-performance increase in reads and 136% in updates
• Even out-of-memory workloads see significant performance gains
• Allows customers to scale down instances while maintaining performance, leading to cost savings

Serverless

• Serverless DocumentDB automatically scales compute resources (CPU, memory) based on workload requirements
• Scales from less than 1 CPU and 2GB of memory up to 64 CPUs and 512GB of memory
• Uses "Database Capacity Units" (DCUs) to measure and provision resources
• Includes intelligent buffer cache resizing to avoid performance issues during scaling
• Allows easy switching between serverless and provisioned instances without data movement

DocumentDB 8.0 and the New Query Planner

• Key improvements in DocumentDB 8.0 include the new query planner and vector index support
• New query planner:
  • Chooses better indexes, including for negation operations
  • Optimizes complex aggregation pipelines by combining stages (e.g., lookup and unwind)
  • Provides up to 90% reduction in switchover time for multi-region deployments

Key Takeaways

• DocumentDB's architectural innovations, such as separation of compute and storage, enable scalability, durability, and cost optimization
• Enhanced compression with Zstandard provides significant storage savings, with potential for performance improvements
• Graviton 4 instances deliver substantial price-performance benefits, especially for in-memory workloads
• Serverless DocumentDB automatically scales resources to match workload needs, simplifying operations
• The new query planner in DocumentDB 8.0 introduces optimizations that can significantly improve query performance

Real-World Impact

• A production customer saw an 8x compression ratio improvement by using Zstandard, reducing a 13MB document to just 90KB
• Serverless DocumentDB allows customers to scale resources up and down as needed, reducing over-provisioning and improving cost efficiency
• The new query planner optimizations have been shown to reduce switchover time by up to 90% for multi-region deployments, improving reliability and availability