AWS re:Invent 2025 - Transforming AI storage economics with Amazon S3 Vectors (STG318)

Transforming AI Storage Economics with Amazon S3 Vectors

Introduction

• Presenters: Mark Tumi (Senior Solutions Architect, Amazon S3), Vijaya Chakraorti (Product Manager, Amazon S3 Vectors), and Frank Oyou Young (VP of R&D, March Networks)
• Key focus: Introducing Amazon S3 Vectors, a new cloud object store with native support for storing and querying vector data for AI applications

The Problem: Encoding Meaning for Computers

• Computers struggle to understand the meaning of text, images, sounds, and other unstructured data
• Vector embeddings provide a way to encode meaning into mathematical representations that AI systems can understand
• Vector embeddings are the "universal language of meaning in computer science"
• They allow AI systems to analyze and synthesize data by converting it into an ordered array of numbers

The Challenge: The Vector Explosion

• Vector embeddings can quickly accumulate into massive datasets, reaching gigabytes, terabytes, or even petabytes of data
• The more granular and specialized the vector embeddings, the more storage is required
• Traditional databases are not well-suited for vector-based similarity searches and queries
• Storing and managing vector data at scale becomes expensive due to compute costs, memory requirements, and licensing fees

Amazon S3 Vectors: A Cost-Effective Vector Store

• Introduced in July 2022 as the first cloud object store with native support for storing and querying vectors
• Provides sub-second vector query performance at scale, up to 90% lower costs compared to alternative vector solutions
• Allows storing billions of vectors per index, with the ability to create 10,000 indexes per vector bucket

Key Features and Capabilities

• Vector Buckets: Specialized buckets for storing and managing vector data
• Vector Indexes: Data structures that maintain the relationships between vectors to power similarity searches
• Vector APIs: Dedicated APIs for uploading, listing, querying, and deleting vectors
• Metadata Support: Ability to associate filterable and non-filterable metadata with vectors
• Efficient Nearest Neighbor Search: Uses advanced partitioning techniques to quickly prune irrelevant vectors and return results

Pricing and Cost Optimization

• Pay-per-use model with three main components:
  1. Put (upload) cost
  2. Storage cost
  3. Query cost
• Ability to distribute vectors across multiple indexes to optimize ingestion and querying
• Leverages the cost-effectiveness of Amazon S3 for vector storage

Real-World Use Cases

1. Semantic Search on Scientific Literature:

   • Biotech firm uses S3 Vectors to build a semantic search on 30 million research papers and scientific journals
   • Compresses a week-long manual review process into a sub-second semantic search

2. Agent Memory for Agentic Applications:

   • Agents need access to a large number of tools (APIs, Lambda functions, integrations, etc.)
   • S3 Vectors stores the embeddings of these tools, allowing the agent to quickly find the most relevant ones for a given task

Integrations and Partnerships

• Integration with Amazon Bedrock knowledge bases for end-to-end vector workflow management
• Integration with Amazon OpenSearch for hybrid search applications, leveraging the low-latency of S3 Vectors

Future Roadmap and Availability

• General availability release with several improvements:
  • 100ms warm query latency
  • 2 billion vectors per index, 10,000 indexes per vector bucket
  • 1,000 transactions per second for vector ingestion
• Continued expansion to more AWS regions
• Ongoing customer feedback and feature enhancements

Key Takeaways

• Amazon S3 Vectors provides a cost-effective, scalable, and performant solution for storing and querying vector data for AI applications
• It addresses the challenges of the "vector explosion" by leveraging the scale and economics of Amazon S3
• Enables a wide range of vector-based use cases, from semantic search to agent memory management
• Seamlessly integrates with other AWS services like Bedrock and OpenSearch
• Continuously evolving to meet the growing demands of AI-powered applications