AWS re:Invent 2025 - From Trigger to Execution: The Journey of Events in AWS Lambda (CNS423)

Summary of AWS re:Invent 2025 - From Trigger to Execution: The Journey of Events in AWS Lambda (CNS423)

Overview of AWS Lambda

• AWS Lambda is a serverless compute service that allows developers to run code without managing servers
• Lambda handles the underlying infrastructure, scaling, and operational complexity on behalf of customers
• Lambda processes over 15 trillion invokes per month, with 1.7 trillion invokes just on Prime Day alone
• Lambda provides 99.99% availability and resilience built into the service

Lambda Compute Models

• Lambda on-demand: Runs in the Lambda service VPC, with invokes routed to EC2 worker hosts owned by Lambda
• Lambda Managed Instances: Provisions and manages EC2 instances in the customer's VPC, with invokes routed to containers running on those instances
• Both models provide the same Lambda operational simplicity, with the ability to choose from a range of EC2 instance types

Lambda Invoke Types

• Synchronous: Caller waits for the function to complete and receive the response
• Asynchronous: Caller hands off the event to Lambda, which processes it separately and doesn't wait for a response

Event Source Mapping (ESM)

• ESM is a Lambda resource that reads items from a queue or stream and triggers Lambda function invocations
• Supports a variety of event sources like Kinesis, DynamoDB, SQS, and more
• Handles polling, batching, filtering, ordering, retries, and other complex logic on behalf of the customer

Synchronous Invoke Path

• Requests are distributed across a fleet of sync invoke service hosts
• Performs authentication, authorization, and accounting checks before routing to the assignment service
• Assignment service coordinates the placement of new execution environments on worker hosts
• Execution environment is initialized with the function's IAM role and environment variables
• Subsequent invokes are routed directly to the existing execution environment

Asynchronous Invoke Path

• Asynchronous events are sent to internal SQS queues, with the caller receiving an acknowledgment
• A fleet of poller instances read messages from the queues, determine the function and payload, and send the invoke request synchronously to the sync invoke service
• Queue management and poller lifecycle are handled by additional control plane services

Event Source Mapping (ESM) Capabilities

• Supports filtering, batching, ordering, retries, and failure handling for both queues and streams
• Provides options for controlling the consumption rate to protect downstream services
• Automatically scales the poller fleet to match the incoming event rate

Operational Complexity and Resilience Patterns

• Lambda must handle correlated failures, non-linear scaling, and abrupt saturation in a real-world distributed system
• Key resilience patterns include:
  • Dependency outage handling: Gradual ramp-up, customer isolation, and backlog management
  • Scale inversion: Static stability, state replication, and circuit breakers
  • Availability zone outages: Proactive detection, evacuation, and observation

Future of AWS Lambda

• Lambda Durable Functions for building long-running workflows
• Tenant isolation for better multi-tenancy support
• Event Source Mapping enhancements like schema registry and provisioned mode
• Continued focus on observability, new runtimes, pricing, and deeper integrations

Key Takeaways

• Lambda applies lessons from queuing theory to handle the operational complexity of a multi-tenant serverless platform
• Lambda's architecture separates concerns like polling, processing, and control plane functionality for scalability and resilience
• Lambda continues to evolve with new capabilities to better support a wide range of serverless workloads and customer needs