Unleashing Your Serverless Scalability
Introduction
- The speaker is excited to share his experiences and lessons learned from building a large-scale live scoring system using serverless technologies.
- The talk will cover the architecture, design decisions, challenges, and strategies for scaling a serverless system.
Live Scoring System
- The system was an event-driven architecture that processed live scoring events from various sources (tournament control, chair umpires, statisticians).
- The key components were:
- Score processing system that ingested the events and stored them in a database.
- Digital experience API that served the score data to the frontend applications.
- CMS system that hosted the public-facing website.
Patterns and Principles
- The speaker emphasizes the importance of thinking in terms of composable subsystems and patterns when building serverless solutions.
- Infrastructure as code was used to define and deploy the various components of the architecture.
- The event broker pattern was used to decouple the event ingestion and processing.
- A caching service (Redis) was introduced to isolate the database and improve performance.
- CDN caching and websocket notifications were used to handle the high traffic and provide a real-time user experience.
Testing and Deployment
- The speaker built a scoring simulator to test the system and validate the behavior during high-load scenarios.
- Performance testing was conducted to understand the system's limits and identify potential bottlenecks.
When Things Go Wrong
- During the performance testing, the team removed the CDN caching, which caused a significant increase in traffic to the API Gateway and Lambda functions.
- This triggered a cascade of issues, including Redis connection timeouts and Lambda throttling.
- The speaker discusses the importance of understanding scalability boundaries, such as connection-oriented systems like Redis, and how they can become the weakest link in the architecture.
- Other scalability boundaries, such as external API calls and network latency in VPC environments, were also explored.
Controlling Serverless at Scale
- The speaker emphasizes the need to design for control points in the architecture, where you can stop or throttle the processing of events to regain control during a crisis.
- Monitoring Lambda duration and concurrency are critical for identifying and addressing scalability issues early.
- Event Source Mapping's maximum concurrency setting can be used to control the flow of events and prevent overloading downstream components.
- Understanding the expected transaction volumes and non-functional requirements is essential for designing a scalable and reliable serverless system.
Patterns and Blueprints
- The speaker highlights the importance of turning patterns into blueprints that capture all the necessary architectural decisions, scalability considerations, and operational requirements.
- Patterns alone are not enough; you need to understand how to apply them in your specific use case and ensure they work as expected.
Conclusion
- The speaker emphasizes the key takeaways:
- Think in terms of composable subsystems and patterns.
- Identify and address scalability boundaries.
- Design for control points and flow control.
- Perform thorough testing and understand your non-functional requirements.
- Turn patterns into reusable and reliable blueprints.
By applying these principles, the speaker believes you can "Unleash Your Serverless Scalability" and build robust, scalable, and maintainable serverless systems.