Scaling from monoliths to microservices with Amazon Aurora (DAT325)
Monolith to Microservices: Transformation Journey with AWS
Understanding Monoliths
A monolith is a large, complex database where multiple applications are talking to a single database.
Common problems with monoliths:
Isolated Workloads: Unrelated workloads co-hosted with the main application, causing disruption.
Entangled Workloads: Multiple applications sharing the same data, leading to shared state issues and coordination challenges.
Temporal Data: As the database grows over time, large queries become inefficient and destabilize the system.
Asymmetric Workloads: Analytics or other non-core workloads placing heavy load on the database, impacting performance.
Into It's Modernization Journey
Into It is a fintech platform serving over 10 million small and midmarket customers through products like QuickBooks.
The legacy QuickBooks architecture was a monolithic Java application on a commercial database, deployed using a "swim lane" architecture.
The need for modernization arose due to challenges in scaling, development velocity, and increasing costs of managing the monolith.
Modernization Approach
Evaluation Phase:
Evaluated technologies based on requirements like multi-tenancy, handling large tables, online schema evolution, SQL capabilities, and query plan stability.
Chose Amazon Aurora PostgreSQL for its features and scalability.
Execution Phase:
Implemented schema conversion, data migration, and application refactoring to work with the new Aurora PostgreSQL backend.
Used a "capture-replay" technique to extensively test the new system against the production workload.
Roll-out Phase:
Adopted a progressive roll-out strategy, starting with 4-hour and 24-hour trials before the final cutover.
Leveraged data replication techniques like "fall forward" and "fall back" to ensure a smooth transition.
Benefits and Results
Improved scalability, particularly for read-heavy workloads, using Aurora's read replicas.
Achieved better query plan stability, reducing unexpected performance issues.
Unlocked new capabilities like semantic search by moving the search workload to Amazon OpenSearch.
Realized cost savings by moving from a provisioned IOPS model to a pay-per-use model.
Next Steps
Exploring "zero-ETL" integrations between Aurora and Amazon Redshift for real-time analytics.
Evaluating Aurora Serverless and Aurora Limitless (managed sharding) to further simplify database management.
Automating version upgrades using blue-green deployment strategies.
AWS Services Discussed
AWS Database Migration Service (DMS): Helps with schema conversion, data migration, and application refactoring during database migrations.
Amazon Aurora: A purpose-built, cloud-native database service that provides performance, scalability, and manageability benefits.
Amazon Aurora Global Database: Enables cross-region disaster recovery and globally distributed read access.
Amazon Aurora Performance Insights: Provides visibility into database performance, including identifying problematic queries.
Amazon Aurora Query Plan Management (QPM): Ensures query plan stability and performance consistency over time.
Amazon Aurora Serverless: A pay-per-use option for Aurora, allowing the database to scale up and down automatically.
Amazon Aurora Limitless: Managed sharding solution for Aurora, simplifying database scaling and management.
Amazon OpenSearch Service: Used by Into It to power advanced search capabilities beyond simple text search.
These cookies are used to collect information about how you interact with this website and allow us to remember you. We use this information to improve and customize your browsing experience, as well as for analytics.
If you decline, your information won’t be tracked when you visit this website. A single cookie will be used in your browser to remember your preference.