Building event-driven architectures using Amazon ECS with AWS Fargate (SVS339)

Building Event-Driven Architectures using Amazon ECS with AWS Fargate

What is Event-Driven Architecture?

  • An event is a JSON object that represents a change in system state
  • Events cannot be changed, but can be modified by creating new events
  • Events are a contract between the producer and the consumer

Event-Driven Architecture is:

  • Something happens, and we react
  • This allows for decoupling of applications

Integration Patterns in Event-Driven Architecture

Synchronous Request-Response

  • Advantages: Low latency, simple, fail fast
  • Disadvantages: Potential for throttling, added complexity, resiliency issues

Asynchronous Communication

  • Introduces a broker that acknowledges the request but does not do the work
  • Consumers pull the data from the broker when ready

Event Routers

  • Route one event at a time
  • Target multiple endpoints and protocols
  • Filter at the router level

Event Bus

  • Reduces location coupling
  • Efficient for the sender
  • Routing logic handled at the rule level

Event Topics

  • Broadcast to multiple consumers
  • Filter the consumer at the topic level
  • Support multiple protocols (email, SMS, Lambda, etc.)

Event Stores

  • Handle multiple events
  • Ensure order is enforced (FIFO)
  • Support dead-letter queues

SQS and Amazon MQ

  • Fully managed message queues
  • Scale almost infinitely

Event Store Streams

  • All services get all events
  • Filtering must be done at the consumer
  • Order is enforced based on different parameters

Amazon Kinesis Data Streams and Amazon MSK (Kafka)

  • Provide high-throughput streaming capabilities

AWS Fargate for Event-Driven Architectures

  • Fargate is a serverless compute engine for containers
  • Allows you to run containers without managing the underlying infrastructure
  • Inherent qualities that make it well-suited for event-driven architectures:
    • Scalable
    • Supports microservices
    • Integrates with event-based services like EventBridge and Step Functions

Patterns for AWS Fargate

Standalone Tasks

  • Run tasks based on events (e.g., video processing, scheduled processing)
  • Considerations:
    • Use when Lambda is not a good fit
    • Manage task failures separately
    • Burst can cause scaling issues

Long-Running Services

  • Run services that need to be up 24/7
  • Scale dynamically based on metrics (e.g., queue depth, backlog per task)

Push-Based Patterns

  • Use EventBridge to send events to public or private APIs
  • API Destination feature allows sending events to APIs with rate limiting and retry handling

Pull-Based Patterns

  • Process messages from queues (e.g., SQS)
  • Scale task instances dynamically based on queue depth or custom metrics

Orchestration Patterns

  • Use AWS Step Functions to orchestrate workflows
    • Async pattern: Fire-and-forget
    • Sync pattern: Wait for task completion
    • Callback pattern: Get task results
  • Step Functions Activity pattern: Decouple workflow and task processing

Resources

  • Slides and recordings of related sessions: s12d.com/svs339
  • Serverless learning resources: s12d.com/serverless.learning

Your Digital Journey deserves a great story.

Build one with us.

Cookies Icon

These cookies are used to collect information about how you interact with this website and allow us to remember you. We use this information to improve and customize your browsing experience, as well as for analytics.

If you decline, your information won’t be tracked when you visit this website. A single cookie will be used in your browser to remember your preference.

Talk to us