Empowering stateful workloads on Amazon EKS with AWS storage services (KUB324)

Key Takeaways

  • The video presents a solution for building a real-time image generation application using a foundation model like Stable Diffusion.
  • The solution addresses three main challenges:
    1. Ensuring the reliability of the entire solution from the front-end to the back-end.
    2. Ensuring that users don't have to wait too long for the image to be generated.
    3. Scaling the application on-demand to handle varying workloads.

Solution Architecture

  1. Asynchronous Model:

    • The front-end sends a request to an API Gateway, which then triggers a Lambda function.
    • The Lambda function validates the message and sends it to an Amazon SNS (Simple Notification Service) topic.
    • An SQS (Simple Queue Service) queue agent pulls the message from the SNS topic and initiates the image generation process.
    • The agent talks to the Stable Diffusion inference endpoint (running on EKS) to generate the image.
    • The generated image is saved to an S3 bucket, and a callback is sent to the SNS topic to notify the user.
    • The message is then deleted from the SQS queue, completing the process.
  2. Performance and Scalability:

    • To quickly load the model files, the solution leverages an S3 bucket as a persistent volume for the EKS cluster.
    • For scalability, the solution uses the Kubernetes Cluster Autoscaler to automatically scale the number of nodes in the EKS cluster based on the incoming workload.

Demonstration

  • The presenter demonstrates the flexibility of the solution by showing how different foundation models (Stable Diffusion and Latent) can be used to generate images with different styles, using the same prompt.
  • The presenter then simulates a high-load scenario by generating a load test, which triggers the Cluster Autoscaler to quickly scale out the EKS cluster to handle the increased demand.
  • The presenter also shows the end-to-end visibility of the solution, from the user's request entering the system to the final image being generated and stored.

Resources

  • The presenter provides a URL to a solution library where the entire solution can be deployed and explored further.
  • There are also links to additional sessions and blog articles that provide more details on the solution and its implementation.

Your Digital Journey deserves a great story.

Build one with us.

Cookies Icon

These cookies are used to collect information about how you interact with this website and allow us to remember you. We use this information to improve and customize your browsing experience, as well as for analytics.

If you decline, your information won’t be tracked when you visit this website. A single cookie will be used in your browser to remember your preference.

Talk to us