AWS re:Invent 2025 - Accelerating AI: How Sun Life reduces latency and enhances engagement (IND3320)

Accelerating AI: How Sun Life Reduces Latency and Enhances Engagement

Evolution of Enterprise AI and Unique Challenges

• The presentation discusses the evolution of enterprise AI, from retrieval-augment-generation (RAG) systems to deterministic agents and autonomous agents.
• As AI workflows become more complex and move towards autonomy, the response time and user experience become critical challenges.
• Users expect instant responses, but longer-running AI workflows can take 30-60 seconds or more to complete, leading to user frustration and potential abandonment.
• The introduction of Model Context Protocol (MCP) has enabled the rapid creation of these agentic systems, but the latency issue has been amplified.

Addressing Latency with Real-Time Streaming

• To address the latency challenge, the presentation introduces the use of real-time streaming, enabled by AWS AppSync.
• AppSync provides several key capabilities:
  • Flexible security through private APIs, ensuring traffic never leaves the VPC
  • Flexible request handling, allowing for real-time updates to be streamed to the front-end
  • Flexible evolution of the API model, making it easier to adapt to changing backend requirements
• The presentation outlines two key patterns for using AppSync:
  1. Simple Q&A bot pattern, where AppSync calls Bedrock directly
  2. Async pattern for longer-running workflows, where AppSync invokes a Lambda function to orchestrate the backend process and stream updates back to the client

Sun Life's Contact Center Transformation

• Sun Life has leveraged AppSync and MCP to transform their contact center experience, moving from traditional menu-driven IVR to a more conversational, concierge-like system.
• The solution uses Amazon Connect and Amazon Lex for speech recognition and natural language understanding, and then augments it with Bedrock and MCP to enable more contextual awareness, multi-turn memory, and dynamic dialogues.
• The key aspects of the solution include:
  • Deterministic agents with structured JSON outputs to ensure predictable behavior
  • Multi-agent orchestration, where a concierge agent routes requests to specialized sub-agents (e.g., authentication, IT support, claims)
  • Leveraging MCP to discover and orchestrate the various agent tools and services

Content Assist: Leveraging Generative AI for Content Creation

• Sun Life developed a solution called "Content Assist" to leverage generative AI (Bedrock, Anthropic's Claude) for content creation, scoring, and rewriting.
• The key goals were to generate high-quality content through precise prompting, with input from subject matter experts to capture brand guidelines and tone.
• The solution uses a web UI that allows users to select content categories and generates the content based on predefined prompt templates.
• Prompt engineering is critical, with the presentation outlining best practices such as:
  • Capturing brand guidelines, examples, and content trends in a structured format (e.g., XML)
  • Providing step-by-step instructions and reasoning in the prompt
  • Iteratively refining the prompt to improve the output quality
• Challenges included the longer processing time for complex prompts, which was addressed using AppSync's real-time streaming capabilities.

Technical Architecture and Lessons Learned

• The technical architecture leverages several AWS services:
  • AppSync for real-time streaming and managed GraphQL
  • Lambda functions for serverless compute and orchestration
  • SNS for secure pub/sub notifications
  • Bedrock for large language model integration
  • MCP for hosting prompts and content scoring tools
• Key design decisions and lessons learned include:
  • Leveraging AppSync's connection pooling to improve performance
  • Building a modular, adaptable architecture to accommodate new technologies and use cases
  • Involving both technical and business teams in the prompt engineering process

Key Takeaways

• Sun Life has successfully transformed their contact center experience and content creation workflows by leveraging the latest AI and cloud technologies.
• Real-time streaming, enabled by AppSync, was critical in addressing the latency challenges of longer-running AI workflows and keeping users engaged.
• Prompt engineering and the integration of MCP were essential for building deterministic, multi-agent systems and high-quality content generation.
• Collaboration between technical and business teams was key to the success of these initiatives, as it combined technical expertise with domain knowledge and user requirements.