AWS re:Invent 2025 - How Yahoo! Finance built research multi-agent systems with Gen AI (SPS321)

Enabling Enterprise-Scale AI Agents: Lessons from Yahoo Finance's Deep Research Journey

Introduction

• Presentation covered how Yahoo Finance built research multi-agent systems using AWS Gen AI technologies
• Speakers included Rahul Ghosh (AWS), Vidya Sagar (AWS), and Deepen (Yahoo Finance)
• Focused on the architectural patterns, technical challenges, and practical lessons learned in deploying production-ready AI agent solutions

Agentic AI: From Hype to Reality

• Key technological advancements enabling practical AI agents:
  • Improved model reasoning capabilities
  • Enhanced data and knowledge integration
  • Scalable infrastructure and agentic protocols
  • Sophisticated agent development tools and frameworks
• Agentic AI maturity scale:
  1. Rule-based automation
  2. Generative AI assistants with web search
  3. Goal-driven AI agents
  4. Autonomous agents that set their own goals

Agentic Reasoning Fundamentals

• Core agentic loop: React (input -> reason -> act), Review (plan upfront), and Reflection (self-evaluate)
• Tradeoffs of single vs. multi-agent systems:
  • Single agents can become overwhelmed with too many tools
  • Multi-agent patterns like Supervisor-Sub-Agent and Agent Swarm offer advantages

Deep Research Challenges in Finance

• Requires complex multi-step reasoning and hypothesis generation
• Integrates diverse data sources: structured financials, unstructured text, audio/video
• Balances scale vs. quality - need complete audit trail and material information detection

Architecting Deep Research Agents

• Supervisor-Sub-Agent pattern with specialized sub-agents:
  • Structured Database Agent
  • News Agent
  • SEC Filings Agent
  • Internal/Partner API Agents
• Supervisor coordinates workflow, sub-agents handle domain-specific tasks

AWS Bedrock Agent Core for Production-Ready Agents

• Addresses key challenges in scaling agent solutions:
  • Performance and scalability
  • Security and governance
  • Context preservation
  • Observability and auditability
• Provides runtime, gateway, identity, memory, and observability services

Yahoo Finance's Agent Journey

• Goal: Enable retail investors to conduct comprehensive equity research
• Key challenges:
  • Heterogeneous data sources (structured, unstructured, temporal)
  • Avoiding financial advice and predictions
• Architecture:
  • Asynchronous, serverless execution with Lambda and SQS
  • LLM services, Bedrock Agents, and RDS for data
  • Observability with CloudWatch
  • Input/Output guardrails using Bedrock Guardrails

Evaluation and Results

• Hybrid approach: Human evaluation and AI-based judging
• Metrics: Accuracy, Coverage, and Performance
• Scalable to thousands of queries, 5-50 seconds latency, 2-5 cents per query

Key Takeaways

• Agentic AI is a powerful paradigm, but scaling to enterprise production poses challenges
• AWS Bedrock Agent Core provides a comprehensive platform to build, deploy, and operate production-ready AI agents
• Yahoo Finance's deep research agent architecture demonstrates practical application of multi-agent systems
• Rigorous evaluation, including both human and AI-based judging, is crucial for ensuring reliable performance