AWS re:Invent 2025 -Develop AI Agents faster with Amazon SageMaker Studio & Bedrock AgentCore-AIM388

Accelerating AI Agent Development with Amazon SageMaker and AWS Bedrock

Overview

• The presentation discussed how enterprises can accelerate the development of AI agents using Amazon SageMaker and AWS Bedrock.
• Key topics included:
  • Drivers of AI agent adoption in enterprises
  • Techniques for customizing foundation models for agent use cases
  • New capabilities in SageMaker and Bedrock to streamline agent development
  • Demonstration of end-to-end agent development workflow
  • Real-world examples of agent deployment at Robin Hood

Adoption Drivers for AI Agents

• Enterprises are deploying AI agents to:
  • Improve customer experience
  • Automate operational workflows
  • Enhance employee productivity
• Enterprises are increasingly choosing to use custom models to power these agents
  • Customizing off-the-shelf foundation models with private data
  • Enables specialized domain knowledge and task-specific capabilities
• Gartner predicts over 50% of enterprises will use custom models in the next 3 years, up from 1% a year ago
• Key techniques driving this trend:
  • Reinforcement fine-tuning
  • Preference tuning
  • Supervised fine-tuning
• These techniques enable models to develop critical reasoning capabilities for agent workflows

SageMaker and Bedrock for Agent Development

• AWS provides purpose-built services to support the agent development lifecycle:
  • Amazon SageMaker: Model development, tuning, evaluation, and deployment
  • AWS Bedrock: Serverless infrastructure for hosting and running custom models
  • Bedrock Agent Core: Tools and services for building, deploying, and monitoring agents
• New capabilities in SageMaker and Bedrock to accelerate agent development:
  • SageMaker Model Customization Agent
    • Guides users through end-to-end model customization workflows using natural language
    • Generates hardened specifications to reproduce workflows
  • Synthetic Data Generation in SageMaker
    • Generates statistically similar data based on user-provided context
    • Includes data quality analysis and responsible AI metrics
  • Serverless Reinforcement Learning in SageMaker
    • Fine-tunes models using techniques like reinforcement learning, preference tuning
    • Optimized for cost and performance on AWS infrastructure
  • Serverless MLFlow in SageMaker
    • Tracks and compares model customization experiments
    • Provides a familiar MLFlow interface without infrastructure management
  • Serverless Model Evaluation in SageMaker
    • Evaluates and compares models using industry benchmarks, custom scoring, or LLM judges
    • Generates detailed reports comparing model performance

Agent Development Workflow Demonstration

• Demonstrated building a customer service chatbot agent using SageMaker and Bedrock
• Key steps:
  1. Used SageMaker Model Customization Agent to define the use case and success criteria
  2. Leveraged Synthetic Data Generation to create a training dataset
  3. Performed Serverless Reinforcement Learning to fine-tune a base LLM model
  4. Utilized Serverless MLFlow and Model Evaluation to track and compare experiments
  5. Deployed the optimized model to Bedrock for inference
  6. Integrated the model into an agent using the Bedrock Agent Core SDK

Real-World Example: Robin Hood

• Robin Hood, a financial services company, has built a platform for accelerating AI agent development
• Key components:
  • Selective model selection based on task complexity
  • Prompt tuning and "trajectory optimization" to improve quality
  • Targeted fine-tuning to optimize for cost and latency
• Specific results:
  • Over 50% latency reduction for a key agent component
  • Enabled scaling of LLM-powered agents across the business

Key Takeaways

• Enterprises are rapidly adopting custom AI models to power agent applications
• AWS provides a comprehensive set of services to streamline the agent development lifecycle
• New capabilities in SageMaker and Bedrock significantly accelerate model customization, data preparation, and model evaluation
• Careful model selection, prompt tuning, and targeted fine-tuning are critical for deploying cost-effective, high-performance agents
• Leading companies like Robin Hood are leveraging these AWS services to build scalable, mission-critical AI agent platforms