AWS re:Invent 2025 - Build more effective agents through model customization (AIM383)

Building More Effective Agents Through Model Customization

What is an AI Agent?

• AI agents are tools that use large language models (LLMs) in a loop to accomplish specific goals
• Agents have access to a model, a set of tools, and a defined goal, then take actions in an environment to achieve that goal

The Rise of Reasoning Models

• Frontier model intelligence had hit a wall, with diminishing returns from scaling data and model size
• Reasoning models emerged, shifting the focus to test-time and inference-time compute rather than just training
• Reasoning models unlocked new capabilities for agentic AI, enabling deployment at scale and cost-effectiveness

Challenges with Agentic AI

• Agents take a series of steps, introducing the risk of "compounding mistakes" where errors propagate through the workflow
• Domain-specific knowledge and business semantics may not align with internet-average models
• Agents need a deep understanding of the underlying tools and schemas to make effective actions

Customization Techniques

• Prompt engineering: Crafting prompts, examples, and instructions to optimize model output
• Retrieval-augmented generation (RAG): Providing additional context to the model at runtime
• Context engineering: Curating the right information in the right format for the model
• Fine-tuning: Adapting a model to a specific task using a labeled dataset
• Direct preference optimization (DPO): Aligning the model to user preferences without training a reward model
• Proximal policy optimization (PPO): Improving the model based on live feedback and task completion
• Model distillation: Transferring complex behaviors from a larger "teacher" model to a smaller "student" model
• Continuous pre-training: Further training the base model on domain-specific unlabeled data

Amazon Nova Models

• Nova provides a range of understanding, reasoning, and content generation models
• Customization options are available on both Amazon Bedrock and Amazon SageMaker
• Bedrock supports efficient fine-tuning and model distillation with simple workflows
• SageMaker provides more advanced customization techniques and orchestration via Hyperparameter Tuning

Evaluating and Deploying Customized Models

• Evaluation can use rule-based heuristics, LLM-based critique, and task-specific metrics
• Deployment options include Bedrock's on-demand and provisioned throughput, as well as SageMaker endpoints and Hyperparameter Tuning clusters
• Integrating customized models into agent workflows can be done using open-source frameworks like Transformer Agents

Key Takeaways

• Customization is crucial for aligning LLMs to specific business needs and reducing errors in agentic AI workflows
• Amazon Nova provides a range of customization options, from simple prompt engineering to advanced fine-tuning and distillation
• SageMaker and Bedrock offer complementary tools for customizing, evaluating, and deploying models for agent-based applications
• Careful evaluation and integration into agent frameworks are essential for successful production deployments