AWS re:Invent 2025 - Customize Amazon Nova models for enhanced tool calling (AIM380)

Customizing Amazon Nova Models for Enhanced Tool Calling

Overview

• This session covered techniques for customizing Amazon Nova language models to enhance their capabilities for a specific use case - a security agreement legal assistant agent.
• The key focus was on leveraging supervised fine-tuning (SFT) and reinforcement fine-tuning (RFT) to fine-tune a base Nova model for the target application.

Data Preparation

• The presenters generated synthetic data to fine-tune the Nova models, rather than using an existing dataset.
• The data sources included:
  • SEC filings for securities agreements
  • Formal SEC regulations
  • Judicial interpretations of the agreements
• A custom data generation toolkit was used to chunk the data and generate prompts in a format consumable by the Nova models.
• The generated data included a system prompt, user prompt, and expected response in JSON format.

Supervised Fine-Tuning (SFT)

• SFT was performed using Amazon SageMaker, leveraging a parameter-efficient "LoRA" (Low-Rank Adaptation) technique.
• This approach fine-tunes only 1-2% of the model's weights, significantly reducing training time and cost.
• Evaluation metrics tracked included:
  • Precision, recall, and F1 score on tool selection
  • Overall weighted score based on tool selection, parameter usage, and sequence
• The SFT fine-tuned model showed a significant improvement over the base Nova model, with the overall score increasing from 44% to 75%.

Reinforcement Fine-Tuning (RFT)

• RFT was performed on top of the SFT fine-tuned model to further improve performance.
• The key components were:
  • Rollout: Generates multiple candidate responses for a given prompt
  • Reward Function: A custom AWS Lambda function that evaluates the responses and provides a numerical reward score
  • Trainer: Uses the reward scores to update the model's weights and improve performance
• The reward function was carefully designed to provide a gradient-based score, rather than binary pass/fail, to guide the model's learning.
• After RFT, the overall score improved further from 75% to around 90%, demonstrating the effectiveness of the combined SFT and RFT approach.

Deployment to Bedrock

• The final fine-tuned Nova model was exported from the SageMaker training job and deployed to Amazon Bedrock for on-demand inference.
• This involved creating a custom Bedrock model and deployment, allowing the model to be accessed through a simple API call.
• Compared to the base Nova model, the fine-tuned model demonstrated significantly improved performance in selecting the appropriate tools and parameters to answer the input queries.

Key Takeaways

• Synthetic data generation can be a powerful approach to fine-tune foundation models when real-world data is scarce or difficult to obtain.
• Combining SFT and RFT techniques can lead to substantial performance improvements for specialized use cases, beyond what can be achieved with SFT alone.
• Careful design of the reward function is crucial for effective RFT, as it guides the model's learning process.
• Deploying the fine-tuned model to a managed service like Bedrock simplifies the process of making the model available for production use.

Business Impact

• The presented approach enables the creation of highly customized, task-specific language models that can be deployed in regulated industries like legal and healthcare.
• By automating the process of selecting the appropriate tools and parameters to analyze complex documents, the legal assistant agent can significantly improve productivity and compliance for attorneys and paralegals.
• The flexibility to fine-tune foundation models like Nova for specific use cases allows organizations to leverage the latest advancements in language AI while tailoring the models to their unique needs.

Example Use Case

• In the presented example, the legal assistant agent was tasked with analyzing securities agreements to ensure compliance with SEC regulations.
• The agent used the fine-tuned Nova model to:
  • Identify the relevant sections of the agreement
  • Determine the appropriate tools to retrieve and analyze the required information
  • Provide the necessary parameters for the tools to execute the analysis
  • Validate the agreement against the SEC regulations and judicial interpretations
• This automated process helped ensure the agreements were compliant, reducing the manual effort required by legal professionals.