AWS re:Invent 2025 - Implement Agentic AI at the edge for industrial automation (HMC317)

Implementing Agentic AI at the Edge for Industrial Automation

Overview

• Presentation by Karim Ahnuk and Muhammad Salah, AWS Solutions Architects
• Focuses on addressing the challenges of unplanned downtime in manufacturing through the use of Agentic AI and edge computing

Key Challenges in Manufacturing

1. Data Silos: Machines in a production line have their own separate databases, leading to a lack of end-to-end visibility.
2. Skill Gap: Experienced experts who can diagnose issues based on machine sounds or visual cues are often not available.
3. Production Delays: Lack of data integration and expert knowledge prevents timely action to address problems.
4. Operational Disruption: Unstable internet connectivity and lack of cloud access can make manufacturers "blind" to data-driven insights.

Proposed Solution Architecture

1. Unified Data Lake: AWS Outpost is used to consolidate data from various machines and OEMs into a single data lake.
2. Unified API Deployment: An EKS local cluster on the Outpost provides a unified API layer to integrate the different machine interfaces.
3. AI-powered Insights: The consolidated data is used to power AI-driven insights and recommendations, including through the use of Amazon Quicksight dashboards.
4. Edge Intelligence: Multiple small language models are deployed at the edge to provide real-time intelligence and automation, including the ability to chat with operators and take corrective actions.

Data Preparation and Model Fine-tuning

1. Data Ingestion: The solution ingests various data sources, including CSV files, text files, and PDF documents, to prepare the data for model fine-tuning.
2. Structured Data Generation: A large language model (LLM) is used to generate structured question-answer pairs from the unstructured data, which are then validated to ensure accuracy and consistency.
3. Model Fine-tuning: The structured data is used to fine-tune a pre-trained language model (e.g., PaLM 3.2B) using Amazon SageMaker, with the goal of creating an assistant that can provide detailed, instructed steps to operators.

LLM as a Judge Evaluation

• The team conducted experiments to compare the performance of the fine-tuned model against the base LLM model on the task of Retrieval Augmented Generation (RAG).
• Three different LLM models (Claude 4.5 Sonnet, Claude 4.5 Haiko, and Amazon Nova Pro) were used as "judges" to evaluate the accuracy, completeness, and relevance of the responses generated by the fine-tuned and base models.
• The results showed that the fine-tuned model outperformed the base model by an average of 14 percentage points across the three evaluation criteria.

Edge Deployment and Agent Implementation

1. Infrastructure Deployment: Two EC2 instances are used to mimic the AWS Outpost environment, with one instance hosting the Agentic AI application and the other hosting the fine-tuned language model.
2. RAG Retriever Tool: A custom tool is developed using the Strand SDK to perform Retrieval Augmented Generation (RAG), integrating the fine-tuned language model for generating responses.
3. Telemetry Tool: A tool is created to retrieve real-time telemetry data and alarms from the production line machines, allowing the agent to access and analyze this information.
4. Agentic AI Agent: The Agentic AI agent is implemented using the GPT-OSS model and the custom tools, with a detailed system prompt to guide the agent's interactions with the operator.

Key Takeaways

1. Connecting operations through a full deployment on AWS Outpost's EKS local cluster ensures resilience and access to control and data planes, even in the event of disruptions.
2. Deploying small language models at the edge enables real-time intelligence and automation, allowing the agent to provide recommendations and take corrective actions.
3. Continuously updating the data and fine-tuning the model is crucial to maintain the agent's relevance and effectiveness in the evolving manufacturing environment.

Conclusion

The presented solution demonstrates how Agentic AI, powered by fine-tuned language models and deployed at the edge, can address the key challenges in manufacturing by integrating data, expert knowledge, and real-time insights to improve operational efficiency and reduce unplanned downtime.