AWS re:Invent 2025 -From Agentic AI Demos to ROI: Path to Production With Confluent, Anthropic & AWS

From Agentic AI Demos to ROI: Path to Production With Confluent, Anthropic & AWS

Transitioning from Demos to Production

• Many businesses struggle to move AI demos and proofs-of-concept into production use cases
• Challenges include demos not designed for production, data/context issues, and lack of clear ROI

Evolution of AI Models

• Shift from purpose-built predictive models to foundation models/generative AI
• Purpose-built models are tailored to specific tasks but lack reusability
• Foundation models are more versatile but require more context/data to be effective

Prompt Engineering vs. Context Engineering

• Prompt engineering (optimizing prompts for single-shot model outputs) has given way to context engineering
• Context engineering involves managing the full information environment for agents running in a loop
• Key aspects include system prompts, tool definitions, data retrieval, and long-horizon optimizations

Prompt Engineering Challenges

• Overly complex or vague prompts can confuse or overwhelm language models
• Prompts should provide just the right level of detail and instructions

Tool Design for Agents

• Tools represent additional prompting information for agents
• Require simple, accurate names and detailed descriptions to guide agent behavior
• Avoid tool overlap/ambiguity that can confuse the model

Data Retrieval and Context Curation

• Agents need access to relevant, up-to-date business context - not just historical data
• Approaches include:
  • Connecting to operational data sources via APIs/MTP
  • Leveraging data warehouses/lakehouses to create curated data products
  • Streaming data capture and real-time context engines

Confluent's Real-Time Context Engine

• Materializes Kafka topics into managed, low-latency tables accessible via MTP
• Enables AI agents to query fresh, relevant business context on-demand
• Integrates with Confluent's streaming agents and ML functions

Business Impact and Use Cases

• Empowers AI agents to make informed, contextual decisions for operational use cases
• Example: River robo-taxi customer service agent accessing ride requests, vehicle telemetry, and dispatch decisions

Key Takeaways

• Transitioning AI from demos to production requires evolving from prompt engineering to comprehensive context engineering
• Managing the full information environment for agents is critical, including prompts, tools, and access to relevant, up-to-date data
• Confluent's real-time context engine provides a solution for serving fresh business context to power AI agents in operational scenarios