AWS re:Invent 2025 - Building a State of the Art Agentic Infrastructure (AIM297)

Building a State-of-the-Art Agentic Infrastructure (AIM297)

The Promise and Challenges of AI Agents

• AI agents hold significant promise for improving business operations and efficiency
• They have the potential to reduce costs, boost productivity, generate revenue, and increase customer retention
• However, the reality is that AI agents are currently causing significant issues for businesses

The Cost and Margin Challenges of AI Adoption

• 84% of companies see at least a 6% erosion of net business margins due to AI programs
• 25% of companies see greater than 16% margin erosion
• This is not due to over-investment, but rather fragmented visibility and governance across AI models, providers, and infrastructure

The Visibility and Governance Challenges

• 86% of organizations are completely blind to how data flows around their AI systems
• 20% of all incidents are now categorized as "shadow AI" incidents due to lack of governance
• Challenges include:
  • Identifying the right models for the job
  • Ensuring security, quality, and data privacy
  • Measuring and controlling costs of token usage and API consumption

The Complexity of the Agentic Data Path

• AI agents require a complex "agentic data path" involving language models, context providers, APIs, and event streams
• Managing the cost, security, and reliability of this entire data path is critical, but highly challenging due to fragmentation

The Need for a Unified Connectivity Layer

• Fragmentation is the root cause of the cost, risk, and visibility challenges
• Organizations must unify their approach to building, running, discovering, governing, and monetizing their AI infrastructure
• This requires a focus on building a flexible, secure, and reliable "AI connectivity layer" that can adapt to changing AI technologies and use cases

Key Pillars of an AI Connectivity Platform

1. Build: Enable developers to quickly design, test, and deploy AI resources and integration patterns
2. Run: Implement central security and access controls for exposing AI resources
3. Discover: Provide self-service discovery and packaging of AI assets for both human and agent consumers
4. Govern: Unify visibility and enforcement of standards, best practices, and guardrails across the entire AI data path
5. Monetize: Enable real-time cost governance and potential monetization of AI resources

The Imperative for Unified AI Infrastructure

• Organizations that can master a unified, governed approach to AI infrastructure will be able to protect margins and gain competitive advantage
• Those that fail to address fragmentation will face an "agentic innovation death spiral" of increasing costs, risks, and lost market share
• The key is unifying visibility and enforcement across the entire AI data path to enable speed, cost control, and risk mitigation