AWS re:Invent 2025 - The AI Discussion Control Plane: How Your Agentic Team Redefines Ops (AIM418)

Summary of AWS re:Invent 2025 - The AI Discussion Control Plane: How Your Agentic Team Redefines Ops (AIM418)

The Changing Landscape of Software Operations

• In the next 2 years, the way we operate software will change dramatically
• Dario Amade, CEO of Entrophic, predicted that AI will write 90% of the code that software developers are currently responsible for
• Even if the 90% prediction is not fully realized yet, a significant portion of code (potentially 50%) will be generated by AI in the near future
• This shift means that engineers will have less context on the code and systems they are responsible for, as well as the potential impact of issues on other AI-generated components

The Need for a New Operational Pattern

• The current state of modern operations is highly siloed, with people responsible for different parts of the system and reacting to issues in a piecemeal fashion
• Traditional tools and processes are not sufficient to address the challenges of tomorrow's AI-driven software landscape
• There is a need for a new operational pattern that can handle the increased complexity and lack of context introduced by AI-generated components

The AI Discussion Control Plane

• The presenter introduces the concept of an "AI Discussion Control Plane" - a system where AI "teammates" collaborate to address operational issues
• AI Assistants vs. AI Teammates:
  • AI Assistants are passive helpers that respond to prompts and provide information
  • AI Teammates are more proactive, initiating actions and research to address issues without being explicitly asked
  • AI Teammates have a broader context and memory, understanding the overall system and dependencies beyond just the immediate problem
• The AI Discussion Control Plane includes an orchestrator that manages the various AI Teammates, assigns tasks based on their expertise, and coordinates their actions
• The system allows for human approval and control over the AI Teammates' actions, ensuring they operate within defined boundaries

Challenges of Running LLMs on Streaming Data

• Certain data types, such as logs and events, are more compatible with LLMs due to their structured, tokenizable nature
• However, feeding LLMs with massive amounts of raw telemetry data (e.g., metrics, traces) is not scalable or efficient
• Techniques like distributed tail sampling and data distillation are required to selectively feed relevant data to the LLMs

Telemetry Pipelines and Data Governance

• Telemetry pipelines have evolved from simple data archiving to more intelligent data filtering and aggregation
• The next step is using telemetry pipelines to provide high-value data to AI systems while maintaining control and governance over sensitive information
• Telemetry pipelines can be used to mask and filter data, protecting intellectual property, user data, and other sensitive information before it is exposed to AI models

The Evolving Role of Human Operators

• The introduction of AI Teammates does not replace human operators but rather elevates their role
• Humans will be responsible for risk management, monitoring the behavior of AI Teammates, and adjusting the tools, models, and data used by the AI systems
• The focus will shift towards complex problem-solving, proactive issue prevention, and managing the organizational risk associated with AI-driven operations

Key Takeaways

• The way we operate software will undergo a significant transformation in the next 2 years, with AI generating a substantial portion of the code
• The AI Discussion Control Plane is a new operational pattern that leverages proactive AI Teammates to address issues, while maintaining human oversight and control
• Handling the scale and complexity of telemetry data for AI systems requires intelligent data processing and governance techniques
• The role of human operators evolves from reactive problem-solving to strategic risk management and AI behavior control