AWS re:Invent 2025 - Fixing AI’s Confidently Wrong Problem in the Enterprise (AIM269)

Fixing AI's Confidently Wrong Problem in the Enterprise

The Challenge of AI Adoption

• AI has seen rapid adoption, with nearly a billion users in a short period of time
• However, the ability to trust and use AI connected to real business data remains a significant challenge
• Even when AI models can be made to work, business users often cannot trust the outputs due to the problem of "AI being confidently wrong"
• This lack of trust severely limits the adoption and usefulness of AI in enterprise settings

The Importance of Continuous Improvement

• The value of human experts is not just their raw intelligence, but their ability to continuously learn, absorb context, and incrementally improve
• For AI to be truly useful, it needs to have this same capacity for continuous improvement and learning
• The key is that AI can only be taught when it acknowledges what it doesn't know

ProMQL's Approach

Surfacing Uncertainty and Assumptions

• ProMQL's system first surfaces what the AI knows and doesn't know, highlighting areas of uncertainty
• When the AI doesn't know something, it makes explicit assumptions and surfaces them to the user
• This allows users to understand the AI's limitations and bring in expert knowledge to fill the gaps

Collaborative Knowledge Building

• When the AI identifies gaps in its knowledge, users and experts can collaborate to update the underlying knowledge base (wiki)
• This wiki contains structured information about key business concepts, data sources, and analysis methodologies
• As the wiki is collaboratively maintained, the AI's capabilities continuously improve

Handling Complex Schemas and Data

• ProMQL's system is designed to handle large, complex data schemas with thousands of tables and metrics
• It surfaces not just semantic uncertainties, but also technical assumptions and limitations in the analysis
• This allows users and experts to provide feedback and improve the system over time

Key Takeaways

• Overcoming the "confidently wrong" problem is critical for driving real-world AI adoption in enterprises
• Surfacing uncertainty, making assumptions explicit, and enabling collaborative knowledge building are key to building trustworthy AI systems
• ProMQL's approach focuses on creating a continuous improvement loop between AI and human experts
• This allows the system to scale to handle complex, enterprise-grade data and use cases

Business Impact and Use Cases

• ProMQL's approach has enabled the company to scale rapidly across a variety of high-velocity decision-making use cases
• By addressing the trust and adoption challenges, ProMQL empowers business users to leverage AI for tasks like:
  • Improving business performance through better staffing, pricing, and other decisions
  • Analyzing financial data, forecasting, and other finance-related use cases
• The ability to handle large, complex data schemas and continuously improve the system are key differentiators that enable enterprise-grade AI applications