Scaling AI with Confidence: CDL's Journey from Proof of Concept to Production

Overview

This presentation details the journey of CDL, a B2B software company in the UK insurance industry, as they scaled their use of AI from early proof-of-concept to full production deployment. Key topics covered include:

• CDL's business context and the role of observability in their highly interconnected ecosystem
• The strategic approach CDL took to enable AI adoption, including process changes, upskilling, and building an AI steering group
• The technical architecture and tooling CDL leveraged, including AWS services like Amazon Bedrock and Dino Trace's observability platform
• Lessons learned around culture, change management, and measuring success as CDL rolled out AI capabilities

CDL's Business Context

• CDL is a major player in the UK insurance industry, handling around 60% of all online private car, household, and motorcycle insurance transactions
• Their highly interconnected ecosystem involves integrations with aggregators, customers, and government systems, processing over 1 trillion transactions annually
• Observability through Dino Trace was critical to understanding performance and issues across this complex ecosystem, rather than just focusing on CDL's own systems

Enabling AI Adoption

Strategic Approach

• CDL's CEO was a strong proponent of AI, challenging the organization to innovate and adopt the technology
• They formed an AI steering group with cross-functional representation (executives, compliance, legal, architecture) to guide decision-making
• Key priorities were ensuring compliance, security, and ethics, as well as streamlining processes for rapid AI adoption

People and Culture

• All 250 engineers were upskilled with cloud certifications to establish a common technical language
• CDL fostered a culture of exploration and experimentation around AI, with hackathons and incubators to drive innovation
• Educating the executive team and board on AI concepts and capabilities was crucial to align the organization

Technical Foundations

• CDL migrated their entire infrastructure to AWS, enabling agility and access to the latest AI/ML services
• They adopted an API-first, microservices architecture to make data more accessible for AI applications
• This technical foundation, combined with the cultural changes, positioned CDL to rapidly evaluate and deploy AI capabilities

AI Architecture and Tooling

Bedrock and Observability

• CDL leveraged Amazon Bedrock as the core of their AI "engine room", allowing them to easily evaluate and swap out different language models
• Dino Trace's observability platform provided critical insights, tracking not just technical metrics but also AI-specific behaviors like relevance, grounding, and token usage
• This observability was key to ensuring cost control, explainability, and trust in the AI systems

Modular, Composable Approach

• CDL took a modular, "thin slice" approach to AI deployment, focusing on low-risk use cases first (e.g., policy changes not affecting premiums)
• This allowed them to iterate quickly, get customer buy-in, and expand the AI capabilities over time
• The composable nature of services like Amazon Bedrock and Dino Trace enabled this agile, incremental rollout

Leveraging AWS Services

• In addition to Bedrock, CDL utilized other AWS AI/ML services like Amazon Kendra, Amazon Lex, and Amazon Transcribe to build out their AI agent capabilities
• The recent launch of Amazon Agent Core was seen as a key enabler for CDL's next phase of AI expansion, providing a reliable, scalable platform for deploying and managing AI agents

Key Takeaways

1. Define a Clear "Why": CDL's strategic focus on specific business objectives and use cases was critical to driving successful AI adoption, rather than just chasing the latest technology.
2. Prioritize People and Culture: Upskilling teams, fostering experimentation, and aligning leadership were essential to overcoming resistance and fear around AI.
3. Leverage Observability and Guardrails: Comprehensive observability and robust guardrails enabled CDL to deploy AI safely, maintain trust, and ensure compliance.
4. Adopt a Modular, Composable Approach: CDL's incremental, "thin slice" rollout strategy allowed them to deliver value quickly and expand AI capabilities over time.
5. Partner with Cloud Platforms: Tight integration with AWS services like Bedrock, Agent Core, and other AI/ML offerings was a key enabler of CDL's AI journey.

Conclusion

CDL's experience demonstrates the importance of a strategic, people-centric approach to scaling AI, underpinned by robust technical foundations and close collaboration with cloud platform providers. By prioritizing observability, compliance, and an incremental rollout strategy, CDL was able to successfully transition from AI proof-of-concept to full production deployment, delivering tangible business value through their insurance industry ecosystem.