SK Telecom's TelClaude: Redefining telco CX with generative AI on AWS (TLC203)

Here is a detailed summary of the video transcription in Markdown format:

The Collaboration between SK Telecom and Anthropic/AWS

Introduction

• The presentation covers the collaboration between SK Telecom (SKT), Anthropic, and AWS to fine-tune the Claude model for SKT's contact center use cases.
• The format includes a mix of presentation and a pre-recorded segment by Eric Davis, VP of the AI Tech Collaboration Group at SKT.

The Telco Industry and Contact Centers

• Telco customers expect anytime, anywhere, any-modality service, with phone calls still being the primary interaction channel.
• Contact centers handle massive call volumes (billions of minutes per year), presenting opportunities for optimization and customer experience improvement.
• 60-70% of telco calls are related to billing or account issues, which can be mapped, understood, and automated.
• Customer expectations have increased, driven by digital experiences like Amazon, leading to long wait times and suboptimal experiences.
• Contact centers are shifting from cost centers to profit centers, with goals to cross-sell and upsell during interactions.

The Need for a Telco-Specific Language Model

• Base language models are a "jack of all trades, master of none" - they lack the domain-specific knowledge and capabilities required for telco use cases.
• SKT needed a model that could:
  1. Understand telco products and services
  2. Provide product recommendations and selection
  3. Understand customer intents and take appropriate actions

The Telco Large Language Model (TCLM)

• SKT partnered with Anthropic to build the TCLM, a model tailored for the telco domain.
• The TCLM is built as a "pyramid" with multiple layers:
  1. Base model
  2. Fine-tuned model
  3. Tools for tasks like retrieval, API integration, and orchestration
  4. Prompting and prompt engineering

The Collaboration Process

• SKT provided domain expertise and data, while Anthropic handled the model fine-tuning and reinforcement learning.
• AWS provided the Generative AI Innovation Center, including the model customization program and infrastructure support.
• The collaboration involved dedicated resources from each party, including prompt engineers, researchers, and solution architects.

Key Techniques and Improvements

• Optimizations included:
  • "Mega prompts" to call multiple tasks at once for improved speed and cost
  • Curriculum learning for fine-tuning, starting with easier tasks and progressing to harder ones
  • Retrieval Augmented Generation (RAG) to ensure accuracy and reduce hallucination
• The TCLM demonstrated significant improvements over the base model:
  • 38% improvement in Telco Expertise Score
  • Over 90% customer satisfaction from contact center agents

Use Cases and Future Expansion

• Two key use cases:
  1. Real-time Assistance: Automating agent search and response generation
  2. Post-call Analysis: Automating call summarization, intent classification, and topic extraction
• Plans for further expansion to other telco domains (marketing, network support, internal operations) and beyond (B2B partnerships)

Lessons Learned and Benefits

• Improved customer experience with faster, more uniform responses
• Increased contact center agent satisfaction and reduced ramp-up time for new hires
• Successful collaboration through dedicated resources and deep integration between the three parties