AWS re:Invent 2025 - BCC: Hybrid architecture for generative AI to meet regulatory needs (HMC324)

Hybrid Architecture for Generative AI to Meet Regulatory Needs

Overview

• Presentation by Maxim Yang, Head of R&D for DevOps and Cloud Solutions at Bank Center Credit (BCC), a leading bank in Kazakhstan
• Discussed BCC's journey to a secure hybrid multi-cloud architecture to enable innovative AI/ML solutions while meeting regulatory requirements

Drivers for Cloud Adoption

• Growing business needs and limitations of local on-premises solutions
• Desire for:
  • Flexibility and scalability
  • Innovation and competitiveness
  • Reliability and fault tolerance
  • Economic efficiency

Regulatory Requirements

• Data transmission and storage must be encrypted with keys stored on-premises
• Ensure customer data privacy during collection and processing

Hybrid Cloud Architecture

• Leveraged AWS Outpost as a private AWS cloud hosted in BCC's data centers
• Deployed managed AWS services like EC2, Kubernetes, and S3 on Outpost
• Integrated Outpost with other public cloud providers for a multi-cloud Kubernetes architecture
• Used AWS KMS with External Key Store (XKS) to enable encryption with BCC's local keys

Generative AI Use Cases

Fine-tuning Automatic Speech Recognition (ASR) Model

• Reasons for custom model:
  • Handle mixed Kazakh-Russian language
  • Adapt to 8kHz call recordings (vs 16kHz typical models)
  • Leverage low-resource Kazakh language data
• Process:
  1. Extract audio and text from call recordings on-premises
  2. Anonymize text using NER
  3. Convert audio to Mel spectrograms
  4. Transfer data to cloud for fine-tuning on SageMaker
  5. Deploy fine-tuned model back on-premises for inference
• Results:
  • 9-23% accuracy improvement across Russian, Kazakh, and mixed languages
  • 4 million KZT monthly cost savings compared to previous solution

Implementing HR Chatbot

• Goals:
  • Improve quality and velocity of HR responses
  • Promote self-service culture
  • Allow HR team to focus on strategic initiatives
• Approach:
  • Used Retrieval Augmentation Generation (RAG) technique
  • Embedded HR knowledge base on-premises, stored vectors in Postgres
  • Processed user prompts, combined with knowledge base context
  • Sent to AWS Bedrock for language generation
  • Returned response to on-premises chatbot
• Results:
  • 70% of HR requests handled by the chatbot
  • Positive employee feedback on chatbot experience

Key Takeaways

1. Established a secure hybrid multi-cloud platform tailored to banking regulations and business needs
2. Implemented a flexible, scalable, and cost-effective architecture enabling innovative AI/ML solutions
3. Leveraged hybrid architecture for fine-tuning generative AI models, reusing the approach across use cases
4. Demonstrated how generative AI can automate routine tasks, with plans to expand to other functional areas