AWS re:Invent 2025 - Maximize ROI: How Octus migrated CreditAI from Azure to Amazon Bedrock (SMB305)

Migrating CreditAI from Azure to Amazon Bedrock: Maximizing ROI and Scaling Generative AI Workloads

Overview

• Octus, a leading provider of data, news, and analytics products for the credit market, migrated their flagship product CreditAI from Azure to Amazon Bedrock.
• The presentation covers the challenges, design decisions, and results of this migration, providing insights for companies looking to scale their generative AI (GenAI) workloads.

Challenges and Requirements

• Octus faced several challenges with their existing CreditAI architecture, including:
  • Complexity of managing the retrieval-augmented generation (RAG) pipeline components
  • Operational challenges of shuttling data between AWS and Azure
  • Desire to unify their cloud architecture and simplify management
• Octus defined the following non-negotiable requirements for the migration:
  1. Scalability to handle increasing demand and development speed
  2. Minimizing costs by consolidating cloud services
  3. Optimizing response latency for a great user experience
  4. Ensuring 100% uptime and compliance with SOC2 requirements
  5. Ability to rapidly iterate and develop new features

Migration to Amazon Bedrock

• Octus chose to migrate CreditAI to Amazon Bedrock, a managed service for building RAG-based applications, to address their challenges and meet their requirements.
• Key design decisions and architecture:
  • Event-driven data extraction and ingestion pipeline using AWS Lambda, Amazon Textract, and Amazon S3
  • Semantic document chunking and Cohere embedding models for efficient retrieval
  • Multi-tenant isolation using separate knowledge bases for each client
  • Leveraging Amazon Bedrock's built-in guardrails for content safety and performance optimization
  • Iterative improvements to the search, re-ranking, and query reformulation strategies

Data Strategy and Migration Approach

• Octus had a strong, unified data strategy across their product portfolio, which was critical for the success of CreditAI.
  • Master data management, standardized data pipelines, and a centralized API layer enabled a consistent data experience.
• The migration process involved:
  • Infrastructure as code using Terraform
  • Integrated CI/CD pipeline for seamless deployments
  • Parallel running of the existing and new architectures for thorough testing and validation
  • Comprehensive security and compliance measures, including the use of tools like VizStack for cloud security posture management

Results and Lessons Learned

• Octus achieved significant improvements after the migration:
  • 78% reduction in costs
  • 87% decrease in cost-per-question
  • Faster document synchronization (hours to minutes)
  • Increased development velocity and feature delivery
  • Improved user experience with better latency and reliability
• Key lessons learned:
  1. Having clear, non-negotiable requirements from the start helped maintain focus during the migration.
  2. Close collaboration with AWS solution experts was crucial for navigating the complexities.
  3. Constantly evolving AI landscape requires a flexible, simplified architecture to keep up with changes.

AWS Programs to Accelerate GenAI Migrations

• AWS offers several programs to support companies in their generative AI migration and production journeys:
  • Demo Squad and GenAI Innovation Center for exploration and proof-of-concept phases
  • Professional Services and partner consulting for advanced deployment and production phases
  • Migration Acceleration Program (MAP) to assess, validate, and execute end-to-end migrations
• AWS also provides options for easy migration of LLM API endpoints to Amazon Bedrock, as well as support for more complex workloads and full-stack applications.

Key Takeaways

• Successful generative AI deployments require addressing not just the AI components, but also the underlying infrastructure, data strategy, security, and compliance.
• Simplifying the architecture by leveraging managed services like Amazon Bedrock can significantly improve cost, scalability, and operational efficiency.
• A well-defined migration strategy, close collaboration with experts, and a focus on non-negotiable requirements are crucial for a successful GenAI migration.
• AWS offers a range of programs and services to support companies at every stage of their generative AI journey, from exploration to production.