Guiding your customer to production: Generative AI POC guidance & tips (PEX106)

Scaling Generative AI POCs to Production: AWS Methodology

Key Takeaways

  • Less than 30% of POCs are scaling to production
  • Common reasons POCs get stuck include:
    1. Lack of ROI modeling
    2. Lack of advanced optimization
    3. Lack of data strategy
    4. Lack of in-house skills
    5. Lack of strategic priority
    6. Tactical roadblocks like procurement and compliance challenges
  • AWS's 8-step methodology for successful POC-to-production journey:
    1. Business Modeling
    2. Data Strategy
    3. Model Selection
    4. Architecture
    5. Security and Compliance
    6. Optimization
    7. MLOps
    8. Monitoring

Business Modeling

  • Venminder, an AWS partner, streamlined document processing and compliance assessments using generative AI
  • They unlocked 70% of analysts' time and reduced contract review turnaround time from 65 days to under 3 days
  • They focused on tight alignment between business and technical teams, integrating multiple AI models to improve data accuracy from 60% to nearly 100%

Data Strategy

  • Venminder prioritized data accessibility, integration, and security/compliance
  • They provided rich context to the AI models by integrating diverse data sources like SOC statements, security policies, and business continuity plans
  • They leveraged Amazon Kendra to handle a variety of data formats

Model Selection

  • Venminder evaluated multiple models during the POC phase, focusing on accuracy, context understanding, and speed
  • They selected Amazon Claude Sonnet 3.5, which provided the desired data accuracy improvement

Architecture

  • Venminder leveraged serverless and managed services like Amazon Bedrock, AWS Lambda, and Amazon Kendra
  • They implemented a data pipeline to securely move data, task-based routing to use appropriate models for specific document types, and mechanisms to combine outputs from multiple AI models

Security and Compliance

  • Venminder implemented robust access control, data protection, and human oversight for critical processes
  • They also built a business continuity plan and conducted regular audits to improve their AI processes and models

Optimization

  • Venminder used Amazon Bedrock to compare and select the right models for their use cases
  • They built a prompt catalog system to allow users to reuse and customize tunable prompts

MLOps

  • Venminder built a data pipeline using AWS services and followed CSAT methods to deploy new AI models and prompts

Monitoring

  • Venminder implemented a robust monitoring framework to track both business and technical metrics, including customer satisfaction, productivity gains, cost savings, service health, success rates, error rates, and model accuracy

Quantiphi's Solution for ACTO

  • ACTO, a life sciences customer, wanted to improve productivity of their sales representatives and reduce onboarding time for new hires
  • Quantiphi built a generative AI solution using their Bionic platform, following the same 8-step methodology
  • Key aspects include:
    • Robust data strategy for ingestion, storage, and processing
    • Event-driven architecture leveraging serverless services like AWS Lambda and SQS
    • Model selection focused on latency, performance, and cost
    • Comprehensive security and monitoring implementation

Take-home Kit

  • AWS has made available an 8-step guide, sample code, Bedrock workshops, and training resources to help customers and partners accelerate their generative AI adoption

Your Digital Journey deserves a great story.

Build one with us.

Cookies Icon

These cookies are used to collect information about how you interact with this website and allow us to remember you. We use this information to improve and customize your browsing experience, as well as for analytics.

If you decline, your information won’t be tracked when you visit this website. A single cookie will be used in your browser to remember your preference.

Talk to us