AWS re:Invent 2024 -Shipping gen AI faster & safer in high tech with LaunchDarkly & Poka.io (AIM262)

Shipping AI, Fast and Safe

Introduction

  • The presentation is about how companies can take AI concepts and get them from an idea to something that is actually in production and working, useful.
  • Tom Totenberg from LaunchDarkly and Edmund Lam from Poka will share their experiences.

Poka's Connected Worker Platform

  • Poka is a connected worker platform for manufacturers, helping factory workers excel at their jobs by providing the right tools and information in a single integrated platform.
  • Poka has its app running in over 1,500 factories around the world in 70 countries, used by large brands like Nestlé, Bosch, Mars, and L'Oreal.
  • Poka's AI R&D/Enablement team is structured with two roles:
    1. A complicated subsystem team to gather expertise on using AI properly and set standards.
    2. An enablement team to share knowledge and work with product/design teams to prototype ideas quickly.

Poka's PDF to Work Instruction Converter

  • One of the biggest challenges for Poka's clients is creating or importing work instructions, often in PDF format.
  • Poka's vision was to allow clients to drag and drop PDFs, hit a "generate" button, and have work instructions in Poka's native format, ready to review.
  • This would be faster, extract images from PDFs, and set the stage for other AI features like retrieval-augmented generation.

Development Process

  1. Prototyping: Tested if generative AI models could understand complicated PDF work instructions.
  2. Proof of Concept: Created a simple UI to upload PDFs, generate work instructions, and get internal feedback to improve the conversion quality.
  3. Beta Testing: Cleaned up the UI, redid the architecture to use a serverless, asynchronous, and scalable content AI microservice.

Using Feature Flags

  • Poka used Feature Flags from LaunchDarkly in several ways:
    1. Main feature flag to safely commit code to the main branch without exposing the feature.
    2. Ops flags for configuration like file size limits, concurrent file limits, etc.
    3. AI-specific ops flags to quickly switch between different AI models (Haiku, Sonnet, Opus) and adjust prompts.

Prompt Handling and Security

  • Poka added a "prompt hacking detection" step to check if the PDF content contained anything adversarial.
  • Initially, the AI returned a simple 0/1 flag, which was later expanded to return JSON with more details, including a reason for flagging content as adversarial.
  • Poka also added examples to help the AI better detect adversarial prompts.
  • When the AI flagged a "guillotine" PDF as violent, Poka worked around it by introducing a "violent" attribute that they could ignore, while still respecting the AI's guidelines.

Beta Testing and Results

  • Poka's beta involved 20 clients and 50 users, who performed over 1,000 conversions.
  • Users reported a 75% reduction in time to import work instructions, from over an hour to around 15 minutes.
  • One client was able to set up a new production line in 3 months instead of 5 months, and estimated 2 months for the next expansion, thanks to the AI feature.

Lessons Learned

  1. The small, dedicated AI team structure worked well, providing expertise to the main feature teams as needed.
  2. Using Amazon Bedrock and LaunchDarkly's Feature Flags enabled quick iteration and testing of different AI models.
  3. Gathering customer feedback was crucial to confirm the value of the AI feature.

LaunchDarkly's Role

  • LaunchDarkly enables the separation of code deployment and feature release, allowing Poka to iterate quickly on AI models and prompts without being blocked by the deployment process.
  • LaunchDarkly provides features to manage AI-specific configurations, target specific audiences for testing, and collect telemetry and feedback to measure the performance and impact of the AI feature.

Q&A

The presentation concludes with a Q&A session, where the audience can ask questions to Edmund and Tom.

Your Digital Journey deserves a great story.

Build one with us.

Cookies Icon

These cookies are used to collect information about how you interact with this website and allow us to remember you. We use this information to improve and customize your browsing experience, as well as for analytics.

If you decline, your information won’t be tracked when you visit this website. A single cookie will be used in your browser to remember your preference.

Talk to us