Introducing automated reasoning checks in Amazon Bedrock Guardrails (AIM393-NEW)

Here is a detailed summary of the key takeaways from the video transcription, broken down into sections for better readability:

Introduction

  • The speaker is Stefano, a product manager at AWS, who is presenting with Byron, a distinguished scientist on the automated reasoning team.
  • They will discuss how they came up with the idea for automated reasoning checks, provide an introduction to the concept, and then demonstrate how to get started with the feature.

The Journey to Automated Reasoning Checks

  • The speaker shares a common story of how leaders often task someone to figure out a strategy for new technologies like generative AI.
  • The speaker tried to understand large language models (LLMs) and identify use cases, such as automating onboarding and customer support.
  • While the initial experiments were exciting, they found that the LLM outputs were not always accurate, with subtle hallucinations being a top concern for the industry.
  • This led the team to explore automated reasoning checks as a way to ensure factual accuracy, soundness (not claiming something wrong is right), and transparency/explainability of the reasoning.

What is Automated Reasoning?

  • Automated reasoning, also known as symbolic AI, is rooted in mathematical logic and contrasts with the data-driven approach of machine learning.
  • It aims to reason about all possible data, rather than learning from a finite set of examples.
  • The speaker provides a detailed example of how automated reasoning can be used to prove that a simple loop will always terminate, highlighting the key concepts of well-foundedness and using SMT (satisfiability modulo theories) solvers.
  • AWS has been using automated reasoning extensively for over 10 years, applying it to various services and components to prove their correctness.

Getting Started with Automated Reasoning Checks

  • The process to get started with automated reasoning checks has three steps:
    1. Create an automated reasoning policy, which is a structured mathematical representation of the knowledge.
    2. Configure Bedrock Guardrails to use the automated reasoning policy to validate incoming questions and answers.
    3. Use the automated reasoning checks at inference time to validate the accuracy of language model responses.
  • The speaker walks through a practical example of creating an automated reasoning policy for a leave of absence HR policy and demonstrates how to use the feedback provided by the automated reasoning checks.

Key Objectives and Next Steps

  • The key objectives of automated reasoning checks are to help build applications that are:
    1. Accurate in their factual claims
    2. Sound (not claiming something wrong is right)
    3. Transparent with an auditable log of the reasoning
  • The speaker provides QR codes to resources for learning more about automated reasoning and getting started with Bedrock Guardrails and automated reasoning checks.

Your Digital Journey deserves a great story.

Build one with us.

Cookies Icon

These cookies are used to collect information about how you interact with this website and allow us to remember you. We use this information to improve and customize your browsing experience, as well as for analytics.

If you decline, your information won’t be tracked when you visit this website. A single cookie will be used in your browser to remember your preference.

Talk to us