AWS re:Invent 2024 -Unleashing gen AI: Secure & scalable architectures with Snowflake & AWS (AIM267)

Here is a detailed summary of the video transcription in markdown format:

Introduction to Snowflake and AWS for Generative AI

About the Speaker

• The speaker is Matt Marillo, a Partner Engineer at Snowflake.
• He works with product teams at Snowflake and AWS to ensure their platforms work well together for AI and Generative AI workloads.
• He also works with individual customers to help them understand how to better use Snowflake and AWS for Generative AI.

What is Snowflake?

• Snowflake is a cloud-native data platform that has been around for 12 years.
• It started as a data warehouse solution and has since grown into a more comprehensive platform.
• Snowflake offers features such as separation of storage and compute, near-limitless scalability, low administration, and ease of use.
• Snowflake has native integrations with the entire AWS ecosystem, including services like S3, Glue, EMR, Lambda, Sage, and Bedrock.

AWS's Generative AI Stack

• AWS's Generative AI stack includes:
  • Infrastructure: Trinium, Inferentia chips, and Nitro
  • Bedrock: Foundational models and the ability to fine-tune models
  • Amazon Q: A popular new service for Generative AI

Snowflake's Generative AI Capabilities

• Snowflake has Generative AI features and functionality under the "Cortex" brand:
  • LLMs: Snowflake has its own homegrown models (Arctic family), as well as models from NVIDIA and LLaMA.
  • Cortex Search: A Generative AI-powered search service for unstructured data in Snowflake.
  • Cortex Analyst: A Generative AI-powered service for structured data in Snowflake, with high accuracy expectations.
  • Document AI: Ability to parse and structure data from scanned documents using Generative AI.

Generative AI Use Cases in Production

• Snowflake is seeing two main use cases for Generative AI in production:
  1. Automated BI and conversational assistance: Allowing users to chat with data and get insights.
  2. Batch processing of text data: Summarizing and synthesizing unstructured data for better utilization.

Integrating Snowflake and AWS for Generative AI

• Orchestrating from Snowflake:
  • Use Snowflake data as context and connect to external services like Bedrock using an external access connector.
  • Wrap a function around the external service and expose it via a Streamlet app.
• Orchestrating from AWS:
  • Use Bedrock agents with Cortex Analyst or Cortex Search as action items in the agent workflow.
  • Set up a plugin in Amazon Q to connect to Snowflake Cortex services.

Demo: Integrating Cortex Search in Snowflake with Amazon Q

• Demonstrated setting up a Cortex Search service in Snowflake for Airbnb listing data.
• Showed how to integrate the Cortex Search service with Amazon Q using a plugin.
• Allowed prompting the Cortex Search service from Amazon Q and getting relevant search results.

Calls to Action

• Try out the quick starts at snowflake.com/quickstart to test Snowflake and AWS integrations.
• Check out the Bedrock agent workflow demo workshop published by AWS.
• Reach out to the Snowflake and AWS teams for direct support and collaboration.