Accelerate migrations using AWS DMS Schema Conversion with gen AI (DAT347-NEW)

Database Migration with Generative AI

Introduction

  • This session is about the Database Migration Service (DMS) and how it uses generative AI to make database schema conversions more efficient.
  • The speaker, John, is the product lead for the DMS service and has been with the service since its launch in 2015.

Use Cases for DMS

  • Migrating business-critical applications, such as moving a data warehouse to the cloud (e.g. Amazon Redshift)
  • Upgrading database versions with minimal downtime
  • Consolidating multiple databases into a single database (e.g. Aurora)
  • Splitting a monolithic database into microservices
  • Replicating data between databases
  • Modernizing legacy databases to cloud-native or open-source alternatives

How DMS Works

  • DMS focuses on migrating data from one database to another, regardless of whether it's a homogenous (like-to-like) or heterogeneous migration.
  • DMS is a heterogeneous logical replication product, which means it can migrate between different database types while the source database is still operational.
  • DMS is an affordable solution compared to other migration tools in the market.

DMS Schema Conversion

  • DMS Schema Conversion is used for heterogeneous database migrations, where the source and target schemas are different.
  • It performs an assessment to understand how easy or difficult the schema conversion will be.
  • The conversion process involves:
    1. Reading the source schema metadata
    2. Parsing the source code objects (e.g. stored procedures, functions)
    3. Resolving references to local variables and other objects
    4. Converting the source abstract syntax tree to a target abstract syntax tree
    5. Printing the converted schema in the target database language

DMS Schema Conversion with Generative AI

  • The team at DMS conducted several pilots to explore how generative AI could be used to improve schema conversion:
    1. Attempting to convert the entire schema using generative AI, which was too slow and inconsistent
    2. Using AI to determine appropriate data types for parameters and storage objects, but this had potential knock-on effects
    3. Using AI to understand the intent of stored procedures and automatically convert them, but this was too ambitious
    4. Using AI to optimize performance, such as converting subqueries to joins and adding indexes, but the improvements were not substantial enough
  • The final approach was to use a Retrieval Augmented Generation (RAG) technique:
    1. Use Bedrock (Amazon's generative AI service) to understand the context of the SQL code
    2. Query a knowledge base of similar conversion examples
    3. Query a database of conversion rules
    4. Combine this information with the obfuscated SQL code and send it to Bedrock for conversion
  • This approach allows DMS Schema Conversion to leverage generative AI to fill in the gaps where the rule-based engine cannot convert the schema successfully.

Demo 1: Standard Schema Conversion

  • The first demo shows a standard schema conversion from SQL Server to Amazon Aurora Postgres, without using generative AI.
  • The assessment report indicates that the tables converted well, but the code objects (views, procedures, etc.) did not convert as successfully.

Demo 2: Schema Conversion with Generative AI

  • The second demo shows how to enable the generative AI capabilities in DMS Schema Conversion.
  • The conversion ratio improves from 11% to 85% when using generative AI.
  • The results show which parts of the schema were converted using generative AI, with the converted code highlighted between carets.
  • The converted schema is then applied to the target Aurora Postgres database.

Application Code Conversion

  • The demo also shows how to use the Amazon Q Developer Agent to assist with converting the application code from using the SQL Server libraries to the Postgres libraries.
  • This allows the application code to be updated to work with the new target database schema.

Next Steps

  • The DMS team plans to continue improving the schema conversion process, including:
    • More robust testing and validation of the generative AI conversions
    • Expanding the supported database endpoints
    • Continuously updating the generative AI models used (currently using Claude 3.5)
    • Increasing the scope of what can be automatically converted using generative AI

Conclusion

  • The addition of generative AI capabilities to DMS Schema Conversion aims to make database migrations more seamless and efficient.
  • Customers are encouraged to try out the new features and provide feedback to the DMS team.
  • There is much more to come in terms of improving database migration experiences using the latest advancements in generative AI technology.

Your Digital Journey deserves a great story.

Build one with us.

Cookies Icon

These cookies are used to collect information about how you interact with this website and allow us to remember you. We use this information to improve and customize your browsing experience, as well as for analytics.

If you decline, your information won’t be tracked when you visit this website. A single cookie will be used in your browser to remember your preference.

Talk to us