Unified and open governance for data and AI (AIM256)

Introduction

  • Zeashan Pappa, a product manager at Databricks, discussed the importance of governance in the age of generative AI and large language models.
  • Fabien Thiaucourt from Mastercard shared Mastercard's journey in leveraging data and AI for innovation, while emphasizing the need for responsible data usage.

The Importance of Governance

  • Governance is critical for scaling data and AI initiatives, as it ensures accountability, compliance, quality, and transparency.
  • In most organizations, data and AI assets are scattered across different sources, formats, and tools, leading to fragmented governance and various challenges.
  • Fragmented governance can result in increased complexity, security risks, and operational inefficiency, hindering innovation and business decision-making.

Databricks' Approach: Unity Catalog

  • Databricks' Unity Catalog offers a unified and open governance solution for all data and AI assets, addressing the challenges of fragmented governance.
  • Unity Catalog provides:
    • Unified governance across data and AI assets, enabling operational efficiency, compliance, and security.
    • Open connectivity, allowing teams to access data and AI assets from any tool, engine, or platform, avoiding vendor lock-in.
    • Built-in intelligence that connects the dots across diverse data and AI assets, providing context-driven insights and autonomous performance optimizations.

Mastercard's Journey with Data and AI Governance

  • Mastercard, as a technology and data company, is focused on leveraging data and AI to prevent fraud and enable responsible innovation.
  • Challenges faced by Mastercard's data users:
    • Data discovery: Finding the right data for use cases can be daunting, and users often rely on tribal knowledge.
    • Data access: Navigating the patchwork of data usage rules and regulations can be complex.
    • Data utilization: Efficiently using data and AI to solve problems requires documentation, lineage, and insights.

Mastercard's Approach with Databricks

  • Mastercard is leveraging Databricks' Unity Catalog to address these challenges:
    • Data discovery: Unity Catalog provides metadata, lineage, and usage insights to help users find and understand the right data.
    • Data access: Unity Catalog's automated access controls and governance capabilities enable "TSA precheck" for data users.
    • Data utilization: Unity Catalog's integrated AI features, such as natural language querying and automated performance optimizations, empower users to more effectively leverage data and AI.

Key Takeaways

  • Governance is crucial for scaling data and AI initiatives, especially in the age of generative AI and large language models.
  • Databricks' Unity Catalog offers a unified and open governance solution that addresses the challenges of fragmented governance.
  • Mastercard's journey with Databricks showcases the benefits of leveraging a platform that unifies data governance, access, and intelligence to enable responsible innovation.
  • Prioritizing data governance, fostering cross-functional collaboration, and learning by doing are essential lessons from Mastercard's experience.

Your Digital Journey deserves a great story.

Build one with us.

Cookies Icon

These cookies are used to collect information about how you interact with this website and allow us to remember you. We use this information to improve and customize your browsing experience, as well as for analytics.

If you decline, your information won’t be tracked when you visit this website. A single cookie will be used in your browser to remember your preference.

Talk to us