Here is a detailed summary of the video transcription in Markdown format, broken down into sections for better readability:
Introduction
- The speaker, Joey Conway from NVIDIA, will provide a quick introduction and background context before introducing Prem, the guest speaker.
- Joey works at NVIDIA, helping to build deep learning software, particularly around large language models and speech deployment.
Financial Services Industry (FSI) Background
- Technology has been in FSI for a long time, with the history of ATMs, online banking, and mobile banking.
- Today, the focus will be on the use of generative AI in FSI.
- Innovations and challenges faced by FSI include model accuracy, explainability, cybersecurity, data sovereignty, privacy, and regulatory changes.
NVIDIA Software Approach
- NVIDIA's approach is based on a "data flywheel" concept, where a model is just the beginning.
- The NeMo software stack, running on AWS, allows taking advantage of enterprise data and incorporating customer feedback to improve the generative AI interactions.
- NVIDIA AI Enterprise is a software suite that provides enterprise support for deploying large language models, producing a data flywheel, and keeping it up to date.
FSI Use Cases
- Common FSI use cases include know-your-customer, fraud, pricing, algorithmic trading, and customer service.
Introduction of Prem
- Prem is the Executive Vice President, Chief Scientist, and Head of AI at Capital One, leading the technology strategy, architecture, research, and development for their AI initiatives and data technologies.
Prem's Perspective
Successful AI Development and Deployment
- Businesses exist to serve customers, and the better ones use data to derive insights and address unmet customer needs.
- AI can be used not just to understand customer friction points, but also to design and deliver improved capabilities.
Prerequisite for Effective AI
- Great data infrastructure: Data advantage is AI advantage.
- Ready technology stack: Ability to translate data into technology and deploy it.
- Talent and organizational culture: Readiness to understand, evaluate, and manage AI risks.
- Leverage: Prioritizing use cases based on risk and scale of usage.
Capital One's Journey with AWS
- Capital One went all-in on the public cloud, migrating from 15-16 data centers to zero by the end of 2019.
- This provided benefits like data proximity to compute resources, a modern development stack, and the ability to attract AI/ML talent.
Partnership with NVIDIA
- The partnership with NVIDIA is not just about leveraging their hardware, but also their software stacks like NeMo and NeMo Guardrails, which have been great enablers for Capital One.
Successful Use Cases
- Software development was an easy and horizontal use case, where AI could be deployed with built-in risk mitigation.
- Customer support was another area where generative AI helped improve the accuracy of responses and reduce the cognitive burden on agents, leading to better customer experiences.
Challenging Use Cases
- Credit is a more challenging use case, requiring a lot of strategic patience and the development of the necessary supporting tooling and scaffolding.
Guardrails and Governance
- Guardrails like NeMo Guardrails are crucial for managing risks, ensuring appropriate input and output, and maintaining compliance.
- Capital One has a well-established governance framework, with first, second, and third lines of defense, to ensure responsible development and deployment of AI.
Looking Ahead
- Excited about agentic workflows, where AI agents can execute tasks in partnership with humans or autonomously.
- Interested in the progress in inference optimization, where costs have decreased significantly in the last 20 months.
- Interested in the advancements in game-theoretic or reasoning-style capabilities of models.
