The journey and challenges of operationalizing generative AI at FINRA (WPS320)
Finra's Generative AI Journey
Introduction
Finra is the Financial Industry Regulatory Authority, working with the SEC to ensure investor protection and market integrity.
Finra is a large data organization with over 600 petabytes of data, which they use for modeling and analytics.
Finra started their cloud journey about 10 years ago and is now a cloud-first organization.
Finra's vision for generative AI is that it is transformative in their way of working with data and expands the possibilities of what's achievable.
Guiding Principles for Generative AI
Be Transformative: Generative AI is not just an incremental improvement, but a transformative change in how Finra works with data.
Mitigate Risks: Risks with generative AI are different from traditional models, including business, legal, and technical risks.
Build Purposefully: Ensure that the use of generative AI is fit for the business purpose and will deliver the desired outcomes.
Include Multiple Stakeholders: Generative AI is not just a technology change, but a transformative journey that requires input from various stakeholders.
Challenges in Getting Started
Groundswell of Interest: Everyone in the organization wants to use generative AI, but the risks are different and need to be addressed.
Legal and Technical Risks: Navigating the rapidly changing landscape of legal and technical requirements for using generative AI models.
Operational Proxy and Guard Rails: Implementing controls to moderate the use of generative AI and ensure ethical and acceptable use.
Observability and Value Measurement: Establishing agreement on the value metrics and context for generative AI usage.
Training and Upskilling: Investing in prompt engineering training for both technology and business users.
Revising Model Governance: Updating model governance processes and tooling to handle the use cases and third-party models associated with generative AI.
Initial Pilots and Rollout
Phillip: Finra's large language interactive portal, built on Amazon Bedrock, with additional controls and proxies.
Finis Developer Assistant: Built on Amazon Q, integrated into Finra's development environment.
Lessons Learned from the Amazon Q Pilot
Goals: Integrate large language models into the software development lifecycle, overcome common development challenges, and foster a culture of continuous improvement and innovation.
Evaluation Framework: Comprehensive evaluation across functionality, supporting technologies, security and compliance, enterprise integration, quality, and maintenance.
Challenges: Prompt engineering, lack of best practices, and security monitoring.
Measurable Impact: 30% increase in code quality and integrity, 40% reduction in code transformation, and 20% reduction in cognitive load.
Phased Rollout: Enabling the service for 15% of the organization initially, with a focus on prerequisites, training, and community building.
Next Steps
Observability and Agile Adoption: Analyze the metrics to streamline and accelerate the adoption of generative AI.
Layered Guard Rails: Develop enterprise-wide and use case-specific guard rails to ensure appropriate and ethical use of generative AI.
Maturing to Agentive Technologies: Explore the use of generative AI to not only assist but also automate business processes.
These cookies are used to collect information about how you interact with this website and allow us to remember you. We use this information to improve and customize your browsing experience, as well as for analytics.
If you decline, your information won’t be tracked when you visit this website. A single cookie will be used in your browser to remember your preference.