Innovating with AWS and Generative AI

Criteria Corp's Journey

Introduction

• Valerie Ganfo, the leader of AWS's commercial west region, interviews Chris Dayen, the Chief Technology Officer (CTO) of Criteria Corp, a talent company that helps organizations make efficient and objective hiring decisions.
• Chris discusses Criteria Corp's journey with generative AI, the challenges in the hiring process, and how they have leveraged AWS services to implement generative AI solutions.

Criteria Corp's Background

• Criteria Corp is a talent company that uses data and science through psychology to deliver assessments to millions of candidates around the world. They have over 4,000 global customers and their goal is to help organizations make the best hiring and talent development decisions.
• Criteria Corp has been an early adopter of AI and ML, and they emphasize the importance of data strategy, data integrity, and data organization as the foundation for successful generative AI implementation.

Challenges in the Hiring Process

• The hiring process can be complex, with multiple stakeholders and stages, evaluation inconsistency, and the need to provide a good candidate experience.
• Criteria Corp focuses on addressing these challenges through structured interviewing, data-driven decision-making, and enhancing the candidate experience.

Criteria Corp's Generative AI Journey

• In Q4 2022, Criteria Corp underwent a significant overhaul of their data infrastructure, leveraging AWS services such as S3 data lake, Glue jobs, and Redshift Serverless.
• They adopted a decentralized approach to generative AI implementation, empowering their senior software engineers to become the "generative AI engineers" for the organization.
• Criteria Corp invested in training their engineers, utilizing AWS resources like AWS Skill Builder and internal hackathons, to upskill their team and adopt generative AI.

Criteria Corp's Generative AI Solutions

• Criteria Corp's talent success platform includes assessments, video interviewing, and post-hire development products.
• They have implemented generative AI capabilities in their video interviewing product, such as interview transcription, automated question generation, candidate feedback, and talent signal generation.
• These capabilities have resulted in significant productivity gains, with a 50% reduction in time-to-hire for their customers.

Considerations and Lessons Learned

• Criteria Corp chose to work with AWS, specifically Amazon Bedrock, due to the enterprise-grade data controls, permissions, and flexibility to keep all generative AI model calls within their own AWS account.
• They also leveraged the support of a partner, Kainos, to build their AI core and benefit from their expertise in areas like model diversity.
• Criteria Corp has established a set of AI guidelines to ensure responsible and ethical use of generative AI, including not using personally identifiable information to make talent decisions directly.

Transformational Impact and Future Plans

• Criteria Corp's work with generative AI has unlocked new capabilities and transformed their product roadmap, enabling them to deliver solutions that were not possible a few years ago.
• They are excited about the continued wave of innovation and the transformational impact generative AI will have on their customers' hiring and talent development processes.