Unlocking the Agentic Future: Building Trust in AI-Driven Software Development
Adoption of AI in Software Development
- 90% of developers are now using AI, with 65% using it heavily
- However, only 30% of users fully trust the output of AI-generated code
- More senior engineers tend to have less trust in AI-generated code due to inconsistent quality
Rapid Advancements in AI Capabilities
- AI capabilities are doubling every 7 months, as shown by the Meter research group's capability curve
- This rapid progress is enabling new use cases for AI in software development:
- From single-line code completion to CLI tools, in-editor agents, and even autonomous agent teams
Productivity Gains from AI Usage
- Surveys show that the more frequently developers use AI, the more productivity gains they self-report
- However, there are concerns about the long-term impact on code quality and technical debt
A Framework for Trusted AI Adoption
To maintain trust while leveraging AI, the presentation outlines a 4-part framework:
1. Governance
- Establish clear policies on who can use AI, how much, and with what cost controls
2. Intent Capture and Preservation
- Ensure AI is given clear requirements and that the intent is preserved throughout the development lifecycle
3. Selecting the Right Tools
- Use a mix of in-house and third-party AI tools, tailored to specific needs and constraints
4. Rigorous Experimentation and Measurement
- Measure the impact of AI usage against key metrics like "diffs per developer month" or "cost to serve software"
- Continuously learn and optimize the AI usage based on the results
Case Studies: Meta and Amazon
- Meta optimizes for "diffs per developer month", seeing a 6-12% lift from AI usage
- Amazon optimizes for "cost to serve software", reducing it by 16% while including the cost of AI
Jet Brains' Approach
- Providing governance and intent preservation capabilities in their IDE tools
- Offering an open platform with a wide selection of AI tools that can integrate seamlessly
- Enabling collaboration between human developers and AI agents through the "Agent-to-Client Protocol"
Key Takeaways
- The "agentic future" is here, and organizations need to proactively manage AI adoption
- Establishing the right governance, intent capture, tool selection, and measurement processes is crucial
- Experimenting with AI, even within budget constraints, can provide compounding advantages over time
- AI is an investment, so focus on areas where the potential return is highest (e.g., greenfield, low-complexity projects)
