AWS re:Invent 2025-The state of AI in software development:Insights across 400+ organizations-AIM126

The State of AI in Software Development: Insights Across 400+ Organizations

Current Impact of AI on Developer Productivity

• Conflicting reports on the impact of AI on developer productivity
  • Google claims 10% productivity gains, but study by Meter showed 19% productivity decrease
  • DORA metrics show modest but positive impacts (7.5% doc quality, 3.4% code quality, 3.1% code review speed)
• Analysis of 20,000 developers showed:
  • 2.6% increase in "change confidence" (ability to ship without breaking things)
  • 2.2% increase in code maintainability
  • 1% reduction in change failure rate
• However, significant volatility across companies - some seeing 20%+ gains or losses

Adoption Trends

• 90% overall adoption of AI across 135,000 developers sampled
• Junior engineers using AI the most, while staff engineers save the most time
• Productivity initially dips when moving from no adoption to light adoption, then improves with moderate/heavy use
• Traditional enterprises have higher daily AI usage, likely due to better change management and AI policies
• Smaller companies adopt AI faster, but may lack governance

Quantified Benefits

• AI users saving an average of 3.8 hours per week on code completion
• 22% of code across the sample is now authored by AI
• Daily AI users shipping 60% more PRs, but quality/content of PRs is a concern

Measuring AI ROI and Productivity

• Challenges in measuring productivity and ROI with AI
• Framework using 3 metric types:
  1. Telemetry metrics (utilization, usage stats)
  2. Impact metrics (productivity, quality, speed)
  3. Self-reported/experience sampling
• Importance of correlating utilization to core productivity metrics like DORA
• Examples of metrics used by companies like Microsoft, Dropbox, Booking.com

Key Takeaways

• AI is seeing rapid, widespread adoption, but impact is volatile across organizations
• Junior engineers leading in AI usage, while staff engineers see biggest productivity gains
• Measuring AI ROI requires a balanced approach across utilization, impact, and qualitative metrics
• AI is an accelerant - amplifying both good and bad practices, so governance is critical
• AI complements but does not replace the need to address broader developer experience challenges

Real-World Examples and Use Cases

• Zapier reduced onboarding time from 30 days to 2 weeks using AI agents
• AI enabling "non-builders" like PMs and designers to contribute more code
• AI assisting with tasks like stack trace analysis, refactoring, and test case generation