AWS re:Invent 2025 - Developer Experience Economics: Moving Past Productivity Metrics (DVT207)

Improving Developer Productivity: Moving Beyond Metrics to Developer Experience Economics

Challenges with Traditional Productivity Metrics

• Lines of code, time-based metrics, and individual telemetry data fail to capture the full picture of developer productivity
• These metrics can incentivize undesirable behaviors like verbose solutions, corner-cutting, and missing quality aspects

Amazon's Approach: Focusing on Developer Experience

• Developer productivity is an outcome, while developer experience is the key input to focus on
• Integrating AI into the developer experience is crucial to drive improvements in productivity

The Cost to Serve Software Framework

• Developed by Amazon's Software Builder Experience (ASBX) team to quantify the impact of developer experience improvements
• Inspired by the "cost to serve" metric used in Amazon's retail supply chain operations
• Captures the full economic benefits of improving the software development lifecycle

Key Components of the Cost to Serve Software Framework

1. Unit of Delivery: Tailored to the team's software delivery model (e.g. microservice deployments, pull requests, commits)
2. Cost: Includes all the costs associated with delivering software, not just coding time
3. Tension Metrics: Real-time indicators of quality and velocity, such as high-severity tickets per deployment

Applying the Framework

• Make the framework specific to your team's software delivery model and existing metrics
• Focus teams on the controllable inputs that drive the cost to serve metric, not just the output
• Leverage mechanisms to systematically address recurring challenges and enable innovative thinking

Impact at Amazon

• 18.3% increase in weekly production deployments per builder
• 30% reduction in manual interventions
• 32.5% decrease in high-incident related tickets per deployment
• 15.9% total benefit to Amazon from investments in developer experience

The Shift to AI-Native Software Development

• AI is enabling a fundamental shift in the software development lifecycle (SDLC)
• Faster prototyping, AI-generated plans, and automated maintenance free up developers to focus on innovation
• Requires robust release processes, security mechanisms, and integration of AI into existing tools and systems

Lessons Learned

• AI does not automatically make the entire business go faster - intentional approach is key
• Measuring lines of code does not capture AI-driven productivity gains
• Security and safety mechanisms are critical to protect work done by AI agents
• Qualitative metrics and user feedback are essential to complement quantitative data

Conclusion

• Shifting the focus from developer productivity to developer experience economics can unlock significant business value
• The cost to serve software framework provides a structured way to quantify these improvements
• Integrating AI into the developer experience is a key enabler, but requires a holistic, intentional approach