Maximize cost efficiency with a shift-left approach to FinOps (COP358)

Here is a detailed summary of the video transcription in markdown format, with the key takeaways broken down into sections for better readability:

Overview of Finops and Optimization

• The presenters, Andrew, Rob, and Adam, discuss the importance of optimizing cloud deployments to reduce waste and inefficiency.
• Typical statistics show that up to 30% of cloud spend is wasted, amounting to a huge sum of overspend.
• The goal is to help the audience understand what it means to "shift left" the optimization process, the benefits it can bring, and ideas to make optimization more proactive in their organizations.

The Finops Lifecycle and Optimization

• Optimization is a crucial part of every Finops practice, as it is built into the different phases of the Finops framework.
• Optimization is about understanding the cost, tying it to the value the organization is getting, and ensuring that the organization is only paying for what it needs.
• The top challenges faced by the Finops community include getting engineers to optimize and reduce waste, indicating a visibility problem.

Shifting Left: Origins and Goals

• The concept of "shifting left" was adopted by Finops from the DevSecOps community, where the goal is to push security and defect elimination earlier in the development process.
• The value of shifting left is that it can dramatically reduce the impact of issues by eliminating them earlier in the software development lifecycle.
• There is a similar pattern that may apply to resolving cloud waste, where waste that has made its way to production could be 30 times more costly to resolve.

An Optimized State

The presenters discuss the characteristics of an optimized state, which include:

1. Well-sized VMs, databases, disks, and containers
2. No idle EC2, RDS, EBS, and S3 resources
3. Well-placed containers with balanced workloads
4. Effective use of Dev/Test environments
5. Dynamic scaling of infrastructure and platforms
6. Adoption of cloud-native and modern applications

Shifting Left: The Four Stages

Stage 1: Ownership

• Ensure accurate tagging of resources to align with the business structure and cost centers.
• Use third-party tools to assist where tagging is poor.
• Identify and quantify waste and optimization opportunities at the application or product level.

Stage 2: Institutionalizing Insights

• Implement policies and processes to ensure that actions are taken to address identified waste, such as through Jira tickets.
• Leverage commitment from leadership and create a culture of accountability to drive change.

Stage 3: Shifting Left in the CICD Pipeline

• Integrate optimization insights and guardrails into the CICD pipeline, leveraging infrastructure-as-code.
• Implement solutions that can compare new deployments to past patterns and identify anomalies before production.

Stage 4: Shifting Left to Product Inception

• Involve Finops teams earlier in the design and decision-making process for new products and applications.
• Make decisions about the cost profile and unit economics of new initiatives upfront, rather than reactively addressing issues later.

Considerations When Shifting Left

Organizational Silos

• Finops teams need to collaborate with engineering, product, and finance teams to make informed decisions about building and operating applications.
• Involving Finops teams earlier in the process can have a bigger impact on the cost profile of applications.

Commitment Strategy

• Leverage compute savings plans to have more flexibility in instance family selection without risking unused commitments.
• Consider layering smaller, more frequent commitments to reduce the risk of overcommitment.

Sustainability

• Shift left the consideration of sustainability metrics, such as power consumption and carbon emissions, to enable engineers to make more informed decisions.
• Tools like Cloudability can provide granular reporting on the carbon footprint of cloud resources.

Unit Economics

• Define and track business-relevant unit economics metrics to measure efficiency and optimization beyond just cost reduction.
• Use unit economics as a goal to drive decision-making earlier in the product lifecycle.

Resources

• The Finops book by Jared Sterman and Mike Fuller
• The Finops Foundation website (finops.org)
• The Cloudability podcast "Finops Friday"
• The AWS Well-Architected Framework's Cost Optimization pillar