AWS re:Invent 2025 - Advanced analytics with AWS Cost and Usage Reports (COP401)

Advanced Analytics with AWS Cost and Usage Reports

Optimizing Cloud Spend with the Cost and Usage Report

• The AWS Cost and Usage Report (CUR) is a powerful data source for understanding cloud spend and identifying optimization opportunities.
• Challenges customers still face include:
  • Identifying specific resources driving cost increases (e.g. NAT gateways, data transfer)
  • Allocating costs to the right teams/departments
  • Controlling access and visibility to cost data
• The session demonstrated how to leverage the CUR and integrate additional data sources to address these challenges.

Investigating Cost Spikes with CUR and VPC Flow Logs

• Scenario: An anomaly was detected in NAT gateway and data transfer costs.
• Using the CUR, the presenters created a "quasi pivot table" to identify the specific resources driving the cost increase.
• However, the CUR alone did not provide enough context to determine the root cause.
• By integrating VPC Flow Logs data, the presenters were able to:
  • Identify the specific IP addresses and resources involved in the data transfer
  • Determine that the traffic was between public subnets, which should not have been routed through the NAT gateway
  • Estimate the cost impact of the inefficient configuration
• This example highlighted the importance of combining multiple data sources to fully understand cost drivers and identify optimization opportunities.

Allocating Costs for AI/ML Workloads

• Allocating costs for AI/ML workloads, specifically AWS Bedrock, can be challenging due to:
  1. Billing entities: First-party, second-party, and third-party models are billed differently
  2. Resource IDs: The model name is the only information provided, making it difficult to identify the user or department
• The presenters demonstrated how to use Bedrock Inference Profiles to tag resources and enable cost allocation by department.
• They also showed how to identify untagged Bedrock usage and proactively reach out to teams to ensure proper tagging.

Securing Access to Cost Data

• Sharing the CUR with application teams can be challenging, as you may not want them to see all cost data.
• The presenters showcased how to use AWS Lake Formation to:
  1. Register the S3 bucket containing the CUR data
  2. Create data filters to restrict access to specific columns and rows (e.g., only unblended costs, only specific accounts)
  3. Assign the appropriate permissions to the application team's IAM role
• This approach allows you to securely share the CUR data with other teams while maintaining control over what they can access.

Key Takeaways

• Leverage multiple data sources (CUR, VPC Flow Logs, etc.) to gain a comprehensive understanding of cost drivers and optimization opportunities.
• Implement tagging strategies, such as Bedrock Inference Profiles, to enable accurate cost allocation for AI/ML workloads.
• Use AWS Lake Formation to securely share the CUR data with other teams, controlling access to specific columns and rows.
• Continuously explore new ways to extract insights from the CUR and integrate it with other data sources to drive cloud cost optimization.

Technical Details and Examples

• The presenters used SQL queries in Athena to analyze the CUR data, including:
  • Creating a "quasi pivot table" to identify cost increases by resource
  • Integrating VPC Flow Logs data to determine the root cause of the NAT gateway and data transfer cost spike
  • Querying Bedrock usage and costs, and leveraging Inference Profiles to enable cost allocation by department
• They also demonstrated the use of AWS Lake Formation to restrict access to the CUR data, including:
  • Registering the S3 bucket containing the CUR data
  • Creating data filters to limit access to specific columns and rows
  • Assigning permissions to the application team's IAM role

Business Impact and Real-World Applications

• The techniques showcased in this session can help organizations:
  • Optimize cloud spend by identifying and addressing inefficient resource utilization and configuration
  • Accurately allocate costs for AI/ML workloads, enabling better financial planning and decision-making
  • Securely share cost data with cross-functional teams, empowering them to make informed decisions about resource usage and cloud investments
• By combining the CUR with additional data sources and leveraging advanced analytics capabilities, organizations can gain deeper insights into their cloud footprint and make more strategic, data-driven decisions about cloud resource management and optimization.

Examples and Use Cases

• The presenters shared a real-world example of a customer who experienced a spike in NAT gateway and data transfer costs, and how they used the CUR and VPC Flow Logs to identify and address the root cause.
• They also demonstrated how to use Bedrock Inference Profiles to enable cost allocation by department for AI/ML workloads, and how to use AWS Lake Formation to securely share the CUR data with application teams.