AWS re:Invent 2025 - Sustainable computing for climate solutions (AIM417)

Sustainable Computing for Climate Solutions

Addressing the Complexity of Climate Challenges

• Climate change is a complex, multifaceted issue involving the interaction of various components of the climate system (atmosphere, hydrosphere, cryosphere, lithosphere, biosphere).
• Solving climate challenges is further complicated by the complex nature of human society, with differing socioeconomic, political, and cultural factors across communities and regions.
• The urgency of the situation requires evaluating both short-term and long-term trade-offs to ensure sustainable solutions.

The Role of Technology in Climate Solutions

• Technology is a key enabler and accelerator for climate solutions, with three critical capabilities:
  1. Data: Acquiring high-value data from diverse sources (e.g., supply chain data, satellite imagery, ocean noise) to understand the complex climate system.
  2. Storage: Accommodating and efficiently retrieving large, heterogeneous datasets.
  3. Compute: Performing large-scale data processing and simulation to uncover patterns and insights.

The Importance of High-Performance Computing (HPC) for Climate Research

• HPC is a powerful and efficient technology for tackling complex climate challenges, but traditional on-premises HPC setups can be energy-intensive and difficult to manage.
• AWS provides a sustainable approach to HPC through services like Amazon FSx for Lustre, Elastic Fabric Adapter, and various compute options (AWS ParallelCluster, AWS Batch, AWS Parallel Computing Service).

Sustainable Practices for HPC Workloads on AWS

1. Data Management:

   • Leveraging AWS Open Data Registry to access ready-to-use datasets, reducing the need for data transfer and storage.
   • Implementing lifecycle policies in Amazon S3 to optimize data storage and reduce emissions.
   • Using Amazon FSx for Lustre to provide a high-performance file system with intelligent tiering and automated lifecycle management.

2. Compute Optimization:

   • AWS ParallelCluster: Enables HPC workloads with open-source tools like OpenMP and MPI, reducing carbon footprint by up to 57%.
   • AWS Batch: Efficient for loosely coupled or containerized workloads, reducing compute time by up to 60%.
   • AWS Parallel Computing Service: Provides a fully managed solution for researchers, focusing on job execution rather than infrastructure management.

3. Hardware and Services:

   • AWS offers HPC-optimized instances and accelerated computing instances (e.g., Graviton, Inferentia, Trainium) that are up to 60% more power-efficient than standard EC2 instances.

Case Study: University of Oxford's APAD Project

• The APAD (Air Pollution Asset Level Detection) project at the University of Oxford aims to identify and map brick kilns, a major source of air pollution in the Indogenetic region.
• Challenges:
  • Massive scale of satellite imagery data (1.2 million images)
  • Need for high-resolution imagery to accurately detect brick kilns
• Solution:
  • Leveraged open-source satellite data (Sentinel-2) for initial low-resolution analysis.
  • Acquired high-resolution satellite imagery for targeted areas to annotate and identify brick kilns using computer vision models.
  • Deployed the solution on AWS, optimizing data storage and compute resources.
• Results:
  • Processed 1.2 million satellite images, saving over 17,000 compute hours.
  • Achieved up to 80% reduction in infrastructure costs and 90% reduction in monitoring time and task runtime.

Key Takeaways

1. HPC can be implemented sustainably on the cloud, leveraging services like Amazon FSx for Lustre, Elastic Fabric Adapter, and various compute options.
2. Careful data management, including the use of open data sources and optimized storage, can significantly reduce the carbon footprint of HPC workloads.
3. AWS provides a range of hardware and services (e.g., Graviton, Inferentia, Trainium) that are designed for energy efficiency and performance, enabling more sustainable HPC solutions.
4. The APAD project at the University of Oxford demonstrates how AWS can support complex climate research by optimizing data, storage, and compute resources, leading to significant cost and emissions reductions.