AWS re:Invent 2025 - Accelerating Engineering: Cross-Industry HPC Cloud Transformations (CMP302)

Summary of "Accelerating Engineering: Cross-Industry HPC Cloud Transformations"

Market Trends in HPC and AI

• The HPC and AI market is experiencing significant growth, with a 23.5% increase projected for 2024 according to Hyperion Research.
• This growth is driven by both traditional HPC workloads (8.4% growth) and the rapid expansion of AI infrastructure (166% growth).
• Cloud adoption in the HPC and AI market is also growing faster than on-premises solutions, expected to reach $23.7 billion by 2029 and account for 20% of the overall market.

Reasons for Faster Cloud Adoption

1. Workload Limitations: Some workloads cannot be run on-premises due to capacity constraints.
2. Access to Latest GPUs: The cloud provides immediate access to the latest GPU technologies as they are released.
3. Scaling at Larger Scales: Cloud elasticity allows organizations to scale their infrastructure up or down as needed.
4. Faster Time to Results: Increased cloud infrastructure can shorten queue times and accelerate time to results.
5. Global Collaboration and Sustainability: The cloud enables global collaboration and offers improved sustainability compared to on-premises data centers.

AWS HPC Building Blocks

1. Infrastructure Components: AWS provides HPC-optimized compute instances, accelerators, network fabric, and storage.
2. Consumption Models: AWS offers on-demand, savings plans, spot instances, and capacity reservations to suit various workload needs.
3. Orchestration Services: AWS provides managed services like AWS ParallelCluster, AWS Batch, and AWS NICE DCV to simplify HPC environment management.
4. Quantum Computing: AWS Braket provides access to quantum computing resources, enabling HPC users to experiment with quantum technologies.

Cross-Industry HPC Use Cases

• CAE and EDA: Structural analysis, finite element analysis, IC design, timing, and signal integrity.
• Healthcare and Life Sciences: Genomic sequencing, drug discovery, personalized medicine, and epidemiological modeling.
• Energy and Power: Grid simulation, nuclear reaction design, oil and gas, and renewable energy system modeling.
• Financial Services: Risk analysis, fraud detection, portfolio optimization, and market simulation.
• Automotive: Computer fluid dynamics, crash and safety, advanced driver assistance, and autonomous driving.
• Academia and Research: High-energy physics, astrophysical modeling, computational chemistry, and social science analytics.
• Weather and Climate: Global climate modeling, weather forecasting, hurricane and storm surge, and air quality modeling.

ARM's Cloud Journey

• ARM is a global leader in compute, powering nearly every connected device and enabling innovation across industries.
• ARM's cloud strategy is driven by industry pressures, the explosion of compute demand, strategic goals, and the need for cloud agility.
• ARM has developed cloud-native platforms, such as CloudRunner and Cloud Foundation Platform, to optimize their front-end and back-end engineering workloads on AWS.
• ARM leverages AWS services like Batch, FSx for Luster, and Graviton instances to achieve cost savings, improved performance, and increased productivity.

DTN's Weather Modeling on AWS

• DTN provides decision-grade weather intelligence data to customers in industries like agriculture, aviation, utilities, and energy.
• DTN migrated from on-premises HPC to a fully cloud-based system on AWS, leveraging services like AWS Batch, FSx for Luster, and GPU-enabled EC2 instances.
• DTN runs its global forecast system hourly, generating over 20TB of data daily, enabled by the scalability and efficiency of AWS HPC.
• DTN is also running production AI-based numerical weather prediction (AI-NWP) models on AWS, achieving significant improvements in forecast skill and efficiency.

Key Takeaways

• The HPC and AI market is experiencing rapid growth, driven by both traditional HPC workloads and the expansion of AI.
• Cloud adoption in HPC is outpacing on-premises solutions due to the cloud's ability to address workload limitations, provide access to the latest technologies, enable scalability, and improve collaboration and sustainability.
• AWS offers a comprehensive set of HPC-optimized building blocks, including infrastructure, consumption models, and orchestration services, to support a wide range of cross-industry use cases.
• ARM and DTN have successfully leveraged AWS HPC to transform their engineering and weather modeling workflows, achieving significant improvements in productivity, cost savings, and forecast accuracy.