AWS re:Invent 2025 - Deep Dive into the AWS Nitro System (CMP316)

Summary of AWS re:Invent 2025 - Deep Dive into the AWS Nitro System (CMP316)

Introduction to the Nitro System

• AWS has invested in building custom silicon across various areas, including data center I/O, core compute, and machine learning infrastructure.
• The Nitro system is a fundamental rethinking of how virtualization in the cloud should be done, moving functionality away from a traditional hypervisor to purpose-built Nitro chips.
• Key reasons for building custom hardware include:
  • Specialization to optimize for AWS use cases
  • Faster time-to-market by owning the end-to-end development process
  • Ability to innovate across traditional silos
  • Enhanced security through a harder root of trust and limited operator access

Networking Offload with Nitro Cards

• Nitro cards provide the VPC data plane offload, handling tasks like ENI attachments, security group enforcement, routing, and packet encapsulation.
• Nitro cards also enable transparent 256-bit AES encryption of network packets in transit without performance overhead.
• The Elastic Network Adapter (ENA) has evolved from 10Gbps to over 600Gbps, allowing instances to scale networking bandwidth as needed.
• The Elastic Fabric Adapter (EFA) uses a Scalable Reliable Datagram (SRD) protocol to leverage multiple network paths simultaneously, improving bandwidth and reducing tail latencies for HPC and ML workloads.
• ENA Express further optimizes TCP and UDP traffic by sending it across multiple paths and reassembling packets in the right order on the Nitro card.

Storage Offload and Nitro SSDs

• Nitro cards expose an NVMe interface to translate commands to the EBS data plane, enabling transparent encryption of EBS volumes.
• EBS performance has scaled from 2GB/s a decade ago to 150Gbps and 720,000 IOPS today, enabling high-performance database workloads.
• For local storage, Nitro SSDs integrate the Flash Translation Layer (FTL) into the Nitro cards, providing up to 60% lower latencies, improved reliability, and encryption with ephemeral keys.

The Nitro Hypervisor

• The Nitro hypervisor is stripped down to the bare minimum, focusing only on CPU, memory, and device assignment, with no network stack, storage stack, or additional services.
• This results in a small, lightweight, and secure hypervisor with minimal performance overhead, allowing virtual machines to achieve close to bare-metal performance.
• The Nitro hypervisor also implements "secret hiding", where the hypervisor's address space is minimized, and it only maps the necessary memory when performing services on behalf of a virtual machine, reducing the attack surface.
• This design has been shown to mitigate security vulnerabilities like L1TF Reloaded that can affect traditional hypervisors.

Security Features of the Nitro System

• The Nitro security chip establishes a hardware root of trust, continuously monitoring and validating the firmware and software running on the system.
• Confidential computing is achieved by removing operator access to the Nitro system and Nitro hypervisor, encrypting all communication channels, and implementing secure boot and measured boot.
• Nitro Enclaves provide a secure, isolated environment for running highly trusted code, integrated with AWS KMS for unlocking sensitive data.
• EC2 Instance Attestation builds on top of UEFI Secure Boot and the Nitro TPM to provide a way for customers to cryptographically verify the entire software stack running on an EC2 instance, enabling use cases like confidential inferencing.

Graviton Servers and the Boot Process

• Graviton 4 servers support multiple sockets, with the coherency link between CPUs also encrypted.
• The boot process is designed with an unbroken chain of custody and verification, leveraging the Nitro chip's private/public key pair to validate each stage of the boot.
• This extends to the CPUs themselves, where only validated Graviton 4 CPUs can communicate with each other, and a secure link is established between the CPUs and the Nitro system.
• The Nitro controller and Nitro security chip play a key role in setting up the system and releasing the CPUs from reset, enabling the boot process to proceed without the hypervisor being involved.
• This secure boot process also enables AWS to offer Mac instances, connecting Mac Mini hardware to the Nitro system for customers building Mac-based applications.

Key Takeaways

• The Nitro system represents a fundamental shift in how AWS approaches virtualization and server infrastructure, moving functionality off the host CPU and into purpose-built hardware.
• This has enabled significant improvements in networking performance, storage performance, security, and the ability to rapidly iterate on new instance types.
• The Nitro hypervisor's "secret hiding" design and the broader security features of the Nitro system, like secure boot and confidential computing, provide strong protections against emerging security threats.
• The tight integration between the Nitro hardware, firmware, and software components allows AWS to maintain an unbroken chain of custody and validation, critical for running mission-critical workloads in the cloud.
• The Nitro system's flexibility and composability have enabled AWS to rapidly expand its instance type offerings, from 70 types before 2017 to over 1,000 today.