Dive deep into the AWS Nitro System (CMP301)

Nitro System: A Deep Dive into AWS's Custom Silicon

Overview

1. AWS has invested in building custom silicon in multiple areas, including:
   • Graviton: Host CPUs that deliver the best price-performance for a wide range of cloud workloads.
   • Inferentia and Trainium: Machine learning accelerators built from the ground up for the best price-performance in inference and training workloads.
2. The key reasons for building custom silicon are:
   • Specialization: Tailoring the designs for AWS's specific use cases and requirements.
   • Speed: Owning the end-to-end development process for faster delivery of technology.
   • Innovation: Ability to innovate across the entire stack (silicon, firmware, hypervisor, etc.).
   • Security: Leveraging hardware roots of trust and a narrow set of authenticated APIs.

The Nitro System

1. Nitro is a fundamental rethinking of virtualization in the cloud, combining custom hardware and software.
2. The Nitro System started in 2013 with the goal of offloading functionality from the traditional hypervisor onto purpose-built chips.
3. The Nitro System consists of:
   • Nitro cards: Providing networking, storage, and security functionality.
   • Nitro hypervisor: A lightweight hypervisor running on the host processor.
   • VMs and applications: Running on top of the Nitro hypervisor.

Networking and Storage Offloading

1. Networking:
   • Offloaded the VPC data plane functionality (e.g., ENI attachment, security groups, routing) to Nitro cards.
   • Introduced the Elastic Network Adapter (ENA) device, which can transparently encrypt network traffic.
   • Developed the Elastic Fabric Adapter (EFA) for high-performance computing and machine learning workloads, leveraging Scalable Reliable Datagram (SRD) technology.
   • ENA Express uses SRD to distribute TCP traffic across multiple network paths, improving bandwidth and reducing tail latency.
2. Storage:
   • Nitro cards expose NVMe interfaces and handle the Flash Translation Layer (FTL) functionality, providing up to 60% lower latencies and improved reliability.
   • Introduced the third generation of Nitro-based SSD instances, offering up to 65% higher real-time storage performance and 50% lower latency variability.

Security and Modularity

1. Security:
   • Nitro provides physical separation between customer code/data and AWS code.
   • The Nitro security chip verifies the firmware and unlocks the server's storage during boot.
   • Supports features like UEFI Secure Boot and Trusted Platform Modules (TPM) for additional security.
2. Modularity:
   • The Nitro System's modular design has enabled AWS to rapidly expand the number of EC2 instance types, from 70 to around 850 in 7 years.
   • This flexibility allows for different storage, networking, and accelerator configurations to meet diverse customer needs.

The Nitro Architecture

1. The Nitro cards are the nexus of the system, controlling the host CPU and managing the lifecycle of instances (both virtual and bare metal).
2. The Nitro controllers communicate with the EC2 control plane to allocate resources and attach devices to instances.
3. The Nitro hypervisor is a lightweight hypervisor focused solely on memory and CPU allocation, with most other functionality offloaded to the Nitro cards.
4. The system provides the same functionality and performance, whether running virtual or bare metal instances, thanks to the Nitro architecture.

Conclusion

The Nitro System represents a fundamental rethinking of virtualization in the cloud, driven by AWS's desire to deliver better performance, security, and innovation to customers. By building custom silicon and tightly integrating the hardware and software components, AWS has been able to offload and optimize key functionalities, resulting in a more efficient and secure cloud infrastructure.