New Relic's Journey to the Cloud

Key Takeaways

1. New Relic is an AI-first, all-in-one observability platform with around 85,000 customers.
2. The company's platform processes massive amounts of data, generating around 3 exabytes of data per year.
3. New Relic's migration from on-premises data centers to the cloud was a gradual, iterative process that took around a year.
4. The migration involved adopting a cellular architecture, where the platform is divided into independent, scalable cells.
5. The migration process included various lessons learned, such as the importance of observability, planning for unexpected challenges, continuously building new cells, and investing in planning and communication.

New Relic's Platform Overview

• New Relic's platform is designed to ingest and process vast amounts of data from its customers, including metrics, logs, traces, and more.
• The platform uses a proprietary time-series database (NRDB) to store and query the data, with a focus on low-latency responses.
• The platform is highly distributed, with customer data stored across multiple regions and cells.
• Reliability is critical, with a promise of 99.9% uptime for customers.

The Migration Journey

1. The Need to Modernize: New Relic's on-premises infrastructure was becoming increasingly difficult to manage and scale, prompting the decision to move to the cloud.
2. Choosing AWS: New Relic selected AWS as its cloud provider due to the availability of relevant services, operational excellence, and a strong partnership.
3. Adopting a Cellular Architecture: New Relic implemented a cellular architecture, where the platform is divided into independent, scalable cells to reduce blast radius and improve reliability.
4. The Cumulus Project: The migration project was named "Cumulus" and involved a gradual, iterative approach to moving data types and customers to the cloud.
5. Hybrid Approach to Modernization: New Relic took a hybrid approach, modernizing certain components before lifting and shifting, and in other cases, lifting and shifting first and then modernizing.

Lessons Learned

1. Observability: Ensuring visibility into the migration process and the performance of the platform in the cloud was critical.
2. Planning for the Unexpected: The migration process encountered unexpected challenges, such as the COVID-19 pandemic and changes in New Relic's business model, which required adaptability.
3. Continuously Building New Cells: The cellular architecture allowed New Relic to constantly build and iterate on new cells, improving reliability and compliance.
4. Investing in Planning and Communication: Involving stakeholders across the organization and communicating with customers were key to the successful migration.

Conclusion

New Relic's journey to the cloud was a complex and multi-faceted process, but the company's adoption of a cellular architecture, gradual and iterative approach, and focus on observability and planning enabled a successful migration. The partnership with AWS and the lessons learned along the way have helped New Relic further strengthen its platform and better serve its customers.