AWS re:Invent 2025 - Bronto: Logs at Scale Without Compromise (DVT224)

Summary of AWS re:Invent 2025 - Bronto: Logs at Scale Without Compromise (DVT224)

Overview

• Presentation by Mike Neville O'Neal, Head of Product at Bronto, and AO Mahoney, Engineering Manager at teamwork.com
• Discussed the challenges of managing logs at scale and how Bronto, a logs-first observability platform, addresses these issues.

The 3C Flywheel of Compromises

• Legacy logging solutions become prohibitively expensive at scale, leading to:
  1. Cutting data retention from 30 days to 7 days, then 3 days
  2. Cutting data sources like CDN data and debug logs to reduce costs
  3. Proliferation of tools and increased operational overhead as teams try to work around the coverage gaps

Bronto: Logs at Scale Without Compromise

• Bronto is a logs-first observability platform that aims to break the cycle of compromises:
  • Retains all data for 12 months by default, with costs starting at cents per gigabyte
  • Integrates with any telemetry agent, including proprietary and open-source options
  • Provides AI-powered features for log management, including:
    • Natural language dashboard creation
    • Automated log parsing
    • AI-driven investigations to reduce mean time to resolution (MTTR)
    • Automatic naming of alerts, parsers, and dashboards

Technical Architecture

• Bronto is built entirely on AWS, with a decoupled storage and compute architecture:
  • Data ingested via HTTP and streamed to Kafka (MSK), then written to S3 in a proprietary format
  • Proprietary S3 format leverages techniques like aggressive compression, columnar storage, and bloom filters to enable fast queries
  • Lambda functions used to parallelize queries and deliver sub-second search performance, even on multi-terabyte datasets

Benefits for Teamwork.com

• Teamwork.com, a project management and resource management platform, faced logging challenges with a complex infrastructure and limited coverage:
  • 33TB of logs per month from various systems, requiring significant operational overhead
  • Limited retention in CloudWatch (2 weeks) and noisy, first-in-first-out logs in Elasticsearch
• By adopting Bronto, Teamwork.com was able to:
  • Simplify their logging infrastructure, replacing multiple legacy tools
  • Achieve 90-day standard retention and 12-month retention for customer traffic logs
  • Save 42% on software costs compared to previous solutions
  • Empower non-technical teams (e.g., support, product) to directly access and analyze logs

Key Takeaways

• Bronto's cost-effective, scalable, and feature-rich observability platform addresses the common challenges of managing logs at scale:
  • Retains all data for 12 months by default, with transparent usage and billing
  • Provides AI-powered features to reduce toil and MTTR
  • Delivers sub-second search performance on multi-terabyte datasets through its efficient architecture
• Bronto's approach has enabled customers like Teamwork.com to simplify their logging infrastructure, reduce costs, and empower cross-functional teams to access and analyze log data.