Leveraging Amazon Security Lake and Cribl to Enhance Security Visibility

Key Takeaways:

• Seamen Foundational Services, a team within Seamen, faced challenges in effectively collecting and centralizing critical security logs from their 800+ AWS accounts.
• They leveraged Amazon Security Lake to simplify their data collection, but encountered issues with accessing and transforming the data for their SIEM.
• Cribl provided a solution to easily extract data from Security Lake, reduce and transform the data, and ingest it into their Splunk SIEM.
• The partnership between Seamen and Cribl resulted in significant data reduction (from 5TB to 500GB daily) and improved security visibility, enabling their security analysts to conduct better investigations and threat hunting.

Challenges Faced by Seamen Foundational Services

1. Difficulty in effectively collecting and centralizing security logs from their 800+ AWS accounts.
2. Dealing with different data sources, configurations, and the need to implement infrastructure in each account.
3. Limitations in SIEM licensing, which forced them to selectively collect data and resulted in limited visibility.

Transition to Amazon Security Lake

1. Amazon Security Lake was seen as a promising solution to simplify data collection and centralization.
2. They became early adopters of the service and were able to start collecting various security-relevant logs with just a few clicks.
3. However, they faced challenges in accessing and utilizing the data stored in Security Lake, including:
   • Uncertainty on how to effectively query and leverage the data.
   • Difficulties in ingesting the data into their Splunk SIEM due to the increased data volume and the OCSF data format.

Leveraging Cribl to Enhance Data Utilization

1. Cribl was identified as a solution to address the challenges in extracting, transforming, and ingesting data from Amazon Security Lake.
2. Cribl allowed Seamen to:
   • Easily retrieve data from Security Lake.
   • Reduce and transform the data, including renaming fields to align with their SIEM's requirements.
   • Aggregate the data and send it to their Splunk environment.
3. This enabled Seamen to achieve a significant data reduction (from 5TB to 500GB daily) and gain comprehensive visibility into their environment.

Lessons Learned and Best Practices

1. Manage IAM roles and permissions for accessing Security Lake data across multiple subscriptions.
2. Carefully plan worker group configurations (CPU, memory) based on the expected data throughput.
3. Start small and gradually scale up data sources to avoid resource contention issues.
4. Leverage Amazon Graviton processors for improved performance and efficiency.
5. Utilize cloud formation templates and auto-scaling groups for easier deployment and scaling.
6. Explore the ability to separate data streams by worker groups for better control and isolation.
7. Consider using Cribl to augment your SIEM with Security Lake data, allowing you to selectively ingest high-priority events.

Conclusion

The partnership between Seamen and Cribl enabled them to overcome the challenges of effectively collecting and utilizing security data from their AWS environment. By leveraging Amazon Security Lake and Cribl's data processing capabilities, they achieved significant data reduction, improved security visibility, and empowered their security analysts to conduct more effective investigations and threat hunting.