The London Stock Exchange Group (LSEG) is a leading global financial market infrastructure company delivering data to 44,000 clients in 170 countries.
LSEG's real-time business processes 274 billion messages per day, ingesting data from 575 venues and exchanges globally.
LSEG has been storing every trade tick since 1996, resulting in a 75 petabyte historical data set.
The unprecedented scale and growth of modern financial markets has created significant challenges for LSEG's legacy on-premises infrastructure.
Business Drivers for Transformation
Legacy on-premises infrastructure unable to keep up with exponential data growth and market volatility
Customer demand for real-time insights and faster time-to-market for new products
Cost pressures in competitive markets
Need for increased innovation velocity to stay ahead of the market
LSEG's Transformation Journey
Historical Data Migration:
Migrated 75PB of tick history data from CSV files to more efficient Parquet format in Amazon S3
Enabled direct S3 access for clients to leverage the historical data
Real-Time Platform Development:
Developed a real-time optimized product leveraging AWS services for low-latency, high-throughput delivery
Deployed managed services like RTMDS in public cloud to scale infrastructure on-demand
Matrix Signals Platform:
Built a serverless, cloud-native solution to detect market hotspots and signals at scale
Leveraged AWS services like Amazon Athena, Amazon OpenSearch, and Amazon Bedrock
Technical Architecture
Transforming Trillions of Ticks to Billions of Optimized Data Points:
Used AWS Glue serverless ETL to aggregate tick-by-tick data from S3 into optimized time series formats
Leveraged S3 Access Points for zero-copy data access and cost-effective storage
Flexible Modeling and Insights Generation:
Queried the optimized data using Amazon Athena to build flexible models and detect market patterns
Automatically indexed Athena query results into Amazon OpenSearch for API-based access and integration
Leveraging Generative AI for Deeper Insights:
Integrated Amazon Bedrock to correlate market hotspots with relevant news and events using generative AI
Enabled faster, more contextual decision-making for operational teams
Key Outcomes and Benefits
5x reduction in infrastructure costs for signal detection and hotspot analysis
97% reduction in storage costs by leveraging S3 for historical data
82% reduction in infrastructure costs through serverless, cloud-native architecture
30-45 minute time-to-signal for market hotspots, down from 3 days previously
Enabled new business capabilities like real-time optimized data products and managed services
Improved customer experience through faster onboarding and on-demand capacity scaling
Accelerated innovation cycles by leveraging the latest AWS services and technologies
Conclusion
LSEG's transformation journey demonstrates the power of a cloud-native, serverless architecture to tackle the challenges of massive scale and growth in the capital markets industry. By leveraging AWS services like Amazon S3, Amazon Athena, Amazon OpenSearch, and Amazon Bedrock, LSEG was able to build a highly scalable, cost-effective, and innovative market intelligence platform that delivers real-time insights and enables faster decision-making. This transformation has positioned LSEG for continued leadership in the market data aggregation and distribution business.
These cookies are used to collect information about how you interact with this website and allow us to remember you. We use this information to improve and customize your browsing experience, as well as for analytics.
If you decline, your information won’t be tracked when you visit this website. A single cookie will be used in your browser to remember your preference.
