OpenSearch: A journey from fork to Linux Foundation (OPN201)

Summary of the OpenSearch Presentation

Background and Introduction

  • The presenters are Carl Meadows, Director of Product for OpenSearch at AWS, and Pallavi, Senior Manager leading engineering efforts around OpenSearch.
  • In 2021, AWS forked Elasticsearch 7.10 and created the open-source OpenSearch project after Elastic changed the license on Elasticsearch.
  • OpenSearch consists of three primary software projects:
    1. OpenSearch Core - the distributed search and analytics engine with vector database capabilities.
    2. OpenSearch Dashboards - the front-end component for log discovery, visualization, and anomaly detection.
    3. Data Prepper - a lightweight data collector for ingesting, transforming, and routing data to OpenSearch.

OpenSearch Use Cases

  • OpenSearch is used for a variety of use cases, including:
    1. Search engines - leveraging its rich search functionalities like fuzzy search, faceting, autocomplete, and hybrid/vector search.
    2. AI/ML applications - taking advantage of its vector search capabilities.
    3. Recommendation systems, fraud detection, and other AI/ML-driven search problems.
    4. Analytics and observability - performing aggregations, forensic analysis, and security analytics on large datasets.

OpenSearch Project Growth and Transition

  • The OpenSearch project has grown significantly since the fork, now with over 120 repositories, 75 partners, 21 launches, and over 700 million downloads.
  • To continue the project's growth and innovation, OpenSearch transitioned to the Linux Foundation in 2024, establishing a Technical Steering Committee and an OpenSearch Software Foundation for financial support.

Innovation and Roadmap

  • Pallavi discussed the key innovation themes for OpenSearch's future roadmap:
    1. Vector and Generative AI
    2. Core Search Improvements
    3. Observability and Security Analytics
    4. Cost, Performance, and Scalability
    5. Stability, Availability, and Resiliency
    6. Security
    7. Modular Architecture
    8. Releases and Project Health

Getting Involved

  • Carl outlined ways for the community to get involved in OpenSearch:

    1. Use the product and provide feedback
    2. Engage with the community (e.g., provide feedback on roadmaps, write technical blogs, contribute code)
    3. Attend OpenSearchCon events
    4. (Optional) Join the OpenSearch Software Foundation
  • The presentation provided various resources for the community, including the public roadmap, triage meetings, and the OpenSearchCon event.

Your Digital Journey deserves a great story.

Build one with us.

Cookies Icon

These cookies are used to collect information about how you interact with this website and allow us to remember you. We use this information to improve and customize your browsing experience, as well as for analytics.

If you decline, your information won’t be tracked when you visit this website. A single cookie will be used in your browser to remember your preference.

Talk to us