What is OpenSearch?<\/h1>\n\n\n\n
In short, it’s the same but different. OpenSearch is also based on the Lucene search software, is designed to be a distributed search and analytics application, and ingests\/stores\/indexes data. If it’s essentially the same thing, why<\/em> does OpenSearch exist? ElasticSearch was initially licensed under the open-source Apache 2.0 license \u2013 a rather permissive free software license. ElasticCo did not agree with how their software was being used by Amazon<\/a>; and, in 2021, the license for ElasticSearch was changed to Server Side Public License<\/a> (SSPL). One of the requirements of SSPL is that anyone who implements the software and sells their implementation as a service needs to publish their source code under the SSPL license \u2013 not just changes made to the original program but all<\/em> other software a user would require to run the software-as-a-service environment for themselves. Amazon used ElasticSearch for their Amazon Elasticsearch Service offering, but was unable\/unwilling to continue doing so under the new license terms. In April of 2021, Amazon Web Services created a fork of ElasticSearch as the basis for OpenSearch<\/a>.<\/p>\n\n\n\n After the OpenSearch fork was created, the product roadmap for ElasticSearch was driven by ElasticCo and the roadmap for OpenSearch<\/a> was community driven (with significant oversight and input from Amazon) \u2013 this means the products are not identical although they provide the same core functionality. Elastic publishes a list of features unique to ElasticSearch<\/a>, and the underlying machine learning algorithms are different. However, the important components of the “unique” feature list have been implemented in OpenSearch over time.<\/p>\n\n\n\n The biggest differences are price and support. OpenSearch is free software \u2013 there is no purchasing a license to unlock features. It does<\/em> appear that Amazon has an internal iteration of OpenSearch as their as-a-service offering provides features not available in the open-source OpenSearch code base, but that is only available for cloud customers. ElasticCo offers ElasticSearch as free software with a limited feature set. One critical limitation is user authentication mechanisms \u2013 we are unable to implement PingID as an authentication source with the free feature set. Advanced features not currently used today \u2013 machine learning based anomaly detection, as an example – are also unavailable in the free iteration of ElasticSearch. With an ElasticSearch license, we would also get vendor support. OpenSearch does not offer vendor support, although there are third party companies that will provide support services.<\/p>\n\n\n\n Both OpenSearch and ElasticSearch have community-based support forums available \u2013 I have gotten responses from developers on both forums for questions regarding usage nuances.<\/p>\n\n\n\n Most companies have a list differentiating their product from the products offered by competitors \u2013 but the important thing is how the products differ as it relates to how an individual customer uses the product. A car that can have a fresh cup of espresso waiting for you as you leave for work might be amazing to some people, but those who don’t drink coffee won’t be nearly as impressed. So how do the two products compare for Windstream<\/em>?<\/p>\n\n\n\n Data ingestion<\/strong> \u2013 Data is ingested using the same mechanisms \u2013 ElasticCo’s filebeat and logstash are important components of data ingestion, and these components remain unchanged. This means existing processes that feed data into ElasticSearch today would not need to be changed to begin ingesting data into OpenSearch.<\/p>\n\n\n\n Data storage<\/strong> \u2013 Both products distribute searchable data over a cluster of servers. Data storage is “tiered” as hot, warm, and cold which allows less used data to reside on slower, less expensive resources. We have confirmed that ingested data is properly housed on cluster nodes designated for ‘hot’ storage and moved to ‘warm’ and ‘cold’ storage as dictated by defined policies. The item count to size ratio is similar between both products (i.e. storing ten million documents takes about<\/em> the same amount of disk space). OpenSearch provides the ability to alert on transition failures (moving from hot to warm, for instance) which will reduce the amount of manual “health checking” required for the environment.<\/p>\n\n\n\n Search and aggregation<\/strong> \u2013 Both products allow both GUI and API searches of indexed data. Data can be aggregated as it is searched \u2013 returning the max\/min\/average value from a search, a count of records matching search criterion, creating sub-aggregations. ElasticSearch does have aggregations not available in OpenSearch, although these could be handled through custom scripted aggregations and many have corresponding GitHub issues requesting such an aggregation be added to OpenSearch (e.g. weighted average<\/a>, geohash grid<\/a>, or geotile grid<\/a>)<\/p>\n\n\n\n Alerting<\/strong> \u2013 ElastAlert2 can be used to provide the same index monitoring and alerting functionality that ElastAlert currently provides with ElasticSearch. Additionally, OpenSearch includes a built-in alerting capability that might allow us to streamline the functionality into the base OpenSearch implementation. <\/p>\n\n\n\n API Access<\/strong> \u2013 Both ElasticSearch and OpenSearch provide API-based access to data. Queries to the ElasticSearch API endpoint returned expected data when directed to the OpenSearch API endpoint. The ElasticSearch python module can be used to access OpenSearch data, although there is a specific OpenSearch module as well.<\/p>\n\n\n\n UX<\/strong> \u2013 ElasticSearch allows users to search and visualize data through Kibana; OpenSearch provides graphical user access in OpenSearch Dashboard. While the “look and feel” of the GUI differs (Kibana 8 looks different than the Kibana 7 we use today, too), the user functionality remains the same.<\/p>\n\n\n\nDifferences Between OpenSearch and ElasticSearch<\/h1>\n\n\n\n
Salient Feature Comparison<\/h1>\n\n\n\n
<\/tr><\/thead><\/table><\/figure>\n\n\n\n <\/tr><\/thead><\/table><\/figure>\n\n\n\n auto-interval date histogram<\/td> x<\/td> <\/td><\/tr> categorize text<\/td> x<\/td> <\/td><\/tr> children<\/td> x<\/td> <\/td><\/tr> composite<\/td> x<\/td> <\/td><\/tr> frequent items<\/td> x<\/td> <\/td><\/tr> geohex grid<\/td> x<\/td> <\/td><\/tr> geotile grid<\/td> x<\/td> <\/td><\/tr> ip prefix<\/td> x<\/td> <\/td><\/tr> multi terms<\/td> x<\/td> <\/td><\/tr> parent<\/td> x<\/td> <\/td><\/tr> random sampler<\/td> x<\/td> <\/td><\/tr> rare terms<\/td> x<\/td> <\/td><\/tr> terms<\/td> x<\/td> <\/td><\/tr> variable width histogram<\/td> x<\/td> <\/td><\/tr> boxplot<\/td> x<\/td> <\/td><\/tr> geo-centroid<\/td> x<\/td> <\/td><\/tr> geo-line<\/td> x<\/td> <\/td><\/tr> median absolute deviation<\/td> x<\/td> <\/td><\/tr> rate<\/td> x<\/td> <\/td><\/tr> string stats<\/td> x<\/td> <\/td><\/tr> t-test<\/td> x<\/td> <\/td><\/tr> top metrics<\/td> x<\/td> <\/td><\/tr> weighted avg<\/td> x<\/td> <\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
![\"\"](\"https:\/\/www.rushworth.us\/lisa\/wp-content\/uploads\/2022\/10\/ES-77.png\")
<\/td>![\"\"](\"https:\/\/www.rushworth.us\/lisa\/wp-content\/uploads\/2022\/10\/OS-22.png\")
<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n![\"\"](\"https:\/\/www.rushworth.us\/lisa\/wp-content\/uploads\/2022\/10\/ES-Scan.png\")
<\/td>![\"\"](\"https:\/\/www.rushworth.us\/lisa\/wp-content\/uploads\/2022\/10\/OS-Visualization.png\")
<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n![\"\"](\"https:\/\/www.rushworth.us\/lisa\/wp-content\/uploads\/2022\/10\/ES-Dashboard.png\")
<\/td>![\"\"](\"https:\/\/www.rushworth.us\/lisa\/wp-content\/uploads\/2022\/10\/OS-Dashboard.png\")
<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n