22. ES详解 - 资料：Awesome Elasticsearch
# 22. ES详解 - 资料：Awesome Elasticsearch
# General
# Elastic Stack
- Elasticsearch 在新窗口打开 (opens new window) official website
- Logstash 在新窗口打开 (opens new window) is a data pipeline that helps you process logs and other event data from a variety of systems
- Kibana 在新窗口打开 (opens new window) is a data analysis tool that helps to visualize your data; Kibana Manual docs 在新窗口打开 (opens new window)
- beats 在新窗口打开 (opens new window) is the platform for building lightweight, open source data shippers for many types of data you want to enrich with Logstash, search and analyze in Elasticsearch, and visualize in Kibana.
# Books
- Deep Learning for Search 在新窗口打开 (opens new window) - teaches you how to leverage neural networks, NLP, and deep learning techniques to improve search performance. (2019)
- Relevant Search: with applications for Solr and Elasticsearch 在新窗口打开 (opens new window) - demystifies relevance work. Using Elasticsearch, it teaches you how to return engaging search results to your users, helping you understand and leverage the internals of Lucene-based search engines. (2016)
- Elasticsearch in Action 在新窗口打开 (opens new window) - teaches you how to build scalable search applications using Elasticsearch (2015)
# Related (awesome) lists
- frutik/awesome-search 在新窗口打开 (opens new window) I am building e-commerce search now. Below are listed some of my build blocks
# Open-source and free products, based on Elasticsearch
- Fess 在新窗口打开 (opens new window) is an open source full featured Enterprise Search, with a web-crawler
- Yelp/elastalert 在新窗口打开 (opens new window) is a modular flexible rules based alerting system written in Python
- etsy/411 在新窗口打开 (opens new window) - an Alert Management Web Application https://demo.fouroneone.io (credentials: user/user)
- appbaseio/mirage 在新窗口打开 (opens new window) is a 🔎 GUI for composing Elasticsearch queries
- exceptionless/Exceptionless 在新窗口打开 (opens new window) is an error (exceptions) collecting and reporting server with client bindings for a various programming languages
- searchkit/searchkit 在新窗口打开 (opens new window) is a UI framework based on React to build awesome search experiences with Elasticsearch
- appbaseio/reactivemaps 在新窗口打开 (opens new window) is a React based UI components library for building Airbnb / Foursquare like Maps
- appbaseio/reactivesearch 在新窗口打开 (opens new window) is a library of beautiful React UI components for Elasticsearch
- appbaseio/dejavu 在新窗口打开 (opens new window) The missing UI for Elasticsearch; landing page 在新窗口打开 (opens new window)
- Simple File Server 在新窗口打开 (opens new window) is an Openstack Swift compatible distributed object store that can serve and securely store billions of large and small files using minimal resources.
- logagent 在新窗口打开 (opens new window) a log shipper to parse and ship logs to Elasticsearch including bulk indexing, disk buffers and log format detection.
- ItemsAPI 在新窗口打开 (opens new window) simplified search API for web and mobile (based on Elasticsearch and Express.js)
- Kuzzle 在新窗口打开 (opens new window) - An open-source backend with advanced real-time features for Web, Mobile and IoT that uses ElasticSearch as a database. (Website 在新窗口打开 (opens new window))
- SIAC 在新窗口打开 (opens new window) - SIAC is an enterprise SIEM built on the ELK stack and other open-source components.
- Sentinl 在新窗口打开 (opens new window) - Sentinl is a Kibana alerting and reporting app.
- Praeco 在新窗口打开 (opens new window) - Elasticsearch alerting made simple
# Elasticsearch developer tools and utilities
# Development and debugging
- Sense (from Elastic) 在新窗口打开 (opens new window) A JSON aware developer console to Elasticsearch; official and very powerful
- ES-mode 在新窗口打开 (opens new window) An Emacs major mode for interacting with Elasticsearch (similar to Sense)
- Elasticsearch Cheatsheet 在新窗口打开 (opens new window) Examples for the most used queries, API and settings for all major version of Elasticsearch
- Elasticstat 在新窗口打开 (opens new window) CLI tool displaying monitoring informations like htop
- Elastic for Visual Studio Code 在新窗口打开 (opens new window) An extension for developing Elasticsearch queries like Kibana and Sense extention in Visual Studio Code
- Elastic Builder 在新窗口打开 (opens new window) A Node.js implementation of the Elasticsearch DSL
- Bodybuilder 在新窗口打开 (opens new window) A Node.js elasticsearch query body builder
- enju 在新窗口打开 (opens new window) A Node.js elasticsearch ORM
- Peek 在新窗口打开 (opens new window) An interactive CLI in Python that works like Kibana Console with additional features
# Import and Export
- Knapsack plugin 在新窗口打开 (opens new window) is an "swiss knife" export/import plugin for Elasticsearch
- Elasticsearch-Exporter 在新窗口打开 (opens new window) is a command line script to import/export data from Elasticsearch to various other storage systems
- esbulk 在新窗口打开 (opens new window) Parallel elasticsearch bulk indexing utility for the command line.
- elasticdump 在新窗口打开 (opens new window) - tools for moving and saving indices
- elasticsearch-loader 在新窗口打开 (opens new window) - Tool for loading common file types to elasticsearch including csv, json, and parquet
# Management
- Esctl 在新窗口打开 (opens new window) - High-level command line interface to manage Elasticsearch clusters.
- Vulcanizer 在新窗口打开 (opens new window) - Github's open sourced cluster management library based on Elasticsearch's REST API. Comes with a high level CLI tool
# Elasticsearch plugins
# Cluster
- sscarduzio/elasticsearch-readonlyrest-plugin 在新窗口打开 (opens new window) Safely expose Elasticsearch REST API directly to the public
- mobz/elasticsearch-head 在新窗口打开 (opens new window) is a powerful and essential plugin for managing your cluster, indices and mapping
- Bigdesk 在新窗口打开 (opens new window) - Live charts and statistics for elasticsearch cluster
- Elastic HQ 在新窗口打开 (opens new window) - Elasticsearch cluster management console with live monitoring and beautiful UI
- Cerebro 在新窗口打开 (opens new window) is an open source(MIT License) elasticsearch web admin tool. Supports ES 5.x
- Kopf 在新窗口打开 (opens new window) - Another management plugin that have REST console and manual shard allocation
- Search Guard 在新窗口打开 (opens new window) - Elasticsearch and elastic stack security and alerting for free
- ee-outliers 在新窗口打开 (opens new window) - ee-outliers is a framework to detect outliers in events stored in an Elasticsearch cluster.
- Elasticsearch Comrade 在新窗口打开 (opens new window) - Elasticsearch admin panel built for ops and monitoring
- elasticsearch-admin 在新窗口打开 (opens new window) - Web administration for Elasticsearch
# Other
- SIREn Join Plugin for Elasticsearch 在新窗口打开 (opens new window) This plugin extends Elasticsearch with new search actions and a filter query parser that enables to perform a "Filter Join" between two set of documents (in the same index or in different indexes).
# Integrations and SQL support
- NLPchina/elasticsearch-sql 在新窗口打开 (opens new window) - Query elasticsearch using familiar SQL syntax. You can also use ES functions in SQL.
- elastic/elasticsearch-hadoop 在新窗口打开 (opens new window) - Elasticsearch real-time search and analytics natively integrated with Hadoop (and Hive)
- jprante/elasticsearch-jdbc 在新窗口打开 (opens new window) - JDBC importer for Elasticsearch
- pandasticsearch 在新窗口打开 (opens new window) - An Elasticsearch client exposing DataFrame API
- monstache 在新窗口打开 (opens new window) - Go daemon that syncs MongoDB to Elasticsearch in near realtime
# You know, for search
- jprante/elasticsearch-plugin-bundle 在新窗口打开 (opens new window) A plugin that consists of a compilation of useful Elasticsearch plugins related to indexing and searching documents
# Kibana plugins and applications
- elastic/timelion 在新窗口打开 (opens new window) time-series analyses application. Overview and installation guide: Timelion: The time series composer for Kibana 在新窗口打开 (opens new window)
- Kibana Alert App for Elasticsearch 在新窗口打开 (opens new window) - Kibana plugin with monitoring, alerting and reporting capabilities
- VulnWhisperer 在新窗口打开 (opens new window) - VulnWhisperer is a vulnerability data and report aggregator.
- Wazuh Kibana App 在新窗口打开 (opens new window) - A Kibana app for working with data generated by Wazuh 在新窗口打开 (opens new window).
- Datasweet Formula 在新窗口打开 (opens new window) - A real time calculated metric plugin Datasweet Formula 在新窗口打开 (opens new window).
# Kibana Visualization plugins
- nbs-system/mapster 在新窗口打开 (opens new window) - a visualization which allows to create live event 3d maps in Kibana
- Kibana Tag Cloud Plugin 在新窗口打开 (opens new window) - tag cloud visualization plugin based on d3-cloud plugin
- LogTrail 在新窗口打开 (opens new window) - a plugin for Kibana to view, analyze, search and tail log events from multiple hosts in realtime with devops friendly interface inspired by Papertrail
- Analyze API 在新窗口打开 (opens new window) - Kibana 6 application to manipulate the _analyze API graphically
- kbn_network 在新窗口打开 (opens new window) - This is a plugin developed for Kibana that displays a network node that link two fields that have been previously selected.
# Discussions and social media
- /r/elasticsearch 在新窗口打开 (opens new window)
- Elasticsearch forum 在新窗口打开 (opens new window)
- Stackoverflow 在新窗口打开 (opens new window)
- Books on Amazon 在新窗口打开 (opens new window) does not fit well into this category, but worth checking out!
- TODO: Put some good twitter accounts
# Tutorials
- Centralized Logging with Logstash and Kibana On Ubuntu 14.04 在新窗口打开 (opens new window) everything you need to now when you are creating your first Elasticsearch+Logstash+Kibana instance
- dwyl/learn-elasticsearch 在新窗口打开 (opens new window) a getting started tutorial with a pack of valuable references
- Make Sense of your Logs: From Zero to Hero in less than an Hour! by Britta Weber 在新窗口打开 (opens new window) demonstrates how you can build Elasticsearch + Logstash + Kibana stack to collect and discover your data
- $$ Elasticsearch 7 and Elastic Stack 在新窗口打开 (opens new window) - liveVideo course that teaches you to search, analyze, and visualize big data on a cluster with Elasticsearch, Logstash, Beats, Kibana, and more.
# Articles
# System configuration
- A Useful Elasticsearch Cheat Sheet in Times of Trouble 在新窗口打开 (opens new window)
- The definitive guide for Elasticsearch on Windows Azure 在新窗口打开 (opens new window)
- Elasticsearch pre-flight checklist 在新窗口打开 (opens new window)
- 9 Tips on Elasticsearch Configuration for High Performance 在新窗口打开 (opens new window)
- Best Practices in AWS 在新窗口打开 (opens new window)
- How to Secure Elasticsearch and Kibana 在新窗口打开 (opens new window) with NGINX, LDAP and SSL 🔒
- Elasticsearch server on Webfaction using NGINX with basic authorization and HTTPS protocol 在新窗口打开 (opens new window)
- Elasticsearch Guides 在新窗口打开 (opens new window) Useful Elasticsearch guides with best practices, troubleshooting instructions for errors, tips, examples of code snippets and more.
# Docker and Elasticsearch
# Java tuning
- Elasticsearch Java Virtual Machine settings explained 在新窗口打开 (opens new window)
- Tuning Garbage Collection for Mission-Critical Java Applications 在新窗口打开 (opens new window)
- G1: One Garbage Collector To Rule Them All 在新窗口打开 (opens new window)
- Use Lucene's MMapDirectory on 64bit platforms, please! 在新窗口打开 (opens new window)
- Black Magic cookbook 在新窗口打开 (opens new window)
- G1GC Fundamentals: Lessons from Taming Garbage Collection 在新窗口打开 (opens new window)
- JVM Garbage Collector settings investigation 在新窗口打开 (opens new window) PDF Comparison of JVM GC
- Garbage Collection Settings for Elasticsearch Master Nodes 在新窗口打开 (opens new window) Fine tunine your garbage collector
- Understanding G1 GC Log Format 在新窗口打开 (opens new window) To tune and troubleshoot G1 GC enabled JVMs, one must have a proper understanding of G1 GC log format. This article walks through key things that one should know about the G1 GC log format.
How to start using G1
#ES_JAVA_OPTS=""
ES_JAVA_OPTS="-XX:-UseParNewGC -XX:-UseConcMarkSweepGC -XX:+UseG1GC"
1
2
3
2
3
# Scalable Infrastructure and performance
- The Authoritative Guide to Elasticsearch Performance Tuning (Part 1) 在新窗口打开 (opens new window) Part 2 在新窗口打开 (opens new window) Part 3 在新窗口打开 (opens new window)
- Tuning data ingestion performance for Elasticsearch on Azure 在新窗口打开 (opens new window) - and not only for Azure. That's a great article about Elasticsearch Performance testing by example
- Elasticsearch Indexing Performance Cheatsheet 在新窗口打开 (opens new window) - when you plan to index large amounts of data in Elasticsearch (by Patrick Peschlow)
- Elasticsearch for Logging 在新窗口打开 (opens new window) Elasticsearch configuration tips and tricks from Sanity
- Scaling Elasticsearch to Hundreds of Developers 在新窗口打开 (opens new window) by Joseph Lynch @yelp
- 10 Elasticsearch metrics to watch 在新窗口打开 (opens new window)
- Understanding Elasticsearch Performance 在新窗口打开 (opens new window)
- Our Experience of Creating Large Scale Log Search System Using Elasticsearch 在新窗口打开 (opens new window) - topology, separate master, data and search balancers nodes
- 📂 Elasticsearch on Azure Guidance 在新窗口打开 (opens new window) it is 10% on Azure and 90% of a very valuable general information, tips and tricks about Elasticsearch
- How to avoid the split-brain problem in Elasticsearch 在新窗口打开 (opens new window)
- Datadog's series about monitoring Elasticsearch performance:
- Performance Monitoring Essentials - Elasticsearch Edition 在新窗口打开 (opens new window)
- Operator for running Elasticsearch in Kubernetes 在新窗口打开 (opens new window)
# Integrations
- Apache Hive integration 在新窗口打开 (opens new window)
- Connecting Tableau to Elasticsearch (READ: How to query Elasticsearch with Hive SQL and Hadoop) 在新窗口打开 (opens new window)
- mradamlacey/elasticsearch-tableau-connector 在新窗口打开 (opens new window)
# Logging
- 5 Logstash Alternatives 在新窗口打开 (opens new window) and typical use cases
# Alerts
- ElastAlert: Alerting At Scale With Elasticsearch, Part 1 在新窗口打开 (opens new window) by engineeringblog.yelp.com
- ElastAlert: Alerting At Scale With Elasticsearch, Part 2 在新窗口打开 (opens new window) by engineeringblog.yelp.com
- Elastalert: implementing rich monitoring with Elasticsearch 在新窗口打开 (opens new window)
# Time series
- Elasticsearch as a Time Series Data Store 在新窗口打开 (opens new window) by Felix Barnsteiner
- Running derivatives on Voyager velocity data 在新窗口打开 (opens new window) By Colin Goodheart-Smithe
- Shewhart Control Charts via Moving Averages: Part 1 在新窗口打开 (opens new window) - Part 2 在新窗口打开 (opens new window) by Zachary Tong
- Implementing a Statistical Anomaly Detector: Part 1 在新窗口打开 (opens new window) - Part 2 在新窗口打开 (opens new window) - Part 3 在新窗口打开 (opens new window) by Zachary Tong
# Machine Learning
- Classifying images into Elasticsearch with DeepDetect 在新窗口打开 (opens new window) (forum thread with discussion 在新窗口打开 (opens new window)) by Emmanuel Benazera
- Elasticsearch with Machine Learning 在新窗口打开 (opens new window) (English translation 在新窗口打开 (opens new window)) by Kunihiko Kido
- Recommender System with Mahout and Elasticsearch 在新窗口打开 (opens new window)
# Use cases for Elasticsearch
- Data Infrastructure at IFTTT 在新窗口打开 (opens new window) Elasticsearch, Kafka, Apache Spark, Redhsift, other AWS services
- OFAC compliance with Elasticsearch 在新窗口打开 (opens new window) using AWS
- Building a Streaming Search Platform 在新窗口打开 (opens new window) - Streaming Search on Tweets: Storm, Elasticsearch, and Redis
# Other
- LogZoom, a fast and lightweight substitute for Logstash 在新窗口打开 (opens new window)
- Graylog2/graylog2-server 在新窗口打开 (opens new window) - Free and open source log management (based on ES)
- Fluentd vs. Logstash for OpenStack Log Management 在新窗口打开 (opens new window)
- Building a Directory Map With ELK 在新窗口打开 (opens new window)
- Structured logging with ELK - part 1 在新窗口打开 (opens new window)
- Search for 😋 Emoji with Elasticsearch 🔎 在新窗口打开 (opens new window)
- Complete Guide to the ELK Stack 在新窗口打开 (opens new window)
- logiq 在新窗口打开 (opens new window) - Simple WebUI Monitoring Tool for Logstash ver. 5.0 and up
- ElasticSearch Report Engine 在新窗口打开 (opens new window) - An ElasticSearch plugin to return query results as either PDF,HTML or CSV.
- Elasticsearch Glossary 在新窗口打开 (opens new window) - explanations of Elasticsearch terminology, including examples, common best practices and troubleshooting guides for various issues.
# Videos
# Overviews
- Elasticsearch for logs and metrics: A deep dive -- Velocity 2016 在新窗口打开 (opens new window) by Sematext Developers
- Elasticsearch in action Thijs Feryn 在新窗口打开 (opens new window) a beginner overview
- Getting Down and Dirty with ElasticSearch by Clinton Gormley 在新窗口打开 (opens new window)
- How we scaled Raygun 在新窗口打开 (opens new window)
- Getting started with Elasticsearch 在新窗口打开 (opens new window)
- Speed is a Key: Elasticsearch under the Hood 在新窗口打开 (opens new window) introduction + basic performance optimization
- $$ Pluralsight: Getting Started With Elasticsearch for .NET Developers 在新窗口打开 (opens new window) this course will introduce users to Elasticsearch, how it works, and how to use it with .NET projects.
- $$ Complete Guide to Elasticsearch 在新窗口打开 (opens new window) Comprehensive guide to Elasticsearch, the popular search engine built on Apache Lucene
- How Elasticsearch powers the Guardian's newsroom 在新窗口打开 (opens new window)
- Elasticsearch Query Editor in Grafana 在新窗口打开 (opens new window)
- Scale Your Metrics with Elasticsearch 在新窗口打开 (opens new window) 2019 by Philipp Krenn (Elastic) optimization tips and tricks
# Advanced
- #bbuzz 2015: Adrien Grand -- Algorithms and data-structures that power Lucene and Elasticsearch 在新窗口打开 (opens new window)
- Rafał Kuć - Running High Performance Fault-tolerant Elasticsearch Clusters on Docker 在新窗口打开 (opens new window) and slides 在新窗口打开 (opens new window)
- Working with Elasticsearch - Search, Aggregate, Analyze, and Scale Large Volume Datastores 在新窗口打开 (opens new window) - O'Reilly Media
- End-to-end Recommender System with Spark and Elasticsearch 在新窗口打开 (opens new window) by Nick Pentreath & Jean-François Puget. Slide deck 在新窗口打开 (opens new window)
# Code, configuration file samples and other gists
- Elasticsearch config for a write-heavy cluster 在新窗口打开 (opens new window) - reyjrar/elasticsearch.yml
- chenryn/ESPL - Elastic Search Processing Language 在新窗口打开 (opens new window) PEG parser sample for SPL to Elasticsearch DSL
- thomaspatzke/EQUEL 在新窗口打开 (opens new window) an Elasticsearch QUEry Language, based on G4 grammar parser
# Who is using elasticsearch?
Yelp 在新窗口打开 (opens new window), IFTTT 在新窗口打开 (opens new window), StackExchange 在新窗口打开 (opens new window), Raygun 在新窗口打开 (opens new window), Mozilla 在新窗口打开 (opens new window), Spotify 在新窗口打开 (opens new window), CERN 在新窗口打开 (opens new window), NASA 在新窗口打开 (opens new window)Zalando 在新窗口打开 (opens new window)
# I want more! (Elasticsearch related resources)
- Technology Explained Blog 在新窗口打开 (opens new window)
- EagerElk 在新窗口打开 (opens new window)
- Tim Roes Blog 在新窗口打开 (opens new window)
# Contributing
- Make sure you are about to post a valuable resource that belongs to this list
- Do NOT group ++Add and --Remove changes in same PR. Make them separate pull requests
- Use spellchecker
- All spelling and grammar corrections are welcome (except for the rule above)
- Fork this repo, do your edits, send the pull request
- Feel free to create any new sections
- Do not even try to add this repo to any awesome-awesome-* lists