Scaling Massive ElasticSearch Clusters

This talk describes how we've used ElasticSearch to build massive search clusters capable of indexing several thousand documents per second while at the same time serving a few hundred QPS over billions of documents in well under a second. We'll talk about building clusters that continuously grow in terms of both indexing and search rates. You will learn about finding cluster nodes that can handle more documents, about managing shard and replica allocation and prevention of shard rebalancing, about avoiding expensive distributed queries, etc. We'll also describe our experience doing performance testing of several ElasticSearch clusters and will share our observations about what settings affect search performance and how much. In this talk you'll also learn how to monitor large ElasticSearch clusters, what various metrics mean, and which ones to pay extra attention to.

Watch the video of Rafał Kuć's talk here.