Monthly Archives: March 2019

Tuning ElasticSearch for running on one node

The default settings for shards and replicates are 5 and 2 respectively. This is a bit overkill with my tiny data. I’m running on one node only, and I have < 1GB data per month.

First off, turn off replication by doing

This will take effect immediately.

Tuning down the number of shards used per index is a bit more complex. We need to create an index template with this setting, and it will only affect future indices.

If the database already contains indices split over several shards, this data needs to be reindexed.

Example: Reindex multiple (e.g. daily) logs:

First reindex into temporary indicies

Delete the originals

Reindex back to the original name

Any of these might time out during the call, so it could be a good idea to keep track of the document count manually.

Example: Reindex to aggregate logs

Reindexing can also be used to aggregate daily logs into monthly. I started out with daily logs but this was really overkill.

Doing this I cut down on the number of shards used from ~3800 to 28, and the system suddenly feels responsive again – even for Kibana dashboards with 10+ visualizations pulling data from 2 years back.