Just dragged our ELK stack up to 7.0 from 6.5 and I'm unimpressed with @elastic cluster.max_shards_per_node being set to 1000, we generate about 1000 shards per month and on our 4 node cluster that works really well for us.
1
1) it's a soft limit. change it if you have to 2) that's a surprising number of shards. we generally recommend elastic.co/blog/how-many-sha… for sizing that 3) why do you need so many? also rollover / ILM might be a better pattern to use?
2
1
3
Replying to @xeraa @elastic
WRT sizing; seems to be fine so far, this cluster has been production for several years now, started out at ES 1.4 I think? We just have a huge volume of data being dumped into it. Rollover is something I have thought about in the past however, ain't broke don't fix?

Jul 4, 2019 · 2:38 PM UTC

1
Replying to @imranh2 @elastic
soft limit as in: it can be changed according to your needs sure, if it's working keep doing that. it's just not really what we plan for or test, so no guarantees. just trying to point out other approaches for moving forward :)
1
We'll agree to disagree on the definition then :) You're not the first person from @Elastic to tell me my clusters not orthodox. Thanks for reaching out though, I tweeted a faceless corporation, didn't expect someone to try and help!
1
more replies