lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tommaso Teofili <tommaso.teof...@gmail.com>
Subject merge policy vs commit rates
Date Tue, 01 Aug 2017 08:58:45 GMT
Hi all,

lately I am looking a bit closer at merge policies, of course particularly
at the tiered one, and I was wondering if we can mitigate the amount of
possibly avoidable merges in high commit rates scenarios, especially when a
high percentage of the commits happens on same docs.
I've observed several evolutions of merges in such scenarios and it seemed
to me the merge policy was too aggressive in merging, causing a large IO
overhead.
I've then tried the same with a merge policy which was tentatively looking
at commit rates and skipping merges if such a rate is higher than a
threshold which seemed to give slightly better results in reducing the
unneeded IO caused by avoidable merges.

I know this is a bit abstract but I would like to know if anyone has any
ideas or plans about mitigating the merge overhead in general and / or in
similar cases.

Regards,
Tommaso

Mime
View raw message