lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Schedule merges during off-peak hours?
Date Mon, 02 May 2016 04:45:09 GMT
Bram:

I'd look at other reasons that merging impacts performance. The first
place I'd look would be frequency of commits and autowarming. My
suspicion is that what you're noticing.

If "commit intervals of around one minute" mean that you're taking 60
seconds for the commit to finish, then perhaps it's the opposite and
you're autowarming too much. I usually start my autowarm counts
around 16 or so for filterCache and queryResultCache...

FWIW,
Erick

On Sun, May 1, 2016 at 11:04 AM, Bram Van Dam <bram.vandam@intix.eu> wrote:
> Hey folks,
>
> Disclaimer: I haven't given this too much thought, so feel free to shoot
> me down if this is a stupid idea.
>
> We have a lot of Solr instances with a lot of different customers, and
> most of them fit into a patterb of 9-5 searching and slow but steady
> addition of documents 24/7, at a rate of about 10docs/second.
>
> This means there's hardly any searching going on at night. This also
> means that merges during the day can be a problem when indexes grow
> large, especially with commit intervals of around one minute. This seems
> wasteful, because the index doesn't grow *that* much larger during a
> single working day.
>
> I was wondering if it would be a good idea to create a MergePolicy that
> only merges segments during off-peak hours (or maybe according to a
> cron-style pattern or something).
>
> Any thoughts?
>
> Thanks,
>
>  - Bram
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message