lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "harish.agarwal" <harish.agar...@gmail.com>
Subject Commits, facet autowarming and hung searches
Date Wed, 12 May 2010 02:26:42 GMT

I'm trying to debug an issue I have surrounding a facet heavy 12M document
index I'm running, sharded across three nodes.  In summary, occasionally one
of the nodes will slowdown to a snail's pace on returning search results
(10's of seconds out to minutes), causing search on the index to hang. 
Sometimes the node recovers, but it can take up to an hour and even then its
no guarantee.  

In detail - the problematic behavior I'm observing occurs when I update the
index (sometimes even with a small number of changes) and run a commit.  I'm
not updating frequently, about once every 2 hours, and only commit once
after all of my changes have been made (which are usually as few as 10,000). 
I'm curious if my problem is related to a simple mismanagement of
configuration variables.  As mentioned, I'm running a very facet heavy index
and most searches will require every facet.  In general, when a new searcher
is registered in the logs, the first thing which happens is all of my
faceted fields are uninverted before searches  can be served.  This can take
as long as 30 seconds.  So, my question is:

-Should I be warming the new searcher with a search which uninverts all of
my fields?
-If so, will the warming happen before the new searcher is registered?
-Will searches coming in to the index before the new searcher is registered
but after its created go to the old searcher?
-Will my memory usage spike during this interval?

I'd really appreciate any help with this issue.

-Harish
-- 
View this message in context: http://lucene.472066.n3.nabble.com/Commits-facet-autowarming-and-hung-searches-tp811584p811584.html
Sent from the Solr - User mailing list archive at Nabble.com.

Mime
View raw message