lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <markrmil...@gmail.com>
Subject Re: Recovery - too many updates received since start
Date Thu, 26 Apr 2012 17:22:40 GMT

On Apr 24, 2012, at 9:31 AM, Trym R. Møller wrote:

> Hi
> 
> I experience that a Solr looses its connection with Zookeeper and re-establish it. After
Solr is reconnection to Zookeeper it begins to recover.
> It has been missing the connection approximately 10 seconds and meanwhile the leader
slice has received some documents (maybe about 1000 documents). Solr fails to update peer
sync with the log message:
> Apr 21, 2012 10:13:40 AM org.apache.solr.update.PeerSync sync
> WARNING: PeerSync: core=mycollection_slice21_shard1 url=zk-1:2181,zk-2:2181,zk-3:2181
too many updates received since start - startingUpdates no longer overlaps with our currentUpdates

You can configure the timeout here - I may have chosen a default that is too low based on
some reports.

> 
> Looking into PeerSync and UpdateLog I can see that 100 updates is the maximum allowed
updates that a shard can be behind.
> Is it correct that this is not configurable and what is the reasons for choosing 100?

Yonik chose this - I'll let him expand on it if he see this. I think it's not configurable
currently, but perhaps with the right caveats as doc, it should be.


> 
> I suspect that one must compare the work needed to replicate the full index with the
performance loss/resource usage when enhancing the size of the UpdateLog?

Yeah, I think that is the gist of it. There may be another gotchya or two, I just don't remember
at the moment. Yonik?

> 
> Any comments regarding this is greatly appreciated.
> 
> Best regards Trym

- Mark Miller
lucidimagination.com












Mime
View raw message