lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Re: Upgrading cluster from 4 to 5. Slow replication detected.
Date Tue, 18 Apr 2017 13:28:45 GMT
On 4/14/2017 2:10 AM, Himanshu Sachdeva wrote:
> We're starting to upgrade our solr cluster to version 5.5. So we
> removed one slave node from the cluster and installed solr 5.5.4 on it
> and started solr. So it started copying the index from the master.
> However, we noticed a drop in the replication speed compared to the
> other nodes which were still running solr 4. To do a fair comparison,
> I removed another slave node from the cluster and disabled replication
> on it till the new node has caught up with it. When both these nodes
> were at the same index generation, I turned replication on for both
> the nodes. Now, it has been over 15 hours since this exercise and the
> new node has again started lagging behind. Currently, the node with
> solr 5.5 is seven generations behind the other node.

Version 5 is capable of replication bandwidth throttling, but unless you
actually configure the maxWriteMBPerSec attribute in the replication
handler definition, this should not happen by default.

One problem that I think might be possible is that the heap has been
left at the default 512MB on the new 5.5.4 install and therefore the
machine is doing constant full garbage collections to free up memory for
normal operation, which would make Solr run EXTREMELY slowly. 
Eventually a machine in this state would most likely encounter an
OutOfMemoryError.  On non-windows systems, OOME will cause a forced halt
of the entire Solr instance.

The heap might not be the problem ... if it's not, then I do not know
what is going on.  Are there any errors or warnings in solr.log?

Thanks,
Shawn


Mime
View raw message