lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: DIH Speed
Date Thu, 27 Apr 2017 23:40:41 GMT
I'm unclear why DIH an deep paging are  mixed. DIH is
indexing and deep paging is querying.

If it's querying, consider cursorMark or the /export handler.
https://lucidworks.com/2013/12/12/coming-soon-to-solr-efficient-cursor-based-iteration-of-large-result-sets/

If it's DIH, please explain a bit more.

Best,
Erick

On Thu, Apr 27, 2017 at 3:37 PM, Vijay Kokatnur
<kokatnur.vijay@gmail.com> wrote:
> We have a new solr 6.5.0 cluster, for which data is being imported via DIH
> from another Solr cluster running version 4.5.0.
>
> This question comes back to deep paging, but we have observed that after 30
> minutes of querying the rate of processing goes down from 400/s to about
> 120/s.  At that point it has processed only 500K of 1.3M docs.  Is there
> any way to speed this up?
>
> And, I can't go back to the source for the data.
>
> --

Mime
View raw message