lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Re: DIH Speed
Date Fri, 28 Apr 2017 00:07:05 GMT
On 4/27/2017 5:40 PM, Erick Erickson wrote:
> I'm unclear why DIH an deep paging are mixed. DIH is indexing and deep paging is querying.
>
> If it's querying, consider cursorMark or the /export handler. https://lucidworks.com/2013/12/12/coming-soon-to-solr-efficient-cursor-based-iteration-of-large-result-sets/

Very likely they are using SolrEntityProcessor.

Vijay, if the source server were running 4.7 (or later) instead of 4.5,
you could enable cursorMark for SolrEntityProcessor in 6.5.0 as Erick
mentioned, and pagination would be immensely more efficient. 
Unfortunately, 4.5 doesn't support cursorMark.

https://issues.apache.org/jira/browse/SOLR-9668

Any chance you could upgrade the source server to a later 4.x version?

Thanks,
Shawn


Mime
View raw message