lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexandre Rafalovitch <arafa...@gmail.com>
Subject Re: How to limit the number of result sets of the 'export' handler
Date Wed, 07 Jan 2015 14:49:38 GMT
I believe export is streaming and it avoids building various caches,
so it will not blow up Solr's memory on large datasets.

You can read a lot more details in the JIRA that introduced it:
https://issues.apache.org/jira/browse/SOLR-5244

I am not sure how it compares with deep-paging though.

Regards,
   Alex.
----
Sign up for my Solr resources newsletter at http://www.solr-start.com/


On 7 January 2015 at 01:26, Sandy Ding <sandy.dingxin@gmail.com> wrote:
> Thanks Alexandre.
> I actually need the whole result set. But it is large(perhaps 10m-100m) and
> I find select is slow.
> How does export differ from select except that select will make distributed
> requests and do the merge?
> Will select with ‘distrib=false’ have comparable performance with export?
>
>
> 2015-01-06 20:55 GMT+08:00 Alexandre Rafalovitch <arafalov@gmail.com>:
>
>> Export was specifically designed to get everything which is very
>> expensive otherwise.
>>
>> If you just want the subset, you might be better off with normal
>> queries and/or with deep paging (cursor).
>>
>> Regards,
>>    Alex.
>> ----
>> Sign up for my Solr resources newsletter at http://www.solr-start.com/
>>
>>
>> On 6 January 2015 at 00:30, Sandy Ding <sandy.dingxin@gmail.com> wrote:
>> > Using rows=xxx doesn't seem to work.
>> > Is there a way to do this?
>>

Mime
View raw message