lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Potter <thelabd...@gmail.com>
Subject Re: Processing a lot of results in Solr
Date Wed, 24 Jul 2013 01:08:47 GMT
Hi Matt,

This feature is commonly known as deep paging and Lucene and Solr have
issues with it ... take a look at
http://solr.pl/en/2011/07/18/deep-paging-problem/ as a potential
starting point using filters to bucketize a result set into sets of
sub result sets.

Cheers,
Tim

On Tue, Jul 23, 2013 at 3:04 PM, Matt Lieber <mlieber@impetus.com> wrote:
> Hello Solr users,
>
> Question regarding processing a lot of docs returned from a query; I
> potentially have millions of documents returned back from a query. What is
> the common design to deal with this ?
>
> 2 ideas I have are:
> - create a client service that is multithreaded to handled this
> - Use the Solr "pagination" to retrieve a batch of rows at a time ("start,
> rows" in Solr Admin console )
>
> Any other ideas that I may be missing ?
>
> Thanks,
> Matt
>
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged
or otherwise protected by law. The message is intended solely for the named addressee. If
received in error, please destroy and notify the sender. Any use of this email is prohibited
when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity
of this communication has been maintained nor that the communication is free of errors, virus,
interception or interference.

Mime
View raw message