lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis.gospodne...@gmail.com>
Subject Re: How does solr 4.2 do in returning large datasets ?
Date Mon, 01 Apr 2013 21:49:22 GMT
Don't forget you could also tell Solr to return a CSV in response,
which will be much lighter in terms of response size, though disk IO
would still be there.  Is returning large result sets recommended? No.
:)  But the same can be said for Lucene.

Otis
--
Solr & ElasticSearch Support
http://sematext.com/





On Mon, Apr 1, 2013 at 10:26 AM, Swati Swoboda
<sswoboda@igloosoftware.com> wrote:
> It really depends on what you are returning (how big is each document? Just a document
ID? Pages and pages of data in fields?).
>
> It can take a long time for Solr to render an XML with 60,000 results. Solr will be serializing
the data and then you'd (presumably) be de-serializing it. Depending on how big each field
actually is, this could take a while or even cause DOS on your server.
>
> Your client would also need a fair bit of memory to parse a document with 60,000 results....
>
>
> -----Original Message-----
> From: Liz Sommers [mailto:lizsworks@gmail.com]
> Sent: Monday, April 01, 2013 9:39 AM
> To: solr-user
> Subject: How does solr 4.2 do in returning large datasets ?
>
> I thought I remembered reading that Solr is not good for returning large datasets.  We
are currently using lucene 3.6.0 and returning datasets of
> 10,000 to 60,000 results.  In the future we might need to return even larger datasets.
>
> Would you all recommend going to Solr for this, or should we stick with Lucene (which
has given us no problems in this regard)?  I am a bit wary of using a web service to return
datasets of this size.
>
> Thanks a lot
> Liz
> lizsworks@gmail.com

Mime
View raw message