lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lance Norskog <goks...@gmail.com>
Subject Re: Solr searching performance issues, using large documents (now 1MB documents)
Date Thu, 26 Aug 2010 02:46:29 GMT
How much disk space is used by the index?

If you run the Lucene CheckIndex program, how many terms etc. does it report?

When you do the first facet query, how much does the memory in use grow?

Are you storing the text fields, or only indexing? Do you fetch the
facets only, or do you also fetch the document contents?

On Wed, Aug 25, 2010 at 11:34 AM, Peter Spam <pspam@mac.com> wrote:
> This is a very small number of documents (7000), so I am surprised Solr is having such
a hard time with it!!
>
> I do facet on 3 terms.
>
> Subsequent "hello" searches are faster, but still well over a second.  This is a very
fast Mac Pro, with 6GB of RAM.
>
>
> Thanks,
> Peter
>
> On Aug 25, 2010, at 9:52 AM, Yonik Seeley wrote:
>
>> On Wed, Aug 25, 2010 at 11:29 AM, Peter Spam <pspam@mac.com> wrote:
>>> So, I went through all the effort to break my documents into max 1 MB chunks,
and searching for hello still takes over 40 seconds (searching across 7433 documents):
>>>
>>>        8 results (41980 ms)
>>>
>>> What is going on???  (scroll down for my config).
>>
>> Are you still faceting on that query also?
>> Breaking your docs into many chunks means inflating the doc count and
>> will make faceting slower.
>> Also, first-time faceting (as with sorting) is slow... did you try
>> another query after  "hello" (and without a commit happening
>> inbetween) to see if it was faster?
>>
>> -Yonik
>> http://lucenerevolution.org  Lucene/Solr Conference, Boston Oct 7-8
>
>



-- 
Lance Norskog
goksron@gmail.com

Mime
View raw message