lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Avlesh Singh <avl...@gmail.com>
Subject Re: mergeFactor / indexing speed
Date Sun, 09 Aug 2009 08:59:03 GMT
>
> And - indexing 160k documents now takes 5min instead of 1.5h!
>
Awesome! It works for all!

(Now I can go relaxed on vacation. :-D )
>
Take me along!

Cheers
Avlesh

On Fri, Aug 7, 2009 at 3:58 PM, Chantal Ackermann <
chantal.ackermann@btelligent.de> wrote:

> Juhu, great news, guys. I merged my child entity into the root entity, and
> changed the custom entityprocessor to handle the additional columns
> correctly.
> And - indexing 160k documents now takes 5min instead of 1.5h!
>
> (Now I can go relaxed on vacation. :-D )
>
>
> Conclusion:
> In my case performance was so bad because of constantly querying a database
> on a different machine (network traffic + db query per document).
>
>
> Thanks for all your help!
> Chantal
>
>
> Avlesh Singh schrieb:
>
>> does DIH call commit periodically, or are things done in one big batch?
>>>
>>>  AFAIK, one big batch.
>>
>
> yes. There is no index available once the full-import started (and the
> searcher has no cache, other wise it still reads from that). There is no
> data (i.e. in the Admin/Luke frontend) visible until the import is finished
> correctly.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message