lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fuad Efendi" <>
Subject RE: frequency of commit when building index from scratch
Date Tue, 25 Aug 2009 21:43:18 GMT
I do commit once a day, millions of small docs... it takes 20 minutes in
average... why OOM? I see only reduced I/O...

-----Original Message-----
From: Edward Capriolo [] 
Sent: August-25-09 5:35 PM
Subject: Re: frequency of commit when building index from scratch

On Tue, Aug 25, 2009 at 5:29 PM, Bill Au<> wrote:
> Just curious, how often do folks commit when building their Solr/Lucene
> index from scratch for index with millions of documents?  Should I just
> and do a single commit at the end after adding all the documents to the
> index?
> Bill

Bill in most cases you probably cannot do one large commit as you will
hit OOM. How many documents can be uncommitted is based on the size of
the documents. Committing every document is slow. I have done a commit
every 10,000 mostly. Results may vary. Someone might have a better
answer then me.

View raw message