lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <>
Subject Re: commit frequency guideline?
Date Fri, 02 Dec 2016 11:27:07 GMT
On Wed, Nov 30, 2016 at 9:37 AM, Rob Audenaerde
<> wrote:
> Thanks for the quick reply!
>>What do you mean by "Lucene complain about too-many uncommitted docs"?
> --> good question, I was thoughtlessly echoing words from my colleague. I
> asked him and he said that it was about taking very long to commit and
> memory issues. So maybe this wasn't the best opening statement :)

Ahhh.  Well, fsync (which Lucene invokes on multiple files when you
call commit) is inherently a costly (in wall clock time) operation
since the OS and IO system may have to move many bytes to the
underlying durable storage.  But you should not hit memory issues.

> For the other part of the question: we need users to see the changed
> documents immediately, but I think we have this covered by using NRT Readers
> and the SearcherManager.
> Am I correct to conclude calling commit() is not necessary for finding
> recently changed documents?

That's correct: NRT readers give you fast visibility of changes, while
commit gives you durability.  The two are entirely orthogonal in
Lucene, as long as you don't use a single thread to call commit and
call refresh.

> I think we can then switch to a time based commit() where we just call
> commit every 5 minutes, in effect losing a maximum of 5 minutes of work
> (which we can mitigate in another way)
>  when the server somehow stops working.


> Thank you,

You're welcome!

Mike McCandless

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message