lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <markrmil...@gmail.com>
Subject Re: A few operations questions about the tlog (UpdateLog)
Date Fri, 01 Mar 2013 03:25:06 GMT
To add:

Current best practice is to do a hard commit with openSearcher=false every minute or so. AutoCommit
is great for this. It shouldn't affect your overall indexing performance and it will constrain
the transaction log.

- Mark

On Feb 28, 2013, at 8:35 PM, Erick Erickson <erickerickson@gmail.com> wrote:

> A new tlog gets created and the current one is closed on a hard commit
> (openSearcher=true or false doesn't matter). The old one will be kept
> around for a bit, I suspect if you'd done it a third time the first one
> would go away. For all I know, the code might read if (tlog docs > 100) and
> you may be hitting an off-by-one situation.
> 
> But you're right. If you follow older practices and _never_ commit until
> the end of a really long indexing process, you'll see these logs grow huge.
> Even worse, anytime they're replayed they'll then re-index a lot of
> documents.
> 
> So I'd try the experiment over again with 110 docs.... But I can say
> they're getting purged on my machine quite regularly with some stress
> testing I'm doing.
> 
> Best,
> Erick
> 
> 
> On Thu, Feb 28, 2013 at 7:59 AM, adityab <aditya_bakle@yahoo.com> wrote:
> 
>> whats the life cycle of a tlog file. Is it purged after commit (even with
>> soft commit) ?
>> I posted 100 docs to solr (standalone) did hard commit. Observed a new tlog
>> file is created.
>> re-posted the same 100 docs and did hard commit. Observed a new tlog file
>> is
>> created. Old one still exists.
>> 
>> When do they get purged. Concern is we have at least 20K docs published
>> every 2hrs so need to understand if its safe to put them in a different
>> location where we can have a script to purge old files at regular interval.
>> 
>> thanks
>> 
>> 
>> 
>> 
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/A-few-operations-questions-about-the-tlog-UpdateLog-tp4042560p4043618.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>> 


Mime
View raw message