lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Lea <>
Subject Re: Rendexing problem: Indexing folder size is keep on growing for same remote folder
Date Tue, 01 Oct 2013 16:51:48 GMT
I'm still a bit confused about exactly what you're indexing, when, but
if you have a unique id and don't want to add or update a doc that's
already present, add the unique id to the index and search (TermQuery
probably) for each one and skip if already present.

Can't you change the log rotation/copying/indexing so that you only
index new data?

To start a fresh index, use IndexWriterConfig.OpenMode.CREATE.


On Tue, Oct 1, 2013 at 4:51 PM, gudiseashok <> wrote:
> Hi
> Basically my log folder consists of four log files like
> abc.log,abc1.log,abc2.log,abc3.log, as my log appender is doing. Every 30
> minutes content will be changed of all these file , for example after 30
> minutes refresh my conent of abc1.log will be replaced with existing abc.log
> content and abc.log will have new content (Timestamp is DD-MM-YYYY MM-ss:S).
> Since I am goingthrough the re-indexing for every 30 minutes, I dont want to
> re-index the same record which was already present with same timstamp.
> Also if I want to do clean-up for every week, (clean up in the sense I want
> to delete all indexes , and I want to do a fresh indexing for these 4
> files), how to do this efficiently.
> I really appreciate your time reading this, and kindly suggest a better way.
> Regards
> Ashok Gudise
> --
> View this message in context:
> Sent from the Lucene - Java Users mailing list archive at
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message