lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Lea <ian....@gmail.com>
Subject Re: Rendexing problem: Indexing folder size is keep on growing for same remote folder
Date Tue, 01 Oct 2013 16:51:48 GMT
I'm still a bit confused about exactly what you're indexing, when, but
if you have a unique id and don't want to add or update a doc that's
already present, add the unique id to the index and search (TermQuery
probably) for each one and skip if already present.

Can't you change the log rotation/copying/indexing so that you only
index new data?

To start a fresh index, use IndexWriterConfig.OpenMode.CREATE.


--
Ian.


On Tue, Oct 1, 2013 at 4:51 PM, gudiseashok <gudise.ashok@gmail.com> wrote:
> Hi
>
> Basically my log folder consists of four log files like
> abc.log,abc1.log,abc2.log,abc3.log, as my log appender is doing. Every 30
> minutes content will be changed of all these file , for example after 30
> minutes refresh my conent of abc1.log will be replaced with existing abc.log
> content and abc.log will have new content (Timestamp is DD-MM-YYYY MM-ss:S).
> Since I am goingthrough the re-indexing for every 30 minutes, I dont want to
> re-index the same record which was already present with same timstamp.
>
> Also if I want to do clean-up for every week, (clean up in the sense I want
> to delete all indexes , and I want to do a fresh indexing for these 4
> files), how to do this efficiently.
>
> I really appreciate your time reading this, and kindly suggest a better way.
>
>
> Regards
> Ashok Gudise
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Rendexing-problem-Indexing-folder-size-is-keep-on-growing-for-same-remote-folder-tp4092835p4092963.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message